01RAG in one breath — then what "agentic" adds

Retrieval-augmented generation (RAG) gives a language model an open book. Instead of answering from memory alone, the model first retrieves relevant passages from an outside source — a search index, a vector database, your documents — and uses them to ground its answer. The original formulation (Lewis et al., 2020) combined the model's built-in parametric memory with a non-parametric retrieval index, and that pairing became the standard recipe for grounding answers in real sources.

"Naive" or traditional RAG runs that recipe as a single, fixed pass: take the question → retrieve once → generate. It works well when the right passage is one hop away. But it has no way to ask "should I even retrieve for this?", "is what I got back actually good enough?", or "do I need to look again with a better query?" — so when retrieval comes back thin or off-topic, naive RAG answers anyway.

Agentic RAG embeds an autonomous agent into that pipeline. According to the 2025 survey on the topic, it applies agentic design patterns — reflection, planning, tool use, and multi-agent collaboration — to dynamically manage retrieval rather than running one fixed pass. The retrieval substrate (vectors, chunks, indexes) is the same; what changes is that a decision-maker now sits on top of it, looping until the evidence is good enough to answer.

• RAG = retrieve relevant text, then generate an answer grounded in it (parametric memory + a retrieval index).
• Naive RAG is one fixed pass: retrieve once, then answer — no checking, no second look.
• Agentic RAG adds an agent that decides, grades, and iterates — reflection, planning, and tool use on top of the same index.

02See it: one-shot vs. a self-correcting loop

Here's the same question — "What changed in our refund policy last quarter?" — run through both styles. In naive mode it retrieves once and answers from whatever came back. In agentic mode the agent first decides retrieval is needed, grades the chunks it gets, finds them off-topic, reformulates the query, retrieves again, and only then answers. Pick a mode and step through it.

• Naive reaches the answer in two moves — but if the one retrieval misses, it grounds the answer on weak evidence.
• Agentic spends extra steps deciding, grading, and re-querying — trading latency and cost for a higher chance the evidence is right.
• The branch point is the grader: "is this good enough to answer?" Naive RAG never asks; agentic RAG loops on a "no".

03The four decisions an agent adds to retrieval

"Agentic RAG" is an umbrella term, not one algorithm — different papers contribute different decision points. Switch between them to see the four that show up most often, and the technique each one comes from.

• These are distinct techniques — Adaptive-RAG, FLARE, CRAG, Self-RAG — not one paper that defines the field.
• The reasoning loop tying them together is often ReAct: interleave a reasoning trace with actions (retrieval calls), observe the result, and revise.

04What the agent routes over: beyond flat chunks

An agent's re-querying is only as good as what it can reach. Naive RAG usually searches one flat set of text chunks in a vector index. Agentic RAG can plan multi-step retrieval across different kinds of knowledge structures, picking the one that fits the question:

• Flat vector chunks — the default: split documents into pieces, embed them, and retrieve by similarity. Fast and simple, but a single chunk rarely answers a broad, corpus-wide question.
• Hierarchical (RAPTOR) — recursively embed, cluster, and summarize passages into a tree, so the agent can retrieve at the right level of abstraction (a detailed leaf, or a high-level summary).
• Graph-based (GraphRAG) — build an entity knowledge graph plus community summaries, which suits global, "what are the themes across everything?" questions that flat retrieval handles poorly.

A practical framework pattern (LlamaIndex) wraps each of these retrievers as a tool — a QueryEngineTool — so a ReAct agent can choose which one to call, or decompose a complex query into sub-queries that run against several at once. OpenAI's file-search tooling similarly breaks complex queries into multiple parallel searches and reranks the combined results. The agent's job is routing and synthesis; the structures are what it routes over.

05Better evidence in, better loops out

An agent that re-queries still depends on the retrieval being able to surface the right passage at all. Two layers help: retrieval-quality techniques that make each lookup stronger, and a framework control loop that wires the decisions together.

On quality: Anthropic's Contextual Retrieval prepends chunk-specific context to each chunk before embedding and BM25 indexing (it calls these Contextual Embeddings + Contextual BM25). Anthropic reports up to a 49% reduction in failed retrievals, and up to 67% when combined with reranking. Treat those as vendor-reported first-party results, and note that retrieval-accuracy numbers are dataset- and configuration-dependent — no single percentage is a universal property of agentic RAG.

On the loop: frameworks like LangGraph model agentic RAG as a graph. An agent node decides whether to call the retriever tool; conditional edges route the flow; and document-grading nodes (a router, a grader, sometimes a hallucination checker) enable the re-retrieve-and-validate cycle. That graph is the concrete shape of the "decide → grade → re-query → answer" loop you stepped through above.

• Quality first: stronger embeddings/indexing (e.g., contextual retrieval + reranking) reduce how often the agent has to loop at all.
• The loop in code: agent node + conditional edges + grader nodes = a re-retrieve-and-validate cycle (LangGraph).
• Read vendor numbers carefully: first-party benchmarks are configuration-dependent; cite them as vendor-reported, not as universal truths.

07Take it with you & go deeper