RAG architecture: one-pass search, agents, or GraphRAG?
Start with one retrieval pass that supplies relevant source passages to the model. Add iterative search when an answer needs a second lookup whose query depends on the first result. Test GraphRAG when questions need relationships or a view of the whole collection that ordinary passage retrieval misses.
These are choices about how to gather evidence, not stages every RAG system must grow through. I would first inspect failed questions and the passages the model actually received. More search cannot repair text that the document parser discarded.
Last reviewed: 2026-09-08.
Match the search to the question
| Question shape | First approach to test | Added work to justify |
|---|---|---|
| “What is the reset command for this device?” | One retrieval pass, then an answer with a source | A relevant passage must reach the answer model |
| “Which replacement fits the part named in this report?” | Iterative search: find the part, then query the parts catalog | More calls, a stopping rule, and tests of both lookups |
| “Which themes recur across these reports?” | GraphRAG global search, compared with a simpler summary approach | Building and refreshing graph-derived summaries |
| “How are these suppliers connected?” | Graph-aware retrieval, compared with repeated passage search | Correct entity links, relationships, and source support |
Microsoft’s GraphRAG query documentation distinguishes local search over extracted entities plus source chunks from global search over generated community reports. A community report summarizes a group in the extracted graph. Global search uses those reports to address questions about the collection as a whole. It is not just a vector lookup with a new name.
Iterative retrieval is a separate design choice and can use ordinary text search. Adaptive-RAG studies routing questions between no retrieval, a single step, and multiple retrieval steps. That supports testing different paths; it does not establish which one wins on your corpus.
Check the missing evidence before changing architecture
For each failed question, keep the expected source and the passages retrieved. Then locate the first loss:
- If the source never entered the index, fix ingestion or parsing.
- If an optional topic filter excluded it, test the filter or retain a broader search path. Keep permission filters enforced on every path.
- If the passage was retrieved but ranked too low, test retrieval and reranking changes.
- If a needed name appears only in the first result, test a second lookup using that name.
- If the model received enough evidence but answered incorrectly, inspect the answer step before adding more retrieval.
The search stack answer covers BM25, embeddings, and rerankers within a retrieval pass. Those components can also serve an iterative system; they do not decide how many searches the system should make.
Make the larger design earn its cost
Compare the paths on the same questions and allowed documents. Give each an explicit time and model-call budget. Measure whether it found the required evidence, answered correctly, and cited passages that support its claims. Keep timeouts and exhausted search budgets among the failures.
For a graph, inspect extracted entities and relationships as well as final answers. A wrong supplier link can produce a coherent answer about the wrong company. Include index-building and refresh costs in the comparison; query cost alone misses that work.
If one-pass retrieval meets the target, stop there. Add another path for the question types where a measured failure gives it a job.
Deeper reading
- RAG evaluation metrics locates evidence loss in parsing, filtering, retrieval, and generation.
- LLM engineering field guide maps the ingestion and query paths and introduces GraphRAG.
- Enterprise RAG Challenge 3 examines public entries, routing, evidence, and their limits.