Retrieval-augmented generation (RAG) and its graph-based extensions (GraphRAG) are effective paradigms for improving large language model (LLM) reasoning by grounding generation in external knowledge. However, most existing RAG and GraphRAG systems operate under static or one-shot retrieval, where a fixed set of documents is provided to the LLM in a single pass. In contrast, recent agentic search systems enable dynamic, multi-round retrieval and sequential decision-making during inference, and have shown strong gains when combined with vanilla RAG by introducing implicit structure through interaction. This progress raises a fundamental question: can agentic search compensate for the absence of explicit graph structure, reducing the need for costly GraphRAG pipelines? To answer this question, we introduce RAGSearch, a unified benchmark that evaluates dense RAG and representative GraphRAG methods as retrieval infrastructures under agentic search. RAGSearch covers both training-free and training-based agentic inference across multiple question answering benchmarks. To ensure fair and reproducible comparison, we standardize the LLM backbone, retrieval budgets, and inference protocols, and report results on full test sets. Beyond answer accuracy, we report offline preprocessing cost, online inference efficiency, and stability. Our results show that agentic search substantially improves dense RAG and narrows the performance gap to GraphRAG, particularly in RL-based settings. Nevertheless, GraphRAG remains advantageous for complex multi-hop reasoning, exhibiting more stable agentic search behavior when its offline cost is amortized. Together, these findings clarify the complementary roles of explicit graph structure and agentic search, and provide practical guidance on retrieval design for modern agentic RAG systems.
Figures & tables
Figure 1. Explicit vs. Implicit Structure in RAG Systems. GraphRAG relies on explicit graph construction, whereas agentic search over dense RAG can induce implicit evidence structure through multi-round retrieval and reasoning.
Figure 2. Overview of the RAGSearch benchmark. RAGSearch models agentic search as an LLM agent interacting with interchangeable retrieval backends (dense RAG or GraphRAG) through a unified interface, and benchmarks both training-free agentic search and RL-based agentic search under standardized protocols.
System
Methods
General QA
Multi-Hop QA
NQ †
PopQA ⋆
TriviaQA ⋆
HotpotQA †
2Wiki ⋆
Musique ⋆
Single-shot
Qwen-2.5-7B-Dense
46.62
32.14
58.60
19.00
35.53
20.99
Qwen-2.5-7B-GraphRAG ♠
48.31 ( ↑1.69 )
32.82 ( ↑0.68 )
57.65 ( ↓0.95 )
46.70 ( ↑27.70 )
62.56 ( ↑27.03 )
47.95 ( ↑26.96 )
Training-free
Search-o1-7B-Dense
38.20
25.57
58.74
33.76
29.64
12.62
Search-o1-7B-GraphRAG ♠
38.34 ( ↑0.14 )
28.01 ( ↑2.44 )
59.50 ( ↑0.76 )
42.75 ( ↑8.99 )
65.56 ( ↑35.92 )
32.44 ( ↑19.82 )
GraphSearch-7B-Dense
58.27
36.29
68.70
38.22
47.43
13.33
Table 1. Overall Contain Exact Match (EM) for single-shot inference, training-free agentic systems, and RL-based agentic systems. ♠ denotes the best result among the five GraphRAG variants. ↑ and ↓ indicate, for the same method, the performance difference between graph-based and dense-retrieval-based RAG; † denotes the in-domain dataset, and ⋆ denotes the cross-domain dataset.
Method
Knowledge Base
General QA
Avg Rank
Multi-Hop QA
Avg Rank
NQ
PopQA
TriviaQA
HotpotQA
2Wiki
Musique
Search-o1
Dense
38.20
25.78
58.74
2.33
33.76
29.64
12.62
5.33
HypergraphRAG
33.02
25.57
56.72
5.33
33.90
50.58
28.05
3.67
HippoRAG2
38.34
28.01
59.50
1.00
42.75
65.56
32.44
1.00
LinearRAG
34.32
25.69
57.03
4.00
35.76
58.94
29.46
2.33
RAPTOR
34.82
23.20
52.52
5.33
29.51
29.87
29.50
4.33
Table 2. Overall Contain EM results of training-free agentic systems across different retriever backends. and indicate the best and second-best results within each method, respectively. means dense-RAG, means tree-based GraphRAG, means entity graph based GraphRAG, means hypergraph based GraphRAG and means tri-graph based GraphRAG. Avg. Rank is computed within each method (lower is better).
Figure 3. Performance of different retrieval infrastructures on RL-based agentic systems
Method
HotpotQA
PopQA
Search Turn
Recall
Variance
Search Turn
Recall
Variance
Search-o1-Dense
2.20
79.38
33.65 ± 1.03
1.53
76.33
25.62 ± 0.61
Search-o1-HippoRAG2
2.03
80.27
42.36 ± 0.22
1.52
78.12
27.81 ± 0.36
Search-R1
1.82
81.67
34.82 ± 0.95
1.36
77.15
33.15 ± 0.54
Graph-R1-HippoRAG2
1.71
83.50
53.71 ± 0.18
1.38
78.61
36.13 ± 0.32
Table 3. Sensitivity Analysis on agentic systems
Figure 4. The impact of different RL algorithms
Method
HotpotQA
Musique
NQ
PopQA
Search-R1-3B
23.67
4.96
30.50
25.34
Graph-R1-3B ♠
42.75
31.73
37.72
31.08
Search-R1-7B
35.76
12.35
46.35
32.90
Graph-R1-7B ♠
51.75
36.53
42.12
35.95
Table 4. 3B vs 7B on RL-based agentic systems.
Method
HotpotQA
Musique
NQ
PopQA
Search-o1-Dense-7B
33.76
12.62
38.20
25.78
Search-o1-GraphRAG-7B ♠
42.75
32.44
38.34
28.01
Search-o1-Dense-32B
40.85
18.95
51.80
36.20
Search-o1-GraphRAG-32B ♠
47.01
42.37
50.50
36.69
Table 5. 7B vs 32B on training-free systems.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Task
Knowledge Source
#Train
#Dev
#Test
NQ
General QA
Wiki
79,168
8,757
3,610
PopQA
General QA
Wiki
-
-
14,267
TriviaQA
General QA
Wiki & Web
78,785
8,837
11,313
HotpotQA
Multi-hop QA
Wiki
90,447
7,405
-
Musique
Multi-hop QA
Wiki
19,938
2,417
-
2Wiki
Multi-hop QA
Wiki
15,000
12,576
-
Appendix
Table 6. Dataset Statistics
System
Methods
General QA
Multi-Hop QA
NQ †
TriviaQA ⋆
HotpotQA †
Musique ⋆
Single-shot
Qwen-2.5-7B-Dense
39.3
61.12
20.12
29.08
Qwen-2.5-7B-GraphRAG ♠
27.92 ( ↓11.38 )
49.25 ( ↓11.87 )
33.72 ( ↑13.60 )
41.13 ( ↑12.05 )
Training-free
Search-o1-7B-Dense
36.53
59.73
39.08
17.46
Search-o1-7B-GraphRAG ♠
36.62 ( ↑0.09 )
58.18 ( ↓1.55 )
41.57 ( ↑2.49 )
37.01 ( ↑19.55 )
GraphSearch-7B-Dense
8.70
13.12
6.02
2.36
Appendix
Table 7. Overall F-1 of different systems
Method
Knowledge Construction
Offline Inference
TM
CM
RT/s
CT/token
HypergraphRAG
1.37h
3.93$
0.77
1680
HippoRAG2
1.19h
2.85$
1.00
3229
LinearRAG
0.68h
0$
1.18
4600
RAPTOR
1.70h
6.38$
8.4
814
GraphRAG
1.72h
13.19$
1.16
22160
Appendix
Table 8. Comparison on cost of GraphRAGs of NQ. TM indicates construction time per 1M token; CM is the cost per 1M token, RT is the average retrieval time and CT is the average context length.
Question
"Are the Laleli Mosque and Esma Sultan Mansion located in the same neighborhood?"
Golden Answer
"No"
qwen2.5-7B-Ins
Search-o1-Dense
Search-R1
Graph-R1-HippoRAG2
Generation
To answer the question of whether the Laleli Mosque and Esma Sultan Mansion are located in the same neighborhood, I need to find the locations of both structures.Assistant: <|begin_search_query|> location of Laleli Mosque <|end_search_query|> <|begin_search_result|> The Laleli Mosque is located in Laleli, Fatih, Istanbul, Turkey… <|end_search_result|> Assistant: Based on the information gathered…{No}
<think> To answer the question “Are the Laleli Mosque and Esma Sultan Mansion located in the same neighborhood?”, I will first reason through what I know about these two landmarks. I know that Laleli Mosque is a significant historical site in Istanbul, Turkey. I need more information about the location of Esma Sultan Mansion to determine if it shares a neighborhood with Laleli Mosque. </think> <search> “Esma Sultan Mansion location Istanbul Turkey” </search> <information> >Doc 1(Title: “Esma Sultan Mansion”):… </information> <think> … Based on my search results: Laleli Mosque is in Laleli neighborhood. Esma Sultan Mansion is in Ortaköy neighborhood. Since Laleli Mosque is in Laleli neighborhood and Esma Sultan Mansion is in Ortaköy neighborhood, they are not located in the same neighborhood. </think> <answer> No </answer>
<think> To answer this question, I need to determine the neighborhoods where the Laleli Mosque and Esma Sultan Mansion are located. I will first search for the location of the Laleli Mosque and then for the Esma Sultan Mansion, and finally compare their neighborhoods. </think> <query> {"query": "Laleli Mosque neighborhood"} </query> <knowledge> Wikipedia Title: Laleli Mosque… </knowledge> <think> According to the Wikipedia information retrieved, the Laleli Mosque is located in Laleli, Fatih, Istanbul. Now, I need to find the neighborhood where Esma Sultan Mansion is located. </think> … <answer> No </answer>
As advanced RAG variants like GraphRAG and Agentic RAG emerge, one leading question is when and how to use them. Here, we introduce a framework for different RAG scenarios evaluation and comparison on semi-structured knowledge bases, including regular RAG, GraphRAG, Modular RAG and Agentic RAG. We provide implementation for 9 standardized RAG scenarios, and conduct experiments for a comprehensive comparison. These scenarios are designed for real use cases regarding data and domain restrictions, spanning from simple document-based retrieval to advanced features such as hybrid text-graph retrieval, integration with computed or pre-defined domain knowledge graphs, agentic multi-step planning, and agent-graph integration. Besides, we present a novel context engineering method for GraphRAG and Agentic RAG, addressing the context/memory overflow issues, efficiently managing text and graph retrievals with new representations and agentic loop design, leading to 19%-53% reduction on token usage. Moreover, further analysis identifies a retrieval-generation gap where expanded retrieval does not proportionally improve generation quality, suggesting retrieval-oriented metrics overstate advanced retrieval benefits. This work provides data-driven insights on when and how to use them for building production-ready intelligent RAG systems.
Long Chen, Ryan Razkenari, Yuxuan Zhou +5
1Generative AI Innovation Center, Amazon Web Services (AWS) · 2Cisco Systems, Inc.
Retrieval-Augmented Generation (RAG) was introduced to enhance the capabilities of Large Language Models (LLMs) beyond their encoded prior knowledge. This is achieved by providing LLMs with an external source of knowledge, which helps reduce factual hallucinations and enables access to new information not available during pretraining. However, inconsistent retrieved information can negatively affect LLM responses. The Retrieval-Augmented Generation Benchmark (RGB) was introduced to evaluate the robustness of RAG systems under such conditions. In this work, we use the RGB corpus to evaluate LLMs in four scenarios: noise robustness, information integration, negative rejection, and counterfactual robustness. We perform a comparative analysis between the RGB RAG baseline and GraphRAG, a knowledge graph based retrieval system. We test three GraphRAG customizations to improve robustness. Results show improvements over the RGB baseline and provide insights for designing more reliable RAG systems for real world scenarios.
Hazem Amamou, Stéphane Gagnon, Alan Davoust +1
Institut national de la recherche scientifique (INRS-EMT), Montréal, Québec, Canada · INRS-UQO Mixed Research Unit on Cybersecurity, Gatineau, Québec, Canada · Université du Québec en Outaouais, Gatineau, Québec, Canada
Retrieval-Augmented Generation (RAG) enhances large language models (LLMs) by incorporating external knowledge, yet traditional single-round retrieval struggles with complex multi-step reasoning. Agentic RAG addresses this by enabling LLMs to dynamically decide when and what to retrieve, but current RL-based training methods suffer from sparse outcome rewards that discard intermediate signals and low sample efficiency where failed samples contribute nothing. We propose Search-P1, a framework that introduces path-centric reward shaping for agentic RAG training, comprising two key components: (1) Path-Centric Reward, which evaluates the structural quality of reasoning trajectories through order-agnostic step coverage and soft scoring that extracts learning signals even from failed samples, and (2) Dual-Track Path Scoring with offline-generated reference planners that assesses paths from both self-consistency and reference-alignment perspectives. Experiments on multiple QA benchmarks demonstrate that Search-P1 achieves significant improvements over Search-R1 and other strong baselines, with an average accuracy gain of 7.7 points.