Retrieval-augmented generation (RAG) can improve factual grounding without guaranteeing that every generated claim is supported. Previous work has studied claim-level conformal factuality in single-stage RAG and, more recently, multi-hop RAG. We examine its behavior in agentic RAG, where a language-model policy adaptively issues follow-up retrieval queries or terminates retrieval before response generation. We retain the retrieval-based relevance-scoring formulation and post-generation conformal filtering and evaluate six model-dataset configurations: GPT-6 Luna and Llama 3.1 8B on HotpotQA, Natural Questions, and TriviaQA. Each configuration uses 60 fixed questions, 100 seeded 30/30 calibration-test partitions, five conformal target levels, and a no-conformal-prediction baseline. Across the six configurations, higher target levels are accompanied by higher observed full-support rates and reduced answer availability. At the 95% target, non-empty response coverage ranges from 17.23% to 60.17%, claim retention from 10.72% to 43.16%, and full support among non-empty outputs from 89.14% to 95.56%. We formulate the adaptive retrieval trajectory explicitly and characterize the reliability-informativeness trade-off when conformal factuality is applied after agent termination.
Figures & tables
Dimension
Multi-hop RAG
Agentic RAG
Follow-up query
Produced by a query-rewriting stage using prior evidence
Selected as the argument of an LLM-policy retrieve action
Retrieval control
Iterative refinement within a hop budget
Explicit retrieve / finish decision after accumulated evidence is inspected
Early termination
Configured hop/novel-evidence stopping conditions
Policy finish , repeated-query safeguards, or configured decision/retrieval limits
Evidence
Aggregated across a sequence of retrieval hops
Aggregated across adaptively selected retrieval actions
Table 1: Operational differences between the evaluated multi-hop and agentic retrieval procedures. Both apply post-generation claim scoring and conformal filtering.
Figure 1: Agentic RAG using the relevance-scoring formulation in eq. 7 , evaluated across GPT-6 Luna and Llama 3.1 8B on three QA datasets. The five points on each curve are the conformal targets 80%, 85%, 90%, 92.5%, and 95%; separate crosses at 100% non-empty coverage denote the no-CP baselines. The y-axis is response-level full support and includes vacuously supported empty filtered outputs. The vertically zoomed scale emphasizes reliability–coverage differences.
Model
Dataset
Non- empty
Claims retained
Supported claims
Full support ∣ non-empty
GPT-6 Luna
HotpotQA
36.47
23.57
97.14
95.23
NQ
17.23
10.72
90.16
89.14
TriviaQA
51.20
43.16
96.59
95.56
Llama 3.1 8B
HotpotQA
38.73
27.06
97.33
93.26
NQ
21.00
11.77
96.27
91.47
TriviaQA
60.17
39.13
97.72
94.56
Table 2: Agentic RAG using the relevance-scoring formulation in eq. 7 at the 95% CP target. All values are percentages averaged over 100 calibration–test partitions. “Full support ∣ non-empty” conditions on retaining at least one claim.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Target
Non-empty
Claims retained
Supported claims
Fully supported ∣ non-empty
HotpotQA
No CP
100.00
100.00
87.68
81.40
80%
97.73
92.57
88.63
82.66
85%
88.30
73.98
91.51
86.41
90%
79.60
62.28
93.33
88.82
92.5%
67.30
49.10
95.08
91.50
95%
36.47
23.57
97.14
95.23
Appendix
Table 3: Agentic RAG with GPT-6 Luna using the relevance-scoring formulation in eq. 7 : no-CP baseline and complete conformal target sweep (%).
Dataset
Target
Non-empty
Claims retained
Supported claims
Fully supported ∣ non-empty
HotpotQA
No CP
100.00
100.00
87.98
70.87
80%
90.73
84.19
91.66
78.26
85%
83.17
69.32
94.54
84.82
90%
77.63
60.33
95.61
87.88
92.5%
72.17
52.62
96.73
91.13
95%
38.73
27.06
97.33
93.26
Appendix
Table 4: Agentic RAG with Llama 3.1 8B using the relevance-scoring formulation in eq. 7 : no-CP baseline and complete conformal target sweep (%).
Retrieval-augmented generation (RAG) evaluation measures how well systems retrieve, use, and cite evidence while balancing reliability, cost, and speed.