URAG: A Benchmark for Uncertainty Quantification in Retrieval-Augmented Large Language Models
Authors: Vinh Nguyen, Cuong Dang, Jiahao Zhang, Hoa Tran, Minh Tran, Trinh Chau, Thai Le, Lu Cheng, +1 more
Organizations: Uppsala University · Virginia Tech · The Pennsylvania State University · FPT Software, AI Center · University of Science, VNU-HCM · VNU University of Engineering and Technology · Indiana University · University of Illinois at Chicago
Retrieval-Augmented Generation (RAG) has emerged as a widely adopted approach for enhancing LLMs in scenarios that demand extensive factual knowledge. However, current RAG evaluations concentrate primarily on correctness, which may not fully capture the impact of retrieval on LLM uncertainty and reliability. To bridge this gap, we introduce URAG, a comprehensive benchmark designed to assess the uncertainty of RAG systems across various fields like healthcare, programming, science, math, and general text. By reformulating open-ended generation tasks into multiple-choice question answering, URAG allows for principled uncertainty quantification via conformal prediction. We apply the evaluation pipeline to 8 standard RAG methods, measuring their performance through both accuracy and prediction-set sizes based on LAC and APS metrics. Our analysis shows that (1) accuracy gains often coincide with reduced uncertainty, but this relationship breaks under retrieval noise; (2) simple modular RAG methods tend to offer better accuracy-uncertainty trade-offs than more complex reasoning pipelines; and (3) no single RAG approach is universally reliable across domains. We further show that (4) retrieval depth, parametric knowledge dependence, and exposure to confidence cues can amplify confident errors and hallucinations. Ultimately, URAG establishes a systematic benchmark for analyzing and enhancing the trustworthiness of retrieval-augmented systems. Our code is available on GitHub.
Figures & tables
Figure 1: Illustration of uncertainty in RAG. When asked “Can pregnant women drink coffee?” , the retriever surfaces multiple documents, one stating that “coffee is safe in pregnancy” and another warning that “too much coffee can be risky.” The LLM, however, places disproportionate attention on the first document and produces an overconfident yet incomplete answer , overlooking contradictory evidence. This example highlights how noisy or one-sided retrieval can amplify model overconfidence, leading to potentially harmful or misleading responses.
Figure 2: Illustration of dataset construction and evaluation pipeline for URAG . The upper flow shows how we construct the benchmark, which involves retrieving documents from the database and prompting LLMs to generate a supporting document and wrong answers. After that, we use an NLI model to confirm the difficulty before having annotators check the final result. During the evaluation process, RAG systems decide between a list of wrong answers and the correct answer before measuring performance and uncertainty.
Dataset
Naive
Iter. 1
Iter. 2
Iter. 3
Prompt F
Prompt F
Prompt F
Prompt F
LCA
0%
100%
100%
100%
CRAG
31%
87%
93%
96%
NewsSum
10%
98%
100%
100%
Table 1: Progressive improvement in valid distractor generation using iterative NLI-based filtering. Starting from a naive prompt, each iteration applies Prompt F to regenerate distractors that fail the entailment-based difficulty check.
Figure 3: Accuracy and uncertainty difference under self-aware prompting. When the LLM is exposed to its confidence scores, RAT exhibits the strongest degradation in accuracy and stability, while most other RAG methods show reduced uncertainty, particularly on math (Olympiad) and general-domain tasks (CRAG).
Figure 4: Accuracy and uncertainty difference under wrong-aware prompting. Misleading confidence cues cause a consistent drop in accuracy and an increase in uncertainty, indicating a higher rate of hallucinated decisions.
Figure 5: Reverse Correlation between Accuracy and Uncertainty across RAG methods. Each point represents the average accuracy and predictive uncertainty (set size) of one RAG method evaluated on ten datasets. A strong negative correlation ( r=−0.76 ) shows that higher accuracy aligns with lower predictive uncertainty across RAG methods.
Figure 6: Impact of Irrelevant Contexts on Accuracy and Uncertainty. We add irrelevant information along with retrieved information on DialFact, Odex, and LCA. Bars show absolute differences between with/without noisy information.
Figure 7: Comparison of RAG accuracy and uncertainty on LLM-Correct and LLM-Incorrect cases. “LLM-Only Correct” and “LLM-Only Incorrect” in the below chart show the Set Size of LLM (w/o retrieval) in two sets. The charts report mean values on the HealthVer benchmark. The extended results are in Table A6 .
Figure A1: Tasks covered by URAG. URAG includes tasks spanning 5 domains, general text, research, code, math, and healthcare, for examining the accuracy and uncertainty in daily-use/abstractive-reasoning/high-stakes tasks.
Dataset
Description
Calibration
Test
Total
Commit Message QA
Evaluating RAG system performance in project understanding
81
82
163
CRAG †
Evaluating RAG system performance across multiple domains
1,181
1,149
2,330
DialFact
Multiple choice QA subset for dialogue fact verification
1,000
1,000
2,000
HealthVer
Health claim verification in MCQA format
665
667
1,332
Multi-NewsSum
MCQA generated from Multi-News summarization dataset
475
475
950
ODEX
Evaluating RAG systems in simple coding tasks
220
219
439
Table A1: Overview of RAG Benchmark Datasets.
Category
Full Question with Options
Answer
Commit Message QA
Given the context, which commit message best describes the following code changes? A. Improve speed of rebalance script - This removes the call to nodetool ring , which can get unreasonably slow as the amount of data in a BOP cluster increases. It also adds a couple flags that allow the nodetool status call to be skipped if the user is already sure the sanity checks will pass. B. Reduce pressure on memory in stream tests - This change runs the python garbage collector before and after each stream test. The garbage collector is disabled in the CI since it has a significant impact on the duration of the jobs. C. Refactor FFmpegSource - Using 2 queues for video packets and audio packets. Whenever the queues have space, more packets are read from the stream. This work will allow to remove more easily the audio thread in favor of a scheduled call to refill the audio player. D. Support CUDA stream on memory pool - Now, memory pool will have an arena (bins) for each stream to avoid concurrent streams touch the same memory block
D
Multinewsum
Which of the following best summarizes the given document? A. Today’s 11 gubernatorial races are primarily focused on local issues, with Democrats poised for significant gains, possibly taking control of a majority of state offices… B. Republicans already control the North Carolina governorship, having wrested it from Democratic control. They also solidified their hold on Utah, North Dakota, and Indiana… C. Today’s gubernatorial elections are proving to be a difficult day for Republicans, who are struggling to hold onto their seats in Utah, North Dakota, and Indiana… D. It’s a race for the governor’s mansion in 11 states today, and the GOP could end the night at the helm of more than two-thirds of the 50 states. The GOP currently controls 29 of the country’s top state offices…
D
OlympiadBench
Let n be a positive integer and fix 2n distinct points on a circumference. Split these points into n pairs and join the points in each pair by an arrow (i.e., an oriented line segment). The resulting configuration is good if no two arrows cross, and there are no arrows AB and CD such that ABCD is a convex quadrangle oriented clockwise. Determine the number of good configurations. Options: A. Cn=n+11(n2n) B. 2nn!(2n)! C. (22n)(22n−2)...(22) D. (n2n)
D
DialFact
Given the conversation context, evaluate this claim: “Well, it’s a little different. but if you’re looking for something different, you can try gluten-free pasta. it’s wheat-flavored and comes in a variety of textures and shapes. i really like the rice flour pasta” What is the verification status? A. Supports B. Refutes C. Not Enough Information
B
Healthver
Can taking medication to lower fever, such as paracetamol (tylenol) and ibuprofen (advil) worsen COVID-19? A. Supported B. Refuted C. Not Enough Information
A
Scifact
Is the following scientific claim supported by evidence? Claim: The risk of female prisoners harming themselves is ten times that of male prisoners. A. Supported B. Refuted C. Not Enough Information
A
Table A2: Dataset Examples with Full Questions and Options . Wrong DialFact and Wrong Odex share similar question-answer, but we use a different database.
k
Olympiad
LCA
CRAG
ODEX
Acc
SS
Acc
SS
Acc
SS
Acc
SS
FiD
10
0.30
3.98
0.21
4.76
0.31
3.69
0.28
3.63
50
0.30
3.98
0.22
4.82
0.31
3.70
0.26
3.78
100
0.30
4.00
0.22
4.79
0.31
3.69
0.26
3.76
500
0.28
3.99
0.23
4.79
0.31
3.70
0.26
3.75
Fusion
10
0.37
3.36
0.82
2.19
0.66
2.38
0.86
1.73
Table A3: Effect of retrieval depth k on accuracy and uncertainty (SS) across representative datasets from different domains. Bold value indicates the best result on the benchmark for each RAG method.
Dataset file
Samples
Total docs
Avg docs/sample
Total text chars
Avg chars/doc
Total words
Avg words/doc
OlympiadBench
661
20,672
N/A (shared)
51,167,805
2,475.22
8,910,741
431.05
LCA
163
163
1.00
42,177,274
258,756.28
3,879,366
23,799.79
CRAG
2,330
11,650
5.00
4,208,072,919
361,207.98
179,542,626
15,411.38
DialFact
2,000
2,435
1.22
11,835,703
4,860.66
1,877,746
771.15
Healthver
1,332
4,191
3.15
88,159,278
21,035.38
13,167,923
3,141.95
NewsSum
950
950
1.00
6,109,017
6,430.54
993,322
1,045.60
Table A9: Statistics of datasets used in the benchmark.
Retrieval-Augmented Generation (RAG) was introduced to enhance the capabilities of Large Language Models (LLMs) beyond their encoded prior knowledge. This is achieved by providing LLMs with an external source of knowledge, which helps reduce factual hallucinations and enables access to new information not available during pretraining. However, inconsistent retrieved information can negatively affect LLM responses. The Retrieval-Augmented Generation Benchmark (RGB) was introduced to evaluate the robustness of RAG systems under such conditions. In this work, we use the RGB corpus to evaluate LLMs in four scenarios: noise robustness, information integration, negative rejection, and counterfactual robustness. We perform a comparative analysis between the RGB RAG baseline and GraphRAG, a knowledge graph based retrieval system. We test three GraphRAG customizations to improve robustness. Results show improvements over the RGB baseline and provide insights for designing more reliable RAG systems for real world scenarios.
Hazem Amamou, Stéphane Gagnon, Alan Davoust +1
Institut national de la recherche scientifique (INRS-EMT), Montréal, Québec, Canada · INRS-UQO Mixed Research Unit on Cybersecurity, Gatineau, Québec, Canada · Université du Québec en Outaouais, Gatineau, Québec, Canada
Allowing large language models (LLMs) to retrieve information from a set of trusted documents can increase reliability and reduce hallucination. However, recent work has demonstrated that retrieval-augmented generation (RAG) can have unintended side effects on the overall safety of the generated responses, when prompted for harmful or dangerous content. A clearer understanding of the mechanisms leading to this result is needed, as increasing numbers of end users turn to RAG to incorporate corporate documents and knowledge bases into LLM-based systems. We introduce RAG-Safety-Bench, a benchmark to measure the safety impact of RAG on LLM models. By removing the confounding effect of retriever quality, and cleanly separating the problem into four conditions -- non-RAG, RAG with an oracle document containing the answer to the harmful request, RAG with documents related to the harmful request but without the specific answer, and RAG with random, safe documents -- the benchmark isolates the impacts of different factors in the observed safety degradation. We report results across five open-source LLMs, showing an inverse relationship between benign and unsafe capability, strong evidence that baseline safety guardrails do not lead to downstream safety guarantees in the RAG case, and model-specific support for previous findings that even benign documents can lead to unsafe generation in retrieval-enabled systems.
Adithiyan Rajan Indira Saravanan, Kathleen C. Fraser
Faculty of Engineering University of Ottawa Ottawa, Canada
Retrieval augmented generation (RAG) depends critically on the quality and granularity of retrieved evidence. Large retrieval units preserve context but often introduce irrelevant content, which can dilute answer bearing evidence and worsen long context utilization. Fine-grained units are more compact, but they may be difficult to retrieve reliably because short chunks can lack semantic, lexical, or bridging cues needed to match the query. We propose Uncertainty-aware Multi-Granularity RAG (UMG-RAG), a training-free hybrid retrieval framework that treats chunk granularity as query-specific reliability estimation. Instead of training a new retriever or modifying the generator, UMG-RAG uses existing dense and sparse retrievers as complementary experts across multiple chunk granularities. For each query, it converts each expert-granularity score list into an evidence distribution, estimates reliability from distribution entropy, and fuses candidates according to query-specific semantic, lexical, and granularity confidence. We further introduce UMGP-RAG, a parent promotion variant that uses fine-grained hits to locate relevant evidence while returning broader non-redundant parent chunks for local coherence. Experiments on question answering benchmarks show that uncertainty-aware fusion and parent promotion improve generation quality while maintaining a lightweight, plug-and-play retrieval pipeline.
Hoin Jung, Xiaoqian Wang
Elmore Family School of Electrical and Computer Engineering Purdue University West Lafayette, IN 47907