The Bottleneck Is Retrieval, Not Reading: Diagnosing Evidence-Based Generation with RAG for Long Context Question Answering

Debopriyo Banerjee; Owais Aijaz; Ahmer Tabassum; Zhuohan Xie; Preslav Nakov; Veselin Stoyanov

cs.CLOct 18, 2026

The Bottleneck Is Retrieval, Not Reading: Diagnosing Evidence-Based Generation with RAG for Long Context Question Answering

Authors: Debopriyo Banerjee, Owais Aijaz, Ahmer Tabassum, Zhuohan Xie, Preslav Nakov, Veselin Stoyanov

Organizations: Inception42 · MBZUAI

Abstract

Retrieval-augmented generation (RAG) is the dominant paradigm for grounding large language models in external corpora, but its similarity-driven, chunked pipeline has structural blind spots: it cannot anticipate which passages a question requires and discards document-level context that resolves cross-passage references. We introduce evidence-based generation (EBG), requiring models to return an answer with line-level citations, and a joint Ans+Evid metric that scores correctness and grounding simultaneously. We reformulate two corpora into evidence-grounded multi-document QA benchmarks- MultiHop-RAG and QASPER-MP, a multi-paper adaptation of QASPER with realistic short paper cues, and pair them with a controlled synthetic two-hop haystack and an evidence-augmented NoLiMa spanning 10K-1M characters. Across ten proprietary frontier and three open-weight LLMs, EBG isolates two RAG limitations that long-context models avoid: similarity retrieval surfaces any relevant chunk far more reliably than all required chunks for multi-fact queries, and chunking destroys document meta-information needed to resolve identity-bearing queries. EBG also exposes grounding failures invisible to answer-only evaluation, including incomplete citations in two-needle settings and a widening gap between Ans and Ans+Evid when context contradicts parametric knowledge. Data and code are available at https://github.com/debobanerjee/LC-EBG.

Figures & tables

Model	LC	flat@3	struct@3	$\Delta$ LC (best)
GPT-5	0.605	0.500	0.553	$+0.05$
GPT-5.5	0.671	0.500	0.539	$+0.13$
Opus-4.6	0.697	0.474	0.566	$+0.13$
Sonnet-4.5	0.592	0.513	0.474	$+0.08$
Gemini-3-Fl	0.658	0.487	0.539	$+0.12$

Table 1: QASPER-MP answerable-subset LLM-judge scores (

n{=}38

, EBG) . LC wins for all five models; the largest gaps are for the strongest readers ( Opus-4.6 and GPT-5.5 ). Higher (

\uparrow

) values are better. Full breakdown in Appendix E .

Figure 1: NoLiMa-based LC benchmark comparing EBG ( Ans ) and answer-only (Ans ∗ ) prompting across eight proprietary and three open-weight models over 10K–1M context lengths. Values report the accuracy difference ( $\Delta$ ) , with larger positive values ( $\uparrow$ ) indicating greater EBG gains; N/A denotes inputs exceeding the model’s context window.

Figure 2: Two-needle citation breakdown: rate of correctly citing the primary needle vs. the secondary needle under NoLiMa-based LC benchmark. Higher (

\uparrow

) values are better.

Figure 3: TN vs. CTN , Ans+Evid . TN uses a factual secondary needle; CTN uses a contradictory secondary needle, conflicting with world knowledge, requiring context-faithful reasoning and citation. Models to the right of the dashed line are open-weight. Higher (

\uparrow

) values are better. Ans accuracy for the same runs is reported in Figure 6 .

Appendix figures & tables26 assets

Supplementary material from the paper’s appendix.

Appendix

Original NoLiMa primary needles (under SN)
Variant	Needle Template
0402	“A message came in from { CHAR } saying, ‘I’m lactose intolerant,’ and nothing more.”
0402Inv	“A message came in saying, ‘I’m lactose intolerant,’ from { CHAR } .”
0405	“Then { CHAR } mentioned that he has been vegan for years.”
0405Inv	“There was a vegan guest, named { CHAR } .”
Gender neutral rephrased primary needle (under SN)

Appendix

Table 2: Needle templates used in the evaluation. Original NoLiMa needles under the single-needle ( SN ) setting evaluate both one-hop and two-hop reasoning. We add 0405-R , a gender-neutral rephrased version of the 0405 primary needle template under SN, to examine the influence of gendered wording in the needle template. Explicit secondary needles under the two-needle ( TN ) setting directly provide the required world-knowledge fact and evaluate two-hop reasoning under fully contextualized evidence. The contradictory two-needle ( CTN ) setting instead introduces secondary needles that conflict with world knowledge, enabling evaluation of whether models remain strictly faithful to the provided context during two-hop reasoning.

Model	$\textit{Ans}^{*}$	Ans	$\mathbf{\Delta}$
	(ans-only)	(EBG)
GPT-5	82.0%	86.0%	$+4$ pp
Claude Sonnet-4.5	68.0%	82.0%	$\mathbf{+14}$ pp
Gemini-3-Flash	80.0%	78.0%	$-2$ pp

Appendix

Table 3: Ans ∗ (answer-only) vs. Ans (EBG) on the 50 pilot queries, full mini-haystack (

\sim 241

K characters, 20 distractors). Requiring citations raises answer accuracy for two of the three models, with a striking

+14

pp improvement for Sonnet-4.5 . This replicates the citation-as-scaffold effect that was invisible in our (saturated) synthetic benchmark. Higher (

\uparrow

) values are better for Ans ∗ and Ans .

any gold
Config	H@1	H@4	H@10	H@50	H@100	MAP@10
small + sent.	0.29	0.52	0.67	0.90	0.95	0.22
small + tok256	0.27	0.59	0.80	0.97	0.99	0.24
large + sent.	0.32	0.55	0.71	0.94	0.97	0.24
large + tok256	0.33	0.66	0.82	0.98	1.00	0.27
all gold (faithful-RAG ceiling)

Appendix

Table 4: Corpus-wide retrieval metrics on MultiHop-RAG (

N{=}2255

); any gold reports the fraction of queries for which at least one annotated gold fact appears in the top-

k

retrieved set; all gold reports the fraction for which every annotated gold fact appears. The latter measures strict annotated-gold coverage, not an upper bound on answer accuracy. Under the stronger assumptions that the gold annotations are exhaustive and that every annotated fact is necessary, all gold upper-bounds on what any faithful RAG pipeline can achieve at top-

k

: if even one of the 2–4 gold facts is missing from the retrieved set, downstream answers cannot be faithful. The token-windowed large configuration dominates and is used for all RAG experiments below. Higher (

\uparrow

) values are better.

$K$	% queries w/ all needles	% gold facts retrieved
4	14.0%	36.3%
10	28.0%	49.6%
25	44.0%	68.1%
50	58.0%	76.3%
100	64.0%	84.4%
200	90.0%	96.3%

Appendix

Table 5: Needle coverage on the 50 pilot queries (retriever: text-embedding-3-large + 256-token chunks). At realistic RAG budgets (

K{\leq}25

) fewer than half of queries have all gold facts retrieved; only at

K{=}200

does coverage approach saturation. Higher (

\uparrow

) values are better.

Answer-only prompt ( $\textit{Ans}^{*}$ / Joint $\equiv$ Ans)
Model	Metric	LC (oracle)	RAG@4	RAG@10	RAG@25	RAG@50	RAG@100	RAG@200
GPT-5	Ans	82.0	58.0	60.0	70.0	72.0	74.0	78.0
Sonnet-4.5	Ans	68.0	24.0	36.0	38.0	56.0	58.0	68.0
Gemini-3-Flash	Ans	80.0	38.0	44.0	50.0	58.0	58.0	70.0
EBG prompt : Ans / Evid (strict) / Joint
GPT-5	Ans	86.0	58.0	60.0	64.0	72.0	76.0	76.0

Appendix

Table 6: Oracle-containing LC versus open-corpus RAG on the 50-query MultiHop-RAG pilot. LC receives every gold-evidence article plus 20 distractor articles; RAG retrieves from the full 609-article corpus. The conditions use the same queries, models, prompt templates, and scoring procedure but are not pool-matched. All values are percentages over

n{=}50

( Sonnet-4.5 RAG@200 EBG:

n{=}49

). Boldface marks the per-row maximum within the corresponding prompt-mode block. Higher (

\uparrow

) values are better.

Answer-only prompt ( $\textit{Ans}^{*}$ )
Model	Metric	LC (oracle)	RAG@10	RAG@200
GPT-OSS-120B	Ans	75.0	47.5	67.5
GLM-5.2	Ans	88.0	53.0	72.5
Nemo-3.5-L	Ans	75.5	43.5	51.0
EBG prompt : Ans / Evid (strict) / Joint
GPT-OSS-120B	Ans	54.0	41.5	52.0

Appendix

Table 7: LC-EBG vs. RAG-EBG for open-weight models on a

200

-query stratified sample. All numbers are percentages over

n{=}200

(a handful of cells return

n{=}198

–

199

where 1–2 queries exceed a provider’s context limit). Boldface marks the per-row maximum within each mode block. Higher (

\uparrow

) values are better. At

n{=}200

the

95\%

bootstrap CI half-width on Ans / Joint is

\approx

6

–

7

pp. LC is the oracle-containing gold-article-plus-20-distractor condition; RAG@10 and RAG@200 use open-corpus retrieval over all 609 articles.

Model	Condition	$n$	Ans-F1	Evid-F1	Judge
GPT-5	LC	50	0.377	0.415	0.480
GPT-5	flat_pool11@3	50	0.357	0.543	0.460
GPT-5	struct_pool11@3	50	0.437	0.607	0.550
GPT-5.5	LC	50	0.378	0.433	0.530
GPT-5.5	flat_pool11@3	50	0.386	0.560	0.460
GPT-5.5	struct_pool11@3	50	0.419	0.600	0.490

Appendix

Table 8: QASPER-MP aggregate scores (50 pilot questions, EBG mode). Best condition per model in bold . Judge scores use the GPT-5 judge described in Appendix E.5 ;

n

reports how many queries returned valid output. Higher (

\uparrow

) values are better.

		Answerable ( $n=38$ )			Unanswerable ( $n=12$ )
Model	Condition	Judge	Ans-F1	Refuse %	Judge	Ans-F1	Refuse %
GPT-5	LC	0.605	0.469	15.8	0.083	0.083	33.3
GPT-5	flat_pool11@3	0.500	0.364	31.6	0.333	0.333	58.3
GPT-5	struct_pool11@3	0.553	0.390	31.6	0.583	0.583	66.7
GPT-5.5	LC	0.671	0.471	13.2	0.083	0.083	16.7
GPT-5.5	flat_pool11@3	0.500	0.402	23.7	0.333	0.333	41.7

Appendix

Table 9: QASPER-MP results split by answerability (EBG mode, LLM-judge). “Refusal” is the fraction of model outputs that started with Unanswerable . Best LC vs. best RAG variant per model in bold within each subset. Higher (

\uparrow

) values are better.

		Answerable ( $n=150$ )			Unanswerable ( $n=50$ )
Model	Condition	Judge	Ans-F1	Refuse %	Judge	Ans-F1	Refuse %
GPT-OSS-120B	LC	0.701	0.467	8.7	0.500	0.475	42.0
GPT-OSS-120B	flat_pool11@3	0.497	0.388	32.7	0.660	0.666	64.0
GPT-OSS-120B	struct_pool11@3	0.537	0.418	24.7	0.640	0.646	62.0
GPT-OSS-120B	flat_full@10	0.497	0.369	28.0	0.670	0.661	64.0
GLM-5.2	LC	0.735	0.553	6.7	0.500	0.457	36.0

Appendix

Table 10: QASPER-MP open-weight results split by answerability (EBG mode, GPT-5 judge), over a

200

-query stratified sample. Answerable subset is

n{=}150

and unanswerable

n{=}50

; a handful of cells return 1–2 fewer queries where a provider error dropped a row ( † GLM-5.2 flat_full@10 returned

n{=}148/49

). Best LC vs. RAG variant per model shown in bold within the answerable subset. “Refuse” is the fraction of outputs starting with Unanswerable . Higher (

\uparrow

) values are better.

		full@10 plain		full@10 meta
Model	LC	Judge	Ref%	Judge	Ref%
GPT-5	0.757	0.473	27	0.607	20
Sonnet-4.5	0.623	0.490	29	0.587	24
GPT-OSS-120B	0.701	0.497	28	0.623	19
GLM-5.2	0.735	0.497	30	0.597	23
Nemotron-3.5-30B	0.523	0.433	23	0.467	17

Appendix

Table 11: Metadata-in-chunk retrieval on flat_full@10 (QASPER-MP, EBG, GPT-5 judge, answerable subset,

n{=}200

). “meta” prepends the paper title to each chunk before embedding. Judge improves over plain retrieval for every model ( bold ); “Ref%” is the answerable-question refusal rate. LC (full document neighborhood) is shown for reference. The

95\%

bootstrap CI half-width on the judge scores is

\approx

0.07

Panel of Figure 4 — Figure 4: Synthetic two-hop benchmark comparing Ans ∗ (answer-only prompting) and Ans (under EBG prompting) for three frontier models across 10K–1M characters. (a) Absolute Ans ∗ and Ans accuracies. Higher ( $\uparrow$ ) values are better. (b) Accuracy difference between the two prompting settings. Cells report percent accuracy averaged over $\geq 50$ trials per context length. Sonnet-4.5’s 1M entry is marked N/A because the input exceeds its 200K-token context window. The two prompting settings produce nearly identical performance on this controlled synthetic benchmark, indicating that the citation requirement neither substantially helps nor hurts answer accuracy. Larger ( $\uparrow$ ) positive difference indicates effectiveness of EBG over answer-only prompting.

Table 12: Performance of models across different context lengths, for one-hop and two-hop questions, averaged across

n=16

(4 needles

\times

4 depths) cases, comparing Ans ∗ (answer-only) with Ans (EBG) .

\Delta

denotes the difference

\texttt{Ans}-\texttt{Ans}^{*}

; positive values are shown in bold blue , negative values in bold red . Higher (

\uparrow

) values are better. Positive values indicate that EBG prompting outperforms answer-only prompting, while negative values indicate that answer-only prompting improves over EBG prompting. Zero indicates no difference between the two prompting strategies. “N/A” denotes unavailable entries due to unsupported context lengths. Here we present scores correct upto two decimal places.

Figure 5: Performance of different models on Ans+Evid (EBG) accuracy for one-hop and two-hop questions across varying context lengths, averaged over needle depths. Cells marked with “N/A” indicate no available results for that model-context combination. Higher ( $\uparrow$ ) values are better).

Figure 6: TN vs. CTN Ans accuracy under EBG prompting . We report an accuracy for the same runs whose Ans+Evid accuracy appears in Figure 3 . Higher (

\uparrow

) values indicate better performance.

Panel of Figure 7 — Figure 7: Comparison of answer-accuracy differences between EBG vs. CoT and EBG vs. CoT-EBG prompting on the NoLiMa-based dataset, with results reported separately for one-hop and two-hop questions. (a) EBG vs. CoT , where each cell reports $\Delta=\texttt{Ans}-\texttt{Ans}^{c}$ in percentage points; positive values favor EBG and negative values favor CoT. (b) EBG vs. CoT-EBG, where each cell reports $\Delta=\texttt{Ans}-\texttt{Ans}^{ce}$ ; positive values favor EBG and negative values favor CoT-EBG. Cells marked N/A indicate that no results are available for the corresponding model–context combination.

Full-coverage (one-hop)
	Context Length
	10K	20K	30K	40K	50K	60K	70K	80K	90K	100K	200K	300K	400K	500K	600K	700K	800K	900K	1M
# Vectors	66	124	173	247	297	381	395	470	513	584	1,179	1,739	2,275	2,852	3,468	4,034	4,658	5,247	5,730
# Vectors ( $\downarrow$ )	2	4	4	4	4	9	11	8	9	10	20	35	43	48	59	62	83	92	96
Chars ( $\downarrow$ )	350	823	818	607	584	1,902	2,343	1,876	1,985	2,452	4,725	7,126	8,244	11,548	12,410	13,981	17,771	20,234	22,910
Full-coverage (two-hop)

Appendix

Table 13: Retrieval analysis of RAG under NoLiMa , showing average total vector count; minimum full-coverage retrieval depth for one-hop and two-hop queries (i.e., the smallest top-

k

such that all required needles/evidence facts are contained within the retrieved set for every sample) and corresponding retrieved character budget across varying haystack context lengths, averaged over three books. Lower (

\downarrow

) values are better.

Figure 9: Ranking of the 0405 needle instantiated with 10 different character names based on cosine similarity to a one-hop test query. Although the sentence semantics remain unchanged, similarity scores vary across character tokens. Higher (

\uparrow

) values are better.

Figure 10: Cosine similarity comparison between the original 0405 needle and the rephrased 0405-R variant across 10 characters . Removing the pronoun consistently increases similarity scores across different characters. Higher positive difference (

\uparrow

) indicates greater effectiveness of the rephrased version compared to the original.

Full-coverage (one-hop)
	Context Length
	10K	20K	30K	40K	50K	60K	70K	80K	90K	100K	200K	300K	400K	500K	600K	700K	800K	900K	1M
# Vectors	71	135	169	226	290	392	407	453	515	542	1,161	1,775	2,319	2,858	3,502	4,086	4,641	5,229	5,671
Embedding-RAG
# Vectors ( $\downarrow$ )	2	4	4	3	1	11	13	4	7	14	17	41	57	48	68	45	90	94	104
Chars ( $\downarrow$ )	118	850	758	363	62	1,556	2,258	1,415	1,721	3,284	4,985	7,879	10,185	11,106	12,182	9,589	18,310	22,216	23,589

Appendix

Table 14: Retrieval analysis of three RAG pipelines under NoLiMa , showing total vector count; minimum full-coverage retrieval depth for one-hop and two-hop queries (i.e., the smallest top-

k

B^{c}_{1}

). Lower (

\downarrow

) values are better.

LC	Full-coverage ( $k$ )		LC (matched)
(Chars)	(Vectors)	(Chars)	(Chars)
200K	176	45,742	50K
400K	340	72,493	70K
500K	422	93,694	90K
600K	530	113,052	100K
1M	920	217,152	200K

Appendix

Table 15: Character-budget matching procedure . For each long-context (LC) haystack used during the retrieval stage on book

B^{c}_{1}

, we compute the minimum top-

k

such that the retrieved set contains all required needles and evidence facts for every sample. We then measure the character count of this retrieved set and match it to the LC condition with the closest character count. The columns report, from left to right, the original LC haystack size, the minimum top-

k

(in vectors), its character count, and the matched LC budget used for comparison.

Figure 11: Retrieval answer accuracy under the EBG prompting for the matched top- $k$ retrieval contexts . Columns denote the long-context character budgets we use for comparison after matching each retrieved top- $k$ context to the nearest long-context haystack length. Rows denote the evaluated models. Each cell reports answer accuracy, $\mathrm{Ans}_{\mathrm{RAG}}$ , after we condition the model on the retrieved top- $k$ chunks rather than the randomly sampled long-context haystack. Higher ( $\uparrow$ ) values are better.

Figure 12: Answer accuracy difference between RAG-EBG and LC-EBG , compared under approximately matched character budgets (Table 15 ). Positive values indicate that RAG-EBG outperforms LC-EBG; negative values indicate that LC-EBG outperforms RAG-EBG. Higher ( $\uparrow$ ) positive values are better for RAG.

	50K		70K		90K		100K		200K
Model	Emb-RAG	Reranker-RAG	Emb-RAG	Reranker-RAG	Emb-RAG	Reranker-RAG	Emb-RAG	Reranker-RAG	Emb-RAG	Reranker-RAG
Sonnet-4.5	100.0	100.0	87.5	100.0	75.0	100.0	75.0	100.0	75.0	87.5
GPT-4.1	75.0	100.0	75.0	87.5	62.5	87.5	87.5	75.0	75.0	87.5
GLM-5.2	62.5	50.0	62.5	75.0	75.0	75.0	50.0	75.0	50.0	75.0
GPT-OSS-120B	62.5	75.0	75.0	87.5	87.5	87.5	62.5	100.0	50.0	87.5

Appendix

Table 16: Comparison of Embedding-RAG and Reranker-RAG Ans accuracy under EBG prompting. Columns are indexed by the LC-matched retrieved-context budgets from Table 15 ; for example, 200K denotes the retrieval budget matched to the 1M LC haystack. Each cell reports answer accuracy over

n=8

samples (4 needles

\times

one-hop/two-hop). Emb: Embedding-RAG. Higher (

\uparrow

) values indicate better performance.

Dataset	Source	License	Eligible / Constructed Pool	Samples Used
Synthetic	Ours	CC BY 4.0	66,834 distractor facts; 10 two-hop needles	100
NoLiMa -based	Modarressi et al. (2025)	Adobe Research License	4 needle variants $\times$ 4 insertion depths	16
MultiHop-RAG	Tang and Yang (2024)	ODC-By	2,255 answerable queries	50, 200
QASPER-MP	Dasigi et al. (2021)	CC BY 4.0	Dev split: 1,005 questions over 281 papers	50, 200

Appendix

Table 17: Dataset statistics. For the synthetic benchmark, we construct 66,834 unique single-sentence distractor facts and curate 10 two-hop needles, evaluating 100 independently generated trials per context length. For the NoLiMa -based benchmark, we use four primary needle variants at four insertion depths, yielding 16 needle-placement conditions per context length per model. For MultiHop-RAG , removing 301 null_query examples leaves 2,255 answerable queries; we evaluate 50 queries with proprietary models and 200 with open-weight models. For QASPER-MP , experiments use the QASPER development split containing 1,005 questions over 281 papers. Each gold paper is paired with ten distractor papers. We evaluate 50 queries (38 answerable and 12 unanswerable) with proprietary models and 200 queries (150 answerable and 50 unanswerable) with open-weight models.

Model	Provider	Source URL	License
GPT-4o	OpenAI	https://platform.openai.com/docs/models/gpt-4o	Proprietary
GPT-4.1	OpenAI	https://platform.openai.com/docs/models/gpt-4.1	Proprietary
o3-mini	OpenAI	https://platform.openai.com/docs/models/o3-mini	Proprietary
GPT-5	OpenAI	https://platform.openai.com/docs/models/gpt-5	Proprietary
GPT-5.5	OpenAI	https://platform.openai.com/docs/models/gpt-5.5	Proprietary
Sonnet-4	Anthropic	https://www.anthropic.com/claude/sonnet	Proprietary

Appendix

Table 18: Models benchmarked in this work, along with their providers, source URLs, and licenses.

RAG Evaluation

Retrieval-augmented generation (RAG) evaluation measures how well systems retrieve, use, and cite evidence while balancing reliability, cost, and speed.

See all 352 papers

Relevance Is Not Sufficient Evidence: Detecting Evidence Gaps Before Generation in RAG
Sep 27, 2026Suting Chen, Peichun Hua, Yunming XiaoRetrieval-Augmented Generation Evidence-Grounded QA
Northwestern University · The Chinese University of Hong Kong, Shenzhen
LIT-RAGBench: Benchmarking Generator Capabilities of Large Language Models in Retrieval-Augmented Generation
Mar 6, 2026Koki Itai, Shunichi Hasegawa, Yuta Yamamoto +2RAG Evaluation LLM Evaluation
neoAI Inc., Tokyo, Japan · Tokyo Metropolitan University, Tokyo, Japan · The University of Tokyo, Tokyo, Japan
MASS-RAG: Multi-Agent Synthesis Retrieval-Augmented Generation
Apr 20, 2026Xingchen Xiao, Heyan Huang, Runheng Liu +1Retrieval-Augmented Generation Multi-Agent LLM Systems
School of Computer Science and Technology, Beijing Institute of Technology · Department of Mathematical Sciences, Tsinghua University

Sep 27, 2026J/K move · Enter open · S save

Relevance Is Not Sufficient Evidence: Detecting Evidence Gaps Before Generation in RAG

Suting Chen, Peichun Hua, Yunming Xiao

Northwestern University · The Chinese University of Hong Kong, Shenzhen

Retrieval-Augmented Generation Evidence-Grounded QA RAG Evaluation

Open paper AskPDF arXiv

Retrieval-augmented generation (RAG) grounds large language models in external sources, but retrieved passages often name the right entities without providing the facts needed to answer. Even when instructed to abstain, 12 generators answer 40.0-99.3% of insufficient-evidence questions. Training generators to abstain ties the decision to model weights, may reward answers recalled from parametric knowledge, and still requires a full generator call. Can sufficiency be judged from the question and evidence alone, before any answer exists? We identify pitfalls in constructing insufficient-evidence tests: removing relevant evidence or pairing evidence with unrelated questions can reveal labels through lexical overlap or evidence position. We build a paired benchmark using substitution, deletion, and question-swap constructions that vary answer support while controlling selected surface features, such as word use. Sufficiency can be judged without generating an answer, but no single signal works across all datasets. We introduce RINSE (Relevance Is Not Sufficient Evidence), which combines three signals: whether every part of the question is covered, whether any passage offers an answer, and whether a small language model reading the passages together judges them sufficient. Across six datasets, RINSE ranks sufficient above insufficient evidence with a score of 0.837 (chance 0.5), exceeding the best of 10 prior methods (0.746) and a frontier model queried through an API (0.784). Its weakest dataset scores higher than any other method's weakest (0.684 vs. 0.676). RINSE runs locally before generation, taking 36.5 ms per question on a single GPU.

Abstract

RAG Evaluation

Explore similar work

Relevance Is Not Sufficient Evidence: Detecting Evidence Gaps Before Generation in RAG

LIT-RAGBench: Benchmarking Generator Capabilities of Large Language Models in Retrieval-Augmented Generation

MASS-RAG: Multi-Agent Synthesis Retrieval-Augmented Generation