Scan or Index? large Evidence Coverage, Answer Quality, and Cost in Long-Document RAG

Juan Terven; Diana-Margarita Córdova-Esparza; Julio-Alejandro Romero-González; Edgar Chavez-Urbiola; Juan Irving Vasquez; Julian Andres Ramirez-Bautista; Silvia L. Chaparro-Cárdenas; Gendry Alfonso-Francia

cs.CLOct 20, 2026

Scan or Index? large Evidence Coverage, Answer Quality, and Cost in Long-Document RAG

Authors: Juan Terven, Diana-Margarita Córdova-Esparza, Julio-Alejandro Romero-González, Edgar Chavez-Urbiola, Juan Irving Vasquez, Julian Andres Ramirez-Bautista, Silvia L. Chaparro-Cárdenas, Gendry Alfonso-Francia

Organizations: CICATA Qro., Instituto Politécnico Nacional, México · Facultad de Informática, Universidad Autónoma de Querétaro, México · CIDETEC, Instituto Politécnico Nacional, México · Fundación Universitaria de San Gil - Unisangil, Colombia ·

Abstract

Scanning every passage in a long document seems thorough, but does it preserve more useful evidence or produce better answers? We compare 11 retrieval configurations spanning lexical and dense search, hybrid reranking, and full-document scanning with Jev, OpenAI Decisions, and Cloudflare Clef. Our evaluation uses 396 QASPER questions and 80 questions from eight complete textbooks, with common passages, a five-passage limit, and a fixed answer generator. On books, Hybrid + Jev achieves the highest observed evidence coverage (94.1%), compared with 88.7% for Jev full scan and 81.3% for OpenAI dense retrieval. Full scans examine every passage initially, yet later reduction steps can discard relevant evidence. Higher evidence coverage also does not consistently produce better answers: Hybrid + Decisions achieves the highest observed book Answer Accuracy among retrieval methods (0.892), despite recovering less reference evidence. Full scanning incurs substantially higher latency on books under the tested deployments, while reusable dense retrieval has lower projected API cost than paid decision-based methods as queries accumulate. Most QASPER articles already fit within the hybrid candidate pool, limiting the benefit of scanning. These results highlight evidence retention, answer quality, and reuse cost as complementary criteria for choosing a long-document retrieval strategy.

Figures & tables

Method	Configuration
Retrieval baselines
BM25	$k_{1}=1.2$ , $b=0.75$ ; English tokenization and stemming
OpenAI dense	text-embedding-3-small ; exact cosine search
BGE-M3 dense	BAAI/bge-m3 ; exact cosine search
Hybrid RRF	BM25 + BGE-M3, combined by reciprocal rank fusion (constant 60)
Hybrid reranking

Table 1: The 11 retrieval configurations. All use common passages and return up to five results; hybrid rerankers receive the same candidate pool.

Panel of Figure 1 — Figure 1: The three retrieval routes share a document, question, and passage boundaries. Indexed retrieval ranks from a reusable index; hybrid reranking selects from a common pool of up to 50 candidates; full scanning considers every passage before hierarchical reduction. Each route returns up to five passages, packed under the same 2,500-token budget for the fixed generator.

Method	QASPER EC@5 (%)	Books EC@5 (%)
BM25	84.7 [80.8, 88.4]	72.1 [63.6, 80.2]
OpenAI dense	90.4 [87.0, 93.5]	81.3 [72.8, 89.4]
BGE-M3 dense	89.9 [86.5, 93.0]	74.2 [62.1, 85.6]
Hybrid RRF	88.5 [85.0, 91.7]	77.4 [68.6, 86.0]
Hybrid + BGE	88.9 [85.8, 91.9]	86.6 [80.0, 93.1]
Hybrid + Jev	97.0 [95.3, 98.4]	94.1 [91.0, 97.2]

Table 2: Evidence coverage in the final five passages, averaged within documents and then across documents. QASPER includes 320 answerable questions from 230 articles; books include 80 questions from eight textbooks. Failed retrievals receive zero coverage. Brackets show pointwise 95% document-bootstrap intervals. Bold marks the highest observed mean in each corpus.

Figure 2: Evidence retained through completed textbook full scans: Jev 76, Decisions 72, and Clef 80. Curves are document-macro means over each method’s completed cases, not a common successful subset. Failed scans are excluded here but included in Table 2 . Initial access to every passage does not prevent loss during reduction.

	QASPER		Books
Method	Median / p95 (s)	Completed	Median / p95 (s)	Completed
Hybrid + Jev	0.338 / 0.751	396/396	1.724 / 2.372	79/80
Hybrid + Decisions	0.621 / 1.163	395/396	4.514 / 5.577	80/80
Hybrid + Clef	5.453 / 10.133	396/396	54.691 / 62.580	80/80
Jev full scan	0.339 / 0.717	396/396	25.103 / 45.711	76/80
Decisions full scan	0.621 / 1.038	396/396	63.361 / 112.586	72/80

Table 3: Median and 95th-percentile latency using the first execution of each assigned query, with failures excluded. Completed counts are relative to assigned queries. Hybrid times cover reranking only, excluding candidate generation; full-scan times include selection and reduction. Jev and Decisions allow two concurrent API calls. Clef runs serially in BF16 on a shared GB10 and includes local logging. Measurements describe these deployments, with different dates and hardware conditions.

Figure 3: Projected cumulative retrieval API cost per book,

C(N)=P+N\overline{Q}

, from measured preparation and query costs. Broken lines are known-charge lower bounds. Dense and Hybrid + Jev cross at

N\approx 5.21

, so dense is cheaper by the sixth query; unknown charges could move the crossover earlier. Both axes are logarithmic. Clef’s zero API charges are noted separately; generation, judging, and unpriced local resources are excluded.

Method	QASPER: $C(1000)$	Books: $C(1000)$
OpenAI dense	0.000296	0.01351
Hybrid + Jev	0.30214	$\geq 2.52$
Hybrid + Decisions	$\geq 0.69345$	6.63
Jev full scan	0.30537	$\geq 64.43$
Decisions full scan	0.70142	$\geq 155.85$
Clef (both variants)	0	0

Table 4: Projected retrieval API cost in US dollars for 1,000 queries per document, including preparation:

C(1000)=P+1000\overline{Q}

. Query costs average all assigned base queries within each document and then across documents. Unknown consumption produces lower bounds, rounded down. Clef and the other local retrieval methods incur no API charge; local computing resources are unpriced. Generation and judging are excluded.

Figure 4: Textbook evidence coverage versus automatic Answer Accuracy. EC@5 uses all 80 retrieval assignments; Answer Accuracy uses 53–60 successfully scored responses from 60 generation assignments per method. Both axes show document-macro means over different eligible subsets. Points are descriptive, not paired question-level observations; uncertainty in method differences is assessed by the paired comparisons in Table 11 .

Method	QASPER AA	Books AA	Books FA
BM25	0.581 [100/100]	0.828 [60/60]	0.891 [59/59]
OpenAI dense	0.614 [99/100]	0.835 [60/60]	0.910 [60/60]
BGE-M3 dense	0.623 [100/100]	0.864 [60/60]	0.901 [60/60]
Hybrid RRF	0.592 [100/100]	0.829 [60/60]	0.938 [59/59]
Hybrid + BGE	0.608 [100/100]	0.842 [59/59]	0.947 [59/59]
Hybrid + Jev	0.607 [99/99]	0.864 [60/60]	0.963 [60/60]

Table 5: Document-macro automated answer scores on 100 answerable QASPER questions and 60 book questions assigned per method. AA: Answer Accuracy; FA: Faithfulness. Brackets show scored/requested evaluations. Missing answers and failed evaluations are excluded from these means; FA also requires supplied evidence and a non-abstaining answer. Bold marks the highest observed mean among retrievers, excluding controls.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Book	Tokens	Passages	Timing Q.	Answer Q.
Test: 80 questions
Astronomy 2e	749,826	1,765	4	9
Biology 2e	954,082	2,245	4	9
Psychology 2e	515,399	1,213	9	6
Principles of Economics 3e	635,781	1,496	8	8
American Government 3e	509,472	1,199	8	5

Appendix

Table 6: Book sizes and test subsets. Timing questions receive three assigned retrieval executions; this column does not define the full-test primary timing comparison used for Jev, Decisions, and Clef. Answer questions are shared by all 13 answer arms. Development books are outside these test subsets. Token counts use the common text tokenizer.

Method	Primary failures QASPER / books	Valid repeated runs QASPER / 150	Valid repeated runs Books / 150
BM25	0 / 0	150	150
OpenAI dense	0 / 0	150	150
BGE-M3 dense	0 / 0	150	150
Hybrid RRF	0 / 0	150	149
Hybrid + BGE	0 / 0	150	150
Hybrid + Jev	0 / 1	143	146

Appendix

Table 7: Primary failures and repeated end-to-end timing availability. Every method has 396 primary QASPER and 80 primary book assignments. The repeated benchmark assigns 50 questions per corpus and three runs per question. Hybrid RRF has one completed run with invalid timing; other exclusions in the repeated-run columns are failed runs. Dashes indicate that Clef has no comparable completed repetition summary.

Full scan	Completed	Initial	First selection	Final input	Final five
Jev	76/80	100.00	99.38	95.75	93.18
Decisions	72/80	100.00	98.61	88.86	85.23
Clef	80/80	100.00	100.00	98.75	91.61

Appendix

Table 8: Book evidence coverage (%) through the reduction stages, averaged within documents and then across eight books. Each row uses that method’s completed questions; the rows are not restricted to a common successful subset. Stages describe candidate sets, and no confidence intervals are inferred for these trace summaries.

Corpus	Documents	Local mean / median (s)	OpenAI mean / median (s)
QASPER	264	0.60 / 0.47	1.14 / 0.64
Books	8	56.66 / 51.49	28.98 / 23.68

Appendix

Table 9: Document preparation, measured separately from retrieval and answer generation. Local time covers shared indexes; OpenAI time covers document embedding calls.

Method	QASPER: USD/query	Books: USD/query
OpenAI dense	$1.8625\times 10^{-7}$	$3.8050\times 10^{-7}$
Hybrid + Jev	0.00030214	$\geq 0.00252247$
Hybrid + Decisions	$\geq 0.00069345$	0.00662575
Jev full scan	0.00030537	$\geq 0.06443530$
Decisions full scan	0.00070142	$\geq 0.15585864$
Clef (both variants)	0	0

Appendix

Table 10: Mean retrieval API cost per assigned base query, averaging within documents and then across documents. Values use the rates recorded during execution; lower bounds are rounded down. The other local retrieval methods also have zero API charges, with computing resources unpriced.

Comparison	QASPER: difference [95% CI]; $n$	Books: difference [95% CI]; $n$
Jev scan $-$ OpenAI dense	$-0.011$ [ $-0.051$ , 0.030]; 99	$+0.048$ [0.003, 0.098]; 57
Hybrid + Decisions $-$ OpenAI dense	$-0.002$ [ $-0.041$ , 0.038]; 98	$+0.057$ [0.019, 0.103]; 60
Decisions scan $-$ OpenAI dense	$+0.003$ [ $-0.036$ , 0.043]; 99	$+0.043$ [0.015, 0.075]; 53
Hybrid + Clef $-$ OpenAI dense	$+0.000$ [ $-0.038$ , 0.039]; 99	$+0.028$ [ $-0.009$ , 0.065]; 59
Clef scan $-$ OpenAI dense	$+0.021$ [ $-0.018$ , 0.062]; 99	$+0.011$ [ $-0.013$ , 0.035]; 60
Hybrid + Decisions $-$ Hybrid + Jev	$+0.001$ [ $-0.028$ , 0.031]; 98	$+0.028$ [ $-0.005$ , 0.062]; 60

Appendix

Table 11: Paired Answer Accuracy differences, first method minus second. “Scan” denotes full scan. Each cell reports the difference, its 95% interval, and the number of jointly scored questions

n

. Intervals use 10,000 whole-document bootstrap resamples with seed 42. An interval containing zero does not establish equivalence.

Method	QASPER answerable 100 assigned	Books 60 assigned	QASPER unanswerable 40 assigned
BM25	100 (2)	60 (1)	40 (12)
OpenAI dense	100 (1)	60 (0)	40 (17)
BGE-M3 dense	100 (2)	60 (0)	39 (16)
Hybrid RRF	100 (2)	60 (1)	40 (12)
Hybrid + BGE	100 (1)	59 (0)	40 (15)
Hybrid + Jev	99 (1)	60 (0)	40 (12)

Appendix

Table 12: Each cell gives completed answers, with abstentions in parentheses. Abstention is an observed response behavior; these are question counts, not document-macro percentages. Rates use completed answers as the denominator: BGE-M3, for example, abstains in 16 of its 39 completed unanswerable cases. Missing answers remain separate.

Questions	Metric	Scored / requested	N/A	Errors
QASPER answerable	Faithfulness	1,172 / 1,177	121	5
	Answer Accuracy	1,296 / 1,298	0	2
Books	Faithfulness	705 / 707	62	2
	Answer Accuracy	767 / 769	0	2
QASPER unanswerable	Faithfulness	293 / 293	226	0
	Answer Accuracy	0 / 0	519	0

Appendix

Table 13: Evaluation outcomes across all 13 answer arms. N/A counts metrics not requested for completed answers; it excludes missing generations. Requested evaluations equal scored evaluations plus errors. Failed and inapplicable metrics receive no imputed score.

Long-Context Retrieval

Long-context retrieval finds and uses relevant evidence across lengthy documents, conversations, and model contexts.

See all 45 papers

VELD-QA: A Large-Scale Dataset for Visual Evidence Grounding and Question Answering in Long Documents
Oct 14, 2026Zhivar Sourati, Marianne Menglin Liu, Rongguang Wang +12Long-Document VQA Document QA
University of Southern California · Oracle AI · Turing Enterprise Inc +1
LENS: In-Context Search via Latent Evidence Exploration over Dynamic Raw Documents
Aug 17, 2026Xingjun Wang, Gongsheng Li, Qi Fan +3In-Context Retrieval Evidence Retrieval
ModelScope Team, Alibaba Group Hangzhou, China
Re:CAP - Auditing Retrieval Coverage in Production RAG Pipelines
Sep 21, 2026Aviral Joshi, Hanoz Bhathena, Max Nelson +1RAG Evaluation IR Evaluation
Machine Learning Center of Excellence, JPMorgan Chase & Co.

Oct 14, 2026J/K move · Enter open · S save

VELD-QA: A Large-Scale Dataset for Visual Evidence Grounding and Question Answering in Long Documents

Zhivar Sourati, Marianne Menglin Liu, Rongguang Wang, Zheng Wang+11

University of Southern California · Oracle AI · Turing Enterprise Inc+1

Long-Document VQA Document QA Evidence-Grounded QA Visual Question Answering VLM Evaluation Evidence Retrieval

Open paper AskPDF arXiv

Existing long-document QA benchmarks rarely combine large-scale human-authored questions with detailed visual evidence annotations, limiting diagnosis of evidence localization and interpretation failures. We present VELD-QA, a large-scale benchmark for Visual Evidence grounding and question answering in Long Documents. It contains 15,000 fully human-authored question-answer pairs across 5,859 real-world PDFs totaling 372,445 pages, balancing chart, table, and complex-layout tasks. Human annotations identify supporting pages and evidence elements and describe cited visual content, enabling separate retrieval and interpretation evaluation. We evaluate nine vision-language models under four evidence settings and seven retrieval-augmented and agentic systems. Our analysis identifies bottlenecks in evidence localization and interpretation. Retrieval-based methods substantially underperform the oracle setting with ground-truth supporting pages; full-document access does not close the gap. Localization alone is insufficient: oracle accuracy peaks at 81.7%, highlighting persistent visual interpretation and reasoning challenges. Enriching inputs with our visual-element annotations narrows the performance gap by 17-24%, demonstrating value beyond evaluation and motivating data enrichment for multimodal document QA. Our code is available at https://github.com/oracle-samples/veld-qa.

Abstract

Long-Context Retrieval

Explore similar work

VELD-QA: A Large-Scale Dataset for Visual Evidence Grounding and Question Answering in Long Documents

LENS: In-Context Search via Latent Evidence Exploration over Dynamic Raw Documents

Re:CAP - Auditing Retrieval Coverage in Production RAG Pipelines