Retrieval-Augmented Generation (RAG) systems critically depend on retrieval quality, yet no systematic comparison of modern retrieval methods exists for heterogeneous documents containing both text and tabular data. We benchmark ten retrieval strategies spanning sparse, dense, hybrid fusion, cross-encoder reranking, query expansion, index augmentation, and adaptive retrieval on a challenging financial QA benchmark of 23,088 queries over 7,318 documents with mixed text-and-table content. We evaluate retrieval quality via Recall@k, MRR, and nDCG, and end-to-end generation quality via Number Match, with paired bootstrap significance testing. Our results show that (1) a two-stage pipeline combining hybrid retrieval with neural reranking achieves Recall@5 of 0.816 and MRR@3 of 0.605, outperforming all single-stage methods by a large margin; (2) BM25 outperforms state-of-the-art dense retrieval on financial documents, challenging the common assumption that semantic search universally dominates; and (3) query expansion methods (HyDE, multi-query) and adaptive retrieval provide limited benefit for precise numerical queries, while contextual retrieval yields consistent gains. We provide ablation studies on fusion methods and reranker depth, actionable cost-accuracy recommendations, and release our full benchmark code.
Figures & tables
Category
Method
R@1
R@3
R@5
R@10
MRR@3
nDCG@10
MAP
Single-method
BM25 (sparse)
0.293
0.552
0.644
0.735
0.411
0.515
0.449
Dense (text-embed-3-large)
0.248
0.481
0.587
0.703
0.351
0.466
0.398
Query expansion
HyDE (gpt-4.1-mini)
0.221
0.441
0.544
0.671
0.318
0.433
0.365
Multi-Query + RRF
0.283
0.539
0.640
0.734
0.397
0.506
0.439
Index augment.
Contextual Dense
0.266
0.508
0.615
0.732
0.373
0.490
0.420
Contextual Hybrid
0.327
0.610
0.717
0.818
0.454
0.571
0.497
TABLE I: Main retrieval results on T 2 -RAGBench (23,088 queries, 7,318 documents). Methods are grouped by category. Hybrid RRF with cross-encoder reranking dominates all methods. Contextual Hybrid outperforms vanilla Hybrid RRF. CRAG provides moderate gains through adaptive query correction. HyDE underperforms vanilla dense retrieval. All pairwise differences between adjacent methods are statistically significant ( p<0.001 ). Best results in bold .
Subset
Method
R@5
R@10
MRR@3
FinQA
BM25
0.729
0.834
0.389
Dense
0.611
0.748
0.308
Hybrid
0.737
0.856
0.389
ConvFinQA
BM25
0.696
0.781
0.500
Dense
0.654
0.781
0.410
Hybrid
0.754
0.850
0.519
TABLE II: Per-subset retrieval results. TAT-DQA is the most challenging subset across all methods. Hybrid RRF provides the largest improvement on TAT-DQA (+8.1pp Recall@5 over BM25).
Fig. 1: Recall@ k curves for BM25, dense (text-embedding-3-large), and hybrid RRF retrieval. Hybrid fusion consistently outperforms both single-method baselines, with the largest gains at small k .
Fig. 2: Grouped comparison of retrieval methods across five metrics. Hybrid RRF (green) dominates, while BM25 (blue) outperforms dense retrieval (orange) on this financial text-and-table benchmark.
Fig. 3: Recall@5 heatmap across retrieval methods and dataset subsets. Darker colors indicate higher retrieval quality. TAT-DQA is consistently the most challenging subset.
Retrieval
GPT-4.1-mini
GPT-5.4
BM25
0.251
†
Dense
0.257
†
Hybrid RRF
0.282
0.346
Oracle
0.350
0.403
TABLE III: End-to-end Number Match (NM) by retrieval method and LLM. Better retrieval consistently leads to better generation quality. GPT-5.4 outperforms GPT-4.1-mini by 6–7pp on the same retrieval. † Not evaluated.
Fig. 4: Correlation between retrieval quality (Recall@5) and generation quality (Number Match). The strong positive correlation ( r>0.99 ) confirms that better retrieval leads to better answers.
Fig. 5: Fusion method ablation. Left: Convex Combination with varying α (dense weight); α=0.5 is optimal. Right: RRF with varying k ; lower k yields slightly better results.
Fig. 6: Reranker depth ablation. More candidates yield better results, with a critical threshold at 50. Format: candidates → top-N returned.
Failure Category
%
Table structure mismatch
73
Numerical reasoning
20
Vocabulary mismatch
5
Ambiguous query
1
Long document
1
TABLE IV: Failure mode categorization ( n=100 sampled failures from hybrid RRF top-5). Table structure mismatch is the dominant failure mode.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Parameter
Value
Notes
BM25
k1
1.2
Term-frequency saturation
b
0.75
Document-length normalization
Tokenizer
whitespace split
Via rank_bm25 library
Dense Retrieval
Embedding model
text-embedding-3-large
OpenAI via Azure AI Foundry
Dimensions
3,072
Full dimensionality, no reduction
Index type
FAISS IndexFlatIP
Exact inner-product (cosine) search
Appendix
TABLE V: Complete hyperparameter settings for all retrieval methods. All LLM calls use gpt-4.1-mini via Azure AI Foundry. Embedding uses text-embedding-3-large (3,072 dimensions) for all dense components.
Method
R@1
R@3
R@5
R@10
R@20
MRR@3
MRR@5
nDCG@5
nDCG@10
MAP
HyDE
0.221
0.441
0.544
0.671
0.767
0.318
0.341
0.392
0.433
0.365
Dense
0.248
0.481
0.587
0.703
0.798
0.351
0.375
0.428
0.466
0.398
Contextual Dense
0.266
0.508
0.615
0.732
0.817
0.373
0.398
0.452
0.490
0.420
Multi-Query
0.283
0.539
0.640
0.734
0.820
0.397
0.420
0.475
0.506
0.439
BM25
0.293
0.552
0.644
0.735
0.797
0.411
0.432
0.485
0.515
0.449
CRAG
0.302
0.556
0.658
0.788
0.788
0.415
0.439
0.493
0.536
0.456
Appendix
TABLE VI: Full retrieval results for all methods and metrics on T 2 -RAGBench (23,088 queries, 7,318 documents). Methods are sorted by nDCG@10 in ascending order. Best results in bold . ∗ Hybrid + Rerank returns at most 10 documents, so R@20 is not applicable.
Retrieval-augmented generation (RAG) evaluation measures how well systems retrieve, use, and cite evidence while balancing reliability, cost, and speed.