cs.IRApr 14, 2026

Adaptive Query Routing: A Tier-Based Framework for Hybrid Retrieval Across Financial, Legal, and Medical Documents

Authors: Afshan Hashmi

Organizations: TRDC, Tuwaiq Academy, Riyadh, Saudi Arabia

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Retrieval-Augmented Generation (RAG) has become the standard paradigm for grounding Large Language Model outputs in external knowledge. Lumer et al. [1] presented the first systematic evaluation comparing vector-based agentic RAG against hierarchical node-based reasoning systems for financial document QA across 1,200 SEC filings, finding vector-based systems achieved a 68% win rate. Concurrently, the PageIndex framework [2] demonstrated 98.7% accuracy on FinanceBench through purely reasoning-based retrieval. This paper extends their work by: (i) implementing and evaluating three retrieval architectures: Vector RAG, Tree Reasoning, and the proposed Adaptive Hybrid Retrieval (AHR) across financial, legal, and medical domains; (ii) introducing a four-tier query complexity benchmark; and (iii) employing GPT-4-powered LLM-as-judge evaluation. Experiments reveal that Tree Reasoning achieves the highest overall score (0.900), but no single paradigm dominates across all tiers: Vector RAG wins on multi-document synthesis (Tier 4, score 0.900), while the Hybrid AHR achieves the best performance on cross-reference (0.850) and multi-section queries (0.929). Cross-reference recall reaches 100% for tree-based and hybrid approaches versus 91.7% for vector search, quantifying a critical capability gap. Validation on FinanceBench (150 expert-annotated questions on real SEC 10-K and 10-Q filings) confirms and strengthens these findings: Tree Reasoning scores 0.938, Hybrid AHR 0.901, and Vector RAG 0.821, with the Tree--Vector quality gap widening to 11.7 percentage points on real-world documents. These findings support the development of adaptive retrieval systems that dynamically select strategies based on query complexity and document structure. All code and data are publicly available.

Figures & tables

TierTypeOptimal StrategyExample
T1Single-fact lookupVector similarityWhat was Q3 revenue?
T2Multi-section reasoningTree navigationRevenue vs. guidance?
T3Cross-referenceHierarchical traversalAppendix details?
T4Multi-document synthesisHybrid fusionCompare across filings
Table 1: Query Complexity Tier Classification
MethodQualityRecallPrecisionF1Latency
Tree Reasoning0.9000.9770.8500.8393.40s
Hybrid AHR (Proposed)0.8450.9550.6550.6703.10s
Vector RAG0.8450.9320.3410.4721.61s
Table 2: Overall Performance Comparison (22 Queries, 3 Domains)
Figure 1: (a) LLM-as-Judge quality score and (b) retrieval recall by query complexity tier. No single method dominates all tiers, supporting the case for adaptive retrieval.
Figure 2: Domain-wise performance comparison across financial, legal, and medical documents. Tree Reasoning dominates legal and medical domains; Vector RAG leads in financial.
DomainVector RAGTree ReasoningHybrid AHR
Financial0.9100.8900.880
Legal0.8000.8830.850
Medical0.7900.9330.800
Table 3: LLM-as-Judge Quality Score by Domain
Figure 3: Latency vs. answer quality trade-off. Vector RAG clusters in low-latency with high variance; Tree Reasoning shows higher but more consistent quality.
Figure 4: Cross-reference resolution capability on Tier 3 queries. Tree-based approaches achieve 100% recall vs. 91.7% for Vector RAG.
Figure 5: Answer quality heatmap (Domain ×\times Tier) for each retrieval method. Vector RAG shows weakness on medical T1 (0.50); Tree Reasoning achieves 1.00 on medical T2.
MethodOverallAccuracyCompletenessRelevanceLatency
Tree Reasoning0.9380.9360.9140.9644.15s
Hybrid AHR (Proposed)0.9010.8840.8760.9424.36s
Vector RAG0.8210.7980.7680.8982.40s
Table 4: Performance on FinanceBench real SEC filings ( n=50n=50 , same LLM-as-judge protocol).
Question typeVector RAGTree ReasoningHybrid AHR
Information extraction0.9260.9900.938
Numerical reasoning0.8920.9740.956
Logical reasoning (numerical)0.5000.5330.500
Logical reasoning (numerical + logical)0.7340.9500.900
Numerical + logical reasoning0.6110.8890.911
Numerical + information extraction0.4440.8440.867
Table 5: FinanceBench performance by question reasoning type.
Figure 6: FinanceBench evaluation on real SEC filings. (A) Overall LLM-as-Judge quality scores: Tree Reasoning 0.938, Hybrid AHR 0.901, Vector RAG 0.821. (B) Performance by question type showing numerical reasoning as the most challenging category. (C) Latency vs. quality: Tree Reasoning shows higher and more consistent quality; Vector RAG is faster with higher variance.
More in this topic

Adaptive RAG

Adaptive retrieval-augmented generation (RAG) adjusts when, what, and how much evidence a language model retrieves for each query.

See all 59 papers

Explore similar work

CardsList
Mar 26, 2026

Sustainable Hybrid Document-Routed Retrieval for Financial RAG: Resolving the Robustness-Precision Trade-off

Retrieval-Augmented Generation (RAG) systems for financial document QA typically follow a chunk-based paradigm: documents are split into fragments, embedded, and retrieved by similarity. In structurally homogeneous corpora such as regulatory filings, this suffers from cross-document chunk confusion. Semantic File Routing (SFR), which uses LLM structured output to route queries to whole documents, reduces catastrophic failures but sacrifices targeted-chunk precision. We identify this robustness-precision trade-off on the FinDER benchmark (1,500 queries across five groups): SFR achieves higher average scores (6.45 vs. 6.02) and fewer failures (10.3% vs. 22.5%), while chunk-based retrieval (CBR) yields more perfect answers (13.8% vs. 8.5%). To resolve it, we propose Hybrid Document-Routed Retrieval (HDRR), a two-stage architecture that uses SFR as a document filter followed by chunk retrieval scoped to the identified document(s), eliminating cross-document confusion while preserving chunk precision. HDRR achieves the best performance on every metric: an average score of 7.54 (25.2% above CBR, 16.9% above SFR), a 6.4% failure rate, 67.7% correctness (+18.7 pp over CBR), and a 20.1% perfect-answer rate (+6.3 pp over CBR, +11.6 pp over SFR), simultaneously attaining the lowest failure rate and highest precision across all five groups. Beyond accuracy, HDRR is also the most efficient of the high-quality systems: it preserves CBR's compact per-query token budget (~5K-15K, an order of magnitude below SFR's ~50K-200K), incurs no indexing-time LLM spend (versus the one-time ~$100 cost of contextual indexing), and uses fewer per-query LLM calls than self-correcting agentic baselines, translating directly to lower API spend and inference-time energy at deployment scale.

Zhiyuan Cheng, Longying Lai, Yue Liu

School of Engineering, Stanford University, Stanford, CA, USA · bSimon Business School, University of Rochester, Rochester, NY, USA · cAccounting & Information Systems, Rutgers University, Newark, NJ, USA

Retrieval-Augmented GenerationFinancial QAInformation RetrievalHybrid RetrievalFinance
AskPDF
Apr 2, 2026

From BM25 to Corrective RAG: Benchmarking Retrieval Strategies for Text-and-Table Documents

Retrieval-Augmented Generation (RAG) systems critically depend on retrieval quality, yet no systematic comparison of modern retrieval methods exists for heterogeneous documents containing both text and tabular data. We benchmark ten retrieval strategies spanning sparse, dense, hybrid fusion, cross-encoder reranking, query expansion, index augmentation, and adaptive retrieval on a challenging financial QA benchmark of 23,088 queries over 7,318 documents with mixed text-and-table content. We evaluate retrieval quality via Recall@k, MRR, and nDCG, and end-to-end generation quality via Number Match, with paired bootstrap significance testing. Our results show that (1) a two-stage pipeline combining hybrid retrieval with neural reranking achieves Recall@5 of 0.816 and MRR@3 of 0.605, outperforming all single-stage methods by a large margin; (2) BM25 outperforms state-of-the-art dense retrieval on financial documents, challenging the common assumption that semantic search universally dominates; and (3) query expansion methods (HyDE, multi-query) and adaptive retrieval provide limited benefit for precise numerical queries, while contextual retrieval yields consistent gains. We provide ablation studies on fusion methods and reranker depth, actionable cost-accuracy recommendations, and release our full benchmark code.

Meftun Akarsu, Recep Kaan Karaman, Christopher Mierbach

Technische Hochschule Ingolstadt · Uludag University · Radiate

RAG EvaluationHybrid RetrievalCross-Encoder RerankingFinancial QATable QA
AskPDF
Oct 8, 2026

RIT-RAG: Navigating Document Corpora with Retrieval-Induced Trees

Retrieval-augmented generation (RAG) grounds language models in external corpora. Agentic RAG enables iterative search, yet exposes the model to isolated chunks without document structure, making it difficult to distinguish relevant evidence from chunks that merely resemble the query. Structure-aware methods such as PageIndex navigate document structure but cannot scale to the structures of large corpora, which do not fit in the LLM context. Hence, they first commit to a single document using a document retriever and cannot recover from a wrong choice. We propose RIT-RAG (Retrieval-Induced Tree RAG), which combines content retrieval with structural navigation. Offline, RIT-RAG builds a tree for each document from its table of contents or sitemap. At query time, it retrieves a broad set of chunks and uses their positions to induce manageable sub-trees, potentially across multiple documents. An LLM agent navigates these sub-trees, selectively reads promising nodes, and reformulates queries when needed. Thus, retrieval proposes where to look, while the agent decides what to read. Across financial, scientific, and customer-support benchmarks, RIT-RAG achieves the highest answer accuracy among vanilla, graph-based, and agentic baselines. On EntQABench, our new benchmark of 2.84 million technical-documentation webpages, it improves accuracy by 6.8 to 11.4 points over the strongest baseline across three LLMs.

Meghanadh Pulivarthi, Swaraj Kumar Biswal, Kushagra Bhushan +3

IBM · IIT Kharagpur

Retrieval-Augmented GenerationStructured RAGAgentic RAGQuestion Answering
AskPDF