Multi-hop question answering requires aggregating information from multiple documents, a critical capability for knowledge-intensive applications. A fundamental challenge lies in efficiently identifying the minimal relevant document set from retrieved candidates while maintaining high recall. We present an efficient dual-view cascaded reranking framework for multi-hop document reranking. Operating as a lightweight post-retrieval stage over E5-base-v2 candidates, our architecture comprises: (1) a Local Scorer employing stacked cross-attention for fine-grained query-document relevance; and (2) a Global Scorer modeling inter-document dependencies via Transformer-based context aggregation. These views are dynamically fused through an Adaptive Gate conditioned on query semantics. Under the fixed candidate set reranking setting with offline cached embeddings, our model achieves competitive results, particularly outstanding on MuSiQue with 99.4% Top-4 Recall and 97.8% Full Hit accuracy at 4.0 ms latency (249 QPS). It substantially outperforms 600M-parameter cross-encoders (BGE-Large: 92.0% Recall, Jina-v3: 90.1% Recall) while maintaining 5 to 6 times lower latency. Ablation studies validate that both Local and Global views contribute substantially to multi-hop performance.
Figures & tables
Figure 1. Overview of our lightweight cascaded reranker. Given a query and candidate documents encoded by a frozen E5 encoder, the Local Scorer captures fine-grained query-document interactions, the Global Scorer models inter-document contextual relationships through a 2-layer Transformer, and the Adaptive Gate dynamically fuses both views conditioned on query semantics. Architecture diagram showing the three-stage pipeline: (1) Frozen E5 encoder produces 768-dim embeddings for query and documents; (2) Local Scorer applies stacked cross-attention to compute query-document relevance; (3) Global Scorer processes the document set via induced set attention to capture context; (4) Adaptive Gate fuses local and global scores with query-conditioned weighting. The final output is a reranked document list for downstream LLM generation.
Figure 2. Architecture of the Local and Global Scorers. The frozen E5 encoder produces embeddings for both query and documents. The Local Scorer (left) applies stacked cross-attention to capture fine-grained query-document interactions, while the Global Scorer (right) employs Transformer-based context aggregation over local features to model inter-document dependencies. Architecture diagram showing Local Scorer with cross-attention layers and Global Scorer with Transformer-based aggregation.
Figure 3. Adaptive Gate architecture for dynamic view fusion. The gate receives local features, global features, and query embedding as inputs. Through query-conditioned projections, it computes a fusion weight gi for each document to dynamically balance local and global scores based on query semantics. Adaptive Gate architecture showing projection layers and query-conditioned fusion mechanism.
Dataset
Split
Samples
Candidates
2WikiMultihopQA ( Ho et al., 2020 )
Train
7,500 (50%)
10
2WikiMultihopQA
Dev
7,785
10
HotpotQA ( Yang et al., 2018 )
Train
10,000 (11.1%)
10
HotpotQA
Dev
7,405
10
MuSiQue ( Trivedi et al., 2022 )
Train
9,969 (50%)
6
MuSiQue
Dev
2,417
6
Table 1. Dataset statistics. All candidate sets are pre-constructed with fixed size limits and pre-computed E5 embeddings.
Model
Params
R@4
FH@4
N@4
M@4
P@4
E5-Cosine
0M
83.5
62.8
84.6
97.5
49.1
E5-MLP
0.6M
94.2
86.0
92.2
96.0
57.4
Ours
11M
93.9
85.0
92.8
97.4
57.1
MiniLM-L6
22M
78.6
54.2
81.0
97.8
45.9
BGE-Large
560M
87.8
69.6
89.4
99.8
52.2
BGE-v2-m3
568M
86.5
67.1
88.1
99.6
51.3
Table 2. Retrieval performance on 2WikiMultihopQA (in-domain, Top-4). Bold : best overall; underlined : best ≤ 22M params.
Model
Params
R@4
FH@4
N@4
M@4
P@4
E5-Cosine
0M
89.0
78.8
86.6
94.0
44.6
E5-MLP
0.6M
88.2
77.7
83.9
89.9
44.2
Ours
11M
90.8
82.5
87.8
93.6
45.6
MiniLM-L6
22M
83.4
67.8
83.0
95.0
41.9
BGE-Large
560M
95.8
91.9
94.6
98.1
48.1
BGE-v2-m3
568M
95.3
90.9
93.9
97.9
47.8
Table 3. Retrieval performance on HotpotQA (held-out dev set, Top-4).
Model
Params
R@4
FH@4
N@4
M@4
P@4
E5-Cosine
0M
71.9
38.3
67.2
83.2
44.9
E5-MLP
0.6M
98.4
94.7
97.8
99.6
64.9
Ours
11M
99.4
97.8
99.2
99.9
65.7
MiniLM-L6
22M
77.3
48.0
78.5
94.9
49.8
BGE-Large
560M
92.0
78.1
90.9
97.6
59.7
BGE-v2-m3
568M
89.4
71.4
88.7
97.2
57.9
Table 4. Retrieval performance on MuSiQue (held-out dev set, most challenging, Top-4). Our 11M model achieves SOTA with 54 × fewer parameters than cross-encoders.
Model
Params
Mean (ms)
P95 (ms)
QPS
Type
E5-Cosine
0M
3.1
3.2
327
bi-encoder
E5-MLP
0.6M
3.3
3.6
303
MLP reranker
Ours
11M
4.0
4.1
249
cascaded
MiniLM-L6
22M
3.7
6.2
271
cross-encoder
BGE-Large
560M
21.8
36.8
46
cross-encoder
BGE-v2-m3
568M
21.9
37.6
46
cross-encoder
Table 5. Inference efficiency (online, doc embeddings cached). Lower latency and higher QPS are better. All measurements on NVIDIA RTX 4090, batch size 1.
Configuration
R@4
FH@4
N@4
M@4
P@4
Full-Model (Ours)
99.4
97.8
99.2
99.9
65.7
− Adaptive Gate (avg fusion)
96.1
87.7
95.2
99.0
63.0
− Global Scorer
95.9
87.1
94.9
98.9
62.9
− Local Scorer (global-only)
84.0
61.1
79.3
87.8
53.7
E5-MLP (0.6M baseline)
98.4
94.7
97.8
99.6
64.9
Table 6. Ablation study on MuSiQue development set (Top-4 metrics). MuSiQue’s 2–4 hop complexity better reveals component contributions for multi-hop reasoning.
Multi-hop question answering studies how systems combine evidence across documents, facts, or modalities to answer questions requiring linked reasoning.
Retrieval-Augmented Generation (RAG) systems for multi-hop Question Answering (QA) must balance retrieval quality with computational cost. This cost is incurred during indexing time, through the use of expensive Knowledge Graphs (KGs) or Large Language Models (LLMs) to generate summaries, or during querying, through iterative LLM-driven retrieval. To reduce it while maintaining retrieval quality, we present MatRAG, a hierarchical framework that combines RAG systems with Matryoshka Representation Learning (MRL). MatRAG addresses both kinds of cost by aligning the semantic hierarchy of a clustering structure with the nested structure of MRL. Specifically, it organizes the corpus of documents into a Directed Acyclic Graph (DAG) of clusters with progressively coarser granularity. Each level is indexed by a lower Matryoshka dimension. MatRAG pairs an iterative, top-down traversal of the DAG with an entity-driven mechanism that controls the hop budget and re-ranks candidates. We evaluated MatRAG on three standard multi-hop QA benchmarks against seven representative baselines. MatRAG outperforms its strongest competitors in terms of retrieval quality; furthermore, it reduces indexing costs by avoiding KG construction and LLM-based summarization, and lowers query-time costs through dimension-aware similarity.
Gianluca Bonifazi, Christopher Buratti, Michele Marchetti +5
Polytechnic University of Marche, Ancona, Italy · University of Modena and Reggio Emilia, Modena, Italy
Retrieval-augmented generation (RAG) critically depends on retrieving the evidence necessary for effective reasoning. However, this remains particularly challenging in multi-hop question answering (QA), where supporting passages are often linked through intermediate entities and relations that must be progressively uncovered. Existing retrieval approaches typically rely on a single retrieval intent or one-shot query expansion, limiting their ability to adapt to newly retrieved evidence and potentially introducing noisy or redundant retrieval signals. To address these limitations, we propose a training-free multi-hop retrieval framework that integrates evidence-conditioned exploration, passage-specific contrastive refinement, and coverage-aware final ranking. During offline indexing, the framework constructs passage-specific contrastive facets that characterize each passage relative to its semantically similar neighbors, providing fine-grained signals to distinguish closely related candidates. At inference time, the framework iteratively retrieves evidence, generates probes targeting unresolved information needs, refines candidate relevance using the contrastive facets, and selects a complementary set of passages that collectively cover diverse evidence-seeking intents. Experiments on MuSiQue, HotpotQA, and 2WikiMultihopQA demonstrate consistent improvements in retrieval quality and downstream QA performance over baselines.
Reasoning-based reranking with Large Language Models (LLMs) has shown promising improvements in text ranking. However, current methods predominantly rely on a single reasoning trajectory, resulting in rankings that are susceptible to reasoning errors and inherently constrained in modeling the multifaceted signals underlying document relevance. To resolve this dilemma, we propose MERIT-Rank(Multi-perspective Evidence and Reasoning Integration for Text Reranking), a framework that models complementary reasoning trajectories to improve reranking robustness. MERIT-Rank formulates a Multi-Trajectory Reasoning Space (MTRS) that evaluates query-document relevance from multiple perspectives and introduces a joint reranker that consolidates these reasoning paths into a unified ranking decision. We further develop Progressive Rank Policy Optimization (PRPO), a progressive training framework that stabilizes reasoning trajectories while continually improving ranking quality through staged optimization objectives. Experiments on both reasoning-intensive and traditional retrieval benchmarks show that MERIT-Rank consistently achieves superior performance over competitive baselines. The 4B model notably outperforms most 7B and even 32B rerankers on BRIGHT.