Attention-based rerankers score each candidate by the attention mass that retrieval heads route to it from the question, but the question, serving as a fixed query, carries limited information for locating relevant evidence. We propose Re3Rank, a listwise reranker built on reasoning-conditioned retrieval heads (RRHeads). A generator first writes a reasoning trace over the question and the candidates, and a scorer then uses the trace tokens as the attention query of a few selected RRHeads, reading relevance as continuous scores. We present a straight-through estimator to propagate the ranking loss through the discrete trace, so that the generator and the scorer can be trained jointly for ranking. Averaged over 9 benchmarks, Re3Rank outperforms 11 baseline rerankers: our 4B variant exceeds the strongest baseline by 3.11 NDCG@10 points on average, and even our 0.6B variant surpasses rerankers with up to 32B parameters.
Figures & tables
Figure 1: NDCG@10 (%) of rerankers averaged over 9 datasets, against model size (left) and reranking time per query at a batch size of 50 (right). † † footnotemark:
Figure 2: Overview of Re 3 Rank . A first-stage retriever returns candidates for the question from the corpus. The generator writes a reasoning trace from the question and the candidates, and the scorer reads the same input followed by the reasoning trace. Of all attention heads in the scorer, only a few selected RRHeads are used: each candidate is scored by the attention mass that these heads route from the reasoning trace positions to it. The ranking loss Lrank updates both components and reaches the generator through a straight-through estimator.
Source
#Questions
Avg. Gold
Avg. Tokens
NarrativeQA
12,000
2.47
10,213
MuSiQue
12,000
1.83
7,378
ConvoMem
7,000
2.06
10,390
QAConv
5,000
1.05
9,857
FriendsQA
2,000
1.01
11,940
Total
38,000
1.93
9,394
Table 1: Statistics of the training mixture by source dataset. Avg. Gold is the mean number of in-pool relevant candidates per question; Avg. Tokens is the mean prompt length before the generated trace.
Figure 3: Conditional R@3 (%) of rerankers across model sizes on the nine benchmarks and their macro average (Macro-9). Lines connect sizes of the same model family, and the dashed line marks the first-stage ranking of Qwen3-Embedding-8B.
Figure 4: NDCG@10 gain of Re 3 Rank (4B) over QRRanker on each benchmark, measured as a difference and shown with the mean number of in-pool relevant chunks per question from Table 12 .
Method
R@1
R@3
R@5
N@10
QR (base)
41.14
69.16
77.06
76.12
RR (base)
41.15 +0.01
71.73 +2.57
79.60 +2.54
77.52 +1.40
QR (trained)
42.76
74.05
81.98
80.33
RR (trained)
44.20 +1.44
76.70 +2.65
84.01 +2.03
82.23 +1.90
Table 2: Comparison of query construction for Qwen3-4B, reported as Macro-9 scores (%). QR uses a single model with the question tokens as attention queries and its top-2 QRHeads, (23,10) and (15,11) . RR uses the generator’s reasoning trace as attention queries with the RRHeads (23,10) and (21,8) . Base rows kept all weights untrained, and trained rows were trained with Lrank in Eq. 13 . Subscripts give the gain of RR over QR under the same training setting.
Figure 5: Ablation of joint training for Re 3 Rank (4B) trained with Lrank , reported as Macro-9 scores (%). Four settings refer to whether the generator ( G ) and the scorer ( S ) were trained or kept at their base weights.
Objective
R@1
R@3
R@5
N@10
LCE
43.91
76.57
83.87
82.05
LCE , α=12
43.77 -0.14
75.60 -0.97
83.09 -0.78
81.35 -0.70
Lrsk
43.79 -0.12
76.81 +0.24
83.75 -0.12
81.99 -0.06
LCE+Lrsk
44.53 +0.62
76.49 -0.08
83.72 -0.15
82.28 +0.23
LCE+2Lrsk †
44.20 +0.29
76.70 +0.13
84.01 +0.14
82.23 +0.18
LCE+4Lrsk
44.15 +0.24
76.62 +0.05
83.98 +0.11
82.19 +0.14
Table 3: Ablation of ranking objectives for Re 3 Rank (4B), reported as Macro-9 scores (%). Lrsk used K={3,10} and τ1=τ2=1 , and all rows used the score scale α=8 in Eq. 7 unless noted. Subscripts give the difference from the first row, with gains in green and losses in red. † Main configuration. Bold marks the best result in each column.
Figure 6: Macro-9 R@3 (left) and NDCG@10 (right) against reranking time per query. For each test set, 50 sampled queries were submitted as one batch, and the wall-clock time until all were scored was divided by 50 and averaged over the nine test sets.
Backbone
R@1
R@3
R@5
N@10
Qwen3-0.6B
42.34
73.55
81.43
79.67
Qwen3-4B
44.20
76.70
84.01
82.23
Qwen3-8B
44.12
76.68
84.36
82.36
Gemma-4-E2B
44.00
75.44
82.99
81.47
Gemma-4-E4B
44.51
77.14
84.06
82.54
Table 4: Macro-9 results (%) of Re 3 Rank with Qwen3 and Gemma 4 backbones, all trained with Lrank . Bold marks the best result in each column.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Rank
Head
Gold mass
Gold share
R@3
N@10
1
(23,10)
0.1813
0.395
59.42
70.48
2
(21,8)
0.1669
0.332
59.24
69.78
3
(24,29)
0.1526
0.336
59.14
70.01
4
(3,5)
0.1487
0.178
52.32
60.91
5
(20,15)
0.1412
0.330
58.73
68.75
6
(15,11)
0.1386
0.297
59.42
70.14
Appendix
Table 5: Top-12 Qwen3-4B heads on Dseed , sorted by gold mass. R@3 and N@10 are zero-shot single-head Recall@3 and NDCG@10 (%). Bold marks the two highest gold masses, whose heads form HRR .
Figure 7: Gold mass γ(l,h) of all 36×32 Qwen3-4B heads on Dseed . Red boxes mark the selected RRHeads (23,10) and (21,8) .
n
1
2
4
8
16
32
64
R@3
59.42
59.96
60.93
59.58
58.56
58.14
55.58
N@10
70.48
70.71
70.55
70.42
69.35
67.92
63.99
Appendix
Table 6: Zero-shot R@3 and NDCG@10 (%) of Qwen3-4B on Dseed when the masses of the top- n heads by gold mass are summed. n=1 is (23,10) alone, and n=2 gives HRR . Bold marks the best value in each row.
Ranking
#Heads
Shared
Gain
Covered
Rank 1–16 (excl. (23,10) )
15
92.6
9.0
10.9
Rank 17–64
48
91.9
9.2
18.8
Rank 65–256
192
88.0
9.0
38.2
Rank 257–1152
896
74.1
8.1
52.9
Random ranking
–
54.5
6.2
55.2
First-stage
–
77.6
23.6
35.0
Appendix
Table 7: Overlap of top-3 hits between (23,10) and other rankings on Dseed (%). Heads other than (23,10) are grouped by their rank in single-head R@3 among all 1,152 heads, and each group reports the median over its heads; the random ranking averages 20 random orders. Shared is the share of a ranking’s hits that (23,10) also hits, Gain is the share of the misses of (23,10) that the ranking hits, and Covered is the share of the ranking’s misses that (23,10) hits.
Wikipedia multi-hop
Method
HotpotQA
2WikiMultihopQA
MuSiQue
R@1
R@3
R@5
N@10
R@1
R@3
R@5
N@10
R@1
R@3
R@5
N@10
First-stage
46.75
83.08
90.29
89.40
53.42
86.08
89.05
91.38
41.55
64.62
73.83
76.42
QRRanker (4B)
49.05
96.40
98.80
96.62
54.22
98.38
99.58
98.87
44.58
81.44
89.67
88.84
Qwen3-Reranker (0.6B)
48.10
86.49
91.84
91.23
54.20
84.78
87.66
90.38
41.29
66.02
73.69
74.91
Qwen3-Reranker (4B)
49.10
91.09
96.05
94.49
53.85
87.31
90.19
92.28
42.25
69.92
78.64
79.71
Appendix
Table 8: Reranking results on the Wikipedia multi-hop datasets. R@ k is conditional Recall@ k and N@10 is NDCG@10 (%). First-stage is the Qwen3-Embedding-8B ranking before reranking. Re 3 Rank was trained with the mixed ranking loss in Eq. 13 . Bold and underline mark the best and second-best reranker for each metric. The Δ row gives the difference between the best Re 3 Rank result and the best baseline result in each column, with gains in green and losses in red. The significance row reports two-sided paired randomization tests on per-question scores (100,000 sign flips) for the same pair: *** p<0.001 , ** p<0.01 , * p<0.05 , and n.s. not significant.
Open-domain factoid
Method
NQ
PopQA
R@1
R@3
R@5
N@10
R@1
R@3
R@5
N@10
First-stage
25.80
54.57
72.45
81.49
54.65
89.90
91.90
79.66
QRRanker (4B)
23.76
55.18
73.82
79.30
54.50
90.70
96.70
81.84
Qwen3-Reranker (0.6B)
23.14
50.79
69.45
78.29
57.55
89.55
91.55
79.92
Qwen3-Reranker (4B)
26.76
56.79
75.00
83.02
53.60
89.75
93.00
80.05
Appendix
Table 9: Reranking results on the open-domain factoid datasets. Metrics and markers follow Table 8 .
Long narrative
Method
NarrativeQA
DetectiveQA
R@1
R@3
R@5
N@10
R@1
R@3
R@5
N@10
First-stage
16.27
38.59
52.65
52.91
9.38
19.87
27.17
29.65
QRRanker (4B)
20.88
42.30
56.01
55.95
11.33
24.76
33.50
34.35
Qwen3-Reranker (0.6B)
15.85
30.09
39.33
40.59
6.75
17.94
25.59
25.96
Qwen3-Reranker (4B)
19.04
36.81
47.81
48.65
10.15
19.53
29.05
29.85
Appendix
Table 10: Reranking results on the long-narrative datasets. Metrics and markers follow Table 8 .
Long dialogue and overall
Method
LoCoMo
LongMemEval-S
Macro-9
R@1
R@3
R@5
N@10
R@1
R@3
R@5
N@10
R@1
R@3
R@5
N@10
First-stage
33.88
53.66
62.00
55.08
40.90
68.48
79.10
76.47
35.84
62.09
70.94
70.27
QRRanker (4B)
71.73
88.43
92.01
88.56
50.75
82.94
90.31
87.72
42.31
73.39
81.16
79.12
Qwen3-Reranker (0.6B)
61.98
82.97
88.12
81.62
44.26
71.40
78.29
74.98
39.24
64.45
71.72
70.88
Qwen3-Reranker (4B)
63.77
82.65
87.63
82.27
44.59
72.89
80.77
77.56
40.35
67.42
75.35
74.21
Appendix
Table 11: Reranking results on the long-dialogue datasets and the macro average over all nine datasets (Macro-9). Metrics and markers follow Table 8 .
Figure 8: NDCG@10 (%) of rerankers across model sizes on the nine benchmarks and their macro average (Macro-9). Lines and markers follow Figure 3 .
Gold / question
Dataset
Regime
#Questions
Retrieval scope
#Chunks
Tok./chunk
All
In-pool
HotpotQA
Wiki multi-hop
1,000
Shared corpus
9,811
138
2.00
1.96
2WikiMultihopQA
Wiki multi-hop
1,000
Shared corpus
6,119
118
2.47
1.98
MuSiQue
Wiki multi-hop
1,000
Shared corpus
11,656
121
2.65
2.21
NQ
Open-domain
1,000
Shared corpus
9,633
148
3.79
3.79
PopQA
Open-domain
1,000
Shared corpus
8,676
146
2.00
1.17
Appendix
Table 12: Evaluation benchmarks. For each question, Qwen3-Embedding-8B served as the first-stage retriever and returned the top-50 candidates from the retrieval scope; parentheses give the number of scopes. #Chunks counts all chunks in the retrieval scopes before first-stage retrieval, and Tok./chunk is the mean chunk length under the Qwen3 tokenizer. All is the mean number of gold chunks per question (Appendix C.3 ); In-pool is the mean number within the top-50 candidates, computed over questions with at least one such chunk.
Table 13: Type and source of the gold labels. Annotated labels come from the evidence annotations of each dataset or of the HippoRAG 2 release ( Gutiérrez et al., 2025 ) ; derived labels are computed from annotated answers or entities; silver labels are selected by GPT-5.4 following MiA-RAG ( Li et al., 2026b ) . A candidate chunk is relevant if it contains any of the listed evidence.