Generative listwise reranking leverages global context for superior retrieval but is plagued by intrinsic position bias, where models exhibit structural sensitivity to input order independent of relevance. Existing mitigations present a dilemma: inference-time aggregation incurs prohibitive latency, while training-based methods often fail to eradicate ingrained priors, particularly in compact models. To resolve this dilemma, we propose CapCal (Content-Agnostic Probability Calibration), a training-free framework that mechanically decouples positional bias from ranking decisions. By estimating the bias distribution via content-free placeholders, CapCal rectifies output logits through an entropy-adaptive contrastive mechanism. Evaluations across 10 benchmarks confirm that CapCal achieves superior performance among training-free methods while preserving single-pass efficiency. Notably, it unlocks the latent potential of lightweight models (e.g., 0.6B), delivering absolute NDCG gains exceeding 10 points and outperforming both permutation-based aggregation and data-augmentation baselines.
Figures & tables
Figure 1: Overview of the proposed CapCal framework. The framework decouples position bias by utilizing an empty-passage query to capture the input-agnostic prior. We then apply contrastive decoding to subtract this prior from the standard inference logits, achieving a calibrated ranking.
MSv1
MSv2
Avg.
BEIR
Avg.
Model
Method
DL19
DL20
DL21
DL22
DL23
InD
COVID
Clim.
NF
Argu.
FiQA
OOD
Qwen3-0.6b
Base
0.4916
0.3415
0.6278
0.4858
0.5457
0.4985
0.6074
0.1622
0.4243
0.1097
0.1807
0.2969
CapCal
0.5454
0.4126
0.6666
0.5457
0.5762
0.5493
0.6400
0.2086
0.4504
0.2448
0.2364
0.3560
Qwen3-8b
Base
0.6807
0.5405
0.8091
0.6129
0.6945
0.6675
0.7525
0.2486
0.4981
0.2436
0.2877
0.4061
CapCal
0.7141
0.5613
0.8228
0.6391
0.6979
0.6870
0.7683
0.2714
0.5094
0.2464
0.2932
0.4177
Qwen2.5-7b
Base
0.7114
0.5151
0.8142
0.6293
0.6317
0.6603
0.7542
0.2325
0.4858
0.2181
0.2846
0.3950
Table 1: Performance comparison on MS MARCO and BEIR benchmarks. All metrics are reported in NDCG@10.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Method
DL19
DL20
DL21
DL22
DL23
BM25
Base
49.16
34.15
62.78
48.58
54.57
CapCal
54.54
41.26
66.66
54.57
57.62
Random
Base
47.61
32.27
59.07
47.26
43.60
CapCal
56.33
37.57
65.93
53.16
52.63
BGE-M3
Base
52.32
45.23
72.06
59.07
59.68
CapCal
62.05
47.21
75.32
65.24
64.23
Appendix
Table 2: Detailed results of retrieval and ordering robustness on Qwen3-0.6B.
Placeholder Content
DL19
DL20
DL21
DL22
DL23
Base (No Calibration)
49.16
34.15
62.78
48.58
54.57
Fixed String
54.54
41.26
66.66
54.57
57.62
Passage[1] Content
56.12
40.78
64.05
50.31
52.98
Empty (a space)
55.82
44.77
63.50
50.06
50.48
Space ×20
57.97
40.43
64.12
52.96
53.28
Random ×20
54.15
38.92
64.58
49.03
51.43
Appendix
Table 3: Impact of different placeholder content and length on calibration performance using Qwen3-0.6B.
Method
DL19
DL20
DL21
DL22
DL23
Base
49.87
33.50
54.05
45.60
40.96
CapCal
52.16
36.01
67.70
53.35
46.80
Appendix
Table 4: Robustness results using alphabetical identifiers (A, B, C…) on Qwen3-0.6B.
Model
Method
DL19
DL20
DL21
DL22
DL23
COVID
Clim.
NF
Argu.
FiQA
Qwen3-0.6b
Base
0.4916
0.3415
0.6278
0.4858
0.5457
0.6074
0.1622
0.4243
0.1097
0.1807
PSC
0.5382
0.4074
0.6937
0.5246
0.5264
0.5842
0.1892
0.4047
0.1910
0.1637
CapCal
0.5454
0.4126
0.6666
0.5457
0.5762
0.6400
0.2086
0.4504
0.2448
0.2364
Qwen3-8b
Base
0.6807
0.5405
0.8091
0.6129
0.6945
0.7525
0.2486
0.4981
0.2436
0.2877
PSC
0.7404
0.5429
0.8473
0.6244
0.6948
0.7753
0.2627
0.4980
0.2527
0.2856
CapCal
0.7141
0.5613
0.8228
0.6391
0.6979
0.7683
0.2714
0.5094
0.2464
0.2932
Appendix
Table 5: Comparison with Permutation Self-Consistency (PSC, k=10 ). All metrics are reported in NDCG@10.
Model
Method
DL19
DL20
DL21
DL22
DL23
Zephyr
Base
48.50
36.91
58.20
51.44
46.72
CapCal
51.85
36.61
62.83
53.81
47.90
RankZephyr
Base
76.67
60.73
89.20
75.11
76.78
CapCal
76.25
60.96
89.15
74.58
75.55
Qwen3-0.6B
Base
49.16
34.15
62.78
48.58
54.57
CapCal
54.54
41.26
66.66
54.57
57.62
Appendix
Table 6: Comparative analysis between standard models and those fine-tuned with permutation-based data augmentation.