Vision-language models (VLMs) are powerful listwise rerankers for multimodal retrieval, but high inference costs restrict them to evaluating small local candidate views. Existing multi-call strategies rely on fixed schedules, wasting expensive VLM calls on uninformative candidate pairs and easy queries. To address this, we propose Adaptive Multi-view Budgeted Elo Reranking (AMBER), an online, budgeted multi-view reranking framework that dynamically optimizes global resource allocation. AMBER treats fragmented listwise VLM outputs as local tournaments, using continuous Elo updates to maintain a lightweight global ranking state. Building on this, it allocates computation at two levels: dynamically constructing candidate views with high score ambiguity, and scheduling queries to maximize expected information gain. We show that each Elo update corresponds to a stochastic gradient ascent step on the Bradley-Terry log-likelihood, and provide a submodular information-theoretic motivation for the query-level allocation strategy. Experiments on CIRR, CIRCO, and PhotoBench demonstrate that AMBER achieves the strongest overall performance among the compared multi-call VLM reranking methods under comparable VLM-call budgets, while remaining effective in lower-budget settings. Our code is publicly available at https://github.com/wnlfc/AMBER.
Figures & tables
Figure 1: The overall architecture of AMBER.
Dataset
# Images
# Queries
Targets / Query
Language
CIRR
36,000+
36,554
Single
English
CIRCO
123,403
1,020
Multiple (Avg. 4.53)
English
PhotoBench
3,582
1,188
Intent-driven
EN / CN
Table 1: Summary of dataset statistics, scales, and evaluation characteristics.
CIRR
Method
Calls
R@1
R@2
R@5
R@10
R@50
Avg. Rank
QQMM-Embed-v2
–
38.43
59.80
80.36
89.23
97.59
3.40
RzenEmbed-v2-7B
–
19.30
44.96
70.36
83.08
96.77
8.00
Qwen3-VL-Embedding-8B
–
48.80
63.81
78.39
86.55
96.94
4.00
AcuRank
∼ 24
50.63
63.78
77.90
85.06
96.94
5.00
Sliding Window
24
51.11
63.86
76.05
86.55
96.94
4.00
TourRank
24
51.37
64.63
77.13
85.52
96.94
4.00
Table 2: Main reranking results across datasets. R@ K denotes Recall@ K , and N@ K denotes NDCG@ K . QQMM-Embed-v2, RzenEmbed-v2-7B, and Qwen3-VL-Embedding-8B are embedding-only retrieval baselines; all VLM reranking methods are evaluated on the same top-50 candidate pools retrieved by Qwen3-VL-Embedding-8B. For CIRCO, we report R@5 and mAP metrics, while for PhotoBench, Avg. N@10 averages Chinese and English NDCG@10 scores. All PhotoBench values are multiplied by 100. Avg. Rank is computed over the displayed metrics within each dataset; lower is better. Best and second-best results are shown in bold and underlined, respectively. Full results are reported in Appendix A.1 .
CIRR
Method
Calls
R@1
R@2
R@5
R@10
R@50
Avg. Rank
Qwen3-VL-Embedding-8B
–
48.80
63.81
78.39
86.55
96.94
3.20
AcuRank
10.53
49.95
62.89
75.23
80.60
96.94
4.40
Sliding Window
8
49.71
62.94
77.18
88.63
96.94
3.40
TourRank
8
51.57
63.69
74.36
81.74
96.94
3.80
UniRank
8
53.23
66.68
79.49
86.55
96.94
2.00
AMBER
8
54.31
67.35
79.98
88.72
96.94
1.00
Table 3: Low-budget reranking results on CIRR, CIRCO, and PhotoBench. Fixed-budget methods use 8 VLM calls per query, while AcuRank reports its average calls under native stopping. Best and second-best results are shown in bold and underlined, respectively.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Calls
R@5
R@10
R@25
R@50
mAP@5
mAP@10
mAP@25
mAP@50
QQMM-Embed-v2
–
71.12
79.75
90.25
95.26
45.92
47.13
50.39
51.45
RZENEmbed-v2-7B
–
60.75
72.38
85.75
92.25
32.20
34.19
37.41
38.61
Qwen3-VL-Embedding-8B
–
63.38
76.50
87.88
93.25
44.60
45.96
48.96
50.08
AcuRank
23.28
73.75
83.38
87.38
93.25
56.28
56.88
58.49
59.52
Sliding Window
24
71.50
84.50
91.38
93.25
54.52
55.64
58.33
58.86
TourRank
24
74.00
82.38
91.12
93.25
56.45
56.74
59.30
59.95
Appendix
Table 4: Full results on CIRCO. Best results are shown in bold and second-best results are underlined.
Lang.
Method
Calls
R@1
R@5
R@10
R@20
NDCG@1
NDCG@10
NDCG@20
CN
QQMM-Embed-v2
–
23.56
44.53
51.63
58.51
46.69
49.65
51.62
CN
RZENEmbed-v2-7B
–
24.02
44.30
51.59
58.05
48.54
50.07
51.78
CN
Qwen3-VL-Embedding-8B
–
22.04
41.05
47.26
52.65
44.52
45.54
46.94
CN
AcuRank
24.3
25.95
46.48
52.25
54.51
51.97
52.27
52.18
CN
Sliding Window
24
25.85
46.14
52.76
56.58
51.38
52.35
52.89
CN
TourRank
24
24.89
46.13
52.65
56.67
51.30
51.98
52.60
Appendix
Table 5: Full PhotoBench results in the merged-album setting under our evaluation protocol. Queries with empty ground-truth sets are retained and assigned zero metric values instead of being skipped. All metric values are multiplied by 100. Best results are shown in bold and second-best results are underlined.
Aggregation Method
mAP@5
mAP@10
mAP@25
mAP@50
Borda Count
54.12
54.84
57.54
58.11
Average Rank
55.87
56.12
58.88
59.51
Batch BT-MLE
56.78
57.04
59.63
60.19
Elo
58.80
58.73
61.17
61.78
Appendix
Table 6: Comparison of cross-view aggregation methods on CIRCO. All methods use the same cached candidate views and VLM outputs, with only the aggregation rule changed. Best and second-best results are shown in bold and underlined, respectively.
Rating Range
R@5
R@10
R@25
R@50
mAP@5
mAP@10
mAP@25
mAP@50
100
73.00
82.25
89.38
93.25
56.36
56.39
58.52
59.42
200
73.25
82.38
89.38
93.25
56.87
56.84
58.88
59.76
400
73.12
82.25
89.50
93.25
56.85
56.79
58.87
59.75
800
73.12
82.25
89.38
93.25
56.69
56.74
58.79
59.68
Appendix
Table 7: Sensitivity analysis of the Elo initialization range on CIRCO under a fixed budget of 8 VLM calls per query. The rating range controls the strength of the embedding-prior initialization: the top-ranked candidate from the first-stage retriever receives the highest initial Elo score, while the last candidate receives the base score.
Calls / Query
R@5
R@10
R@25
R@50
mAP@5
mAP@10
mAP@25
mAP@50
4
71.88
81.00
88.00
93.25
55.65
55.42
57.33
58.43
8
73.12
81.88
89.50
93.25
56.80
56.76
58.85
59.70
12
73.38
82.62
90.25
93.25
57.50
57.54
59.72
60.50
16
73.75
83.00
90.62
93.25
58.02
58.05
60.38
61.07
24
74.75
83.50
90.75
93.25
58.80
58.73
61.17
61.78
Appendix
Table 8: Performance of AMBER on CIRCO under different VLM-call budgets. Best and second-best results are shown in bold and underlined, respectively. Ties shared by all budget settings are left unmarked.
Method
Calls / Query
Images / Call
Input Tokens / Call
Output Tokens / Call
Latency / Call (s)
Cost / Call (CNY)
Projected Charge (CNY)
AMBER
8
11.00
3327.6
39.2
1.899
0.001742
11.15
UniRank
8
11.00
3361.6
38.0
1.763
0.001757
11.24
TourRank
8
11.63
3419.1
41.4
1.957
0.001829
11.70
Sliding Window
8
11.00
3301.6
38.0
1.825
0.001727
11.05
AcuRank
10.52
10.12
3067.5
35.2
1.700
0.001604
13.50
Appendix
Table 9: Practical inference costs measured on the same CIRCO subset using a shared API backend. The projected charge estimates the cost of evaluating the full 800-query CIRCO test split.
Method
Calls
R@5
R@10
R@25
R@50
mAP@5
mAP@10
mAP@25
mAP@50
Sliding Window
8
75.38
84.75
90.50
93.25
59.75
59.32
61.61
62.29
AcuRank
10.60
75.00
81.75
88.50
93.25
59.27
58.22
60.01
60.97
UniRank
8
73.50
76.50
87.88
93.25
57.94
55.29
58.21
59.33
TourRank
8
72.25
81.50
91.62
93.25
58.26
58.28
61.03
61.56
AMBER
8
75.75
84.12
89.88
93.25
60.63
60.01
61.83
62.68
Appendix
Table 10: Results on CIRCO using Qwen3.5-Flash as the VLM reranker. Qwen3-VL-Embedding-8B remains the first-stage retriever, and all methods rerank the same top-50 candidate pools. Best and second-best results are shown in bold and underlined, respectively. Ties shared by all methods are left unmarked.