Remote sensing (RS) image-text retrieval plays a critical role in understanding massive RS imagery. However, the dense multi-object distribution and complex backgrounds in RS imagery make it difficult to simultaneously achieve fine-grained cross-modal alignment and efficient retrieval. Existing methods either rely on complex cross-modal interactions that lead to low retrieval efficiency, or depend on large-scale vision-language adaptation, which requires massive data and computational resources. To address these issues, we develop a fast-then-fine (FTF) retrieval framework centered on interaction-free multi-granular recall and parameter-free text-guided cross-modal reranking with the Balanced Text-guided Interaction Block (BTIB). Specifically, interaction-free multi-granular recall pre-computes gallery embeddings and matches them with the corresponding query embedding without full-gallery token-level text-image interaction, while text-guided cross-modal reranking with BTIB refines local semantic correspondence within the top-K candidate set. Furthermore, an inter- and intra-modal loss is designed to jointly optimize cross-modal alignment across multi-granular representations. Extensive experiments on public benchmarks demonstrate that the FTF achieves competitive retrieval accuracy while significantly improving retrieval efficiency compared with existing methods.
Figures & tables
Figure 1. Comparison of three paradigms for RS image–text retrieval. (a) Task-specific methods perform fine-grained matching via complex full-gallery token-level text–image interaction, but suffer from high latency. (b) VLM-based methods achieve strong generalization, but often require massive data and computational resources. (c) Our FTF performs interaction-free multi-granular recall followed by text-guided cross-modal reranking with BTIB, achieving a better balance between retrieval accuracy and efficiency. Three retrieval paradigms are compared: full-gallery task-specific interaction, large vision–language model adaptation, and the proposed two-stage FTF framework.
Figure 2. The overall FTF two-stage retrieval framework. Multi-granular visual embeddings v1 are used for interaction-free multi-granular recall, where query texts are matched with pre-computed gallery features without full-gallery token-level text–image interaction, while BTIB generates v2 and v3 in the text-guided cross-modal reranking stage. I 2 M Loss jointly optimizes cross-modal alignment and intra-modal structural consistency. The FTF pipeline contains interaction-free multi-granular recall, top-K candidate selection, BTIB-based text-guided reranking, and inter- and intra-modal training objectives.
Method
Year
I2T
T2I
mR ↑
Lat. (ms) ↓
Thr. (q/s) ↑
R@1
R@5
R@10
R@1
R@5
R@10
LW-MCR
2022
9.43
26.18
37.87
8.52
34.13
53.80
28.32
172.8
5.8
AMFMN
2022
10.31
29.21
37.93
9.85
33.30
52.26
28.81
211.3
4.7
GaLR
2022
14.69
32.20
41.84
10.68
36.90
52.44
31.46
246.8
4.1
KAMCL
2023
14.95
29.47
44.39
9.76
36.41
55.48
31.74
185.6
5.4
PIR
2023
17.54
40.50
52.55
12.71
41.14
63.55
38.00
198.5
5.0
Table 1. Comparison of state-of-the-art methods on RSITMD. For FTF, Full denotes full-gallery BTIB reranking without candidate pruning, while K>0 denotes interaction-free multi-granular recall followed by text-guided cross-modal reranking with BTIB over the top- K candidate set.
Figure 3. Accuracy–efficiency comparison on RSITMD using the methods reported in Table 1 . FTF provides favorable operating points under different candidate sizes K . Scatter plot comparing mean recall and inference latency on RSITMD; FTF variants provide low-latency operating points while retaining competitive mean recall.
Dataset
Method
I2T
T2I
mR ↑
Lat. (ms) ↓
Thr. (q/s) ↑
R@1
R@5
R@10
R@1
R@5
R@10
RSICD
LW-MCR
3.34
13.00
21.95
3.00
15.38
27.96
14.11
386.4
2.6
AMFMN
4.59
15.33
27.99
4.93
17.42
28.33
16.43
472.8
2.1
PIR
10.38
27.03
38.84
6.33
24.20
39.90
24.45
435.6
2.3
RemoteCLIP
16.79
34.94
48.25
13.91
40.79
56.58
35.21
333.5
3.0
FSSN
18.94
39.65
51.71
16.30
39.32
56.92
37.14
548.9
1.8
Table 2. Comparison of retrieval performance and efficiency on RSICD and UCM.
Figure 4. Relative inference speedup over RemoteCLIP on additional datasets. Speedup is computed as the latency of RemoteCLIP divided by the latency of each method on the same dataset. Gray bars denote compared baselines, and blue bars denote FTF variants under different candidate sizes K . Bar chart of relative speedup over RemoteCLIP on RSICD and UCM, showing the largest gains for FTF with small candidate sets.
Figure 5. Latency growth ratio from UCM to RSICD. The growth ratio is computed as the latency on RSICD divided by the latency on UCM for each method. Smaller values indicate better scalability as the gallery size increases. Bar chart of latency growth from UCM to RSICD; two-stage FTF variants grow substantially more slowly than full-gallery methods.
Dataset
Direction
Cov@50 (%)
Cov@100 (%)
Cov@200 (%)
RSITMD
I2T
90.6
94.7
97.6
RSITMD
T2I
89.8
94.1
97.0
RSITMD
Avg.
90.2
94.4
97.3
RSICD
I2T
88.4
92.6
96.1
RSICD
T2I
87.6
92.0
95.5
RSICD
Avg.
88.0
92.3
95.8
Table 3. Stage-1 candidate coverage before text-guided cross-modal reranking.
Variant
I2T R@1
T2I R@1
mR
Coarse only
18.42
16.87
39.26
Fine only
24.83
22.94
45.31
w/o BTIB
27.96
25.21
46.67
w/o intra-modal term
29.08
26.11
48.72
Full FTF
29.83
26.42
50.49
Table 4. Ablation study of FTF on RSITMD with K=100 .
Reranking strategy
Params
I2T R@1
T2I R@1
mR ↑
Lat. (ms) ↓
Thr. (q/s) ↑
No reranking
0
27.96
25.21
46.67
12.2
82.0
Score-level fusion
0
26.87
24.61
48.12
15.8
63.3
Text-to-image softmax
0
28.16
25.84
49.31
18.9
52.9
Lightweight cross-attention
0.52M
29.31
26.18
50.21
33.4
29.9
BTIB
0
29.83
26.42
50.49
22.9
43.7
Table 5. Comparison between BTIB and lightweight reranking alternatives on RSITMD with K=100 . The reported parameters denote the additional parameters introduced by the reranking module.
Figure 6. Accuracy–efficiency balance of FTF under different candidate sizes K . The plot shows latency and throughput versus mean recall under different candidate sizes K . Point color encodes K , and the figure highlights the balance between retrieval accuracy and efficiency. Dual-axis plot showing that mean recall rises with candidate size while latency increases and throughput decreases.
Figure 7. Qualitative comparisons with representative baselines. For each text query, we show the top-5 retrieved results of FSSN, EBAKER, and the proposed method. The numbers denote retrieval ranks, and ground-truth images are highlighted with green boxes. The proposed method retrieves results that are more consistent with both global semantics and fine-grained spatial details. Top-five retrieval examples for airport and bridge queries compare FSSN, EBAKER, and FTF, with ground-truth images marked by green boxes.
Figure 8. Qualitative examples illustrating the effectiveness of the FTF two-stage retrieval framework. In the interaction-free multi-granular recall stage, the ground-truth image is retrieved into the top- K candidate set but may appear at a relatively low rank. After text-guided cross-modal reranking with BTIB, the same image is promoted to a much higher position. Two examples show ground-truth images moving from low ranks after recall to substantially higher ranks after BTIB reranking.
Figure 9. Qualitative examples illustrating the effectiveness of multi-granular learning. Coarse queries capture global scene structures and dominant semantic categories, while fine queries focus on local objects and structural details. Ground-truth retrievals are highlighted with green boxes. Retrieval examples contrast coarse scene-level queries and fine object-level queries, showing their complementary behavior.