Finding specific objects, structures, and spatial patterns supports critical work in agriculture, infrastructure planning, environmental monitoring, and disaster response. These features, however, are often dispersed across very large regions and span dramatically different spatial scales, which defines a search space that is costly to embed, maintain, and search naively. We introduce GeoDIVE, a model that treats a multiresolution satellite image gallery as a forest and uses best-first traversal to determine which tiles to encode. At each visited node, a lightweight model then predicts an optimistic estimate of the best query similarity hidden among its descendants using the query and subtree representations. In evaluations with image and open-vocabulary text queries, GeoDIVE matches exhaustive search at 10% of the encoding cost for image queries and leads all tested baselines at every budget in both modalities. To support future evaluation of multiscale geospatial search, we also release object-level image and text queries for Git-10M, constructed from Microsoft building footprints and OpenStreetMap polygons with VLM-generated captions. Source code and models will be made publicly available.
Figures & tables
Figure 1: Encode everything, or descend. A real descent for the query baseball field over one gallery site, zoom level 15 (z15) to z18: the imagery (top) and the descent’s similarity map of the same levels (bottom) under a 2% encode budget. Colored cells were encoded and scored. Grey cells were never passed through the encoder. The red outline marks the answer tile and arrows follow the descent. Exhaustive cosine (right) must encode every leaf of the site to score the same target.
Figure 2: Bound architecture. Image and text queries are evaluated separately: crop and tile use DINOv3, while caption and tile use GeoRSCLIP’s respective towers. One path computes the cosine similarity sim(q,t) ; the other feeds both embeddings and subtree size to an MLP that outputs a residual r . Their sum is the traversal priority c^(q,t) of Eq. 2 , fit directly to the subtree-maximum target: the tile’s own similarity plus how far above it the best hidden descendant can plausibly sit (right panel).
Figure 3: Encode–quality frontiers. Image-crop queries at R@10 (left) and text-caption queries at R@50 (right) against encode budget. Every curve ranks its own visited set by a standard inner product. Stars mark the adaptive stop of § 4.3 .
Figure 4: Qualitative traversals. Left: two full descents. Darkened tiles were pruned without being encoded, red boxes mark the followed branch, and yellow boxes mark candidate matches down to the target. Right: three further queries, shown as retrieved site, descended branch, and query.
Object image → tile
Object text → tile
Estimate of c∗
R@5
R@10
MRR
R@50
R@100
MRR
Cosine
62.5
67.4
52.8
32.9
41.7
8.0
Variance
75.2
81.0
63.4
32.3
40.9
8.2
Node zoom
78.3
84.3
66.0
33.1
41.7
8.3
Scalar bound
77.6
83.6
65.4
33.1
41.7
8.4
+ ZoomEye sched. [ 32 ]
78.4
84.5
66.1
33.6
42.7
8.6
Table 1: Estimator ablation. Each row replaces the estimate of the subtree maximum that orders expansion. The search, the folds, the encoders, and the cosine ranking of visited tiles are fixed at 10% encode. Rows prefixed + each add one signal to the scalar bound and do not stack. The oracle substitutes the exact subtree maximum for the estimate.
Method
Encode
R@1
R@5
R@10
GeoDIVE
10%
59.8
83.4
89.9
+ MaxSim
83.3
93.8
95.3
GeoDIVE + Adaptive Stop
6.1%
59.0
81.9
88.0
+ MaxSim
81.6
91.6
93.0
Exhaustive cosine
100%
62.0
86.5
93.2
+ MaxSim
86.4
97.3
98.9
Table 2: Late-interaction reranking results. Each + MaxSim row rescores the cosine top 200 of the visited set above it with a ColBERT-style MaxSim and adds no additional encoder inference. Adaptive encode is the mean over queries. The exhaustive + MaxSim row requires patch tokens for the entire gallery.
System
< 1%
1–2%
2–5%
≥ 5%
all
Unique queries
2,582
1,311
783
324
5,000
VAS [ 31 ]
5.8
5.8
6.7
6.2
6.0
Cosine
57.0
52.9
55.2
65.1
56.2
Behavior Cloning [ 14 ]
50.6
47.4
47.1
61.5
49.9
OTM [ 49 ]
70.0
68.1
68.6
79.1
69.9
Scalar bound
70.7
68.9
69.4
79.6
70.6
Table 3: R@10 at 5% encode by object size (fraction of tile area). Baseline adaptations are detailed in the supplement.
Figure 5: Results on reranking. R@1 (left) and R@10 (right) by encode budget for GeoDIVE with and without reranking. Star: the adaptive stop of § 4.3 .
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: The gallery is a forest of spatial quadtrees. 20,608 tiles in 256 disjoint trees; a coarse tile’s four children partition its footprint at the next zoom.
Figure 7: Caption queries. Left: the target tile with the instance box marked. Centre: the padded crop shown to the captioner. Right: the OSM tags supplied as context and the resulting caption, which is the query. Tags never appear in the query. The prompt asks for details visible in the crop, such as the rubble pile and blue container (top) and the white pitch lines (bottom).
Cosine
Scalar bound
Size-aware
Oracle
Enc. %
Reach
R@50
Reach
R@50
Reach
R@50
Reach
R@50
2
11.63
11.48
10.28
10.19
12.81
12.76
28.17
27.78
5
33.44
24.81
31.14
24.34
38.15
28.91
62.03
44.49
10
54.89
32.93
52.90
33.08
62.34
37.98
82.53
45.13
15
67.62
36.71
67.23
37.41
74.91
40.77
90.20
45.13
25
81.67
40.67
84.22
42.18
88.54
43.51
96.55
45.13
Appendix
Table 4: Text-query budget grid. Reach and R@50 by encode budget.
R@50 at 10% encode
Text tower
Exhaustive
Cosine
Scalar
Size-aware
Δ
CLIP ViT-L/14
11.92
9.64
9.64
9.71
+0.07
OpenCLIP ViT-L/14
15.24
12.72
12.54
14.01
+1.29
RemoteCLIP ViT-L/14
14.35
12.56
11.78
12.48
-0.08
GeoRSCLIP ViT-L/14
15.92
13.22
13.95
13.96
+0.74
SigLIP2 so400m
15.17
12.52
13.64
13.78
+1.26
Appendix
Table 5: Text-tower swap. R@50 on the 74-site text setting with the tile tower and protocol fixed: the exhaustive ceiling and the descent priorities at 10% encode.
Encoder
Cosine
Scalar
Size-aware
Oracle
Gain
Closed
DINOv3-SAT + our LoRA
67.4
83.6
89.9
93.2
+6.3
66%
DINOv3-SAT, frozen
30.0
31.9
33.9
36.7
+1.97
42%
GeoRSCLIP ViT-L/14 [ 44 ]
7.3
8.3
8.6
9.5
+0.36
30%
RemoteCLIP ViT-L/14 [ 24 ]
4.6
5.6
6.2
8.8
+0.62
19%
DINOv2 ViT-L/14
11.9
19.2
21.6
25.0
+2.43
42%
CLIP ViT-B/32
5.7
6.4
6.6
7.6
+0.23
20%
Appendix
Table 6: The size input across encoders. R@10 at 10% encode with only the encoder changed; Closed is the fraction of the scalar-to-oracle gap that the size input recovers.
Enc.
Area (px 2 )
N
Δ Reach
Δ R@1
Δ R@10
5%
[300, 600)
6,990
+9.2 [5.2, 13.0]
+3.9 [2.1, 5.8]
+7.7 [4.3, 11.2]
[600, 1200)
4,266
+17.3 [11.0, 22.6]
+10.2 [7.0, 13.4]
+15.8 [10.2, 20.8]
[1200, 2400)
2,256
+20.7 [12.9, 27.3]
+15.2 [9.6, 19.8]
+19.7 [12.4, 26.0]
[2400, inf)
1,488
+11.7 [5.0, 17.4]
+10.7 [4.5, 16.1]
+11.7 [5.0, 17.4]
10%
[300, 600)
6,990
+4.7 [2.3, 7.5]
+2.3 [0.9, 4.1]
+4.2 [2.0, 7.0]
[600, 1200)
4,266
+7.9 [3.8, 12.2]
+4.2 [2.0, 6.7]
+6.8 [3.1, 10.9]
Appendix
Table 7: Size-aware minus scalar bound by object area and by target depth. Paired deltas with site-blocked 95% CIs.
System
Encode
Time (s)
TFLOPs
R@1
R@5
R@10
MRR
Uniform subsample
10.0%
4.5
259
8.6
9.7
9.8
.091
Single coarse level
5.0%
2.2
129
2.9
3.2
3.2
.031
TDM tile classifier [ 47 ]
10.0%
4.5
259
40.5
56.5
60.7
.476
Raw cosine descent
10.0%
4.5
259
44.8
62.5
67.4
.528
Cascade ( K=100 )
21.3%
9.6
551
53.2
72.5
77.0
.617
GeoDIVE (5% budget)
5.0%
2.2
130
55.63
77.51
83.37
.654
Appendix
Table 8: Extended object-to-tile quality at budget. Encoder work priced at batch-64 A100 throughput (2.179 ms per forward). The cascade row is priced in forward-equivalents; adaptive rows report achieved mean encode.
Figure 8: The subtree maximum rises with subtree size. Descendant similarities standardized per subtree; the expected maximum of n draws sits κ(n)σ above the mean, with κ measured per size bucket (Table 9 ). Densities are illustrative.
n bucket
Geom. n
Empirical κ
Blom κ
Mean gap
q99 gap
2–8
5
1.240
1.180
0.042
0.170
8–32
21
1.945
1.890
0.083
0.292
32–128
85
2.533
2.441
0.124
0.424
128–512
341
3.015
2.906
0.195
0.528
Appendix
Table 9: Order-statistic calibration on 750,000 cached query–tile pairs. κ is the standardized gap to the subtree maximum; the last two columns are the mean and 99th percentile of c∗−sim .
Encode %
1.5
2
3
5
10
25
Reach
2
4
10
12
32
56
R@10, native confidence ranking
0
0
2
2
2
2
R@10, cosine ranking of the same tiles
2
4
10
12
32
54
Appendix
Table 10: ZoomEye (LLaVA-OV-7B), 50-query cohort. R@10 under ZoomEye’s native ranking and under plain-cosine ranking of the same visited tiles.
System
Enc.
Reach
R@1
R@10
Δ R@10
s/query
GeoDIVE
5%
89.08
55.63
83.37
ref.
—
OTM
5%
74.33
46.66
69.90
−13.47 [-18.07,-8.50]
0.0013
VAS
5%
6.22
4.24
5.97
−77.40 [-81.70,-72.29]
0.0028
Behavior cloning
5%
53.17
33.45
49.91
−33.46 [-40.03,-26.62]
0.223
Pointwise control
5%
53.24
33.13
49.83
−33.53 [-40.58,-25.96]
0.016
GeoDIVE
10%
96.36
59.76
89.90
ref.
—
Appendix
Table 11: Learned policies versus GeoDIVE. Δ is policy minus size-aware R@10 with a paired site-block interval; s/query is decision-only time on one H100.
Mechanism
Enc. %
Reach
R@10
Best-first R@10
Per-level beam, W=16
2.31
14.24
14.13
11.57
Per-level beam, W=64
5.44
38.53
37.37
39.39
Per-level beam, W=128
9.43
56.60
54.25
59.19
Per-level beam, W=512
27.22
86.29
81.28
85.15
Greedy single-path
1.31
7.56
7.55
8.19
Attention sampling
10.00
49.05
46.11
83.26
Appendix
Table 12: Other search strategies. Each rule against best-first descent at the same achieved encode: on the same TDM scores for the beam rows, size-aware for the rest. Level-synchronous rows fix the width at 512.
Figure 9: Three measured descents. Per case: the site quadtree (blue: encoded; orange: target, hollow if not reached); the root tile per zoom with the tiles the search expanded bright, tiles encoded but not expanded dimmed, and never-encoded tiles dark; the path from root to target; and the query area cut from each path tile. Queries 2702 (5%) and 1785 (10%) rank first; 3798 is not reached at 10%.
2% encode
5% encode
10% encode
Depth
Leaves
Cosine
Size lift
Cosine
Size lift
Cosine
Size lift
3
64
73.0
62.6
100
100
100
100
4
256
94.8
98.0
98.0
100
99.3
100
5
1,024
85.0
78.8
92.0
99.9
95.2
100
6
4,096
70.4
63.6
81.9
99.6
88.1
100
7
16,384
54.2
99.9
68.4
100
77.4
100
Appendix
Table 13: Synthetic depth scaling. Reach (%) of the target leaf by forest depth and budget for raw-cosine and size-lift descent. The oracle reaches 100 in every cell except 74.1 at depth 3 and 2%.
Query
Count
Encoder
Image crop
5,000
DINOv3 ViT-L/16 + LoRA
Caption
4,000
GeoRSCLIP ViT-L/14 + LoRA
Appendix
Table 14: Query modalities and reported evaluation populations. Each query uses its modality’s encoder; image and text embeddings are not fused into a single query.
Family
Objective and fitting settings
TDM-style [ 47 ]
A shared 256-hidden-unit MLP on query and tile embeddings predicts a binary logit. Strict ancestors of the target are positives; four negatives per positive are sampled at the same level, with accidentally positive draws relabelled. Frozen tile embeddings replace free per-tile parameters so the scorer can operate on held-out sites.
OTM-style [ 49 ]
A 400-update pinball warm start is followed by three rounds of data collection and 150-update warm-started refits. Learning rate 10−3 is unchanged across rounds. Collection uses the updating policy; rows are weighted by trajectory counts, with no replay-support mixture. The adaptation retains subtree-value regression in place of the original recommendation objective.
VAS-style [ 31 ]
The flat-action adaptation uses target-hit reward, a training horizon of 64 actions, 30 training epochs, learning rate 3×10−4 , 32-dimensional keys, and a 64-unit hidden representation. It uses masked sequential actions and a REINFORCE objective with a random-action control variate; the gallery is the multiscale tile set rather than a fixed image grid.
Behavior cloning and pointwise control
Both fit oracle-action demonstrations with AdamW, learning rate 3×10−4 , weight decay 10−4 , and logical batches of 256 states. Training runs for at most 30 epochs, with patience five and selection by validation oracle-action negative log likelihood. Policy rollouts use the training-site subforest; the pointwise control uses the same demonstrations and fitting schedule.
Appendix
Table 15: Training details for the image-query comparison families. The OTM row in the main paper’s Table 3 reports the warm-start model; subsequent refits are evaluated separately in the on-policy study.
Worldwide image geolocalization, which aims to predict the GPS coordinates of any image on Earth, remains challenging due to global visual diversity. Recent generative approaches based on Retrieval-Augmented Generation (RAG) and Large Multimodal Models (LMMs) leverage candidates retrieved from fixed databases for reasoning, but often struggle with scenes that are absent from the reference set. In this work, we propose GeoSearch, an open-world geolocation framework that integrates web-scale reverse image search into the RAG pipeline. GeoSearch augments LMM prompts with database-retrieved coordinates and textual evidence extracted from web pages. To mitigate noise from irrelevant content, we introduce a two-layer filtering mechanism consisting of image matching, followed by confidence-based gating. Experiments on standard benchmarks Im2GPS3k and YFCC4k demonstrate the superiority of GeoSearch under leakage-aware evaluation. Our code and data are publicly available to support reproducibility.
Tung-Duong Le-Duc, Hoang-Quoc Nguyen-Son, Minh-Son Dao
University of Science, VNU-HCM Ho Chi Minh City, Vietnam · National Institute of Information and Communications Technology Tokyo, Japan
Semantic search of Earth observation archives remains challenging. Visual foundation models such as CLAY produce rich embeddings of satellite imagery but lack the natural-language grounding needed for intuitive query, and full contrastive training of a remote-sensing CLIP-style model requires paired data and compute that are unavailable at global scale. To allow natural language querying at global scales, we present GeoQuery, a zero-shot retrieval system that sidesteps data and compute constraints through a two-stage semantic and visual search, leveraging a natural language embedding of a subset (proxy) of global data. Rather than training a joint encoder, we generate language descriptions for a 100k proxy subset of global Sentinel-2 tiles and optimise the description-generation prompt so that distances in the resulting text-embedding space correlate with distances in the frozen CLAY visual-embedding space. Queries are resolved in two stages, with a text-similarity search over the proxy subset followed by a visual nearest-neighbour search over worldwide CLAY embeddings On 76 disaster-location queries covering UK floods, US wildfires, and US droughts, GeoQuery achieves 31.6% accuracy within 50,km, with the strongest performance on floods (50% within 50,km) where terrain features are well captured by RGB embeddings. Deployed within a crisis response system called \ECHO{}, GeoQuery identified vulnerable areas during Brisbane's 2025 Cyclone Alfred, with downstream flood simulations reproducing historical patterns. Prompt-aligned proxies offer a practical bridge between EO foundation models and operational retrieval when full contrastive training is out of reach.
James Walsh, William Fawcett, Grace Colverd +1
1Trillium Technologies · University of Cambridge · 3Universidad de Antioquia
Cross-view geo-localization (CVGL) retrieves geo-tagged satellite imagery for a ground-view query. Most systems exhaustively search a flat, fixed-resolution gallery, incurring high cost over large areas and adapting poorly to satellite resolution changes. Autoregressive coarse-to-fine alternatives reduce comparisons but bind later predictions to earlier decisions and a predefined hierarchy. We introduce GeoMoE, a sparse mixture-of-experts dual encoder that decouples global multi-scale representation learning from local hierarchical search. Global multi-scale supervision and content-adaptive routing map ground and satellite images across resolutions into a globally comparable embedding space. At inference, each image is encoded once, and probabilistic beam search follows parent--child links to score a small candidate subset. Later levels reuse these descriptors rather than features generated by preceding levels, limiting feature-level error propagation and hierarchy coupling. We further introduce VIGOR-M, a four-city benchmark with an explicit parent--child satellite hierarchy and held-out half-step galleries for single-resolution, cross-resolution, and hierarchical evaluation. GeoMoE achieves 95.78% R@40m on Just Zoom In, 2.77 percentage points above the previous best, and 62.39% R@1 on VIGOR-M. The latter requires 0.885 MMAC/query for descriptor matching, 5.27% of an exhaustive L3 scan, while exceeding the strongest exhaustive baseline by 3.12 percentage points in R@1. One model trained on L1, L2, and L3 also outperforms a matched dense control across all six galleries and transfers to three withheld resolutions. By decoupling globally trained embeddings from local hierarchical search, GeoMoE jointly improves localization accuracy, search efficiency, and cross-resolution transfer.
Ruijie Fan, Junyan Ye, Qi Zhu +1
Tsinghua Shenzhen International Graduate School, Tsinghua University · School of Geospatial Engineering and Science, Sun Yat-sen University