We developed Lens, a training-free system for semantic object and scene search across large satellite mosaics that requires no detector trained for the class of interest. Lens represents each tile with a frozen, self-supervised vision transformer (DINOv3), removes the positional bias in its dense features by projecting them onto the subspace orthogonal to a positional basis estimated once from a constant image, and centers the debiased features on the mean feature computed across the whole area before comparison. Lens forms a query by averaging the centered features of a small neighborhood of patches around a chosen point, or of an entire tile for scene-level search, and matches it against the index by cosine similarity. It stores the centered patch features in a byte-quantized similarity index that supports real-time range search over a mosaic of thousands of patches, and a coarser whole-tile embedding index for scene-level retrieval. A guided upsampling network (AnyUp) runs once per tile to recover pixel-accurate boundaries beyond the native feature grid, and the result is cached so later queries reuse it. We evaluated Lens on the 5,128-tile DFC2020 test set and ran it as an interactive tool on five Sentinel-2 areas spanning agricultural, industrial, and desert terrain; the pipeline is not tied to a specific sensor. On DFC2020 a single click reaches 0.09 to 0.80 pixel F1 across eight land-cover classes at the per-class best threshold, with a mean F1 of 0.39, and it beats a pairwise INSID3 baseline on six of the eight classes at roughly four hundred times lower query cost. Indexing 1.31 million patches takes 169 s on one CPU thread, and a click returns a rendered overlay in 6 to 10 ms.
Figures & tables
Figure 1: Architecture of the Lens indexing and query pipeline.
Site
Location
Bounding box (W, S, E, N)
Date window
Scenes
Dominant content
Al Jawf
Saudi Arabia
38.15, 30.40, 38.30, 30.52
2024-06-01 to 2024-09-01
38
Center-pivot irrigation on desert
Bhadla
India
71.88, 27.48, 72.02, 27.58
2024-01-01 to 2024-04-01
21
Photovoltaic arrays, Thar Desert
Imperial Valley
California, USA
−115.62 , 32.95, −115.48 , 33.07
2024-05-01 to 2024-08-01
14
Dense center-pivot agriculture
Rotterdam
Netherlands
4.05, 51.88, 4.22, 51.98
2024-05-01 to 2024-08-01
8
Port berths, tanks, industry
Atacama
Chile
−69.52 , −23.12 , −69.38 , −22.98
2024-01-01 to 2024-04-01
35
Solar arrays, high desert
Table 1: Sentinel-2 areas of interest used for the deployed-tool experiments.
Figure 2: Patch-to-query cosine over the DFC2020 corpus for one protocol click, with and without centering, target patches shaded; the five patches of the clicked prototype are excluded. Median separation rises from 0.10 to 0.19.
Figure 3: Patch-resolution and refined masks on one Al Jawf tile at threshold 0.35. Intersection over union rises from 0.343 to 0.360 on this tile.
512×512 px at 10 m, giving a 32×32 descriptor grid
Positional subspace rank
64
Patch index
Faiss, inner product, 8-bit scalar quantization, range search
Tile index
Faiss, inner product, float, top- k
Refinement
AnyUp to 256×256 , per-pixel normalized, 8-bit cached
Table 2: Configuration used throughout the experiments.
Corpus
Tiles
Patches
Index time (s)
Encoder share
Patch index (MB)
Descriptor cache (MB)
Refinement cache, warm (MB)
DFC2020 test split
5,128
1,312,768
168.8
97.5%
514.6
1,008.2
n/a
Al Jawf
9
9,216
3.42
97.0%
3.63
14.2
226.5
Atacama
12
12,288
3.89
99.4%
4.84
18.9
302.0
Table 3: Indexing time and memory. DFC2020 is the primary evaluation corpus; the five sites are the deployed-tool areas of Table 1 .
Class
Prevalence
Best threshold
Precision
Recall
F1
AP
F1 at 0.30
INSID3 pairwise F1
Forest
0.263
0.04
0.437
0.756
0.535
0.496
0.226
0.475
Shrubland
0.056
0.13
0.131
0.499
0.193
0.122
0.128
0.247
Grassland
0.094
0.16
0.211
0.524
0.286
0.219
0.200
0.211
Wetlands
0.012
0.25
0.066
0.250
0.090
0.047
0.081
0.128
Croplands
0.209
0.09
0.358
0.716
0.465
0.407
0.279
0.316
Urban / built-up
0.110
0.27
0.501
0.680
0.561
0.531
0.511
0.228
Table 4: Retrieval accuracy on the DFC2020 test split for the deployed configuration, with the INSID3 pairwise baseline. Pixel-level, means over 30 simulated clicks per class.
Configuration
Mean F1
Mean AP
Classes won
Neither correction
0.276
0.232
0
Centered only, no debiasing
0.282
0.231
0
Debiased only, no centering
0.349
0.306
2
Debiased and centered
0.385
0.343
6
Table 5: Ablation over the two corrections on DFC2020, at the per-class best threshold. Means over the eight classes; “classes won” counts classes where an arm has the highest F1 of the four.
Figure 4: Change in F1 from area centering against target prevalence, over 27 class and corpus combinations. The benefit crosses zero near 0.13.
Figure 5: Al Jawf overlays at thresholds 0.30, 0.45, and 0.60 from one click on an irrigation circle. Precision holds at 0.45, 0.46, 0.45 while recall falls from 0.55 to 0.27.
Figure 6: Scene-level retrieval using pooled descriptors. Query tile on the left, top five ranked tiles with cosine values; four of five are relevant.
Query type
Corpus
Cache state
Latency
Click to overlay at 0.30
Five sites, 9.2k to 12.3k patches
Warm index
6.7 to 10.2 ms
Range search at 0.30
DFC2020, 1.31M patches
Warm index
436 ms
Scene-level top-10
Five sites
Warm index
0.014 to 0.02 ms
Refined, first query on area
Five sites
Cold
26.5 to 29.6 s
Refined, later queries
Five sites
Warm, in memory
0.20 to 0.21 s
Refined, later queries
Five sites
Warm, from disk
0.72 to 0.87 s
Table 6: Query latency by configuration. Medians on one CPU thread; five-site rows give the range across the areas of Table 1 .
Site
Classes evaluated
F1 range
Strongest class
Al Jawf
4
0.28 to 0.66
Bare / sparse vegetation
Bhadla
4
0.31 to 0.78
Built-up
Imperial Valley
4
0.33 to 0.75
Cropland
Rotterdam
5
0.37 to 0.78
Permanent water
Atacama
2
0.12 to 0.64
Bare / sparse vegetation
Table 7: Agreement with ESA WorldCover 2021 v200 ( Zanaga et al., 2022 ) on the deployed areas, at the per-class best threshold, for classes with sufficient support. Pixel-level F1, means over 30 simulated clicks.
In satellite applications, user queries often take the form of open-ended natural language, extending beyond a fixed set of predefined categories. This open-vocabulary nature poses significant challenges for retrieving relevant image tiles, as the retrieval system must generalize to a wide range of unseen objects and concepts. While vision-language models (VLMs) such as CLIP are widely used for text-image retrieval, even fine-tuned variants often struggle to accurately align such queries with satellite imagery. To address this, we propose Open-SAT, a training-free query embedding refinement algorithm that operates at inference time to improve alignment between user queries and satellite image content. Open-SAT uses VLMs to compute embeddings for image tiles, which are stored in a vector database for efficient retrieval. At query time, it leverages Large Language Models (LLMs) to refine the text embeddings by incorporating contextual information about objects of interest and their surroundings. A threshold-free retrieval mechanism further enhances accuracy and efficiency. Experimental results in three public benchmarks demonstrate that Open-SAT improves the F1 score by up to 16.04%, while retrieving a comparable number of image tiles. These results demonstrate the effectiveness of Open-SAT in open-vocabulary satellite image retrieval, leveraging LLM guidance without the need for additional training or supervision.
Md Adnan Arefeen, Biplob Debnath, Ravi K. Rajendran +2
Semantic search of Earth observation archives remains challenging. Visual foundation models such as CLAY produce rich embeddings of satellite imagery but lack the natural-language grounding needed for intuitive query, and full contrastive training of a remote-sensing CLIP-style model requires paired data and compute that are unavailable at global scale. To allow natural language querying at global scales, we present GeoQuery, a zero-shot retrieval system that sidesteps data and compute constraints through a two-stage semantic and visual search, leveraging a natural language embedding of a subset (proxy) of global data. Rather than training a joint encoder, we generate language descriptions for a 100k proxy subset of global Sentinel-2 tiles and optimise the description-generation prompt so that distances in the resulting text-embedding space correlate with distances in the frozen CLAY visual-embedding space. Queries are resolved in two stages, with a text-similarity search over the proxy subset followed by a visual nearest-neighbour search over worldwide CLAY embeddings On 76 disaster-location queries covering UK floods, US wildfires, and US droughts, GeoQuery achieves 31.6% accuracy within 50,km, with the strongest performance on floods (50% within 50,km) where terrain features are well captured by RGB embeddings. Deployed within a crisis response system called \ECHO{}, GeoQuery identified vulnerable areas during Brisbane's 2025 Cyclone Alfred, with downstream flood simulations reproducing historical patterns. Prompt-aligned proxies offer a practical bridge between EO foundation models and operational retrieval when full contrastive training is out of reach.
James Walsh, William Fawcett, Grace Colverd +1
1Trillium Technologies · University of Cambridge · 3Universidad de Antioquia
Finding specific objects, structures, and spatial patterns supports critical work in agriculture, infrastructure planning, environmental monitoring, and disaster response. These features, however, are often dispersed across very large regions and span dramatically different spatial scales, which defines a search space that is costly to embed, maintain, and search naively. We introduce GeoDIVE, a model that treats a multiresolution satellite image gallery as a forest and uses best-first traversal to determine which tiles to encode. At each visited node, a lightweight model then predicts an optimistic estimate of the best query similarity hidden among its descendants using the query and subtree representations. In evaluations with image and open-vocabulary text queries, GeoDIVE matches exhaustive search at 10% of the encoding cost for image queries and leads all tested baselines at every budget in both modalities. To support future evaluation of multiscale geospatial search, we also release object-level image and text queries for Git-10M, constructed from Microsoft building footprints and OpenStreetMap polygons with VLM-generated captions. Source code and models will be made publicly available.
Hamza Iqbal, Dan Cher, Eric Xing +4
Washington University in St. Louis, USA · Saint Louis University, USA · The George Washington University, USA