Zero-shot Vision-Language Reranking for Cross-View Geolocalization
Authors: Yunus Talha Erzurumlu, John E. Anderson, William J. Shuart, Charles Toth, Alper Yilmaz
Organizations: Dept. of Electrical and Computer Engineering, The Ohio State University, 281 W Lane Ave, Columbus, Ohio, USA · US Army Corps of Engineers Geospatial Research Lab, Corbin Field Station, Woodford, Virginia, USA · Dept. of Civil Engineering, The Ohio State University, 281 W Lane Ave, Columbus, Ohio, USA
Cross-view geolocalization (CVGL) systems, while effective at retrieving a list of relevant candidates (high Recall@k), often fail to identify the single best match (low Top-1 accuracy). This work investigates the use of zero-shot Vision-Language Models (VLMs) as rerankers to address this gap. We propose a two-stage framework: state-of-the-art (SOTA) retrieval followed by VLM reranking. We systematically compare two strategies: (1) Pointwise (scoring candidates individually) and (2) Pairwise (comparing candidates relatively). Experiments on the VIGOR dataset show a clear divergence: all pointwise methods cause a catastrophic drop in performance or no change at all. In contrast, a pairwise comparison strategy using LLaVA improves Top-1 accuracy over the strong retrieval baseline. Our analysis concludes that, these VLMs are poorly calibrated for absolute relevance scoring but are effective at fine-grained relative visual judgment, making pairwise reranking a promising direction for enhancing CVGL precision.
Figures & tables
Figure 1: Example of the cross-view geolocalization challenge. a. Ground-level query images. b. Several aerial image candidates, only one of which is the correct match, demonstrating the difficulty caused by viewpoint and appearance differences.
Figure 2: The proposed two-stage framework. Stage 1 uses a SOTA retrieval model to get top- K candidates . Stage 2 uses a VLM reranker to produce the final ranked list.
Method
R@1 (%)
R@3 (%)
R@5 (%)
Baseline (Retrieval Only)
61.20
73.80
82.40
Pointwise Methods
LLaVA Direct
61.20
73.80
82.40
Qwen Direct
25.00
49.00
65.20
LLaVA Likert
4.80
15.20
26.20
Qwen Likert
14.80
34.80
50.80
Table 1: Performance comparison of VLM reranking strategies on the VIGOR dataset (500 test queries). Baseline performance is from the initial SOTA retrieval model. Results are shown in Recall@k (%). ’-’ indicates metric not applicable or computed. Best result for each metric is in bold .
Figure 3: Analysis of pointwise reranking failures. Plots (a–b) show score distributions for direct score prediction, (c–d) for Likert-scale prediction, (e–f) for Yes/No prediction, and (g–h) for Yes/No prediction with explicit reasoning. In all cases, the distributions for correct and incorrect candidates strongly overlap, indicating that the VLM-assigned scores (whether direct, Likert, or Yes/No) do not provide a clear, separable signal to distinguish the true match from other plausible-but-incorrect candidates.
Figure 4: Main Recall Performance Comparison. This plot shows Recall@1, Recall@3, and Recall@5 for the baseline retrieval model and all VLM reranking strategies, sorted by R@1 performance. LLaVA Pairwise (64.80% R@1) is the only method to achieve a significant improvement over the Baseline (61.20% R@1). In contrast, all other pointwise methods (Direct, Likert, Yes/No) cause a catastrophic drop in performance, or no change at all.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Some qualitative results comparing the base model’s ranking and the VLM’s final selection. In each example, the satellite images are displayed in the order originally produced by the base model, not in the reranked order. The blue outlined image indicates the VLM’s top-1 selection among these candidates.
Cross-view object geo-localization (CVOGL) locates a target in satellite imagery using drone or street-view queries. Existing methods train separate detectors for each viewpoint, leading to parameter redundancy and impeding cross-view knowledge sharing. Moreover, top-ranked satellite candidates are often visually similar, so visual appearance and categorical labels alone are insufficient to resolve such ambiguity. To address these, we propose MVLGeo, an efficient framework designed to unify multiple viewpoints and reduce model redundancy. First, we introduce environmental contextual text from the query view as cues to distinguish visually similar candidates via Vision-Language Reranking (VL-Rerank). Second, we design a multi-view Mixture-of-Experts architecture (MV-MoE) with a shared encoder and view-specific experts to reduce redundancy and promote knowledge sharing, while cross-view contrastive learning aligns their representations for consistency. Third, we introduce an adaptive elliptical prior (ESAM-Prior) as auxiliary positional encoding for anisotropic geometric perception. Extensive experiments on the CVOGL benchmarks confirm that MVLGeo, as a unified model for multiple query viewpoints, achieves state-of-the-art performance, demonstrating robustness to input degradation and generalization across viewpoints. Code and models will be available on GitHub to facilitate future work.
Xuyu Fan, Qi Ming, Zhu Han +6
College of Computer Science, Beijing University of Technology · Zhengzhou University · Zhejiang University +3
Cross-view geo-localization (CVGL) estimates a camera's location by matching a street-view image to geo-referenced overhead imagery, enabling GPS-denied localization and navigation. Existing methods almost universally formulate CVGL as an image-retrieval problem in a contrastively trained embedding space. This ties performance to large batches and hard negative mining, and it ignores both the geometric structure of maps and the coverage mismatch between street-view and overhead imagery. In particular, salient landmarks visible from the street view can fall outside a fixed satellite crop, making retrieval targets ambiguous and limiting explicit spatial inference over the map. We propose Just Zoom In, an alternative formulation that performs CVGL via autoregressive zooming over a city-scale overhead map. Starting from a coarse satellite view, the model takes a short sequence of zoom-in decisions to select a terminal satellite cell at a target resolution, without contrastive losses or hard negative mining. We further introduce a realistic benchmark with crowd-sourced street views and high-resolution satellite imagery that reflects real capture conditions. On this benchmark, Just Zoom In achieves state-of-the-art performance, improving Recall@1 within 50 m by 5.5% and Recall@1 within 100 m by 9.6% over the strongest contrastive-retrieval baseline. These results demonstrate the effectiveness of sequential coarse-to-fine spatial reasoning for cross-view geo-localization.
Yunus Talha Erzurumlu, Jiyong Kwag, Alper Yilmaz
Photogrammetric Computer Vision Lab, The Ohio State University
Image geolocalization has traditionally been addressed through retrieval-based place recognition or geometry-based visual localization pipelines. Recent advances in Vision-Language Models (VLMs) have demonstrated strong zero-shot reasoning capabilities across multimodal tasks, yet their performance in geographic inference remains underexplored. In this work, we present a systematic evaluation of multiple state-of-the-art VLMs for country-level image geolocalization using ground-view imagery only. Instead of relying on image matching, GPS metadata, or task-specific training, we evaluate prompt-based country prediction in a zero-shot setting. The selected models are tested on three geographically diverse datasets to assess their robustness and generalization ability. Our results reveal substantial variation across models, highlighting the potential of semantic reasoning for coarse geolocalization and the limitations of current VLMs in capturing fine-grained geographic cues. This study provides the first focused comparison of modern VLMs for country-level geolocalization and establishes a foundation for future research at the intersection of multimodal reasoning and geographic understanding.