Spatially-Weighted CLIP for Street-View Geo-localization

Large collections of street-view imagery provide rich visual information about urban environments, but extracting fine-grained geographic information from such data remains challenging. In particular, fine-grained regional geolocalization is challenging because nearby areas often share coarse geographic cues. We study regional geolocalization within a metropolitan area and ask whether pretrained CLIP features are sufficient for regional discrimination, and what visual information supports performance after adaptation. Using 9,085 street-view images from eight Greater Los Angeles regions, we compare zero-shot CLIP, frozen-encoder readouts, partial encoder updating, Low-Rank Adaptation (LoRA), and full fine-tuning. Frozen readouts remain near the 39.03% zero-shot accuracy, whereas encoder adaptation achieves 75.94-82.10%. Full fine-tuning also reduces the mean distance to the predicted region center from 12.30 km to 3.86 km. We probe these gains through semantic cue removal, appearance reduction using edge maps and blur, and scene-configuration disruption using patch scrambling. Adapted models achieve higher edge and blur accuracy and switch 42.92-45.56% of predictions after scrambling, compared with 10.79-14.60% for frozen methods. However, adaptation does not improve the fraction of performance retained after appearance reduction, while vegetation and sky remain influential. A Caltech101 control further shows that scrambling sensitivity is not unique to geolocalization. Overall, encoder adaptation substantially improves nearby-region discrimination and is associated with greater sensitivity to intact scene configuration, without evidence that coarse structure alone becomes sufficient for prediction. These conclusions concern viewpoint variation near known locations rather than geographically disjoint generalization.

Worldwide image geo-localization aims to determine where on Earth a single image was captured. However, visually similar scenes may lie thousands of kilometers apart, so methods that localize primarily by appearance often mistake a distant look-alike for the true location. We attribute this failure to a structural cause: in existing methods, GPS coordinates serve only as training supervision, and the distance relationships among locations never enter the learned representation. To address this, we propose GeoMetric, a retrieval-based framework that encodes GPS coordinates relationally rather than in isolation, injecting the distance structure among locations into both representation learning and inference. GeoMetric comprises three components: (1) a Transformer-based GPS encoder with distance-aware location attention that modulates inter-sample aggregation by great-circle proximity; (2) a trimodal contrastive objective that aligns images, geo-textual descriptions, and GPS embeddings in a unified space; and (3) a retrieval-augmented inference stage that supplies large multimodal models (LMMs) with contrastive candidate context for grounded coordinate reasoning. Extensive experiments on IM2GPS, IM2GPS3k, YFCC4k, and YFCC26k demonstrate that GeoMetric consistently outperforms state-of-the-art methods, improving street-level accuracy (within 1 km) by 1.5%, 0.9%, 6.9%, and 2.5%, respectively. Controlled ablations confirm that the gains originate from the proposed geographic encoding rather than any specific LMM.

Cross-view geo-localization (CVGL) estimates a camera's location by matching a street-view image to geo-referenced overhead imagery, enabling GPS-denied localization and navigation. Existing methods almost universally formulate CVGL as an image-retrieval problem in a contrastively trained embedding space. This ties performance to large batches and hard negative mining, and it ignores both the geometric structure of maps and the coverage mismatch between street-view and overhead imagery. In particular, salient landmarks visible from the street view can fall outside a fixed satellite crop, making retrieval targets ambiguous and limiting explicit spatial inference over the map. We propose Just Zoom In, an alternative formulation that performs CVGL via autoregressive zooming over a city-scale overhead map. Starting from a coarse satellite view, the model takes a short sequence of zoom-in decisions to select a terminal satellite cell at a target resolution, without contrastive losses or hard negative mining. We further introduce a realistic benchmark with crowd-sourced street views and high-resolution satellite imagery that reflects real capture conditions. On this benchmark, Just Zoom In achieves state-of-the-art performance, improving Recall@1 within 50 m by 5.5% and Recall@1 within 100 m by 9.6% over the strongest contrastive-retrieval baseline. These results demonstrate the effectiveness of sequential coarse-to-fine spatial reasoning for cross-view geo-localization.