Global Cross-Modal Geo-Localization: A Million-Scale Dataset and a Physical Consistency Learning Framework

Yutong Hu; Jinhui Chen; Chaoqiang Xu; Yuan Kou; Sili Zhou; Shaocheng Yan; Pengcheng Shi; Qingwu Hu

cs.CVMar 9, 2026

Global Cross-Modal Geo-Localization: A Million-Scale Dataset and a Physical Consistency Learning Framework

Authors: Yutong Hu, Jinhui Chen, Chaoqiang Xu, Yuan Kou, Sili Zhou, Shaocheng Yan, Pengcheng Shi, Qingwu Hu,

Organizations: School of Remote Sensing and Information Engineering, Wuhan University, Wuhan, 430079, China · First Surveying and Mapping Institute of Hunan Province, Changsha, 421001, China

Abstract

Cross-modal Geo-localization (CMGL) matches ground-level text descriptions with geo-tagged aerial imagery, which is crucial for pedestrian navigation and emergency response. However, existing studies are constrained by narrow geographic coverage and simplistic scene diversity, failing to reflect the immense spatial heterogeneity of global architectural styles and topographic features. To bridge this gap and facilitate universal positioning, we introduce CORE, the first million-scale dataset dedicated to global CMGL. CORE comprises 1,034,786 cross-view images sampled from 225 distinct geographic regions across six continents, offering an unprecedented variety of perspectives in varying environmental conditions and urban layouts. We leverage the zero-shot reasoning of Large Vision-Language Models (LVLMs) to synthesize high-quality scene descriptions rich in discriminative cues. Furthermore, we propose a physical-law-aware network (PLANET) for cross-modal geo-localization. PLANET introduces a novel contrastive learning paradigm to guide textual representations in capturing the intrinsic physical signatures of satellite imagery. Extensive experiments across varied geographic regions demonstrate that PLANET significantly outperforms state-of-the-art methods, establishing a new benchmark for robust, global-scale geo-localization. The dataset and source code will be released at https://github.com/YtH0823/CORE.

Figures & tables

Figure 1: Illustration of cross-modal geo-localization. This task retrieves satellite imagery based on text queries of the ground environment for position localization. The physical attribute information presented in the text description can yield valuable clues regarding the location.

Dataset	Image Type	Query Image	Reference Image	Total Image	Text Annotation	Distribution	Environment
CVGL
CVUSA ( Workman et al., 2015 )	Street-view	44,416	44,416	88,832	$\times$	USA	Suburban
CVACT ( Liu and Li, 2019 )	Street-view	128,334	128,334	256,668	$\times$	Canberra	Urban
VIGOR ( Zhu et al., 2021 )	Street-view	90,618	105,214	195,832	$\times$	USA	Urban
University-1652 ( Zheng et al., 2020 )	Drone and Street-view	50,218	55,227	105,445	$\times$	Campus	Urban
CVGlobal ( Ye et al., 2024 )	Street-view	134,233	134,233	268,466	$\times$	Global (7 regions)	Urban

Table 1: Comparison of the CORE dataset with other geo-localization datasets. The comparison includes the number of reference and query images, text annotations, distribution range, and sampling environment.

Figure 2: We evaluate representative methods on the CORE dataset and the existing CVG-Text dataset, with PLANET achieving state-of-the-art performance in 8 settings, demonstrating strong localization capabilities. All results are reported as R@1, and the radial axis is linearly scaled from 20 to 70.

Figure 3: Geographical distribution map of sampled images and corresponding text annotations from the CORE dataset. CORE exhibits unprecedented global scale and spatial heterogeneity. This extensive diversity ensures a comprehensive representation of varying urban and suburban environments worldwide.

Figure 4: Annotation pipeline of the CORE dataset. Manual templates guide Qwen3-VL-Plus generation, and every generated description is manually reviewed and corrected.

Figure 5: Word cloud analysis of text descriptions from the CORE dataset.

Figure 6: Overview of text feature statistics in the CORE dataset. (a) Token length distribution histogram; (b) Text statistics comparison between the CORE dataset and existing CMGL datasets; (c) Text similarity matrix.

Figure 7: Pipeline of the proposed PLANET. (a) During training, image statistics are computed directly from the input satellite images and aligned with the projected text descriptors. (b) During inference, global query and gallery features are matched by similarity without the auxiliary statistical branches.

Method	Global-level		Intercontinental-level
	All		Subset1		Subset2		Subset3		Subset4
	R@1	L@150	R@1	L@150	R@1	L@150	R@1	L@150	R@1	L@150
ViLT ( Kim et al., 2021 )	23.05	26.23	24.43	26.18	22.08	25.34	25.58	30.79	19.65	25.07
X-VLM ( Zeng et al., 2022 )	26.97	30.81	28.90	30.65	25.72	28.74	28.67	33.86	24.31	29.62
X2-VLM ( Zeng et al., 2023 )	34.35	38.33	35.79	38.50	33.73	36.56	36.12	41.25	31.43	36.61
SigLIP-B/16 ( Zhai et al., 2023 )	30.10	34.08	29.13	31.87	30.87	33.92	31.23	36.18	28.98	34.20

Table 2: Quantitative results of PLANET and existing methods on the CORE dataset. We report the retrieval recall (

R@1

) and localization accuracy (

L@150

) across the World-level and four intercontinental subsets. All methods are fine-tuned on the CORE training split under a unified training protocol. Bold indicates the best result, and underline indicates the second best result.

Figure 8: The retrieval results of PLANET on the CORE dataset. The left side displays the original street view image along with its corresponding textual description. The right side illustrates the top four retrieval results, where green highlights indicate correct matches and red highlights signify incorrect results.

Figure 9: Failure cases of PLANET on the CORE dataset. For each query, the right side shows the top four retrieval results in row-major order from Rank-1 to Rank-4. Green boxes denote the ground-truth satellite images, while red boxes denote incorrect retrieved candidates.

Figure 10: Top-3 retrieval results and satellite heatmaps for PLANET and the baseline, together with PLANET’s token-level attention visualization of the query text. Green and red boxes denote correct and incorrect retrievals, respectively, and tokens with stronger PLANET attention responses correspond to discriminative physical attributes in the scene. The introduction of physical consistency constraints enables PLANET to more accurately anchor its attention on crucial geographical regions.

Figure 11: UMAP visualization of the feature distribution of the CORE dataset. Compared to the baseline (left), PLANET (right) significantly enhances the discriminative power of features by introducing physical property constraints. Samples from different subsets (distinguished by color) exhibit higher intra-class cohesion and clearer inter-class boundaries in space, while achieving tighter cross-modal alignment between images (circles) and text (stars).

Method	Subset 1		Subset 2		Subset 3		Subset 4
Method	R@1	L@150	R@1	L@150	R@1	L@150	R@1	L@150
Baseline (w/o Physical)	53.12	55.88	59.36	62.37	46.97	51.27	48.71	54.61
+ Color ( $f_{color}$ )	55.81	58.23	62.58	65.10	47.93	52.20	49.83	55.07
+ Structure ( $f_{struc}$ )	55.78	58.16	62.32	64.98	48.03	52.24	49.52	55.12
+ Texture ( $f_{tex}$ )	55.43	58.10	62.46	65.08	48.21	52.53	49.95	55.48
+ Color + Structure ( $f_{color}+f_{struc}$ )	56.64	59.43	63.98	66.77	48.86	53.21	50.50	56.03

Table 3: Ablation study on the physical consistency mechanism. We investigate the impact of explicit physical constraints by comparing our PLANET variants against the baseline.

Variant	Weights			$\sum$	R@1 (%) on CORE Subsets
Variant	$\lambda_{c}$	$\lambda_{s}$	$\lambda_{t}$	$\sum$	Sub 1	Sub 2	Sub 3	Sub 4
(a) Ablation on Overall Weight (Fixed 1:1:1 Ratio)
Under-regularized	0.1	0.1	0.1	0.3	54.12 $\pm$ 0.07	60.83 $\pm$ 0.08	47.58 $\pm$ 0.12	49.97 $\pm$ 0.04
Under-regularized	0.2	0.2	0.2	0.6	56.43 $\pm$ 0.06	63.15 $\pm$ 0.13	48.76 $\pm$ 0.08	50.66 $\pm$ 0.07
Over-regularized	0.5	0.5	0.5	1.5	57.07 $\pm$ 0.13	63.88 $\pm$ 0.11	49.26 $\pm$ 0.10	50.92 $\pm$ 0.12
Over-regularized	1.0	1.0	1.0	3.0	54.82 $\pm$ 0.15	60.75 $\pm$ 0.17	48.04 $\pm$ 0.15	50.23 $\pm$ 0.14

Table 4: Ablation study on the weighting coefficients of the physical consistency constraints across four subsets of the CORE dataset. We report the R@1 (%) metric.

Bins	Subset 1		Subset 2		Subset 3		Subset 4
Bins	R@1	L@150	R@1	L@150	R@1	L@150	R@1	L@150
8	57.08	59.76	63.88	66.57	48.95	53.37	50.39	55.88
16	57.43	60.27	64.32	67.13	49.28	53.64	50.95	56.13
32	57.90	60.72	64.74	67.58	49.81	54.13	51.35	56.65
64	57.52	60.33	64.38	67.09	49.21	53.58	51.02	56.27
128	57.17	59.91	64.03	66.75	48.88	53.26	50.62	55.95

Table 5: Ablation study on the histogram bin number of physical descriptors. We report R@1 (%) and L@150 (%) across four subsets of the CORE dataset.

Method	Backbone	Params (M)	FLOPs (G)	Memory (GB)	Retrieval Time (ms)	R@1	L@150
Baseline	CLIP-ViT-L/14@336px	428.12	401.03	30.45	49.97	53.12	55.88
PLANET	CLIP-ViT-B/32	155.06	32.58	6.43	31.28	41.03	44.22
	CLIP-ViT-B/16	153.40	57.57	8.40	31.93	44.49	47.22
	CLIP-ViT-L/14@336px	431.79	403.16	30.46	50.08	57.90	60.72

Table 6: Efficiency and accuracy comparison on CORE Subset 1. The baseline uses the original CLIP-ViT-L/14@336px backbone, while PLANET is evaluated with different CLIP backbones. Retrieval time denotes the average retrieval time for a single query.

Setting	R@1	L@150	R@1 Drop	L@150 Drop
Full text	57.90	60.72	–	–
Entity only	10.93	12.52	46.97	48.20
w/o Entity	36.20	38.86	21.70	21.86
w/o Attribute	34.66	36.98	23.24	23.74
w/o Color	49.30	52.05	8.60	8.67
w/o Structure	45.24	47.74	12.66	12.98

Table 7: Text cue ablation study on CORE Subset 1. We remove one cue group from the query during testing, while the Entity only setting retains only entity words. The trained model and retrieval gallery remain unchanged.

Method	Subset1 $\rightarrow$ 4		Subset2 $\rightarrow$ 4		Subset3 $\rightarrow$ 4
Method	R@1	L@150	R@1	L@150	R@1	L@150
TAG-CLIP	12.22	14.13	11.83	13.79	14.79	17.06
BLIP	15.24	17.93	14.17	16.58	16.84	19.35
CLIP-L/14	15.58	18.25	14.34	16.62	17.22	19.83
Long-CLIP	21.68	25.07	19.75	22.86	22.96	26.10
CrossText2Loc	23.13	26.97	21.06	24.64	24.75	28.12

Table 8: Quantitative comparison of out-of-distribution generalization on CORE dataset.

Method	New York		Brisbane		Tokyo
Method	R@1	L@50	R@1	L@50	R@1	L@50
ViLT ( Kim et al., 2021 )	11.58	15.58	11.10	14.50	10.83	15.50
X-VLM ( Zeng et al., 2022 )	15.74	16.86	15.67	17.60	12.46	14.34
X2-VLM ( Zeng et al., 2023 )	23.33	25.17	21.58	23.67	17.83	20.75
SigLIP-B/16 ( Zhai et al., 2023 )	19.67	21.08	19.58	22.00	15.58	17.92
SigLIP-SO400M ( Zhai et al., 2023 )	33.50	34.83	34.25	36.83	28.42	31.50

Table 9: Quantitative comparison of different methods on the CVG-Text dataset.

Training Set	Test Set	R@1	L@50	L@150
CORE	CVG-Text New York	23.17	25.25	29.42
	CVG-Text Brisbane	26.33	29.83	30.50
	CVG-Text Tokyo	19.92	22.75	23.58
CVG-Text	CORE Subset1	8.03	8.85	10.36
	CORE Subset2	9.14	9.98	12.57
	CORE Subset3	7.75	8.52	9.85

Table 10: Cross-dataset generalization of PLANET between CORE and CVG-Text. The model is trained on the source dataset and directly evaluated on the target dataset without additional fine-tuning.

Cross-View Geo-Localization

Cross-view geo-localization matches ground, drone, or aerial images to maps or satellite views to estimate where they were captured.

See all 94 papers

What Words Keep of a Place: Zero-Shot Language Reasoning for Cross-View Geo-Localization
Oct 5, 2026Ayesh Abu Lehyeh, Jay Hwasung Jung, Safwan WshahCross-View Geo-Localization Large Vision-Language Models
Department of Computer Science, University of Vermont
DualGeo: A Dual-View Framework for Worldwide Image Geo-localization
Apr 28, 2026Junchao Cui, Wenqi Shi, Shaoyong Du +4Cross-View Geo-Localization Geospatial Representation Learning
Henan Key Laboratory of Cyberspace Situation Awareness, Zhengzhou, China · Information Engineering University, Zhengzhou, China
GeoBridge++: Fact-Guided Geo-Semantic Bridging for Unified Cross-View Geo-Localization
Dec 2, 2025Zixuan Song, Jing Zhang, Di Wang +6Cross-View Geo-Localization Geospatial Representation Learning
College of Computer Science and Technology and Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun 130012, China · Zhongguancun Academy, Beijing 100094, China · School of Computer Science, Wuhan University, Wuhan, China +1

Oct 5, 2026J/K move · Enter open · S save

What Words Keep of a Place: Zero-Shot Language Reasoning for Cross-View Geo-Localization

Ayesh Abu Lehyeh, Jay Hwasung Jung, Safwan Wshah

Department of Computer Science, University of Vermont

Cross-View Geo-Localization Large Vision-Language Models Geospatial Reasoning

Open paper AskPDF arXiv

Cross-view geo-localization is commonly solved as an image retrieval problem, matching a ground-level image against a database of satellite tiles through a jointly trained embedding. Such models are accurate, but they need large paired supervision and cannot show what evidence supports a match. In this paper, we study a different question: how much of this task can be solved through language alone? We prompt a multimodal large language model (MLLM) to describe each ground panorama and each satellite tile as structured text, and localize by comparing these descriptions. No component is trained. We evaluate on 9,826 VIGOR pairs from four U.S. cities, in three settings. First, the descriptions are faithful but not discriminative. They agree closely across the two views, yet ranking the full pool by description similarity almost never returns the correct tile (0.39% Recall@1). Second, we narrow the pool to ten neighboring tiles, as a coarse prior would do. The same descriptions now become useful: an MLLM judge that scores structural consistency doubles random ranking and matches a strong lexical baseline. It also states which fields of the two descriptions agree and which conflict, which an embedding distance cannot do, and which we see as a step toward interpretable localization. Third, we place the judge on a trained visual retriever. On the queries it ranks wrongly, reranking from images works, while reranking from our descriptions does not (23.5% against 10.7% Recall@1). Scene structure survives the conversion into language, while the fine appearance detail needed to separate nearby places does not. Code and prompts are publicly available at https://github.com/AyeshAbuLehyeh/GeoLingual.

Abstract

Cross-View Geo-Localization

Explore similar work

What Words Keep of a Place: Zero-Shot Language Reasoning for Cross-View Geo-Localization

DualGeo: A Dual-View Framework for Worldwide Image Geo-localization

GeoBridge++: Fact-Guided Geo-Semantic Bridging for Unified Cross-View Geo-Localization