GeoLEON-CLIP: Graph-Anchored Multimodal Contrastive Learning for Geographic Region Representation

Radoslaw Malek; Szymon Soltysiak; Aleksandra Kawala-Sterniuk; Piotr Szymanski

cs.CVOct 23, 2026

GeoLEON-CLIP: Graph-Anchored Multimodal Contrastive Learning for Geographic Region Representation

Authors: Radoslaw Malek, Szymon Soltysiak, Aleksandra Kawala-Sterniuk, Piotr Szymanski

Organizations: Wrocław University of Science and Technology, Department of Artificial Intelligence, Wrocław, Poland

Abstract

Geographic regions are multimodal: satellite imagery captures visual morphology, text conveys human-interpretable semantics, and map structures describe urban context. Existing geospatial contrastive methods typically align image-text or image-location pairs, underusing structured map data. We introduce GeoLEON-CLIP, a graph-anchored contrastive framework that aligns three views of each H3-indexed region: an OpenStreetMap-derived graph embedding, a satellite image crop, and a textual description. Frozen pretrained LEON graph embeddings provide a semantic anchor between vision and language. Frozen graph, image, and text encoders are connected by trainable residual projection heads and trained with a symmetric tri-modal contrastive objective. Experiments show strong graph-image and graph-text alignment; graph-image retrieval achieves the best cross-modal performance in a mixed-resolution pool. To evaluate the anchoring hypothesis, we propose chained cross-modal recall $\mathrm{R}@A@B$ , which routes retrieval through the graph modality and compares it with direct retrieval under a matched candidate budget. At $\mathrm{R}@1@1$ , image $\rightarrow$ graph $\rightarrow$ text and text $\rightarrow$ graph $\rightarrow$ image outperform direct image-text retrieval by 4.8 and 4.4 points, respectively, despite using a single intermediate cell. Downstream probes show task-dependent gains, with the largest improvements on Airbnb and Beijing housing. These results indicate that OpenStreetMap graph structure can anchor multimodal representations combining maps, imagery, and language.

Figures & tables

Panel of Figure 1 — Figure 1. Overview of the GeoLEON-CLIP architecture. Each H3 anchor is represented by three paired views: an OSM-derived graph view encoded by the pretrained LEON graph masked autoencoder, a Sentinel-2 image view encoded by a frozen DINOv3 ViT-L/16 satellite encoder, and a text view encoded by frozen Jina embeddings. Trainable residual projection heads map the three frozen encoder outputs into a shared geo-semantic space optimized with symmetric pairwise contrastive losses.

Resolution	Avg. area	Anchors	Interpretation
7	5.16 km 2	17,104	district / macro urban structure
8	0.74 km 2	119,791	neighborhood-scale context
9	0.10 km 2	838,537	block-scale POI and morphology

Table 1. H3 resolutions used in GeoLEON-CLIP. Average areas follow the official H3 cell statistics.

Modality	Source	Alignment key	Coverage	Temporal scope and statistics
Graph	OpenStreetMap objects aggregated to H3	H3 index	100% of anchors	OSM snapshot from December 2025; $K=31$ functional categories used as graph node features.
Image	Sentinel-2 Level-2A RGB crops	H3 polygon / centroid	100% of anchors	Primary acquisition window: March 2025–February 2026; scenes closest to December 2025 preferred; approximately 5% of crops use the nearest valid observation outside the primary window.
Text	Textual descriptions associated with OSM-derived objects and external web sources	H3 cell or parent H3 cell	100% of anchors	OSM object set fixed to the December 2025 snapshot; external textual content accessed in March 2026; long documents are chunked and averaged before alignment.

Table 2. Dataset components, provenance, and temporal scope. All retained H3 anchors are fully paired across graph, image, and text modalities.

Direction	R@1	R@5	R@10	Med.	MRR
Graph $\rightarrow$ Text	0.742	0.884	0.930	1	0.806
Text $\rightarrow$ Graph	0.741	0.878	0.928	1	0.804
Graph $\rightarrow$ Image	0.800	0.924	0.958	1	0.859
Image $\rightarrow$ Graph	0.813	0.932	0.969	1	0.869
Image $\rightarrow$ Text	0.617	0.778	0.847	1	0.696
Text $\rightarrow$ Image	0.616	0.783	0.846	1	0.697

Table 3. Cross-modal retrieval for the base GeoLEON-CLIP model. The base run uses residual MLP projection heads and 1024 validation queries.

Direction	MGE@1	MGE@5	MGE@10
Graph $\rightarrow$ Text	1541.8	300.9	115.3
Text $\rightarrow$ Graph	1625.5	250.2	103.0
Graph $\rightarrow$ Image	819.4	185.3	78.5
Image $\rightarrow$ Graph	790.2	74.9	37.7
Image $\rightarrow$ Text	1781.1	314.8	127.7
Text $\rightarrow$ Image	1931.6	541.9	325.9

Table 4. Mean minimum geographic error (MGE) for the base GeoLEON-CLIP model. Values are kilometers; lower is better. MGE@

k

uses the closest H3 centroid among the top-

k

retrieved candidates.

Panel of Figure 2 — Figure 2. Qualitative cross-modal retrieval examples involving the graph modality. Each row shows a query from one modality and the top-1 retrieved result from another modality. Graph regions are represented by H3/OSM graph cards, satellite regions by image crops, and text by short semantic descriptions.

Chain	$A@B$	Chained	Direct	$\Delta$
I $\rightarrow$ G $\rightarrow$ T	1@1	66.5	61.7	+4.8
	1@5	78.6	77.8	+0.8
	3@3	81.5	83.9	$-$ 2.3
	5@5	87.6	94.1	$-$ 6.5
	10@1	74.9	84.7	$-$ 9.8
T $\rightarrow$ G $\rightarrow$ I	1@1	66.0	61.6	+4.4

Table 5. Chained recall and direct endpoint retrieval at matched budget

k=AB

(%). G, I, and T denote graph, image, and text. Alternative bridges are reported at

1@1

;

\Delta

is computed from exact hit counts before percentage rounding.

Variant	G $\rightarrow$ T	T $\rightarrow$ G	G $\rightarrow$ I	I $\rightarrow$ G	I $\rightarrow$ T	T $\rightarrow$ I
Base residual head	0.742	0.741	0.800	0.813	0.617	0.616
Linear head	0.523	0.526	0.295	0.295	0.258	0.249
Small MLP	0.615	0.592	0.400	0.413	0.297	0.304
No modality dropout	0.716	0.710	0.585	0.611	0.366	0.350
Multi-positive	0.703	0.700	0.771	0.794	0.526	0.505
Hard negatives	0.701	0.701	0.768	0.783	0.523	0.515

Table 6. Recall@1 for the base GeoLEON-CLIP checkpoint and ablations evaluated under the same protocol.

Task	Best config.	Base $R^{2}$	Best $R^{2}$	$\Delta$
Airbnb	Post / all	0.216	0.287	+0.071
Beijing housing	Pre / all	0.669	0.690	+0.021
Chicago crime	Pre / graph	0.978	0.978	+0.000
King County	Post / image	0.762	0.764	+0.002
Philadelphia crime	Post / image	0.876	0.883	+0.007

Table 7. Best downstream Ridge probe result per task when embeddings are appended to baseline task features.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 3 — Figure 3. Qualitative examples of cross-modal embedding arithmetic in the aligned GeoLEON-CLIP space. For each H3 cell, all three modalities are shown: satellite image, graph-derived region representation, and text description. Thin frames indicate the two modalities used as operands in the arithmetic expression. The thicker frame indicates the retrieved target modality.

Geospatial Representation Learning

Geospatial representation learning builds reusable embeddings of places, imagery, maps, mobility, and Earth data for spatial prediction and reasoning.

See all 133 papers

Apr 13, 2026

UNIGEOCLIP: Unified Geospatial Contrastive Learning

The growing availability of co-located geospatial data spanning aerial imagery, street-level views, elevation models, text, and geographic coordinates offers a unique opportunity for multimodal representation learning. We introduce UNIGEOCLIP, a massively multimodal contrastive framework to jointly align five complementary geospatial modalities in a single unified embedding space. Unlike prior approaches that fuse modalities or rely on a central pivot representation, our method performs all-to-all contrastive alignment, enabling seamless comparison, retrieval, and reasoning across arbitrary combinations of modalities. We further propose a scaled latitude-longitude encoder that improves spatial representation by capturing multi-scale geographic structure. Extensive experiments across downstream geospatial tasks demonstrate that UNIGEOCLIP consistently outperforms single-modality contrastive models and coordinate-only baselines, highlighting the benefits of holistic multimodal geospatial alignment. A reference implementation is available at https://gastruc.github.io/unigeoclip.

Guillaume Astruc, Eduard Trulls, Jan Hosang +2

LASTIG, Univ Gustave Eiffel, IGN, ENSG, France · CNES, France · LIGM, CNRS, Univ Gustave Eiffel, ENPC, Institut Polytechnique de Paris, Marne-la-Vallée, France +1

Geospatial Representation Learning Multimodal Embedding Cross-Modal Alignment Multimodal Contrastive Learning

Ask PDF

Aug 4, 2026

Geo-Embed: Towards Unified Multimodal Embeddings for Urban Understanding

Geospatial and urban applications increasingly require models to compare heterogeneous evidence across street-view imagery, remote-sensing observations, text descriptions, region proposals, and temporal change cues. However, existing multimodal embedding models and benchmarks are still largely designed and evaluated around general-purpose image-text matching, leaving unclear whether unified embedding space can support heterogeneous geospatial tasks involving spatial relationships, fine-grained semantics, and temporal changes. To address this gap, we make three key contributions. First, we introduce GeoMEB, a large-scale multimodal embedding benchmark that standardizes 45 urban evaluation tasks across retrieval, visual question answering, change detection, classification, and visual grounding, together with training collections comprising 1.32M examples and 286K evaluation queries. Second, we present Geo-Embed, a unified embedding model that adapts a shared vision-language backbone to instruction-conditioned query-target matching over heterogeneous geospatial inputs, including single images, multiple images, text, regions, and masks. On GeoMEB, Geo-Embed achieves the strongest overall performance among representative multimodal embedders, with a 15.3% relative improvement over the strongest baseline. These results motivate future geospatial embedders that organize training and evaluation around explicit query-target relations, including semantic, cross-view, region-level, and temporal correspondence.

Jiapeng Li, Yong Li, Junjie Zhou +2

1Peking University · 2Beijing University of Posts and Telecommunications

Geospatial Representation Learning Cross-Modal Retrieval Vision-Language Grounding Multimodal Model Evaluation Geospatial Reasoning Benchmarks

Ask PDF

Jun 5, 2026

Textual Supervision Enhances Geospatial Representations in Vision-Language Models

Geospatial understanding is a critical yet underexplored dimension in the development of machine learning systems for tasks such as image geolocation and spatial reasoning. In this work, we analyze the geospatial representations acquired by three model families: vision-only architectures (e.g., ViT), vision-language models (e.g., CLIP), and large-scale multimodal foundation models (e.g., LLaVA, Qwen, and Gemma). By evaluating across image clusters, including people, landmarks, and everyday objects, grouped based on the degree of localizability, we reveal systematic gaps in spatial accuracy and show that textual supervision enhances the learning of geospatial representations. Our findings suggest the role of language as an effective complementary modality for encoding spatial context and multimodal learning as a key direction for advancing geospatial AI.

Marcelo Sartori Locatelli, Fernando Tonucci, Jea Kwon +5

Max Planck Institute for Security and Privacy (MPI-SP), Bochum, Germany · Universidade Federal de Minas Gerais, Belo Horizonte, Brazil · Changwon National University, Changwon, South Korea +1

Vision-Language Models Geospatial Representation Learning Multimodal Large Language Models Geospatial Reasoning VLM Reasoning

Ask PDF

Abstract

Geospatial Representation Learning

Explore similar work

UNIGEOCLIP: Unified Geospatial Contrastive Learning

Geo-Embed: Towards Unified Multimodal Embeddings for Urban Understanding

Textual Supervision Enhances Geospatial Representations in Vision-Language Models