GeoLEON-CLIP: Graph-Anchored Multimodal Contrastive Learning for Geographic Region Representation

Radoslaw Malek; Szymon Soltysiak; Aleksandra Kawala-Sterniuk; Piotr Szymanski

cs.CVOct 23, 2026

GeoLEON-CLIP: Graph-Anchored Multimodal Contrastive Learning for Geographic Region Representation

Authors: Radoslaw Malek, Szymon Soltysiak, Aleksandra Kawala-Sterniuk, Piotr Szymanski

Organizations: Wrocław University of Science and Technology, Department of Artificial Intelligence, Wrocław, Poland

Abstract

Geographic regions are multimodal: satellite imagery captures visual morphology, text conveys human-interpretable semantics, and map structures describe urban context. Existing geospatial contrastive methods typically align image-text or image-location pairs, underusing structured map data. We introduce GeoLEON-CLIP, a graph-anchored contrastive framework that aligns three views of each H3-indexed region: an OpenStreetMap-derived graph embedding, a satellite image crop, and a textual description. Frozen pretrained LEON graph embeddings provide a semantic anchor between vision and language. Frozen graph, image, and text encoders are connected by trainable residual projection heads and trained with a symmetric tri-modal contrastive objective. Experiments show strong graph-image and graph-text alignment; graph-image retrieval achieves the best cross-modal performance in a mixed-resolution pool. To evaluate the anchoring hypothesis, we propose chained cross-modal recall $\mathrm{R}@A@B$ , which routes retrieval through the graph modality and compares it with direct retrieval under a matched candidate budget. At $\mathrm{R}@1@1$ , image $\rightarrow$ graph $\rightarrow$ text and text $\rightarrow$ graph $\rightarrow$ image outperform direct image-text retrieval by 4.8 and 4.4 points, respectively, despite using a single intermediate cell. Downstream probes show task-dependent gains, with the largest improvements on Airbnb and Beijing housing. These results indicate that OpenStreetMap graph structure can anchor multimodal representations combining maps, imagery, and language.

Figures & tables

Panel of Figure 1 — Figure 1. Overview of the GeoLEON-CLIP architecture. Each H3 anchor is represented by three paired views: an OSM-derived graph view encoded by the pretrained LEON graph masked autoencoder, a Sentinel-2 image view encoded by a frozen DINOv3 ViT-L/16 satellite encoder, and a text view encoded by frozen Jina embeddings. Trainable residual projection heads map the three frozen encoder outputs into a shared geo-semantic space optimized with symmetric pairwise contrastive losses.

Resolution	Avg. area	Anchors	Interpretation
7	5.16 km 2	17,104	district / macro urban structure
8	0.74 km 2	119,791	neighborhood-scale context
9	0.10 km 2	838,537	block-scale POI and morphology

Table 1. H3 resolutions used in GeoLEON-CLIP. Average areas follow the official H3 cell statistics.

Modality	Source	Alignment key	Coverage	Temporal scope and statistics
Graph	OpenStreetMap objects aggregated to H3	H3 index	100% of anchors	OSM snapshot from December 2025; $K=31$ functional categories used as graph node features.
Image	Sentinel-2 Level-2A RGB crops	H3 polygon / centroid	100% of anchors	Primary acquisition window: March 2025–February 2026; scenes closest to December 2025 preferred; approximately 5% of crops use the nearest valid observation outside the primary window.
Text	Textual descriptions associated with OSM-derived objects and external web sources	H3 cell or parent H3 cell	100% of anchors	OSM object set fixed to the December 2025 snapshot; external textual content accessed in March 2026; long documents are chunked and averaged before alignment.

Table 2. Dataset components, provenance, and temporal scope. All retained H3 anchors are fully paired across graph, image, and text modalities.

Direction	R@1	R@5	R@10	Med.	MRR
Graph $\rightarrow$ Text	0.742	0.884	0.930	1	0.806
Text $\rightarrow$ Graph	0.741	0.878	0.928	1	0.804
Graph $\rightarrow$ Image	0.800	0.924	0.958	1	0.859
Image $\rightarrow$ Graph	0.813	0.932	0.969	1	0.869
Image $\rightarrow$ Text	0.617	0.778	0.847	1	0.696
Text $\rightarrow$ Image	0.616	0.783	0.846	1	0.697

Table 3. Cross-modal retrieval for the base GeoLEON-CLIP model. The base run uses residual MLP projection heads and 1024 validation queries.

Direction	MGE@1	MGE@5	MGE@10
Graph $\rightarrow$ Text	1541.8	300.9	115.3
Text $\rightarrow$ Graph	1625.5	250.2	103.0
Graph $\rightarrow$ Image	819.4	185.3	78.5
Image $\rightarrow$ Graph	790.2	74.9	37.7
Image $\rightarrow$ Text	1781.1	314.8	127.7
Text $\rightarrow$ Image	1931.6	541.9	325.9

Table 4. Mean minimum geographic error (MGE) for the base GeoLEON-CLIP model. Values are kilometers; lower is better. MGE@

k

uses the closest H3 centroid among the top-

k

retrieved candidates.

Panel of Figure 2 — Figure 2. Qualitative cross-modal retrieval examples involving the graph modality. Each row shows a query from one modality and the top-1 retrieved result from another modality. Graph regions are represented by H3/OSM graph cards, satellite regions by image crops, and text by short semantic descriptions.

Chain	$A@B$	Chained	Direct	$\Delta$
I $\rightarrow$ G $\rightarrow$ T	1@1	66.5	61.7	+4.8
	1@5	78.6	77.8	+0.8
	3@3	81.5	83.9	$-$ 2.3
	5@5	87.6	94.1	$-$ 6.5
	10@1	74.9	84.7	$-$ 9.8
T $\rightarrow$ G $\rightarrow$ I	1@1	66.0	61.6	+4.4

Table 5. Chained recall and direct endpoint retrieval at matched budget

k=AB

(%). G, I, and T denote graph, image, and text. Alternative bridges are reported at

1@1

;

\Delta

is computed from exact hit counts before percentage rounding.

Variant	G $\rightarrow$ T	T $\rightarrow$ G	G $\rightarrow$ I	I $\rightarrow$ G	I $\rightarrow$ T	T $\rightarrow$ I
Base residual head	0.742	0.741	0.800	0.813	0.617	0.616
Linear head	0.523	0.526	0.295	0.295	0.258	0.249
Small MLP	0.615	0.592	0.400	0.413	0.297	0.304
No modality dropout	0.716	0.710	0.585	0.611	0.366	0.350
Multi-positive	0.703	0.700	0.771	0.794	0.526	0.505
Hard negatives	0.701	0.701	0.768	0.783	0.523	0.515

Table 6. Recall@1 for the base GeoLEON-CLIP checkpoint and ablations evaluated under the same protocol.

Task	Best config.	Base $R^{2}$	Best $R^{2}$	$\Delta$
Airbnb	Post / all	0.216	0.287	+0.071
Beijing housing	Pre / all	0.669	0.690	+0.021
Chicago crime	Pre / graph	0.978	0.978	+0.000
King County	Post / image	0.762	0.764	+0.002
Philadelphia crime	Post / image	0.876	0.883	+0.007

Table 7. Best downstream Ridge probe result per task when embeddings are appended to baseline task features.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 3 — Figure 3. Qualitative examples of cross-modal embedding arithmetic in the aligned GeoLEON-CLIP space. For each H3 cell, all three modalities are shown: satellite image, graph-derived region representation, and text description. Thin frames indicate the two modalities used as operands in the arithmetic expression. The thicker frame indicates the retrieved target modality.

Geospatial Representation Learning

Geospatial representation learning builds reusable embeddings of places, imagery, maps, mobility, and Earth data for spatial prediction and reasoning.

See all 133 papers

UNIGEOCLIP: Unified Geospatial Contrastive Learning
Apr 13, 2026Guillaume Astruc, Eduard Trulls, Jan Hosang +2Geospatial Representation Learning Multimodal Embedding
LASTIG, Univ Gustave Eiffel, IGN, ENSG, France · CNES, France · LIGM, CNRS, Univ Gustave Eiffel, ENPC, Institut Polytechnique de Paris, Marne-la-Vallée, France +1
Geo-Embed: Towards Unified Multimodal Embeddings for Urban Understanding
Aug 4, 2026Jiapeng Li, Yong Li, Junjie Zhou +2Geospatial Representation Learning Cross-Modal Retrieval
1Peking University · 2Beijing University of Posts and Telecommunications
Textual Supervision Enhances Geospatial Representations in Vision-Language Models
Jun 5, 2026Marcelo Sartori Locatelli, Fernando Tonucci, Jea Kwon +5Vision-Language Models Geospatial Representation Learning
Max Planck Institute for Security and Privacy (MPI-SP), Bochum, Germany · Universidade Federal de Minas Gerais, Belo Horizonte, Brazil · Changwon National University, Changwon, South Korea +1

Apr 13, 2026J/K move · Enter open · S save

UNIGEOCLIP: Unified Geospatial Contrastive Learning

Guillaume Astruc, Eduard Trulls, Jan Hosang, Loic Landrieu+1

LASTIG, Univ Gustave Eiffel, IGN, ENSG, France · CNES, France · LIGM, CNRS, Univ Gustave Eiffel, ENPC, Institut Polytechnique de Paris, Marne-la-Vallée, France+1

Geospatial Representation Learning Multimodal Embedding Cross-Modal Alignment Multimodal Contrastive Learning

Open paper AskPDF arXiv

The growing availability of co-located geospatial data spanning aerial imagery, street-level views, elevation models, text, and geographic coordinates offers a unique opportunity for multimodal representation learning. We introduce UNIGEOCLIP, a massively multimodal contrastive framework to jointly align five complementary geospatial modalities in a single unified embedding space. Unlike prior approaches that fuse modalities or rely on a central pivot representation, our method performs all-to-all contrastive alignment, enabling seamless comparison, retrieval, and reasoning across arbitrary combinations of modalities. We further propose a scaled latitude-longitude encoder that improves spatial representation by capturing multi-scale geographic structure. Extensive experiments across downstream geospatial tasks demonstrate that UNIGEOCLIP consistently outperforms single-modality contrastive models and coordinate-only baselines, highlighting the benefits of holistic multimodal geospatial alignment. A reference implementation is available at https://gastruc.github.io/unigeoclip.

Abstract

Geospatial Representation Learning

Explore similar work

UNIGEOCLIP: Unified Geospatial Contrastive Learning

Geo-Embed: Towards Unified Multimodal Embeddings for Urban Understanding

Textual Supervision Enhances Geospatial Representations in Vision-Language Models