The growing availability of co-located geospatial data spanning aerial imagery, street-level views, elevation models, text, and geographic coordinates offers a unique opportunity for multimodal representation learning. We introduce UNIGEOCLIP, a massively multimodal contrastive framework to jointly align five complementary geospatial modalities in a single unified embedding space. Unlike prior approaches that fuse modalities or rely on a central pivot representation, our method performs all-to-all contrastive alignment, enabling seamless comparison, retrieval, and reasoning across arbitrary combinations of modalities. We further propose a scaled latitude-longitude encoder that improves spatial representation by capturing multi-scale geographic structure. Extensive experiments across downstream geospatial tasks demonstrate that UNIGEOCLIP consistently outperforms single-modality contrastive models and coordinate-only baselines, highlighting the benefits of holistic multimodal geospatial alignment. A reference implementation is available at https://gastruc.github.io/unigeoclip.
Figures & tables
Figure 2: Sample from our multimodal geospatial dataset. Each location is represented through five complementary modalities: aerial imagery, street-level imagery, a Digital Surface Model (DSM), geographic coordinates, and an automatically generated text description. 3 3 3 Analytical use of proprietary data sources was done with special permission from Google. Faces and license plates are anonymized by blurring. All modalities are spatially co-registered and jointly contrasted during training.
Figure 3 : Multi-Scale Coordinate Encoder. Latitude–longitude coordinates are first projected using multiple random Fourier feature matrices with increasing spectral bandwidths. Each frequency projection is treated as a token and processed through self-attention to enable inter-scale interaction. The resulting tokens are averaged to produce a unified D -dimensional geographic embedding.
1×22×2DM1∼N(0,σ1)M1∼N(0,σ2)M1∼N(0,σK)1×D Transformer blocks pool
Figure 3: Multi-Scale Coordinate Encoder. Latitude–longitude coordinates are first projected using multiple random Fourier feature matrices with increasing spectral bandwidths. Each frequency projection is treated as a token and processed through self-attention to enable inter-scale interaction. The resulting tokens are averaged to produce a unified D -dimensional geographic embedding.
Figure 4: Geographic Coverage. Spatial distribution of sampled locations across the continental United States. Green regions indicate areas containing samples after spatial filtering and farthest-point sampling.
Table 1: Cross-Modal Street View Retrieval. We report Acc@100 m for cross-modal retrieval and specify the modalities contrasted during training. OOD denotes the out-of-domain evaluation setting. refers to geocells hashcodes. GeoCLIP is scaled up to the parameter count of our model and retrained on our data.
classif / semseg ↦
training modalities
m-pv4ger
m-chesapeake
model
classif (OA)
semseg (mIoU)
SatClip [ 15 ]
ViT-B
93.0
59.3
SigLip2 [ 35 ]
ViT-B
95.7
60.9
UniGeoCLIP
ViT-B
96.9
65.9
UniGeoCLIP
ViT-B
97.0
66.3
SenPaMAE [ 25 ]
ViT-B
87.1
46.9
Table 2: Satellite Image Encoder. We evaluate the ability of our satellite image encoder to analyze aerial and satellite imagery of various image encoders on two geospatial benchmarks: solar panel detection and land-cover segmentation . Models are grouped into cross-modal contrastive frameworks (top), Earth observation foundation models (middle), and general-purpose vision foundation models (bottom). We underline the best performance among contrastive models and bold the best overall performance.
Regression ↦
target
health
social
environmental
mean R²
SigLIP 2 [ 35 ]
34.5
48.5
72.3
40.2
SatClip [ 15 ]
25.7
34.4
70.7
30.1
† SatClip [ 15 ]
32.3
48.5
43.7
36.7
GeoClip [ 38 ]
45.2
65.7
84.2
49.8
† GeoClip [ 38 ]
47.3
67.9
77.9
51.6
Table 3: Evaluation of the Location Encoder. Performance on 27 downstream regression tasks spanning health, socio-economic, and environmental indicators. Models are grouped into contrastive approaches (top) and pre-computed embedding fields (bottom). Variants marked with † are retrained on our dataset. ⋆ denotes models specifically trained for socio-economic prediction. We bold the best overall performance and underline the best performance among contrastive approaches.
Figure 5: PCA of Coordinate Embeddings. Embeddings computed over a dense grid in Manhattan, NYC are projected using PCA, with the top three principal components mapped to RGB. UniGeoCLIP produces spatial patterns that reflect underlying urban structure (e.g., Central Park and surrounding neighborhoods), indicating semantically informed representations. In contrast, SatCLIP and GeoCLIP exhibit smoother, predominantly position-driven gradients.
Table 4 : Evaluation of the DSM Encoder on MDAS. We evaluate the accuracy of land-cover prediction from DSM-only input. Our linearly-probed DSM encoder significantly outperforms U-Net and ViT-B models trained from scratch.
Table 4: Evaluation of the DSM Encoder on MDAS. We evaluate the accuracy of land-cover prediction from DSM-only input. Our linearly-probed DSM encoder significantly outperforms U-Net and ViT-B models trained from scratch.
Figure 6: Location Embedding Visualization. T-SNE projection of embeddings for 48 distinct locations. Each cluster corresponds to a single geographic location and contains embeddings from all modalities.
Computational Methods Lab, HafenCity University Hamburg, Germany. · Institute of Geodesy and Geoinformation, University of Bonn, Germany. · Dept. of Operations & Technology, Technical University of Munich; Heilbronn Data Science Center; Munich Data Science Institute.
Max Planck Institute for Security and Privacy (MPI-SP), Bochum, Germany · Universidade Federal de Minas Gerais, Belo Horizonte, Brazil · Changwon National University, Changwon, South Korea +1