BEACON: Behavioral and Semantic Enrichment of AlphaEarth Embeddings through Tri-Modal Contrastive Learning

Hao Tian; Heng Cai; Yifan Yang

cs.LGAug 30, 2026

BEACON: Behavioral and Semantic Enrichment of AlphaEarth Embeddings through Tri-Modal Contrastive Learning

Authors: Hao Tian, Heng Cai, Yifan Yang

Organizations: Department of Geography Texas A&M University College Station, TX, USA

Abstract

Geospatial foundation models such as the AlphaEarth Foundation produce compact and globally consistent representations of the Earth's surface that transfer effectively to a wide range of downstream tasks. However, because these models are trained primarily on Earth-observation imagery, their embeddings mainly capture physical and spectral characteristics while encoding human activity and urban function only weakly. To address this limitation, we propose BEACON, a tri-modal contrastive learning framework that aligns three complementary views of urban space: physical representations from AE embeddings, semantic representations from point-of-interest (POI) text, and human behavioral representations from hourly POI visitation, while keeping the deployed representation image-only. Using the Houston Metropolitan Area as a case study area, we evaluated the performance of the BEACON framework on nine downstream tasks, including seven regression and two classification tasks against six baselines (raw coordinates, Space2Vec, SatCLIP, TESSERA, Clay and AlphaEarth), using frozen linear and MLP probes over five seeds. Under a linear probe, BEACON improves relative R^2 over AlphaEarth by up to 43% for obesity prevalence, 34% for poor mental health, and 22% for median household income, while remaining competitive in the prediction of physical and environmental variables. These findings highlight the value of augmenting geospatial foundation models with semantic and behavioral signals, extending their applicability from physical Earth observation to human-centered urban analytics.

Figures & tables

Figure 1. Overview of BEACON. Physical, semantic, and behavioral views are aligned through tri-modal contrastive learning to produce human-contextualized embeddings. Schematic of the BEACON framework: AlphaEarth physical embeddings, POI semantic attributes, and hourly mobility profiles are encoded into a shared space and aligned by tri-modal contrastive learning.

Target	Support	$N$	Task
Median household income	Block groups	4,174	Reg.
Obesity prevalence	Tracts	1,613	Reg.
Poor mental health	Tracts	1,613	Reg.
Nighttime lights	500 m px	139,186	Reg.
LST daytime	1 km px	27,424	Reg.
LST nighttime	1 km px	27,424	Reg.

Table 1. Downstream benchmark tasks, spatial supports, and common-support sample sizes.

	Human-related				Physical / Environmental
Model	Income	Obesity	Mental health	NTL	LST day	LST night	PM 2.5
Coordinates	0.012 $\pm$ 0.007	0.032 $\pm$ 0.020	0.040 $\pm$ 0.021	0.002 $\pm$ 0.000	0.246 $\pm$ 0.009	0.304 $\pm$ 0.006	0.110 $\pm$ 0.004
Space2Vec	0.280 $\pm$ 0.027	0.472 $\pm$ 0.038	0.328 $\pm$ 0.060	0.419 $\pm$ 0.071	0.464 $\pm$ 0.014	0.331 $\pm$ 0.009	0.386 $\pm$ 0.007
SatCLIP	0.024 $\pm$ 0.010	0.062 $\pm$ 0.025	0.047 $\pm$ 0.020	0.316 $\pm$ 0.050	0.644 $\pm$ 0.009	0.584 $\pm$ 0.003	0.617 $\pm$ 0.009
TESSERA	0.420 $\pm$ 0.027	0.519 $\pm$ 0.046	0.431 $\pm$ 0.041	0.667 $\pm$ 0.111	0.824 $\pm$ 0.009	0.779 $\pm$ 0.008	0.868 $\pm$ 0.004
Clay	0.413 $\pm$ 0.034	0.557 $\pm$ 0.048	0.429 $\pm$ 0.031	0.603 $\pm$ 0.087	0.860 $\pm$ 0.004	0.813 $\pm$ 0.007	0.905 $\pm$ 0.002

Table 2. Regression performance (

R^{2}

, linear probe) across seven downstream tasks (mean

\pm

s.d. over five seeds). Best in bold, second-best underlined.

Figure 2. Pretraining dynamics of BEACON. (a) Training/validation contrastive loss. (b) Relative validation loss of the four objective terms. (c) Held-out cross-modal retrieval Recall@1/5/10. (d) AlphaEarth–mobility cosine-similarity before/after training. Four-panel figure of BEACON pretraining diagnostics: contrastive loss curves, per-term relative validation loss, cross-modal retrieval recall, and AlphaEarth–mobility cosine similarity before and after training.

	Land cover		Functional land use
Model	Macro-F1	Acc.	Macro-F1	Acc.
Coordinates	0.152 $\pm$ 0.001	0.266 $\pm$ 0.007	0.144 $\pm$ 0.007	0.241 $\pm$ 0.007
Space2Vec	0.190 $\pm$ 0.002	0.253 $\pm$ 0.002	0.270 $\pm$ 0.001	0.333 $\pm$ 0.003
SatCLIP	0.230 $\pm$ 0.002	0.338 $\pm$ 0.005	0.153 $\pm$ 0.002	0.252 $\pm$ 0.008
TESSERA	0.521 $\pm$ 0.003	0.635 $\pm$ 0.004	0.421 $\pm$ 0.003	0.518 $\pm$ 0.003
Clay	0.357 $\pm$ 0.004	0.464 $\pm$ 0.005	0.279 $\pm$ 0.003	0.355 $\pm$ 0.004

Table 3. Classification performance (linear probe; mean

\pm

s.d. over five seeds). Best in bold, second-best underlined.

Variant	Human-related Avg. $R^{2}$	Physical / Environmental Avg. $R^{2}$
Mobility only	0.011 $\pm$ 0.030	0.015 $\pm$ 0.007
POI only	0.288 $\pm$ 0.047	0.199 $\pm$ 0.009
Concat Raw	0.294 $\pm$ 0.059	0.644 $\pm$ 0.007
AlphaEarth	0.401 $\pm$ 0.037	0.743 $\pm$ 0.008
AE-Mobility	0.475 $\pm$ 0.029	0.736 $\pm$ 0.006
AE-POI	0.494 $\pm$ 0.023	0.750 $\pm$ 0.007

Table 4. Ablation results of different modality combinations using linear probes.

Geospatial Representation Learning

Geospatial representation learning builds reusable embeddings of places, imagery, maps, mobility, and Earth data for spatial prediction and reasoning.

See all 133 papers

Earth Embeddings Reveal Diverse Urban Signals from Space
Apr 3, 2026Wenjing Gong, Udbhav Srivastava, Yuchen Wang +6Geospatial Representation Learning Remote Sensing Benchmarks
Texas A&M University, College Station, TX, USA. · The University of Alabama, Tuscaloosa, AL, USA. · Emory University, Atlanta, GA, USA.
Recoverable Geographic Location Information in Earth-Observation Embeddings
Sep 24, 2026Peiwen Zhang, Kristie Hu, Jovana Knezevic +2Geospatial Foundation Models Geospatial Representation Learning
University of Waterloo · University of Cambridge · Aalto University
Geo-Embed: Towards Unified Multimodal Embeddings for Urban Understanding
Aug 4, 2026Jiapeng Li, Yong Li, Junjie Zhou +2Geospatial Representation Learning Cross-Modal Retrieval
1Peking University · 2Beijing University of Posts and Telecommunications

Apr 3, 2026J/K move · Enter open · S save

Earth Embeddings Reveal Diverse Urban Signals from Space

Wenjing Gong, Udbhav Srivastava, Yuchen Wang, Yuhao Jia+5

Texas A&M University, College Station, TX, USA. · The University of Alabama, Tuscaloosa, AL, USA. · Emory University, Atlanta, GA, USA.

Geospatial Representation Learning Remote Sensing Benchmarks Geospatial Foundation Models Urban Representation Learning

Open paper AskPDF arXiv

Conventional urban indicators derived from censuses, surveys, and administrative records are often costly, spatially inconsistent, and slow to update. Recent geospatial foundation models enable Earth embeddings, compact satellite image representations transferable across downstream tasks, but their utility for neighborhood-scale urban monitoring remains unclear. Here, we benchmark three Earth embedding families, AlphaEarth, Prithvi, and Clay, for urban signal prediction across six U.S. metropolitan areas from 2020 to 2023. Using a unified supervised-learning framework, we predict 14 neighborhood-level indicators spanning crime, income, health, and travel behavior, and evaluate performance under four settings: global, city-wise, year-wise, and city-year. Results show that Earth embeddings capture substantial urban variation, with the highest predictive skill for outcomes more directly tied to built-environment structure, including chronic health burdens and dominant commuting modes. By contrast, indicators shaped more strongly by fine-scale behavior and local policy, such as cycling, remain difficult to infer. Predictive performance varies markedly across cities but remains comparatively stable across years, indicating strong spatial heterogeneity alongside temporal robustness. Exploratory analysis suggests that cross-city variation in predictive performance is associated with urban form in task-specific ways. Controlled dimensionality experiments show that representation efficiency is critical: compact 64-dimensional AlphaEarth embeddings remain more informative than 64-dimensional reductions of Prithvi and Clay. This study establishes a benchmark for evaluating Earth embeddings in urban remote sensing and demonstrates their potential as scalable, low-cost features for SDG-aligned neighborhood-scale urban monitoring.

Abstract

Geospatial Representation Learning

Explore similar work

Earth Embeddings Reveal Diverse Urban Signals from Space

Recoverable Geographic Location Information in Earth-Observation Embeddings

Geo-Embed: Towards Unified Multimodal Embeddings for Urban Understanding