LEPA: Learning Geometric Equivariance in Satellite Remote Sensing Data with a Predictive Architecture

Erik Scheurer; Rocco Sedona; Stefan Kesselheim; Gabriele Cavallaro

cs.CVMar 7, 2026

LEPA: Learning Geometric Equivariance in Satellite Remote Sensing Data with a Predictive Architecture

Authors: Erik Scheurer, Rocco Sedona, Stefan Kesselheim, Gabriele Cavallaro

Organizations: Jülich Supercomputing Centre (JSC), Forschungszentrum Jülich, 52428 Jülich, Germany · Institute for Visualization and Interactive Systems (VIS), University of Stuttgart, 70569 Stuttgart · School of Engineering and Natural Sciences (SENS), University of Iceland, 102 Reykjavík, Iceland

Abstract

Geospatial foundation models provide precomputed embeddings that serve as compact feature vectors for large-scale satellite remote sensing data. While these embeddings can reduce data-transfer bottlenecks and computational costs, Earth observation (EO) applications can still face geometric mismatches between user-defined areas of interest and the fixed precomputed embedding grid. Standard latent-space interpolation is unreliable in this setting because the embedding manifold is highly non-convex, yielding representations that do not correspond to realistic inputs. We verify this using Prithvi-EO-2.0 to understand the shortcomings of interpolation applied to patch embeddings. As a substitute, we propose a Learned Equivariance-Predicting Architecture (LEPA). Instead of averaging vectors, LEPA conditions a predictor on geometric augmentations to directly predict the transformed embedding. We evaluate LEPA on NASA/USGS Harmonized Landsat-Sentinel (HLS) imagery and ImageNet-1k. Experiments show that standard interpolation achieves a mean reciprocal rank (MRR) below 0.2, whereas LEPA increases MRR to over 0.8, enabling accurate geometric adjustment without re-encoding.

Figures & tables

Fig. 1: LEPA training architecture: the original input is passed to the student encoder which produces patch embeddings. These patch embeddings form the context to the predictor along with transformation parameters. Predicted embeddings are compared against the teacher output which are computed from a transformed input image. Masking is omitted for visual clarity.

Panel of Figure 2 — Fig. 2: Input and reconstructions of rotated and downsampled images using a finetuned version of Prithvi [ 5 ] . The bottom left images depict reconstructions from a transformed latent, the bottom right images are reconstructions from rotations in image space as a baseline.

Fig. 3: Box-plot of the normalized score based on the results of Table I .

Panel of Figure 4 — Fig. 4: Example images with class-specific noise. Bottom rows show the first two PCA components (per image) mapped to a color wheel. In the ImageNet model without a CLS-token, background embeddings resemble the subject, a pattern not seen consistently in HLS data.

Model	Dataset	CLS	Pos. Enc.	MRR	After Finetune
I-JEPA	IN-1k	No	default	0.1837	N/A
I-JEPA	IN-1k	Yes	default	0.2108	N/A
I-JEPA	HLS	No	default	0.1826	N/A
I-JEPA	HLS	Yes	default	0.1839	N/A
LEPA	HLS	No	CondPos	0.6975	0.8062
LEPA	HLS	Yes	CondPos	0.6630	0.7994

TABLE I: MRR of models given different hyperparameters with and without finetuning of the predictor. For Prithvi and I-JEPA, the predicted embedding is obtained with nearest-neighbor interpolation. Other interpolation methods decrease MRR.

Panel of Figure 6 — Fig. 5: Angle sweep of a selected image comparing the first two PCA components using interpolation in image space (Targets), the predictions using finetuned LEPA, and nearest neighbor interpolation.

Fig. 6: Grid that is used for sinusoidal positional encodings by default, ours without a transformation and ours with a rotation angle of 10°.

Panel of Figure 8 — Fig. 7: Comparison of positional encodings for each embedding dimension out of 768. Each row consists of one line of equation 2 .

Model	Dataset	CLS	PosEnc	Params	HLSBurnscars	Sen1Floods11	MADOS	AI4SmallFarms	PASTIS	DynamicEarthNet	SpaceNet7
I-JEPA	IN-1k	No	default	85.8M	77.15	71.75	59.28	26.58	21.31	35.13	60.02
I-JEPA	IN-1k	Yes	default	85.8M	77.58	74.74	65.59	25.98	24.07	37.68	61.12
I-JEPA	HLS	No	default	86.4M	82.82	86.69	45.74	24.33	33.00	29.95	56.93
I-JEPA	HLS	Yes	default	86.4M	83.08	85.87	46.40	25.48	35.03	31.46	56.67
I-JEPA	HLS	Yes	CondPos	86.4M	81.73	85.88	47.59	24.58	34.43	32.64	56.64
LEPA	HLS	No	CondPos	86.4M	83.03	87.29	51.40	23.82	33.84	33.84	56.17

TABLE II: Mean IoU comparison of pretrained foundation models and JEPA variations of the PANGAEA benchmark [ 22 ] . The best and second best models are marked for each dataset.

Equivariant Representation Learning

Equivariant representation learning builds features that transform predictably with rotations, translations, permutations, and other symmetries.

See all 54 papers

From Pixels to Patches: Pooling Strategies for Earth Embeddings
Mar 2, 2026Isaac Corley, Caleb Robinson, Inbal Becker-Reshef +1Spatial Generalization Geospatial Foundation Models
Wherobots · Microsoft AI for Good Research Lab
Recoverable Geographic Location Information in Earth-Observation Embeddings
Sep 24, 2026Peiwen Zhang, Kristie Hu, Jovana Knezevic +2Geospatial Foundation Models Geospatial Representation Learning
University of Waterloo · University of Cambridge · Aalto University
Location Is All You Need: Continuous Spatiotemporal Neural Representations of Earth Observation Data
Apr 8, 2026Mojgan Madadikhaljan, Jonathan Prexl, Isabelle Wittmann +2Geospatial Representation Learning Implicit Neural Representations
University of the Bundeswehr Munich · IBM Research – Europe · Columbia University +1

Mar 2, 2026J/K move · Enter open · S save

From Pixels to Patches: Pooling Strategies for Earth Embeddings

Isaac Corley, Caleb Robinson, Inbal Becker-Reshef, Juan M. Lavista Ferres

Wherobots · Microsoft AI for Good Research Lab

Spatial Generalization Geospatial Foundation Models Representation Learning Remote Sensing Benchmarks

Open paper AskPDF arXiv

Geospatial foundation models increasingly expose pixel-level embedding products that can be downloaded and reused without access to the underlying encoder. In this setting, downstream tasks with patch- or region-level labels require a post-hoc aggregation step that maps dense pixel embeddings to a single representation. The default choice, mean pooling, discards within-patch variability and can underperform under spatial distribution shift. To study this setting, we introduce EuroSAT-Embed: 81,000 embedding GeoTIFFs derived from three foundation models: AlphaEarth, OlmoEarth, and Tessera. Using these fixed embedding products, we benchmark 11 training-free pooling methods and 2 train-set-fitted baselines under both random and geographically disjoint test splits. Richer pooling schemes reduce the geographic generalization gap by over 50% relative to mean pooling and improve accuracy by up to 6% on spatial splits. We recommend a three-tier strategy: (1) mean as a baseline, (2) stats pooling (min/max/mean/std) as the default at 4x the embedding dimension, and (3) covariance pooling for peak accuracy. Across all three embedding products, simple distributional statistics improve spatial-split performance over mean pooling.

Abstract

Equivariant Representation Learning

Explore similar work

From Pixels to Patches: Pooling Strategies for Earth Embeddings

Recoverable Geographic Location Information in Earth-Observation Embeddings

Location Is All You Need: Continuous Spatiotemporal Neural Representations of Earth Observation Data