OlmoEarth v1.3: Embeddings

Gabriel Tseng; Favyen Bastani; Patrick Alan Johnson; Yawen Zhang; Piper Wolters; Ando Shah; Hadrien Sablon; Joseph Redmon

cs.CVOct 23, 2026

OlmoEarth v1.3: Embeddings

Authors: Gabriel Tseng, Favyen Bastani, Patrick Alan Johnson, Yawen Zhang, Piper Wolters, Ando Shah, Hadrien Sablon, Joseph Redmon,

Organizations: Allen Institute for AI

Abstract

We make some OlmoEarth embeddings. To do this, we train a new model: OlmoEarth v1.3. OlmoEarth v1.3 is designed to condense multimodal, multitemporal remote sensing data into a single digestible 64 or 128-dimensional vector. OlmoEarth v1.3 produces state-of-the-art embeddings. In addition, OlmoEarth v1.3 is a strong encoder backbone -- competitive with OlmoEarth v1 and v1.2 -- while requiring 18% and 53% of the GPU hours to train, respectively. We use OlmoEarth v1.3 to compute global, annualized 10m embeddings, which we release under the OlmoEarth artifact license.

Figures & tables

Panel of Figure 1 — Figure 1: Storage cost per pixel against mean rank (the smaller the better) over 11 evaluations (Section 3.1 ) 2 2 2 For the AEF evaluations, we rank models using a single classifier. We use $k$ NN-20 for consistency with Section 3.2.1 . , all four products ranked together. TESSERA v2 stores 128 int8 values plus a float scale.

Panel of Figure 2 — Figure 2: One OlmoEarth v1.3 training step. In a single forward pass, we compute the LatentMIM Lite and supervision losses (which are used to train the Attention Pooler and Vision Transformer) and the distillation losses (which are used to train the student). This makes distillation essentially free: no additional forward passes are necessary to compute the distillation losses.

Panel of Figure 3 — Figure 3: The Attention Pooler learns a query which cross-attends to the ViT’s output tokens. We repeat this query at the desired output resolution to obtain embeddings at a finer granularity than the ViT’s patch size; this decouple the model’s output resolution from the number of tokens passing through the ViT (and therefore its compute cost).

Panel of Figure 4 — Figure 4: Inference cost of one $16\times 16$ embedding window (12 timesteps, Sentinel-1, Sentinel-2 and Landsat) by patch size, with and without an Attention Pooler outputting at pixel-resolution. Labels give the encoder’s GMACs and the Pooler’s overhead .

Panel of Figure 5 — Figure 5: We use a linear layer to distill the 768-dimensional location tokens to 128-dimensional embeddings. We apply the distillation losses to the 128-dimensional embeddings and to their first 64 dimensions.

Observations	Maps
Sentinel-1	WorldCereal [ 29 ]	OpenStreetMap [ 21 ]
Sentinel-2	WorldCover [ 33 ]	Cropland Data Layer [ 28 ]
Landsat-8	SRTM [ 20 ]	Canopy Height Map [ 24 ]

Table 6

	Africa crop mask				Canada crops (coarse)				Canada crops (fine)				Descals oil palm				Ethiopia crops				GLanCE				LCMAP land use				US trees
	AEF	TES.	v1.3		AEF	TES.	v1.3		AEF	TES.	v1.3		AEF	TES.	v1.3		AEF	TES.	v1.3		AEF	TES.	v1.3		AEF	TES.	v1.3		AEF	TES.	v1.3
Readout	64	128	128	64	64	128	128	64	64	128	128	64	64	128	128	64	64	128	128	64	64	128	128	64	64	128	128	64	64	128	128	64
Ridge, $\lambda{=}0$	88.8	88.1	91.9	92.6	70.3	70.7	73.6	73.3	60.6	64.5	66.2	63.5	77.4	69.8	79.1	79.0	68.5	68.1	63.6	69.1	67.7	68.7	70.9	69.5	85.8	85.2	88.7	88.1	21.1	20.9	23.0	21.2
kNN-5	87.6	88.3	91.7	91.6	71.2	71.0	72.4	71.5	61.3	62.6	64.7	64.0	71.2	69.0	72.0	71.6	66.4	77.4	70.8	69.2	68.0	68.0	70.2	70.0	83.4	83.7	86.3	86.3	18.3	16.1	18.6	18.3
kNN-20	86.9	87.1	91.8	91.6	70.9	70.6	71.7	70.9	61.9	63.3	64.9	63.9	72.2	70.3	73.2	72.8	64.3	74.2	67.0	66.2	68.8	68.8	71.0	70.8	84.2	84.2	87.1	87.0	21.9	19.5	22.0	21.6
Ridge, $\lambda{=}1$	89.1	89.7	93.2	93.0	70.1	71.5	74.5	73.3	60.1	64.5	66.0	63.1	76.4	71.5	80.6	78.4	66.2	79.6	73.8	70.3	67.7	68.7	70.9	69.5	85.7	85.2	88.7	88.1	21.1	20.9	22.9	21.3

Table 1: Balanced accuracy on the eight AEF balanced-trial datasets for every readout. Sub-columns are AEF (64 dims), TESSERA v2 (“TES.”, 128 dims) and OlmoEarth v1.3 at 128 dims and its first 64 dims. The first block is the AEF protocol (ridge with

\lambda{=}0

, kNN with

k{=}5,20

); the second block adds commonly used classifiers at library defaults. Every classifier is fit on the same class-balanced draws at AEF’s Table-1 sizes and scored on the remainder; cells are means over repeated draws. We highlight OlmoEarth cells that beat the better of the two baselines.

		PASTIS	PLANTEUR
Model	Dim.	Random	Random	LOIO
AEF	64	39.1	27.7	15.8
TESSERA v2	128	58.3	34.2	19.1
OlmoEarth v1.3 distilled	128	58.6	36.7	24.9
OlmoEarth v1.3 distilled	64	53.4	35.2	21.5

Table 2: Crop-type segmentation results, measured via mIoU, on the PASTIS and PLANTEUR datasets. We also measure PLANTEUR’s “leave-one-island-out” (LOIO) mIoU performance, which evaluates the embeddings’ geographic generalization capabilities.

Panel of Figure 6 — Figure 6: PASTIS validation mIoU against inference cost per 16 $\times$ 16 $\times$ 12 S1+S2+L8 window, for the 128-dimensional embeddings and their first 64 dimensions (d64). In all cases, the model’s output patch size is 1 (10m). AEF scores 39.1.

Panel of Figure 7 — Figure 7: Performance vs. computational efficiency, averaged across 13 $k$ NN/LP tasks (Sentinel-2 tasks which all models can support). The chart shows average Multiply-Accumulate operations (MACs) to encode one example across all tasks (input size varies by task) - each line connects model sizes within a family. We measure OlmoEarth v1.3 performance without the Attention Pooler and with (+P, square). Without the Attention Pooler, v1.3 has the best average rank of all models, at the same cost as v1.2 Base; on these 13 tasks, the Attention Pooler adds 13% to the MACs. Full results in Table 6 .

m-bigearthnet

m-so2sat

m-brick-kiln

m-forestnet

m-eurosat

BreizhCrops

CropHarvest-PRC

CropHarvest-Togo

m-cashewplant

m-SA-crop-type

PASTIS

MADOS

Sen1Floods11

AWF

Nandi

Modalities

S1,S2

Time series

✘

✔

✘

✔

✘

✔

Method

kNN

Model

Metric

\mu\text{F1}

Acc.

mIOU

Acc.

Best non-OlmoEarth

64.9

60.5

94.1

52.3

95.0

69.7

60.9

81.9

79.3

79.4

81.4

81.0

54.1

31.7

28.0

44.7

68.4

79.4

66.0

67.5

79.0

61.5

26.8

68.2

OlmoEarth v1

ViT Base

62.4

67.7

94.3

41.9

95.5

70.9

56.8

73.4

75.4

80.1

87.3

82.0

32.3

28.9

29.7

50.6

67.2

79.2

77.0

68.5

77.5

67.9

26.5

74.7

Table 3: kNN/Linear probe results on research benchmarks and real-world partner tasks for the OlmoEarth Base models. “Best non-OlmoEarth” is, for each task, the best score among all other models we evaluate (full results in Table 6 ). We run v1.3 with the Attention Pooler (“+P”) and without. We bold the best score among the rows shown and highlight which ViT Base model wins between v1, v1.2, and v1.3.

		m-bigearthnet	m-so2sat	m-brick-kiln	m-forestnet	m-eurosat	m-cashewplant	m-SA-crop-type	PASTIS	MADOS	Sen1Floods11	AWF	AWF	GEA North Africa	Forest Loss Driver	Live Fuel Moisture Content	Live Fuel Moisture Content	Mangrove	Mangrove	Nandi	Nandi	Vessel Detection	Vessel Detection	Vessel Length	Vessel Type	Solar Farm Detection
	Modalities	S2	S2	S2	L8	S2	S2	S2	S2	S2	S1	S2	S2, S1	S2	S2	S2	S2, S1	S2	S2, S1	S2	S2, S1	L8	S1	S2	S2	S2
	Time series	✘	✘	✘	✘	✘	✘	✘	✔	✘	✘	✔	✔	✔	✔	✔	✔	✔	✔	✔	✔	✘	✘	✘	✘	✔
Model	Metric	$\mu\text{F1}$	Acc.	Acc.	Acc.	Acc.	mIOU	mIOU	mIOU	mIOU	mIOU	Acc.	Acc.	Acc.	Acc.	L1	L1	Acc.	Acc.	Acc.	Acc.	F1	F1	L1	Acc.	mIoU
Anysat	ViT Base	68.4	56.7	98.7	51.6	95.9	80.4	34.2	60.9	63.3	77.4	78.0	83.0	59.3	84.6	19.1	19.4	96.9	97.1	78.4	76.7	-	-	73.4	43.5	82.6
Clay	ViT Large	65.7	61.3	98.2	49.2	95.1	73.9	33.4	48.9	68.9	78.5	77.0	-	53.4	93.3	24.8	-	96.6	-	30.9	-	70.7	79.9	16.4	68.3	82.2
CopernicusFM	ViT Base	71.3	66.8	98.3	-	98.4	78.7	33.6	54.6	66.0	78.6	79.0	79.0	58.8	90.0	25.2	24.5	97.1	97.1	68.2	55.9	-	77.4	16.7	69.6	77.6

Table 4: Finetuning results on research benchmarks and partner tasks. Due to resource constraints, we do not fine-tune large models on all tasks. We run v1.3 with the Attention Pooler (“+P”) and without. We highlight which ViT Base model wins between v1, v1.2, and v1.3.

Panel of Figure 8 — Figure 8: Layout of the OlmoEarth embedding dataset. Each UTM zone is one Zarr array over (reference year, embedding dimension, northing, easting) at 10 m, sparse so unwritten regions cost nothing. A shard is the unit of storage and is pinned to one inference window, so a window writes exactly one object and concurrent writers never contend. A chunk is the unit of transfer; chunking the embedding axis at 64 means a Matryoshka read of the first 64 dimensions fetches one chunk rather than two.

Panel of Figure 9 — Figure 9: Tiled inference – even with significant overlap between crops – introduces artifacts between crops, especially on homogeneous areas like water. These artifacts disappear when using searchlight inference.

Panel of Figure 10 — Figure 10: Dissimilarity between adjacent pixels where four inference windows meet (bright is a boundary). Without a margin the window edges show as a faint cross; with a 48-pixel margin they disappear and field boundaries are unchanged.

Panel of Figure 11 — Figure 11: Bytes transferred to read one $256\times 256$ tile of embeddings. Because a whole chunk is the smallest unit the store can move, reading the Matryoshka $[\ldots,{:}64]$ prefix costs half of the full 128 dimensions rather than the same.

				Energy	Carbon	Water
Model	Size	Hardware	GPU Hrs	(kWh)	(tCO 2 eq)	(kL)
OlmoEarth v1	Nano	H100	1,149	195	0.08	0.30
OlmoEarth v1	Tiny	H100	1,149	205	0.08	0.32
OlmoEarth v1	Base	H100	2,989	803	0.32	1.24
OlmoEarth v1.2	Nano	H100	1,151	191	0.09	0.30
OlmoEarth v1.2	Tiny	H100	1,132	225	0.10	0.35

Table 5: Approximate environmental impact of pretraining OlmoEarth v1, v1.2 and v1.3.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

m-bigearthnet

m-so2sat

m-brick-kiln

m-forestnet

m-eurosat

BreizhCrops

CropHarvest-PRC

CropHarvest-Togo

m-cashewplant

m-SA-crop-type

PASTIS

MADOS

Sen1Floods11

AWF

Nandi

Modalities

S1,S2

Time series

✘

✔

✘

✔

✘

✔

Method

kNN

Model

Metric

\mu\text{F1}

Acc.

mIOU

Acc.

Anysat [ 1 ]

ViT Base

54.5

36.5

77.0

34.0

77.6

62.7

57.0

74.1

72.3

69.6

78.4

74.5

24.6

27.2

24.2

41.9

41.3

77.8

60.0

64.0

47.4

20.5

55.8

Clay [ 4 ]

ViT Large

48.8

38.7

92.8

40.3

86.1

57.0

56.7

66.5

63.4

78.4

67.0

67.3

30.8

23.1

19.9

22.6

47.5

78.9

59.5

61.5

56.5

52.2

23.4

57.0

Appendix

Table 6: kNN/Linear probe results on research benchmarks and real-world partner tasks. Not all models can run on all tasks due to incompatible input modalities. We run v1.3 with Attention Pooling (“+P”) and without. We highlight which ViT Base model wins between v1, v1.2, and v1.3.

Geospatial Representation Learning

Geospatial representation learning builds reusable embeddings of places, imagery, maps, mobility, and Earth data for spatial prediction and reasoning.

See all 133 papers

Self-Supervised Multi-Modal World Model with 4D Space-Time Embedding
Mar 7, 2026Lance Legel, Qin Huang, Brandon Voelker +9World Model Learning Geospatial Representation Learning
Ecological Intelligence Lab Ecodash.ai · School of Complex Adaptive Systems Arizona State University · Geosensing Systems Engineering & Sciences Lab University of Houston +6
Any Model, Any Place, Any Time: Get Remote Sensing Foundation Model Embeddings On Demand
Feb 27, 2026Dingqi Ye, Daniel Kiv, Wei Hu +2Remote Sensing Benchmarks Geospatial Foundation Models
University of Illinois Urbana-Champaign, Urbana, Illinois, USA
Earth Embeddings
Aug 4, 2026Adam J. Stewart, Heng Fang, Isaac A. Corley +1Geospatial Representation Learning Land Cover Classification
Chair of Data Science in Earth Observation, Technical University of Munich, Munich, Germany · Munich Center for Machine Learning, Munich, Germany · KTH Royal Institute of Technology, Stockholm, Sweden +1

Mar 7, 2026J/K move · Enter open · S save

Self-Supervised Multi-Modal World Model with 4D Space-Time Embedding

Lance Legel, Qin Huang, Brandon Voelker, Daniel Neamati+8

Ecological Intelligence Lab Ecodash.ai · School of Complex Adaptive Systems Arizona State University · Geosensing Systems Engineering & Sciences Lab University of Houston+6

World Model Learning Geospatial Representation Learning Positional Encoding Multimodal Pretraining

Open paper AskPDF arXiv

We present DeepEarth, a self-supervised multi-modal world model with Earth4D, a novel planetary-scale 4D space-time positional encoder. Earth4D extends 3D multi-resolution hash encoding to include time, efficiently scaling across the planet over centuries with sub-meter, sub-second precision. Multi-modal encoders (e.g. vision-language models) are fused with Earth4D embeddings and trained via masked reconstruction. We demonstrate Earth4D's expressive power by achieving state-of-the-art performance on an ecological forecasting benchmark. Earth4D with learnable hash probing surpasses a multi-modal foundation model pre-trained on substantially more data. Access open source code and download models at: https://github.com/legel/deepearth

Abstract

Geospatial Representation Learning

Explore similar work

Self-Supervised Multi-Modal World Model with 4D Space-Time Embedding

Any Model, Any Place, Any Time: Get Remote Sensing Foundation Model Embeddings On Demand

Earth Embeddings