Decomposing Earth Embeddings with Sparse Autoencoders
Organizations: Max Planck Institute for Biogeochemistry, Jena, Germany · ELLIS Unit Jena, Jena, Germany · ETH Zürich, Zürich, Switzerland
Abstract
Earth observation foundation models produce global metre-resolution embeddings that underpin work across agriculture, ecology, hydrology, and beyond. Yet these embeddings are opaque: each dimension entangles unrelated physical concepts, which makes them difficult to interpret and their robustness hard to assess. Here, we propose sparse autoencoders (SAEs) as a principled way to decompose Earth embeddings into sparse, approximately monosemantic features over an overcomplete dictionary. Because LLM judges of interpretations do not transfer to satellite pixels, and spatial autocorrelation matters, we introduce a tailored evaluation protocol. It scores any SAE on reconstruction at matched sparsity, alignment with reference concepts, and spatial scale, each against baselines that include an SAE whose decoder is a frozen random dictionary. On this protocol, SAEs reconstruct AlphaEarth and Tessera embeddings better than every baseline at matched sparsity. Their features align with far fewer reference concepts than raw embedding dimensions (on AlphaEarth, 88% align with at most one of 20 concept groups, most with none, against a median of 12 per raw dimension). They also carry more of their variance at fine spatial scales (23 to 32% within a 640 m patch for BatchTopK and Matryoshka, against 9%). An SAE with a frozen random decoder shares both properties, which therefore follow from sparsity, whereas only a learned dictionary reconstructs the embedding accurately: on AlphaEarth, trained SAEs leave eight times less variance unexplained. We test the sparse code on two applications, discovering land-cover sub-concepts and detecting out-of-distribution pixels by area-of-applicability estimation. Both indicate how sparse features could be used, but neither is yet reliable enough for practical use. Together, these results are a step toward Earth observation models whose features can be inspected.
Figures & tables
Appendix figures & tables22 assets
Supplementary material from the paper’s appendix.
Appendix
| Name | Type | Resolution | Dim. / classes | Description | Reference |
| Earth embeddings | |||||
| AlphaEarth Foundations | embedding | m | Pixel embedding from Sentinel-1/2 and Landsat imagery, trained to also reconstruct GEDI lidar, MODIS land-surface temperature, ERA5 climate, elevation and other sources; depends on the surrounding pixels. | [ 2 ] | |
| Tessera | embedding | m | Pixel embedding from the annual Sentinel-1/2 time series of each single pixel, trained with Barlow Twins; does not look at neighbouring pixels. | [ 3 ] | |
| Categorical reference products | |||||
| Overture water, natural land | categorical | m | classes | Water and natural-land vector layer (ocean, lake, river, canal, reservoir, wetland, forest, glacier, sand, bare rock, shrub, grass) from Overture Maps. | [ 72 ] |
| Overture roads | categorical | m | classes | Road-class vector layer (motorway, primary, secondary, residential, …). | [ 72 ] |
| Concept group | Member (product, classes) |
|---|---|
| forest | ESA WorldCover: Trees; ESRI: Trees; MODIS IGBP: [Evergreen needleleaf, Evergreen broadleaf, Deciduous needleleaf, Deciduous broadleaf, Mixed]; ESA CCI LC: [Broadleaf evergreen forest, Broadleaf deciduous forest, Broadleaf deciduous closed, Broadleaf deciduous open, Needleleaf evergreen forest, Needleleaf evergreen closed, Needleleaf evergreen open, Needleleaf deciduous forest, Needleleaf deciduous closed, Needleleaf deciduous open, Mixed-leaf forest]; Overture water: forest; Overture land use: forest; LCZ: Dense trees; GLanCE: Trees; GLC_FCS30D: [Open evergreen broadleaf, Closed evergreen broadleaf, Open deciduous broadleaf, Closed deciduous broadleaf, Open evergreen needleleaf, Closed evergreen needleleaf, Open deciduous needleleaf, Closed deciduous needleleaf, Open mixed-leaf, Closed mixed-leaf] |
| shrubland | ESA WorldCover: Shrubland; ESRI: Scrub; MODIS IGBP: [Closed shrubs, Open shrubs]; ESA CCI LC: [Shrubland, Shrubland, evergreen, Shrubland, deciduous]; Overture water: shrub; LCZ: Bush / scrub; GLanCE: Shrublands; GLC_FCS30D: [Shrubland, Evergreen shrubland, Deciduous shrubland] |
| grassland | ESA WorldCover: Grassland; ESRI: Grass; MODIS IGBP: Grasslands; ESA CCI LC: Grassland; Overture water: grass; Overture land use: [meadow, recreation_ground]; LCZ: Low plants; GLanCE: Herbaceous; GLC_FCS30D: Grassland |
| cropland | ESA WorldCover: Cropland; ESRI: Crops; MODIS IGBP: [Croplands, Cropland / vegetation mosaic]; ESA CCI LC: [Cropland, rainfed, Herbaceous cover, Tree/shrub cover, Cropland, irrigated, Mosaic cropland]; Overture land use: [farmland, orchard, vineyard, allotments, plant_nursery]; WorldCereal: Temporary crop; GLC_FCS30D: [Rainfed cropland, Herb. cropland, Tree/shrub cropland, Irrig. cropland] |
| built-up | ESA WorldCover: Built-up; ESRI: Built; MODIS IGBP: Urban; ESA CCI LC: Urban areas; Overture land use: [residential, industrial, commercial, military, construction, retail, education, religious]; LCZ: [Compact high-rise, Compact midrise, Compact low-rise, Open high-rise, Open midrise, Open low-rise, Light low-rise, Large low-rise, Sparsely built, Heavy industry]; GLanCE: Developed; GLC_FCS30D: Impervious surfaces |
| water | ESA WorldCover: Permanent water; ESRI: Water; MODIS IGBP: Water Bodies; ESA CCI LC: Water bodies; Overture water: [ocean, lake, river, canal, reservoir]; Overture land use: [port, aquaculture]; LCZ: Water; GLanCE: Water; GLC_FCS30D: Water body |
| MSE ( ) | FVE ( ) | cos ( ) | ||
| AlphaEarth ( ) | ||||
| Raw ( dims) | ||||
| PCA- | ||||
| ICA- | ||||
| NMF- | ||||
| Frozen decoder | ||||
| Embedding | SAE | frozen | PCA | raw | ||||
|---|---|---|---|---|---|---|---|---|
| Copernicus-FM | 768 | 1024 | 16 | 16 | 0.874 | 0.124 | 0.676 | 0.108 |
| 768 | 1024 | 192 | 185 | 0.952 | 0.490 | 0.931 | 0.431 | |
| OlmoEarth | 768 | 4096 | 16 | 16 | 0.650 ∗ | 0.153 ∗ | 0.295 | 0.021 |
| 768 | 4096 | 192 | 185 | 0.870 | 0.598 | 0.785 | 0.241 | |
| MMEarth | 320 | 4096 | 16 | 18 | 0.641 ∗ | 0.328 ∗ | 0.355 | 0.056 |
| 320 | 4096 | 80 | 84 | 0.809 | 0.639 | 0.697 | 0.263 |
| Embedding | Representation | live | median | |||
|---|---|---|---|---|---|---|
| OlmoEarth | raw dimensions | 768 | 0.01 | 0.01 | 767 | 7 |
| PCA | 768 | 0.35 | 0.25 | 698 | 2 | |
| SAE | 4095 | 0.94 | 0.09 | 609 | 0 | |
| frozen decoder | 4096 | 0.92 | 0.16 | 995 | 0 | |
| MMEarth | raw dimensions | 320 | 0.00 | 0.00 | 320 | 6 |
| PCA | 320 | 0.17 | 0.12 | 306 | 3 |
| AlphaEarth | Tessera | |||
| Raw | ||||
| PCA- | ||||
| ICA- | ||||
| NMF- | ||||
| ReLU+ | ||||
| AlphaEarth | Tessera | |||||||
| GeoMS | MS | GeoMS | MS | |||||
| mean | max | mean | max | mean | max | mean | max | |
| Uniform | – | – | – | – | ||||
| Raw | ||||||||
| PCA | ||||||||
| ICA | ||||||||
| Feature | Köppen-Geiger | RESOLVE biome | PNV | |
|---|---|---|---|---|
| f139 | Af 0.80 (15.6); Am 0.16 (4.2) | Trop. moist broadleaf 0.98 (6.7) | Closed evergr. broadleaf 1.00 (5.5) | |
| f124 | BSh 0.39 (6.3); Aw 0.24 (1.9) | Trop. grassland, savanna, shrub 0.59 (3.7); temp. grassland 0.17 (2.1) | Shrubs 0.69 (2.8) | |
| f35 | Dfc 0.61 (4.6); Dfb 0.30 (4.4) | Boreal forest 0.55 (4.8); temp. conifer 0.23 (8.0) | Closed evergr. needleleaf 0.65 (8.6) |
| train on Europe | leave Europe out | |||
|---|---|---|---|---|
| checkpoint | raw | SAE | raw | SAE |
| BatchTopK, seed 42 | ||||
| BatchTopK, seed 43 | ||||
| BatchTopK, seed 44 | ||||
| Matryoshka | ||||
Interpretable ML
Interpretable machine learning develops models and explanations that make predictions, evidence, and decision processes understandable to people.
See all 325 papers