FairRSFM: A Biome-Aware Benchmark and Debiasing Framework for Remote Sensing Foundation Models

Md Aminur Hossain; Omkumar Vaghasiya; Rajeev Ranjan Dwivedi; Vinod Kurmi; Biplab Banerjee

cs.CVOct 5, 2026

FairRSFM: A Biome-Aware Benchmark and Debiasing Framework for Remote Sensing Foundation Models

Authors: Md Aminur Hossain, Omkumar Vaghasiya, Rajeev Ranjan Dwivedi, Vinod Kurmi, Biplab Banerjee

Organizations: Space Applications Centre, ISRO, Ahmedabad, India · Indian Institute of Science Education and Research Bhopal, Bhopal, India · Centre of Studies in Resources Engineering (CSRE), Indian Institute of Technology Bombay, India

Abstract

Remote sensing foundation models (RSFMs) are commonly evaluated using aggregate metrics, which can hide systematic performance disparities across ecological regions. We introduce FairRSFM, a biome-aware benchmark for evaluating ecological group robustness in RSFMs. FairRSFM maps georeferenced samples from 14 terrestrial biome classes into six ecologically meaningful macro-groups and evaluates models under a unified frozen-backbone evaluation protocol. The benchmark covers four downstream datasets: m-EuroSAT, m-BigEarthNet, m-SA-Crop-Type, and MMEarth20K with Dynamic World label maps. Using Prithvi-EO-2.0, SatMAE, and DOFA across three random seeds, we show that aggregate performance consistently masks biome-dependent disparities across architectures and tasks. For example, Prithvi-EO-2.0 reaches 90.98% overall macro-F1 on m-EuroSAT but a mean worst-group score of only 83.72%, while m-SA-Crop-Type drops from 27.30% overall mIoU to 18.47% in the Xeric and Mineralogical group. We further evaluate Biome-Orthogonal Linear Probing (BOLP), Dynamic Biome Reweighting (DBR), and GroupDRO as complementary mitigation baselines. Their effectiveness is model- and task-dependent; for example, BOLP improves Prithvi-EO-2.0 worst-group F1@opt on m-BigEarthNet from 46.12% to 50.27% without updating the RSFM backbone. FairRSFM provides a reusable protocol for diagnosing and mitigating ecological robustness gaps in remote sensing foundation models. Code and datasets are available at: https://github.com/aminurhossain/FairRSFM.

Figures & tables

Figure 1 : Overview of FairRSFM. Georeferenced samples are mapped from 14 terrestrial biomes to six macro-groups and evaluated under a frozen-backbone downstream evaluation protocol. BOLP removes biome-associated directions from frozen RSFM embeddings before training the task head.

Dataset	Task	Classes	Train	Val	Test	Notes
m-EuroSAT [ 12 ]	Classification	10	2,000	1,000	1,000	Sentinel-2, land cover
m-BigEarthNet [ 12 , 25 ]	Multi-label classification	43	20,000	1,000	1,000	Sentinel-2, multi-label land cover
m-SA-Crop-Type [ 12 ]	Segmentation	10	3,000	1,000	1,000	Sentinel-2, crop type
MMEarth20K [ 17 , 2 ]	Segmentation	9	16,000	2,000	2,000	Dynamic World label maps

Table 1 : FairRSFM dataset suite. GEO-Bench datasets are selected for classification and segmentation; the MMEarth20K subset increases dense land-cover segmentation diversity. Sample counts for GEO-Bench tasks follow the modified benchmark setting.

Configuration	Prithvi-EO-2.0 [ 27 ]	SatMAE [ 3 ]	DOFA [ 33 ]
Backbone	Spatio-Temporal ViT (300M)	ViT-Large (304M)	ViT-Base (86M)
Embedding Dim	1024 ( $16\times 16$ patch)	1024 ( $16\times 16$ patch)	768 ( $16\times 16$ patch)
Input Bands	6 HLS Bands	10 Sentinel-2 Bands	9–12 Wavelength Bands
Encoder Status	Frozen (0 updates)	Frozen (0 updates)	Frozen (0 updates)
Classification Head	Linear / MLP probe	Linear probe	BatchNorm1d + Linear
Segmentation Head	FCN decoder	Conv decoder	Multi-level UPerNet [ 33 ]

Table 2 : Unified benchmarking configurations across foundation models.

Dataset	Split	Pheno.	Biomass	Trans.	Cryo.	Hydro.	Xeric	Unk.
m-EuroSAT	Train	1,430	50	481	–	–	–	39
	Valid	741	15	227	–	–	–	17
	Test	724	25	239	–	–	–	12
m-BigEarthNet	Train	11,269	356	6,291	–	–	–	2,084
	Valid	518	17	335	–	–	–	130
	Test	535	14	336	–	–	–	115

Table 3 : Per-macro-group sample counts across Train/Validation/Test splits for all benchmark datasets. Empty cells indicate absent groups. Unknown or unmatched samples are excluded from worst-group scores.

Dataset	Method	Test metric	$M_{\text{worst}}$	ECE $\downarrow$	NFR $\downarrow$	EOdd $\downarrow$	DPM $\downarrow$
m-EuroSAT	ERM	90.98 $\pm$ 2.36	83.72 $\pm$ 0.87	4.02 $\pm$ 1.64	5.96 $\pm$ 3.28	20.50 $\pm$ 1.75	9.91 $\pm$ 0.21
	BOLP	93.42 $\pm$ 0.35	84.34 $\pm$ 0.00	2.37 $\pm$ 0.04	9.82 $\pm$ 0.30	17.06 $\pm$ 0.16	9.61 $\pm$ 0.03
	DBR	91.36 $\pm$ 1.01	84.35 $\pm$ 0.50	5.50 $\pm$ 2.29	5.85 $\pm$ 0.70	21.23 $\pm$ 1.39	10.90 $\pm$ 0.17
	GroupDRO	94.14 $\pm$ 0.47	84.34 $\pm$ 0.00	3.17 $\pm$ 0.40	10.34 $\pm$ 0.51	17.31 $\pm$ 0.12	10.04 $\pm$ 0.10
m-BigEarthNet	ERM	60.09 $\pm$ 0.99	46.12 $\pm$ 1.51	2.12 $\pm$ 0.28	31.14 $\pm$ 1.99	42.93 $\pm$ 1.90	12.66 $\pm$ 0.27
	BOLP	62.75 $\pm$ 0.16	50.27 $\pm$ 0.63	2.07 $\pm$ 0.08	20.92 $\pm$ 0.39	45.93 $\pm$ 0.76	13.29 $\pm$ 0.11

Table 4 : Prithvi-EO-2.0: Mean and Std over seeds across datasets.

Model	Method	m-EuroSAT (Macro-F1)			m-BigEarthNet (F1@opt)			m-SA-Crop-Type (mIoU)
Model	Method	Phenological	High Biomass	Transitional	Phenological	High Biomass	Transitional	Transitional	Xeric
Prithvi-EO-2.0	ERM	88.96 $\pm$ 3.38	84.56 $\pm$ 0.31	86.71 $\pm$ 3.79	59.67 $\pm$ 0.48	61.14 $\pm$ 6.05	46.04 $\pm$ 1.85	28.55 $\pm$ 0.19	18.47 $\pm$ 0.16
	BOLP	92.28 $\pm$ 0.53	84.34 $\pm$ 0.00	93.02 $\pm$ 0.41	61.85 $\pm$ 0.72	54.03 $\pm$ 1.68	50.27 $\pm$ 0.77	27.70 $\pm$ 0.67	19.38 $\pm$ 0.24
	DBR	88.57 $\pm$ 0.96	85.00 $\pm$ 0.00	85.96 $\pm$ 2.74	55.59 $\pm$ 0.18	64.53 $\pm$ 0.13	48.03 $\pm$ 0.78	26.49 $\pm$ 1.93	19.81 $\pm$ 0.42
	GroupDRO	92.82 $\pm$ 0.70	84.34 $\pm$ 0.00	93.68 $\pm$ 0.50	49.59 $\pm$ 0.67	56.78 $\pm$ 5.15	46.82 $\pm$ 0.60	28.52 $\pm$ 0.11	19.09 $\pm$ 0.26
SatMAE	ERM	93.99 $\pm$ 0.14	74.45 $\pm$ 0.51	93.62 $\pm$ 0.13	52.06 $\pm$ 0.22	47.97 $\pm$ 3.69	41.33 $\pm$ 0.36	27.45 $\pm$ 0.48	18.07 $\pm$ 0.23

Table 7 : Biome-wise performance for m-EuroSAT (Macro-F1), m-BigEarthNet (F1@opt), and m-SA-Crop-Type (mIoU) across foundation models and debiasing methods.

Model	Method	Pheno.	Biomass	Trans.	Cryo.	Hydro.	Xeric
Prithvi-EO-2.0	ERM	44.19 $\pm$ 1.39	41.20 $\pm$ 1.35	44.54 $\pm$ 1.04	43.74 $\pm$ 0.56	37.28 $\pm$ 2.14	35.32 $\pm$ 2.69
	BOLP	40.57 $\pm$ 0.93	35.96 $\pm$ 2.17	41.38 $\pm$ 2.26	39.71 $\pm$ 0.63	32.43 $\pm$ 3.10	32.82 $\pm$ 1.91
	DBR	43.58 $\pm$ 1.24	40.52 $\pm$ 1.05	44.03 $\pm$ 1.56	43.78 $\pm$ 1.23	38.49 $\pm$ 3.13	36.87 $\pm$ 2.85
	GroupDRO	42.77 $\pm$ 0.33	40.27 $\pm$ 0.76	41.53 $\pm$ 0.08	43.44 $\pm$ 0.56	37.16 $\pm$ 0.26	40.30 $\pm$ 0.24
SatMAE	ERM	40.89 $\pm$ 0.81	34.89 $\pm$ 0.55	35.86 $\pm$ 0.35	37.42 $\pm$ 0.50	30.78 $\pm$ 0.57	34.11 $\pm$ 1.08
	BOLP	39.91 $\pm$ 1.31	33.13 $\pm$ 2.08	35.29 $\pm$ 1.23	38.09 $\pm$ 0.93	30.27 $\pm$ 0.14	33.76 $\pm$ 0.58

Table 8 : Biome-wise performance for MMEarth20K (mIoU) across foundation models and debiasing methods.

Figure S1 : Global map of the six biome macro-groups used as primary evaluation strata in FairRSFM. Raw 14-class terrestrial biomes are pooled into spectral, phenological, hydrological, cryospheric, and surface-property regimes.

ID	Biome macro-group	Included raw biome classes	Rationale
1	Aseasonal High-Biomass	1 Tropical Moist Forests; 3 Tropical Conifer Forests; 5 Temperate Conifer Forests	Dense, high-biomass vegetation with relatively persistent canopy structure.
2	High-Amplitude Phenological	2 Tropical Dry Forests; 4 Temperate Broadleaf and Mixed Forests; 6 Boreal Forests/Taiga	Forested regions with strong seasonal or phenological variation.
3	Transitional Herbaceous and Scrub	7 Tropical and Subtropical Grasslands, Savannas and Shrublands; 8 Temperate Grasslands, Savannas and Shrublands; 12 Mediterranean Forests, Woodlands and Scrub	Open or mixed vegetation regimes with strong grass, shrub, soil, and canopy heterogeneity.
4	Cryospheric and Short-Cycle	10 Montane Grasslands and Shrublands; 11 Tundra	Temperature-restricted ecosystems with short growing periods or cryospheric influence.
5	Xeric and Mineralogical	13 Deserts and Xeric Shrublands	Vegetation-sparse surfaces dominated by albedo, exposed soil, and mineral background.
6	Hydrologically Modulated	9 Flooded Grasslands and Savannas; 14 Mangroves	Water-influenced ecosystems where inundation strongly affects spectral response.

Table S1 : Mapping from 14 terrestrial biome classes to six biome macro-groups used for ecological robustness evaluation and mitigation in FairRSFM.

Biome Macro-Group	Biome IDs	Quota / Biome	Total (%)
1. Aseasonal High-Biomass	1, 3, 5	$\approx$ 1,867	5,600 (28.0%)
2. High-Amplitude Phenol.	2, 4, 6	1,700	5,100 (25.5%)
3. Transitional Herbaceous	7, 8, 12	1,200	3,600 (18.0%)
4. Cryospheric & Short-Cycle	10, 11	1,400	2,800 (14.0%)
5. Xeric & Mineralogical	13	1,700	1,700 (8.5%)
6. Hydrologically Modulated	9, 14	600	1,200 (6.0%)

Table S2 : MMEarth20K sample allocation across the 6 macro-groups and 14 terrestrial biomes (

N=20,000

). Biome IDs correspond directly to Table S1 .

Hyperparams		m-EuroSAT (Macro-F1)			MMEarth20K (mIoU)
$\alpha$	$T_{\mathrm{upd}}$	Overall $\uparrow$	Worst $\uparrow$	NFR $\downarrow$	Overall $\uparrow$	Worst $\uparrow$	NFR $\downarrow$
0.2	1	91.73 $\pm$ 0.84	83.89 $\pm$ 0.62	7.18 $\pm$ 0.85	48.02 $\pm$ 0.28	33.91 $\pm$ 0.51	27.84 $\pm$ 0.72
0.4	1	91.52 $\pm$ 0.92	84.26 $\pm$ 0.48	6.13 $\pm$ 0.68	47.91 $\pm$ 0.24	34.53 $\pm$ 0.46	26.68 $\pm$ 0.65
0.5	1	91.36 $\pm$ 1.01	84.35 $\pm$ 0.50	5.85 $\pm$ 0.70	47.84 $\pm$ 0.22	34.76 $\pm$ 0.42	26.42 $\pm$ 0.62
0.6	1	90.82 $\pm$ 1.15	83.74 $\pm$ 0.64	6.48 $\pm$ 0.82	47.36 $\pm$ 0.34	34.18 $\pm$ 0.48	27.02 $\pm$ 0.71
0.8	1	88.94 $\pm$ 1.95	82.38 $\pm$ 1.62	8.64 $\pm$ 1.85	46.12 $\pm$ 0.94	33.15 $\pm$ 1.12	29.48 $\pm$ 1.76

Table S3 : DBR sensitivity sweep across feedback strength

\alpha

and update frequency

T_{\mathrm{upd}}

on Prithvi-EO-2.0 test splits (

\text{mean}\pm\text{std}

across

S=3

random seeds).

Dataset	Rank ( $k$ )	Overall $\uparrow$	Worst $\uparrow$	NFR $\downarrow$
m-EuroSAT ( $\|\mathcal{G}_{d}\|=3$ )	$k=0$ (ERM)	90.98 $\pm$ 2.36	83.72 $\pm$ 0.87	5.96 $\pm$ 3.28
	$k=1$	92.47 $\pm$ 0.48	84.13 $\pm$ 0.18	8.14 $\pm$ 0.42
	$\mathbf{k=2}$ (Canonical)	93.42 $\pm$ 0.35	84.34 $\pm$ 0.00	9.82 $\pm$ 0.30
m-BigEarthNet ( $\|\mathcal{G}_{d}\|=3$ )	$k=0$ (ERM)	60.09 $\pm$ 0.99	46.12 $\pm$ 1.51	31.14 $\pm$ 1.99
	$k=1$	61.24 $\pm$ 0.34	48.16 $\pm$ 0.85	24.53 $\pm$ 0.61
	$\mathbf{k=2}$ (Canonical)	62.75 $\pm$ 0.16	50.27 $\pm$ 0.63	20.92 $\pm$ 0.39

Table S4 : BOLP projection rank (

k

) ablation across datasets on Prithvi-EO-2.0 (

\text{mean}\pm\text{std}

across

S=3

seeds;

k=0

denotes unprojected ERM).

Explore similar work

Scalable and Trustworthy Earth Observation Foundation Models
Jul 8, 2026Syed Usama Imtiaz, Mitra Nasr Azadani, Nasrin AlamdariGeospatial Foundation Models Trustworthy Earth Observation Decisions
Department of Civil and Environmental Engineering, Florida State University
EarthShift: a benchmark for measuring robustness to real-world distribution shifts in Earth observation
May 28, 2026Kelsey Doerksen, Hannah KernerGeospatial Foundation Models Distribution Shifts
School of Computing and Augmented Intelligence Arizona State University
REMSA: Foundation Model Selection for Remote Sensing via a Constraint-Aware Agent
Nov 21, 2025Binger Chen, Tacettin Emre Bök, Behnood Rasti +2Remote Sensing Foundation Model
Humboldt-Universität zu Berlin, Berlin, Germany · 5U AI, Munich, Germany · Technische Universität Berlin & BIFOLD, Berlin, Germany

Jul 8, 2026, cs.LGJ/K move · Enter open · S save

Scalable and Trustworthy Earth Observation Foundation Models

Syed Usama Imtiaz, Mitra Nasr Azadani, Nasrin Alamdari

Department of Civil and Environmental Engineering, Florida State University

Geospatial Foundation Models Trustworthy Earth Observation Decisions Earth Observation Foundation Model

Open paper AskPDF arXiv

Foundation models (FMs) have transformed machine learning from isolated task-specific model development toward general-purpose models pretrained on broad data and adapted to multiple downstream tasks. Earth observation (EO) is an important domain for this paradigm because satellite and airborne archives are large, high-revisit, and increasingly multimodal, while reliable field labels are often sparse. Remote sensing foundation models (RSFMs) cannot be transferred reliably/optimally without domain-specific adaptation. This is because EO data are governed by measurement physics and operational decision constraints. This chapter reviews the design principles arising from these domain-specific constraints. It first defines the FMs paradigm in remote sensing (RS), then synthesizes the current model landscape, pretraining objectives, architecture designs, downstream adaptation and trustworthiness requirements. The chapter also incorporates recent benchmark evidence showing that no single geospatial foundation model is universally best and that inconsistent evaluation remains a major issue to fair comparison and reliable deployment. In addition, two brief environmental monitoring case studies; physics-informed spectral targeted masking for harmful algal bloom prediction and reinforcement learning for adaptive environmental monitoring station selection to illustrate the FMs domain-guided principles in practice. This chapter posits that next-generation RSFMs should be evaluated not only by benchmark accuracy, but also by modality-aware transfer and physically plausible representations for trustworthy EO decisions.

Abstract

Explore similar work

Scalable and Trustworthy Earth Observation Foundation Models

EarthShift: a benchmark for measuring robustness to real-world distribution shifts in Earth observation

REMSA: Foundation Model Selection for Remote Sensing via a Constraint-Aware Agent