Brewing Stronger Features: Dual-Teacher Distillation for Multispectral Earth Observation

Filip Wolf; Blaž Rolih; Luka Čehovin Zajc

cs.CVFeb 23, 2026

Brewing Stronger Features: Dual-Teacher Distillation for Multispectral Earth Observation

Authors: Filip Wolf, Blaž Rolih, Luka Čehovin Zajc

Organizations: University of Ljubljana, Faculty of Computer and Information Science, Slovenia

Abstract

Foundation models are transforming Earth Observation (EO), yet the diversity of EO sensors and modalities makes a single universal model unrealistic. Multiple specialized EO foundation models (EOFMs) will likely coexist, making efficient knowledge transfer across modalities essential. Most existing EO pretraining relies on masked image modeling, which emphasizes local reconstruction but provides limited control over global semantic structure. To address this, we propose a dual-teacher contrastive distillation framework for multispectral imagery that aligns the student's pretraining objective with the contrastive self-distillation paradigm of modern optical vision foundation models (VFMs). Our approach combines a multispectral teacher with an optical VFM teacher, enabling coherent cross-modal representation learning. Experiments across diverse optical and multispectral benchmarks show that our model adapts to multispectral data without compromising performance on optical-only inputs, achieving state-of-the-art results in both settings, with an average improvement of 3.64 percentage points in semantic segmentation, 1.2 in change detection, and 1.31 in classification tasks. This demonstrates that contrastive distillation provides a principled and efficient approach to scalable representation learning across heterogeneous EO data sources. Project page: \textcolor{magenta}{https://wolfilip.github.io/DEO/}.

Figures & tables

Figure 1: DEO , our proposed dual-teacher pretraining approach, results in a model that achieves state-of-the-art performance in multispectral EO tasks while maintaining performance on optical EO tasks. On top, we demonstrate our performance in optical and multispectral semantic segmentation, visualizing model size using colored circles. Below, the first row of images shows qualitative results for the optical SpaceNetv1 [ 53 ] dataset, while the second row shows results for the multispectral m-SA-crop-type dataset [ 33 ] .

Figure 2: Overview of our dual-teacher pretraining approach. The pretraining dataset utilizes a standard fMoW Sentinel-2 dataset, augmented with high-resolution aerial images where possible. Random crops and other augmentation operations are performed on sampled images. Full multispectral and optical channel subsets are used for their corresponding distillation branches. The multispectral branch is a contrastive learning setup where the teacher is updated using EMA. In the optical branch, distillation is done using a frozen VFM teacher. The resulting model can then be used in various downstream tasks.

Figure 3: PCA feature visualization and comparison between Copernicus-FM [ 57 ] , DINOv3-LS [ 50 ] , and DEO (ours). We note the similarity of our method’s features to those of DINOv3. We provide additional alignment experiments in the Supplementary.

Panel of Table 1 — Table 1: Results on optical and multispectral segmentation datasets . Methods take as input either 3-channel optical ( RGB ) bands or multi-channel MS bands. The GB prefix indicates that the dataset is from the GEO-Bench [ 33 ] benchmark, SN represents SpaceNetv1 [ 53 ] , S1F11 is Sen1Floods11 [ 9 ] , and PASTIS is from [ 22 ] . DINOv3-LS is the ViT-L version of DINOv3 pretrained on satellite imagery. All results are expressed in macro mIoU. First and second place results are marked. Further dataset details are available in the Supplementary.

Panel of Table 2 — Table 2: Results on optical and multispectral bi-temporal change detection datasets . Methods take as input either 3-channel optical ( RGB ) bands or multi-channel MS bands. All results are expressed in binary F1 score considering only the change class. First and second place results are marked.

Panel of Table 3 — Table 3: Results on multispectral classification datasets with linear probing . Methods take as input either 3-channel optical ( RGB ) bands or multi-channel MS bands. The GB prefix indicates that the dataset is from the GEO-Bench [ 33 ] benchmark. GB-ben (m-bigearthnet) results are expressed in the F1 metric, while GB-s2s (m-so2sat) and GB-es (m-eurosat) results are expressed in Top-1 accuracy. First and second place results are marked.

Panel of Table 4 — Table 4: Summary of methods and results. Ranks for all tested methods over benchmark datasets, averaged over optical datasets, multispectral (MS) datasets, and overall. First and second place results are marked. Additionally, model size and pretraining corpus size are provided for reference.

Table 8

Panel of Table 7 — Table 7: Results in a low-data regime (10%) . Methods take as input either 3-channel optical ( RGB ) bands or multi-channel MS bands. The GB prefix indicates that the dataset is from the GEO-Bench [ 33 ] benchmark. GB-SA-c. and SpacenNetv1 [ 53 ] results are expressed in macro mIoU, while GB-BEN results are expressed in F1. First and second place results are marked.

Figure 4: Qualitative results for semantic segmentation. The first two columns contain the optical part of the input image and the ground truth. The final three columns contain predictions from two related models, and our own.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Value
$n$	2
$m$	10
$\alpha_{1}$	1
$\alpha_{2}$	0.5
$\alpha_{3}$	0.5
$\gamma$	1

Appendix

Table 8: Details of pretraining hyperparameters.

Element	Value
Swin
Input patch size	4
Embedding dim.	128
Windows size	12
Projection head
Hidden dim.	2048

Appendix

Table 9: Network element sizes.

Segmentation
GEO-Bench [ 33 ]	In paper	Image Size	# Classes	Train	Val	Test	# Bands	RGB res	Sensors
m-pv4ger-seg	GB-pv	320 $\times$ 320	2	3000	403	403	3	0.1	RGB
m-chesapeake-landcover	GB-chesa.	256 $\times$ 256	7	3000	1000	1000	4	1.0	RGBN
m-cashew-plantation	GB-cas.	256 $\times$ 256	7	1350	400	50	13	10.0	Sentinel-2
m-SA-crop-type	GB-SA-c.	256 $\times$ 256	10	3000	1000	1000	13	10.0	Sentinel-2
m-nz-cattle	GB-cattle	500 $\times$ 500	2	524	66	65	3	0.1	RGB

Appendix

Table 10: Details for segmentation datasets used in the paper for evaluation.

Classification
GEO-Bench [ 33 ]	In paper	Image Size	# Classes	Train	Val	Test	# Bands	RGB res	Sensors
m-bigearthnet	GB-ben	120 $\times$ 120	43	20000	1000	1000	12	10.0	Sentinel-2
m-so2sat	GB-s2s	32 $\times$ 32	17	19992	986	986	18	1.0	Sen.-2 + Sen.-1
m-eurosat	GB-es	64 $\times$ 64	10	2000	1000	1000	13	10.0	Sentinel-2

Appendix

Table 11: Details for classification datasets used in the paper for evaluation.

Change detection
GEO-Bench [ 33 ]	In paper	Image Size	# Classes	Train	Val	Test	# Bands	RGB res	Sensors
LEVIR-CD [ 12 ]	LEVIR	256 $\times$ 256	2	7120	1024	2048	3	0.5	Google Earth satellite
OSCD [ 17 ]	OSCD	96 $\times$ 96	2	827	-	385	10	10.0	Sentinel 2

Appendix

Table 12: Details for change detection datasets used in the paper for evaluation.

Parameter	Value
Pool scales	1, 2, 3, 6
Hidden size	512

Appendix

Table 13: Details of the UPerNet segmentation head.

	LR
Dataset	Other methods	SatDiFuser
m-pv4ger-seg	$10^{-4}$	$10^{-2}$
m-chesapeake-landcover	$10^{-4}$	$10^{-2}$
m-cashew-plantation	$10^{-2}$	$10^{-2}$
m-SA-crop-type	$10^{-4}$	$10^{-2}$
m-nz-cattle	$10^{-4}$	$10^{-3}$

Appendix

Table 14: Learning rates for datasets and methods.

Size	Model	Optical	MS	Overall
Large models	DEO (contrast.)	81.98	57.45	69.72
	DEO (MIM)	78.18	56.39	67.28
	DEO (no CR)	82.02	54.47	68.25
Small models	DEO (Swin-T)	79.88	55.45	67.67
	DEO (ViT-S p8)	76.62	52.09	64.35
	DEO (shift)	79.86	55.56	67.71

Appendix

Table 15: Additional experiments.

Model	img/s	Mem. [MB]	TFLOPS	kgCO2eq
DEO (Swin- B )	5.917	5272	10475	34.5
DINOv3- B	9.80	5595	9823	18000
DEO (ViT- S p8)	2.43	18152	31302

Appendix

Table 16: Inference experiments.

Figure 5: Extended qualitative results for Sen1Floods11.

Figure 6: Extended qualitative results for m-SA-crop-type.

Figure 7: Extended qualitative results for SpaceNetv1.

Figure 8: Extended qualitative results for LEVIR and OSCD.

Remote Sensing Image Understanding

Remote sensing image understanding uses AI to interpret satellite and aerial imagery for objects, land cover, change, hazards, and spatial relationships.

See all 208 papers

SpectralEarth-FM: Bringing Hyperspectral Imagery into Multimodal Earth Observation Pretraining
May 20, 2026Nassim Ait Ali Braham, Aaron Banze, Conrad M. Albrecht +3Geospatial Foundation Models Multimodal Pretraining
Chair of Data Science in Earth Observation, Technical University of Munich, Germany · Remote Sensing Technology Institute, German Aerospace Center (DLR), Germany · Department of Aerospace Engineering, University of the Bundeswehr Munich, Germany +3
UniverSat: Resolution- and Modality-Agnostic Transformers for Earth Observation
Jun 22, 2026Yohann Perron, Guillaume Astruc, Nicolas Gonthier +2Geospatial Representation Learning Vision Transformer
LIGM, Ecole Nationale des Ponts et Chaussées, IP Paris, Univ Gustave Eiffel, CNRS · EFEO · LASTIG, Univ Gustave Eiffel, IGN, ENSG +2
Earth-OneVision: Extending Remote Sensing Multimodal Large Language Models to More Sensor Modalities and Tasks
Jun 9, 2026Miaoxin Cai, Guanqun Wang, Wei Zhang +6Multimodal Understanding Multimodal Grounding
National Key Laboratory of Science and Technology on Space-Born Intelligent Information Processing (SBIIP), Beijing Institute of Technology, Beijing 100081, China · Advanced Research Institute of Multidisciplinary Sciences, Beijing Institute of Technology, Beijing 100081, China · School of Mechatronical Engineering, Beijing Institute of Technology, Beijing 100081, China +5

May 20, 2026J/K move · Enter open · S save

SpectralEarth-FM: Bringing Hyperspectral Imagery into Multimodal Earth Observation Pretraining

Nassim Ait Ali Braham, Aaron Banze, Conrad M. Albrecht, Julien Mairal+2

Chair of Data Science in Earth Observation, Technical University of Munich, Germany · Remote Sensing Technology Institute, German Aerospace Center (DLR), Germany · Department of Aerospace Engineering, University of the Bundeswehr Munich, Germany+3

Geospatial Foundation Models Multimodal Pretraining Hyperspectral Imaging Multimodal Sensor Fusion

Open paper AskPDF arXiv

Earth observation (EO) foundation models (FMs) are increasingly trained on multisensor data, spanning multispectral imagery (MSI), synthetic aperture radar (SAR), and derived geospatial layers, but hyperspectral imagery (HSI) remains underrepresented. Conversely, existing hyperspectral FMs are trained on HSI alone, leaving joint pretraining and fusion of HSI with co-located EO sensors unexplored. We introduce SpectralEarth-FM, a hierarchical transformer for multisensor EO input with heterogeneous spectral dimensionality. The architecture combines spectral tokenization for hyperspectral inputs, sensor-specific encoders, a cross-sensor fusion module, and a shared hierarchical encoder, enabling joint processing of HSI and lower-channel observations. To pretrain SpectralEarth-FM, we curate SpectralEarth-MM, a dataset that co-locates HSI from three spaceborne sensors (EnMAP, EMIT, DESIS) with Sentinel-2, Landsat-8/9 optical imagery, Landsat land surface temperature (LST), and Sentinel-1 SAR, over common geographic footprints. It comprises approximately 2M globally distributed locations, 25M georeferenced patches, and over 40TB of data. Pretraining uses a Joint-Embedding Predictive Architecture (JEPA)-style objective that matches representations between global views and single-sensor local views from the same location. We evaluate SpectralEarth-FM on hyperspectral downstream tasks and standard EO benchmarks following the PANGAEA protocol, achieving state-of-the-art results across both evaluation settings.

Abstract

Remote Sensing Image Understanding

Explore similar work

SpectralEarth-FM: Bringing Hyperspectral Imagery into Multimodal Earth Observation Pretraining

UniverSat: Resolution- and Modality-Agnostic Transformers for Earth Observation

Earth-OneVision: Extending Remote Sensing Multimodal Large Language Models to More Sensor Modalities and Tasks