Spectral Gaps and Spatial Priors: Studying Hyperspectral Downstream Adaptation Using TerraMind

Julia Anna Leonardi; Johannes Jakubik; Paolo Fraccaro; Maria Antonia Brovelli

cs.CVMar 4, 2026

Spectral Gaps and Spatial Priors: Studying Hyperspectral Downstream Adaptation Using TerraMind

Authors: Julia Anna Leonardi, Johannes Jakubik, Paolo Fraccaro, Maria Antonia Brovelli

Organizations: Department of Civil and Environmental Engineering, Politecnico di Milano, Milan, Italy · IBM Research Europe, Rüschlikon, Switzerland

Abstract

Geospatial Foundation Models (GFMs) typically lack native support for Hyperspectral Imaging (HSI) due to the complexity and sheer size of high-dimensional spectral data. This study investigates the adaptability of TerraMind, a multimodal GFM, to address HSI downstream tasks without HSI-specific pretraining. Therefore, we implement and compare two channel adaptation strategies: Naive Band Selection and physics-aware Spectral Response Function (SRF) grouping. Overall, our results indicate a general superiority of deep learning models with native support of HSI data. Our experiments also demonstrate the ability of TerraMind to adapt to HSI downstream tasks through band selection with moderate performance decline. Therefore, the findings of this research establish a critical baseline for HSI integration, motivating the need for native spectral tokenization in future multimodal model architectures.

Figures & tables

Figure 1: Experimental framework for benchmarking channel adaptation strategies (Naive Selection vs. SRF Grouping) using the TerraMind backbone on four HSI-specific applications.

		Segmentation Tasks (mIoU $\uparrow$ )			Regression (nMSE $\downarrow$ )
Model	Adaptation	EnMAP-BNETD	EnMAP-CDL	EnMAP-BDForet	Hyperview-1
		(Easy)	(Moderate)	(Hard)	(V. Hard)
TerraMind	Naive Selection	0.465 $\pm$ 0.002	0.693 $\pm$ 0.006	0.657 $\pm$ 0.007	0.813 (Rank #6)
TerraMind	SRF Grouping	0.461 $\pm$ 0.003	0.679 $\pm$ 0.006	0.623 $\pm$ 0.006	0.831 (Rank #25)
SpectralEarth	None (Full HSI)	0.495 $\pm$ 0.001	0.774 $\pm$ 0.003	0.766 $\pm$ 0.005	0.810 (Rank #5)
(Upper Bound)	None (Full HSI)	0.495 $\pm$ 0.001	0.774 $\pm$ 0.003	0.766 $\pm$ 0.005	0.810 (Rank #5)

Table 1: Comparative performance on downstream tasks (mean

\pm

std over 10 runs). Metrics are mIoU for segmentation (higher is better) and Normalized MSE for regression (lower is better). Bold indicates the best performing model; underline indicates the best adaptation strategy for TerraMind.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 2: Prediction examples of TerraMind on the EnMAP-BNETD dataset. Compared to SRF Grouping (right), Naive Band Selection (center) better preserves fine spatial structures.

Figure 3: Prediction examples of TerraMind on the EnMAP-CDL dataset using both band sampling techniques. Note that the background class was set as the ignore index during training.

Figure 4: Prediction examples on EnMAP-BDFORET . The spectral similarity of tree species leads to visible class confusion in the SRF predictions (right), whereas Naive Band Selection (center) achieves better discrimination between pine subtypes. Note the extensive background regions (grey), which were set as the ignore index during training.

Explore similar work

MBTI: A Multi-Branch Efficient Fine-Tuning Framework for Hyperspectral Image Classification with Foundation Models
Jul 14, 2026Mingzhen Xu, Haonan Guo, Di Wang +9Hyperspectral Image Classification Parameter-Efficient Fine-Tuning
School of Computer Science, Wuhan University, Wuhan, China · State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan, China · China Petroleum Pipeline Engineering Corporation, Langfang, China +6
SpectralEarth-FM: Bringing Hyperspectral Imagery into Multimodal Earth Observation Pretraining
May 20, 2026Nassim Ait Ali Braham, Aaron Banze, Conrad M. Albrecht +3Geospatial Foundation Models Multimodal Pretraining
Chair of Data Science in Earth Observation, Technical University of Munich, Germany · Remote Sensing Technology Institute, German Aerospace Center (DLR), Germany · Department of Aerospace Engineering, University of the Bundeswehr Munich, Germany +3
SPECTRA: Band-Routed Embedding and Stage-Wise LoRA for Cross-Sensor Fine-Tuning of Geospatial Foundation Models
Aug 3, 2026Xingyan Li, Jordan A. Caraballo-Vega, Jie Gong +2Remote Sensing Image Segmentation Geospatial Foundation Models
University of Maryland, Baltimore County Baltimore, Maryland, USA · NASA Goddard Space Flight Center Greenbelt, Maryland, USA

Jul 14, 2026J/K move · Enter open · S save

MBTI: A Multi-Branch Efficient Fine-Tuning Framework for Hyperspectral Image Classification with Foundation Models

Mingzhen Xu, Haonan Guo, Di Wang, Yinghua Qu+8

School of Computer Science, Wuhan University, Wuhan, China · State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan, China · China Petroleum Pipeline Engineering Corporation, Langfang, China+6

Hyperspectral Image Classification Parameter-Efficient Fine-Tuning Modular Neural Networks

Open paper AskPDF arXiv

Hyperspectral foundation models learn transferable spectral-spatial representations from large-scale unlabeled data. They provide an effective paradigm for adapting to downstream hyperspectral image (HSI) classification tasks with limited labeled samples. However, spectral band configurations vary substantially across sensors, which makes direct model transfer difficult. Existing adaptation strategies often compress, select, or reshape the original spectra to match model-specific input requirements. These operations may discard useful spectral information and weaken local spectral continuity. To address this problem, we propose MBTI, a Multi-Branch efficient fine-tuning framework for Hyperspectral Image classification. MBTI adapts hyperspectral foundation models to downstream classification tasks while preserving full-band spectral information. First, we introduce a spectral-continuity-preserving multi-branch preprocessing strategy. The original HSI is divided into multiple continuous spectral subsets, and a band reuse mechanism is used when the remaining bands cannot form a complete branch. This avoids invalid padding and unnecessary spectral loss. Second, independent Low-Rank Adaptation (LoRA) modules are inserted into each branch. They enable different spectral intervals to learn task-specific discriminative features while keeping most pre-trained parameters frozen. Finally, a multi-branch channel attention fusion module adaptively recalibrates and integrates features from all spectral branches. Experiments on three public hyperspectral datasets show that MBTI achieves competitive and superior performance compared with representative classification methods. Under the final rank-8 configuration, only about 2.33%--2.36% of the parameters are trainable. The code will be available at https://github.com/Azhenmiddleblock/MBTI/tree/main.

Abstract

Explore similar work

MBTI: A Multi-Branch Efficient Fine-Tuning Framework for Hyperspectral Image Classification with Foundation Models

SpectralEarth-FM: Bringing Hyperspectral Imagery into Multimodal Earth Observation Pretraining

SPECTRA: Band-Routed Embedding and Stage-Wise LoRA for Cross-Sensor Fine-Tuning of Geospatial Foundation Models