SpectralTrain: A Universal Framework for Hyperspectral Image Classification

Meihua Zhou; Liping Yu; Xinyu Tong; Wai Kin Fung; Ruiguo Hu; Jiarui Zhao; Nan Wan

cs.CVNov 20, 2025

SpectralTrain: A Universal Framework for Hyperspectral Image Classification

Authors: Meihua Zhou, Liping Yu, Xinyu Tong, Wai Kin Fung, Ruiguo Hu, Jiarui Zhao, Nan Wan

Organizations: School of Medical Information, Wannan Medical University, Wuhu, China · University of Chinese Academy of Sciences, Beijing, China · The Chinese University of Hong Kong, Hong Kong, China · Northeastern University, Liaoning, China

Abstract

Hyperspectral image (HSI) classification typically involves large-scale data and computationally intensive training, which limits the practical deployment of deep learning models in real-world remote sensing tasks. This study introduces SpectralTrain, a universal, architecture-agnostic training framework that enhances learning efficiency by integrating curriculum learning (CL) with principal component analysis (PCA)-based spectral downsampling. By gradually introducing spectral complexity while preserving essential information, SpectralTrain enables efficient learning of spectral -- spatial patterns at significantly reduced computational costs. The framework is independent of specific architectures, optimizers, or loss functions and is compatible with both classical and state-of-the-art (SOTA) models. Extensive experiments on three benchmark datasets -- Indian Pines, Salinas-A, and the newly introduced CloudPatch-7 -- demonstrate strong generalization across spatial scales, spectral characteristics, and application domains. The results indicate consistent reductions in training time by 2-7x speedups with small-to-moderate accuracy deltas depending on backbone. Its application to cloud classification further reveals potential in climate-related remote sensing, emphasizing training strategy optimization as an effective complement to architectural design in HSI models. Code is available at https://github.com/mh-zhou/SpectralTrain.

Figures & tables

Figure 1: Overview of the proposed SpectralTrain framework. Left subplot illustrates the generalized curriculum learning (CL) pipeline of existing methods for RGB natural images. A single $\mathcal{T}_{t}(\cdot)$ with spatial Fourier frequency transformation is controlled by the Training Curriculum scheduler, which only regulates the spatial resolution level as training epochs increase from $1$ to $T$ . During training, $\mathcal{T}_{t}(\cdot)$ outputs samples with gradually elevated spatial frequency and image size ( $t=1,t,T$ : Low/Mid/Full Frequency + Small/Medium/Full size) and feeds them into deep networks. Only one single spatial control dimension is involved in this paradigm. Right subplot shows our extended spatio-spectral generalized CL named SpectralTrain tailored for hyperspectral image (HSI) cubes. The transformation module $\mathcal{T}_{t}(\cdot)$ contains two parallel branches: PCA Spectral Recovery and Spatial Downsampling. Correspondingly, the Training Curriculum scheduler provides two independent control signals: Control PCA Stage for tuning the number of retained spectral principal components $k$ , and Control Resolution Stage for adjusting spatial resolution $B$ . As epochs grow, $\mathcal{T}_{t}(\cdot)$ progressively generates HSI samples with rising spectral component quantity and spatial size: $t=1$ (Few PCs + Small size), intermediate epoch $t$ (More PCs + Medium size), and final epoch $T$ (Full PCs + Full size), before sending complex spatio-spectral data to deep networks. Different from RGB-only single spatial curriculum, SpectralTrain synchronously exposes richer spectral and spatial features via dual-dimensional soft-selection transformation, forming a joint spatio-spectral generalized curriculum learning strategy for hyperspectral tasks.

Figure 2: Overview of the dual curriculum learning framework for hyperspectral image classification and training process. Along the training epoch axis (1→t→T), the spectral curriculum increases retained PCA components k, and the spatial curriculum enlarges input patch resolution B. The epoch-adaptive transform $\mathcal{T}_{t}(\cdot)$ controls input complexity. A replay buffer curriculum further reuses historical samples by expanding buffer capacity during training. Transformed samples are sent to deep networks for HSI classification, and three key curriculum rules are listed below the diagram.

Dataset	Data unit	Bands	Classes	$B_{0}$	$(B_{1},B_{2},B_{3})$
Indian Pines	$145\times 145$ scene	200	16	22	$(10,15,22)$
Salinas-A	$86\times 83$ scene	204	6	18	$(8,13,18)$
CloudPatch-7	380 patches from 28 scenes	462	7	30	$(14,21,30)$

Table 1: Dataset-specific configurations of SpectralTrain.

B_{0}

denotes the final patch size, and

(B_{1},B_{2},B_{3})

gives the spatial sizes used in the three curriculum stages.

Stage	Equivalent-budget interval	Active PCs $k_{i}$	$B_{i}/B_{0}$	$T_{\mathrm{ft}}$
1	1–100	10	0.45	10
2	101–200	20	0.70	10
3	201–300	30	1.00	–

Table 2: Shared stage-wise configuration of SpectralTrain. Both conventional training and SpectralTrain use 300 equivalent full-complexity epochs, with

N=3

and

\beta=1

. The proxy length

T_{\mathrm{ft}}

is used only during the one-time curriculum configuration procedure.

Figure 3: Motivation example for a spectral curriculum on Indian Pines. (A) Average spectra of two clusters show band-localized discriminative cues, so uniform band dropping risks removing signals. (B) PCA cumulative explained variance illustrates that a small number of components preserve most energy, motivating an information-preserving low-cost start; PCA is used here as a representative linear compressor. (C) Per-epoch transfer and compute scale with the number of retained components $k$ , so beginning with small $k$ saves early cost. (D) Curriculum stages progressively increase both spectral components and spatial size (PC1/PC2/PC3 composites). Panels illustrate one instantiation with PCA; Table 5 shows that alternative reducers (UMAP/ICA) produce comparable behavior under the same staged schedule.

Settings	OA (%)	AA (%)	Kappa	Time (s/epoch)
Ours (PCA curriculum)	99.02	99.04	0.99	8.00
+ Savitzky–Golay	98.82	98.87	0.99	8.00
+ Gaussian	98.80	98.84	0.99	11.00
+ TV Denoising	98.87	98.91	0.99	8.00

Table 3: Performance with different filters added. Time denotes seconds per epoch.

Model	Current SOTA Training Ravikumar et al. (2022)				SpectralTrain
Model	Time (s/epoch)	OA (%)	AA (%)	Kappa	Time (s/epoch)	OA (%)	AA (%)	Kappa
ResNet-34	113	98.55	98.59	0.98	66	98.20	98.05	0.98
ADGAN	664	98.96	98.91	0.98	298	98.28	98.10	0.98
MetaFormer	77	98.93	98.98	0.99	17	98.95	98.94	0.99
ConvNeXt-T	269	99.06	99.14	0.99	94	99.15	99.18	0.99
SQS	356	99.30	99.30	0.99	254	98.17	98.11	0.98

Table 4: Comparison of conventional training and SpectralTrain using PCA downsampling. All models share the same splits and hyperparameters within each column group. Times measured on the same hardware; seeds fixed unless stated.

Dimensionality Reduction Method	OA (%)	AA (%)	Kappa	Time (s/epoch)
PCA	99.15	99.18	0.99	8.00
UMAP	99.36	99.39	0.99	8.00
ICA	98.89	98.86	0.99	8.00

Table 5: Performance with different dimensionality reduction methods.

Loss Function	OA (%)	AA (%)	Kappa	Time (s/epoch)
Cross Entropy Loss	98.96	99.01	0.99	8.00
Multi-Label Soft Margin Loss	98.96	98.97	0.99	8.00
Binary Cross Entropy with Logits	99.02	99.04	0.99	8.00

Table 6: Performance and efficiency of different loss functions.

Optimizer	OA (%)	AA (%)	Kappa	Time (s/epoch)
AdamW	99.15	99.18	0.99	8.00
AdamMini	98.95	98.99	0.99	8.00
SGD	98.82	98.83	0.99	8.00

Table 7: Performance with different optimizers.

Epoch	OA (%)	AA (%)	Kappa
50	98.67	98.67	0.99
100	98.80	98.88	0.99
200	98.87	98.93	0.99
300	99.15	99.18	0.99
400	98.98	98.98	0.99
500	98.86	98.94	0.99

Table 8: Performance comparison across different epochs.

#	Settings	OA (%)	AA (%)	Kappa
S1	Seed = 1	98.87	98.85	0.98
S2	Seed = 10	98.80	98.87	0.98
S3	Seed = 42	98.96	98.99	0.99
S4	Seed = 123	98.96	98.94	0.99
S5	Seed = 666	98.83	98.92	0.98
S6	Mean $\pm$ Std	$98.88\pm 0.07$	$98.91\pm 0.06$	$0.984\pm 0.005$

Table 9: Robustness results under different random seeds and train-test splits. S1-S6: Different Random Seeds (fixed 80% split). D1-D6: Different Training Ratios (fixed seed=42)

Model	OA (%)	AA (%)	Kappa
ADGAN	97.37	94.59	0.97
MetaFormer	94.34	93.31	0.94
ConvNeXt-T	98.72	99.12	0.99
SQS	94.02	97.30	0.94
3D-ConvSST	90.95	84.56	0.91
RCNN	98.33	98.71	0.98

Table 10: Performance comparison on Indian Pines dataset.

Model	OA (%)	AA (%)	Kappa
ADGAN	92.69	94.23	0.93
MetaFormer	94.10	94.22	0.94
ConvNeXt-T	98.09	97.86	0.98
SQS	95.02	95.37	0.95
3D-ConvSST	97.35	96.65	0.97
RCNN	98.09	98.02	0.98

Table 11: Performance comparison on Salinas-A dataset.

Model	OA (%)	AA (%)	Kappa
ADGAN	98.28	98.10	0.98
MetaFormer	98.95	98.94	0.99
ConvNeXt-T	99.15	99.18	0.99
SQS	98.17	98.11	0.98
3D-ConvSST	99.02	99.04	0.99
RCNN	99.44	99.42	0.99

Table 12: Performance comparison on CloudPatch-7 dataset.

Panel of Figure 4 — Figure 4: Backbone-scale performance under SpectralTrain.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Supplementary Figure S1. Class distribution in CloudPatch - 7. Bars show per-class sample counts (c01–c07), revealing moderate imbalance.

Figure 6: Supplementary Figure S2. Spectral reflectance curves. Left (a): class-wise mean spectra; Right (b): a representative single-pixel spectrum. Axes are reflectance (vertical) and wavelength index (horizontal).

Figure 7: Supplementary Figure S3. Average reflectance spectra for the seven classes (c01–c07). Distinct spectral shapes and band-localized differences provide discriminative cues.

Figure 8: Supplementary Figure S4. Dense overlay of per-sample spectra. Overlaid curves emphasize intra-class variability and contiguous, instrument-governed spectral ordering.

Hyperspectral Image Classification

Hyperspectral image classification assigns land-cover, material, or biological labels using the spectral and spatial information in image pixels.

See all 53 papers

MixerSENet: A Lightweight Framework for Efficient Hyperspectral Image Classification
Jun 1, 2026Mohammed Q. Alkhatib, Swalpa Kumar Roy, Ali JamaliEfficient Neural Network Inference Hyperspectral Imaging
College of Engineering and IT, University of Dubai, Dubai, 14143, UAE · Department of Computer Science and Engineering, Alipurduar Government Engineering and Management College, West Bengal 736206, India · Department of Geography, Simon Fraser University, British Columbia 8888, Canada
VP-Hype: A Hybrid Mamba-Transformer Framework with Visual-Textual Prompting for Hyperspectral Image Classification
Mar 1, 2026Abdellah Zakaria Sellam, Fadi Abdeladhim Zidi, Salah Eddine Bekhouche +4Hybrid Mamba-Transformer Architectures Hyperspectral Image Classification
Institute of Applied Sciences and Intelligent Systems (ISASI), CNR, 73100 Lecce, Italy · Dept. of Engineering for Innovation, University of Salento, 73100 Lecce, Italy · VSC Laboratory, Department of Electronics and Automation, University of Biskra, Biskra, Algeria +4
A PyTorch Library for Hyperspectral Image Models: Technical Report
Sep 30, 2026Tanishq Rachamalla, Aryan Das, Srishti Kaushik +1Remote Sensing Benchmarks Hyperspectral Image Classification
Department of Information Technology Siddhartha Academy of Higher Education Vijayawada, Andhra Pradesh 521108, India · Department of Computer Science and Engineering Vellore Institute of Technology Bhopal, Madhya Pradesh 466114, India · Department of Computer and Information Sciences Indira Gandhi National Open University New Delhi 110068, India +1

Jun 1, 2026J/K move · Enter open · S save

MixerSENet: A Lightweight Framework for Efficient Hyperspectral Image Classification

Mohammed Q. Alkhatib, Swalpa Kumar Roy, Ali Jamali

College of Engineering and IT, University of Dubai, Dubai, 14143, UAE · Department of Computer Science and Engineering, Alipurduar Government Engineering and Management College, West Bengal 736206, India · Department of Geography, Simon Fraser University, British Columbia 8888, Canada

Efficient Neural Network Inference Hyperspectral Imaging Convolutional Neural Networks Hyperspectral Image Classification

Open paper AskPDF arXiv

In this paper, a novel framework, MixerSENet, is introduced for hyperspectral image (HSI) classification, designed to address the challenges of computational efficiency and limited labeled data. The proposed model processes hyperspectral image patches while maintaining consistent size and resolution throughout the network, effectively decoupling the mixing of spatial and channel dimensions. Notably, MixerSENet is lightweight and computationally efficient, requiring fewer parameters compared to traditional models, making it suitable for resource-constrained environments. A squeeze and excitation block is incorporated into the model to refine feature extraction, enhancing the network's ability to capture more informative features. Experimental results on two benchmark datasets demonstrate that MixerSENet achieves superior performance, reaching an overall accuracy (OA) of 82.47% on Houston13 dataset and 96.70% on the Qingyun dataset, outperforming state-of-the-art methods including 3D-CNN, HybridKAN, HSIFormer, SimPoolFormer, and MorphMamba. Furthermore, a detailed analysis of computational efficiency shows that MixerSENet achieves a favorable balance between accuracy and efficiency, with only 53,146 parameters and an low inference time, confirming its practicality for real-world applications. At publication, source code will be publicly available at https://github.com/mqalkhatib/MixerSENet.

Abstract

Hyperspectral Image Classification

Explore similar work

MixerSENet: A Lightweight Framework for Efficient Hyperspectral Image Classification

VP-Hype: A Hybrid Mamba-Transformer Framework with Visual-Textual Prompting for Hyperspectral Image Classification

A PyTorch Library for Hyperspectral Image Models: Technical Report