Organizations: Institute of Applied Sciences and Intelligent Systems (ISASI), CNR, 73100 Lecce, Italy · Dept. of Engineering for Innovation, University of Salento, 73100 Lecce, Italy · VSC Laboratory, Department of Electronics and Automation, University of Biskra, Biskra, Algeria · Dept. of Computer Science and AI, University of the Basque Country (UPV/EHU), Spain · VSC Laboratory, Department of Electronics and Automation, University of Biskra, Algeria · Institut Galilée, Université Sorbonne Paris Nord, F-93430, Villetaneuse, France · Sorbonne Center for Artificial Intelligence, Sorbonne University Abu Dhabi, Abu Dhabi, UAE
Accurate classification of hyperspectral imagery (HSI) is often frustrated by the tension between high-dimensional spectral data and the extreme scarcity of labeled training samples. While hierarchical models like LoLA-SpecViT have demonstrated the power of local windowed attention and parameter-efficient fine-tuning, the quadratic complexity of standard Transformers remains a barrier to scaling. We introduce VP-Hype, a framework that rethinks HSI classification by unifying the linear-time efficiency of State-Space Models (SSMs) with the relational modeling of Transformers in a novel hybrid architecture. Building on a robust 3D-CNN spectral front-end, VP-Hype replaces conventional attention blocks with a Hybrid Mamba-Transformer backbone to capture long-range dependencies with significantly reduced computational overhead. Furthermore, we address the label-scarcity problem by integrating dual-modal Visual and Textual Prompts that provide context-aware guidance for the feature extraction process. Our experimental evaluation demonstrates that VP-Hype establishes a new state of the art in low-data regimes. Specifically, with a training sample distribution of only 2%, the model achieves Overall Accuracy (OA) of 99.69% on the Salinas dataset and 99.45% on the Longkou dataset. These results suggest that the convergence of hybrid sequence modeling and multi-modal prompting provides a robust path forward for high-performance, sample-efficient remote sensing.
Figures & tables
Figure 1: Comparison of performance metrics across nine hyperspectral image (HSI) classification models. All scores are normalized relative to the proposed VP-Hype, which demonstrates superior performance across all datasets (IP, UP, SA, LK, HH, HC), overall accuracy metrics (AA, Kappa), and computational efficiency.
Figure 2: Architecture of the proposed VP-Hype framework. The model combines a hybrid Mamba–Transformer backbone with visual–textual prompting for hyperspectral image classification. Task-specific text prompts encoded by CLIP and learnable visual prompts are fused via Text Conditional Spatial Prompt (TCSP) blocks and injected at multiple network stages. The prompt-enhanced features are progressively downsampled and finally passed to a classification head for prediction.
Figure 3: Visualization of hyperspectral data cubes and corresponding ground-truth classification maps for the WHU - Hi - HongH, WHU - Hi - Longkou, and Salinas datasets: (a) hyperspectral image cube; (b) ground - truth map.
Metric
Ours
HSIMAE
HybridSN
ViT
MASSFormer
DBCT
LSGA
OA (%)
99.64
96.24
96.76
96.76
98.14
99.14
99.02
AA (%)
99.23
92.99
92.50
92.50
96.25
98.07
97.54
Kappa (%)
99.55
96.18
96.77
96.77
97.65
98.91
98.75
Table 1: Overall performance comparison on the HongHu dataset (10% training samples).
Metric
Ours
LoLA
MADANet
A 2 MFE
HybridSN
SSFTT
HiT
LANet
OA (%)
99.99
99.87
99.17
98.56
97.05
98.61
97.83
96.67
AA (%)
99.99
99.71
99.12
98.72
97.52
98.97
98.87
97.12
Kappa (%)
99.99
99.49
98.34
97.35
96.72
98.46
97.58
96.39
Table 2: Overall performance comparison on the Salinas dataset (10% training samples).
Metrics
Ours
Cross
Hybrid-ViT
Hir-Trans.
SSMamba
MorpMamba
E-SR-SSIM
LOLA
OA (%)
99.95
99.78
99.75
99.68
99.51
99.70
94.43
99.92
AA (%)
99.83
99.45
99.36
99.14
98.45
99.25
81.78
99.73
Kappa (%)
99.81
99.78
99.68
99.59
99.36
99.61
92.63
99.89
Table 3: Overall, average, and kappa accuracies (%) on the Longkou dataset (10% training samples).
Class
ViT
PiT
HiT
GAHT
AMHFN
Ours
1
89.7
89.4
89.8
95.9
95.9
99.99
2
63.6
87.4
90.2
94.8
95.3
98.82
3
75.1
48.2
88.0
94.4
96.2
98.99
4
90.1
89.6
90.6
95.6
96.3
99.84
5
79.0
76.8
91.6
95.8
94.5
98.48
6
96.4
95.5
96.1
97.9
98.1
99.82
Table 4: Classification accuracy (%) on Longkou dataset (2% training). Best results in bold.
Dataset
Train %
OA (%)
AA (%)
Kappa (%)
Longkou
2
99.45
99.20
99.28
Salinas
2
99.69
99.78
99.65
QUH-Qingyun
10
99.34
99.18
99.13
Table 5: MambaVision-T performance across three datasets with limited training samples.
Dataset
Configuration
Visual
Text
Prompt
OA (%)
AA (%)
Kappa (%)
Longkou
Visual-Only
✓
✗
✓
99.14
97.23
98.88
Text-Only
✗
✓
✓
98.30
99.00
98.10
No Prompt
✓
✓
✗
99.26
97.55
99.02
Full Model (Ours)
✓
✓
✓
99.45
99.20
99.28
Salinas
Visual-Only
✓
✗
✓
98.35
99.03
98.16
Text-Only
✗
✓
✓
98.30
99.00
98.10
Table 6: Ablation Study Results with 2% Training Samples
Dataset
Configuration
Visual
Text
Prompt
OA (%)
AA (%)
Kappa (%)
Longkou
Visual-Only
✓
✗
✓
99.52
99.47
99.46
Text-Only
✗
✓
✓
99.59
99.37
99.13
No Prompt
✓
✓
✗
99.24
99.18
99.09
Full Model (Ours)
✓
✓
✓
99.72
99.59
99.66
Salinas
Visual-Only
✓
✗
✓
99.84
99.82
99.82
Text-Only
✗
✓
✓
99.75
99.76
99.72
Table 7: Ablation Study Results with 10% Training Samples
Figure 4: Comparative classification performance on standard hyperspectral benchmarks: (I) Salinas, (II) WHU-Hi-LongKou, and (III) WHU-Hi-HongHu datasets. Ground truth and method predictions are presented with challenging regions highlighted (red boxes).
Textual Prompts
Visual Prompts
OA (%)
AA (%)
Kappa (%)
None (Baseline)
None
98.28
99.05
98.08
4 Prompts
None
99.31
99.54
99.23
None
4 Prompts
98.82
99.31
98.68
2 Prompts
2 Prompts
99.43
99.60
99.36
4 Prompts
4 Prompts
99.69
99.78
99.65
Table 8: Ablation study on the influence of the number of textual and visual prompts on VP-Hype performance (Salinas Dataset, 2% training samples).
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Dataset / Setting
Params (M)
Madanet [ 12 ]
Salinas
0.16
LANet [ 13 ]
Salinas
5.3
PiT-B [ 42 ]
Longkou (2%)
73.8
ViT-Huge [ 43 ]
Longkou (2%)
632.0
VP-Hype (ours)
All datasets
77.30
Appendix
Table B.1: Parameter counts of representative models, comparing compact CNNs, large-scale transformers, and the proposed VP-Hype, which achieves a favourable balance between model expressivity and computational footprint.
Class
Ours
LoLA
MADANet
A 2 MFE
HybridSN
SSFTT
HiT
LANet
OA (%)
99.99
99.87
99.17
98.56
97.05
98.61
97.83
96.67
AA (%)
99.99
99.71
99.12
98.72
97.52
98.97
98.87
97.12
Kappa (%)
99.99
99.49
98.34
97.35
96.72
98.46
97.58
96.39
Brocoli green weeds 1
100.00
100.00
99.64
99.46
99.30
99.34
100.00
98.32
Brocoli green weeds 2
100.00
100.00
98.95
97.28
99.96
100.00
99.75
98.64
Fallow
100.00
100.00
98.90
97.99
98.16
99.89
100.00
95.09
Appendix
Table B.2: Overall, average, kappa, and per-class accuracies (%) on the Salinas dataset (10% training samples).
Class
Ours
HSIMAE
HybridSN
ViT
MASSFormer
DBCT
LSGA
OA (%)
99.64
96.24
96.76
96.76
98.14
99.14
99.02
AA (%)
99.23
92.99
92.50
92.50
96.25
98.07
97.54
Kappa (%)
99.55
96.18
96.77
96.77
97.65
98.91
98.75
Red roof
99.84
97.40
98.46
97.84
98.99
98.27
99.54
Road
99.71
88.87
75.45
87.22
97.25
97.44
96.82
Bare soil
99.59
95.84
95.61
94.98
98.87
99.01
98.23
Appendix
Table B.3: Overall, average, kappa, and per-class accuracies (%) on the HongHu dataset (10% training samples).
College of Engineering, Shantou University, Shantou, 515063, Guangdong, China · School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan, 430074, Hubei, China · Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun, 130033, Jilin, China +2