Convolution-Free Holistic Multivariance Decomposition Layer for Efficient Hyperspectral Image Classification Tensor Networks

Süha Tuna; Ülker Başar

cs.CVAug 17, 2026

Convolution-Free Holistic Multivariance Decomposition Layer for Efficient Hyperspectral Image Classification Tensor Networks

Authors: Süha Tuna, Ülker Başar

Organizations: Istanbul Technical University, Informatics Institute, Maslak 34469, İstanbul, Türkiye · Istanbul Esenyurt University, Faculty of Business and Management Sciences, Department of Management Information Systems, Esenyurt, 34517, İstanbul, Türkiye

Abstract

Feature extraction for hyperspectral image classification is conventionally addressed using rigid tensor decompositions that fail to capture complex spatio-spectral interdependencies, or heavily parameterized convolutional neural networks that are computationally expensive. To overcome these limitations, this work introduces the Holistic Multivariance Decomposition (HMD) framework as a novel, end-to-end differentiable neural network layer. By explicitly separating independent single mode variations from cooperative higher dimensional interactions via learnable, matrix valued supports, the proposed HMD-0, HMD-1 and HMD-2 approximants are optimized jointly with a downstream classifier via backpropagation. Comprehensive evaluations across three benchmark HS datasets demonstrate that the higher level HMD layers achieve superior classification accuracy compared to classical learnable tensor baselines, including Tucker, Canonical Polyadic, and Tensor Train decompositions. Furthermore, HMD-1 and HMD-2 achieve a generalization capacity and training stability comparable to standard 2D and 3D-CNNs while requiring significantly fewer feature extractor parameters. These results demonstrate that the HMD framework provides a structurally robust substitute for traditional convolution in multidimensional HS image classification, offering high parameter efficiency and stability throughout the optimization process.

Figures & tables

Dataset	Sensor	Spatial Size	Spectral Bands	Classes
Indian Pines	AVIRIS	$145\times 145$	200	16
Pavia University	ROSIS	$610\times 340$	103	9
Loukia	Hyperion	$250\times 1376$	176	14

Table 1: Summary of the HS datasets used in this study.

Figure 1: Ground truth classification maps for the three benchmark HS datasets, from left to right: Indian Pines, Pavia University and Loukia. Black pixels in all images represent unlabelled background regions excluded from training and evaluation.

Figure 2: Schematic representation of the comparative deep learning pipeline for HS image classification. The architecture highlights the eight interchangeable feature extraction layers including the proposed HMD family, classical tensor models, and CNN baselines feeding into an identical, shared classifier head.

Method	OA (%)	AA (%)	F1 (macro)	$\kappa$
HMD-0	94.20	94.67	0.9421	0.9339
HMD-1	99.85	99.90	0.9975	0.9983
HMD-2	99.41	99.64	0.9935	0.9933
Tucker	90.88	90.52	0.9110	0.8960
CP	74.93	64.56	0.6297	0.7159
TT	98.05	98.12	0.9825	0.9777

Table 2: Test-set classification performance on Indian Pines. Best value in bold ; second-best underlined .

Method	OA (%)	AA (%)	F1 (macro)	$\kappa$
HMD-0	78.87	70.78	0.7017	0.7229
HMD-1	98.11	97.31	0.9691	0.9749
HMD-2	99.31	98.91	0.9882	0.9909
Tucker	86.40	74.52	0.6981	0.8175
CP	89.61	82.16	0.8374	0.8590
TT	65.31	59.32	0.5191	0.5438

Table 3: Test-set classification performance on Pavia University. Best value in bold ; second-best underlined .

Method	OA (%)	AA (%)	F1 (macro)	$\kappa$
HMD-0	79.34	70.77	0.7190	0.7554
HMD-1	91.23	87.86	0.8985	0.8950
HMD-2	94.82	90.99	0.9292	0.9384
Tucker	73.27	71.33	0.6737	0.6823
CP	80.27	62.78	0.6490	0.7605
TT	78.71	71.94	0.7342	0.7409

Table 4: Test-set classification performance on Loukia. Best value in bold ; second-best underlined .

Rank	Method	OA (%)	AA (%)	F1	$\kappa$	Parameters
1	HMD-2	97.85	96.51	0.9703	0.9742	908
2	HMD-1	96.40	95.02	0.9550	0.9561	908
3	2D-CNN	94.90	93.15	0.9449	0.9400	12,768
4	3D-CNN	91.24	87.25	0.8723	0.8963	1,032
5	HMD-0	84.13	78.74	0.7876	0.8041	908
6	Tucker	83.51	78.79	0.7609	0.7986	908

Table 5: Mean performance across the three datasets, ranked by mean overall accuracy, alongside mean feature-extractor parameter count.

Figure 3: Sensitivity analysis of Overall Accuracy (OA) and Average Accuracy (AA) with respect to the subspace dimensionality reduction parameter ( $r$ ) on the Indian Pines dataset.

Figure 4: Sensitivity analysis of Overall Accuracy (OA) and Average Accuracy (AA) with respect to the subspace dimensionality reduction parameter ( $r$ ) on the Pavia University dataset.

Figure 5: Sensitivity analysis of Overall Accuracy (OA) and Average Accuracy (AA) with respect to the subspace dimensionality reduction parameter ( $r$ ) on the Loukia dataset.

Figure 6: Training/validation accuracy and loss curves, all eight methods, Indian Pines dataset.

Figure 7: Training/validation accuracy and loss curves, all eight methods, Pavia University dataset.

Figure 8: Training/validation accuracy and loss curves, all eight methods, Loukia dataset.

Method	Formula	Indian Pines	Pavia University	Loukia
HMD-0 *	$r(n_{1}+n_{2}+n_{3})$	1,110	625	990
HMD-1 *	$r(n_{1}+n_{2}+n_{3})$	1,110	625	990
HMD-2 *	$r(n_{1}+n_{2}+n_{3})$	1,110	625	990
Tucker *	$r(n_{1}+n_{2}+n_{3})$	1,110	625	990
CP *	$R(n_{1}+n_{2}+n_{3})$	1,110	625	990
TT	$n_{1}R_{1}+R_{1}n_{2}R_{2}+R_{2}n_{3}R_{3}$	5,330	2,905	4,730

Table 6: Closed-form feature-extractor parameter counts by method and dataset, at the rank/filter configuration used throughout this study (

r=5

; TT:

R_{1}=R_{2}=R_{3}=5

; 2D-CNN:

k=3

f_{1}=8

f_{2}=16

; 3D-CNN:

k=3

f_{1}=4

f_{2}=8

). Indian Pines:

n_{3}=200

; Pavia University:

n_{3}=103

; Loukia:

n_{3}=176

Explore similar work

ConvVitMamba: Efficient Multiscale Convolution, Transformer, and Mamba-Based Sequence modelling for Hyperspectral Image Classification
Apr 20, 2026Mohammed Q. AlkhatibHybrid CNN-Transformer Architectures Hyperspectral Image Classification
College of Engineering and IT, University of Dubai, Dubai 14143, United Arab Emirates
MSCM-net: A hyperspectral image classiffcation method based on multi-scale convolution and Mamba
Jul 30, 2026Jianjun Chen, Linlin Wang, Lifang Chang +4Channel Attention Multi-Scale Feature Fusion
Qingdao University of Technology, Qingdao, 266520, China · Nanyang Technological University, Singapore, 639798, Singapore
Data Efficient Complex Feature Fusion Network For Hyperspectral Image Classification
Jun 3, 2026Maitreya Shelare, Atharva Satam, Poonam Sonar +1Efficient Neural Network Inference Complex-Valued Neural Networks
Department of Electronics and Telecommunication, Rajiv Gandhi Institute of Technology, University of Mumbai, India

Apr 20, 2026J/K move · Enter open · S save

ConvVitMamba: Efficient Multiscale Convolution, Transformer, and Mamba-Based Sequence modelling for Hyperspectral Image Classification

Mohammed Q. Alkhatib

College of Engineering and IT, University of Dubai, Dubai 14143, United Arab Emirates

Hybrid CNN-Transformer Architectures Hyperspectral Image Classification

Open paper AskPDF arXiv

Hyperspectral image (HSI) classification remains challenging due to high spectral dimensionality, redundancy, and limited labeled data. Although convolutional neural networks (CNNs) and Vision Transformers (ViTs) achieve strong performance by exploiting spectral-spatial information and long-range dependencies, they often incur high computational cost and large model size, limiting practical use. To address these limitations, a unified hybrid framework, termed ConvVitMamba, is proposed for efficient HSI classification. The architecture integrates three components: a multiscale convolutional feature extractor to capture local spectral, spatial, and joint patterns; a Vision Transformer based tokenization and encoding stage to model global contextual relationships; and a lightweight Mamba inspired gated sequence mixing module for efficient content-aware refinement without quadratic self-attention. Principal Component Analysis (PCA) is used as preprocessing to reduce redundancy and improve efficiency. Experiments on four benchmark datasets, including Houston and three UAV borne QUH datasets (Pingan, Qingyun, and Tangdaowan), demonstrate that ConvVitMamba consistently outperforms CNN, Transformer, and Mamba based methods while maintaining a favorable balance between accuracy, model size, and inference efficiency. Ablation studies confirm the complementary contributions of all components. The results indicate that the proposed framework provides an effective and efficient solution for HSI classification in diverse scenarios. The source code is publicly available at https://github.com/mqalkhatib/ConvVitMamba

Abstract

Explore similar work

ConvVitMamba: Efficient Multiscale Convolution, Transformer, and Mamba-Based Sequence modelling for Hyperspectral Image Classification

MSCM-net: A hyperspectral image classiffcation method based on multi-scale convolution and Mamba

Data Efficient Complex Feature Fusion Network For Hyperspectral Image Classification