Cosine-Normalized Attention for Hyperspectral Image Classification

Muhammad Ahmad; Manuel Mazzara

cs.CVApr 2, 2026

Cosine-Normalized Attention for Hyperspectral Image Classification

Authors: Muhammad Ahmad, Manuel Mazzara

Organizations: SDAIA-KFUPM, Joint Research Center for Artificial Intelligence (JRCAI), King Fahd University of Petroleum and Minerals, Dhahran, 31261, Saudi Arabia · Institute of Software Development and Engineering, Innopolis University, Innopolis, 420500, Russia

Abstract

Transformer-based methods have improved hyperspectral image classification (HSIC) by modeling long-range spatial-spectral dependencies; however, their attention mechanisms typically rely on dot-product similarity, which mixes feature magnitude and orientation and may be suboptimal for hyperspectral data. This work revisits attention scoring from a geometric perspective and introduces a cosine-normalized attention formulation that aligns similarity computation with the angular structure of hyperspectral signatures. By projecting query and key embeddings onto a unit hypersphere and applying a squared cosine similarity, the proposed method emphasizes angular relationships while reducing sensitivity to magnitude variations. The formulation is integrated into a spatial-spectral Transformer and evaluated under extremely limited supervision. Experiments on three benchmark datasets demonstrate that the proposed approach consistently achieves higher performance, outperforming several recent Transformer- and Mamba-based models despite using a lightweight backbone. In addition, a controlled analysis of multiple attention score functions shows that cosine-based scoring provides a reliable inductive bias for hyperspectral representation learning.

Figures & tables

Panel of Figure 1 — Fig. 1 : Standard dot-product attention mixes vector magnitude and angular alignment through $\mathbf{q}^{\top}\mathbf{k}$ . While after $\ell_{2}$ normalization, the embeddings lie on a unit hypersphere and the score depends only on angular similarity. The proposed squared cosine score further sharpens the distinction between strongly aligned and weakly aligned token pairs.

Panel of Figure 2 — Fig. 2 : The HSI cube is partitioned into 3D patches, embedded into a token matrix, projected into query, key, and value representations, normalized, and scored using a cosine-based overlap function.

Metric	Transformers					State Space Models							Proposed
Metric	MSST	S2CIFT	S2CAT	WF	DF	SSAM	DBMLLA	WDM	MM	GM	PM	KDM	CS 2	CS	SDP	DP	Add	SA	MSA-CS 2	C-SDP	C-CS 2	C-CS	C-Add
SA Dataset
$\kappa$	97.97	95.65	95.78	94.90	94.06	87.80	92.28	89.26	88.38	95.85	98.16	92.44	99.15	98.60	99.18	97.75	98.93	99.24	99.23	98.76	99.33	98.16	98.75
OA	98.17	96.10	96.21	95.42	94.67	89.07	93.06	90.35	89.56	96.27	98.35	93.22	99.23	98.74	99.26	97.98	99.04	99.32	99.31	98.88	99.40	98.35	98.87
AA	97.63	95.56	97.48	95.14	95.87	88.59	95.44	91.80	87.76	96.24	97.55	91.23	99.18	98.83	99.06	98.02	99.03	99.20	99.30	98.99	99.18	98.53	99.09
HH Dataset

TABLE I : Comparison of the proposed framework. ( Highest , Second , and Third highest ).

Panel of Figure 4 — Fig. 3 : Visual comparison of classification maps on SA.

Panel of Figure 5 — Fig. 4 : Visual comparison of classification maps on TD.

Panel of Figure 6 — Fig. 5 : Visual comparison of classification maps on HH.

Panel of Figure 7 — Fig. 6 : t-SNE visualization of the learned feature spaces.

Analysis Group	Configuration	SA	HH	TD
Normalization	None	98.11 $\pm$ 0.12	97.64 $\pm$ 0.15	98.42 $\pm$ 0.11
	Query only	98.74 $\pm$ 0.09	97.93 $\pm$ 0.12	98.58 $\pm$ 0.10
	Key only	98.69 $\pm$ 0.10	97.89 $\pm$ 0.13	98.55 $\pm$ 0.09
	Query + Key	99.24 $\pm$ 0.05	98.32 $\pm$ 0.07	98.84 $\pm$ 0.06
Squaring strategy	Cosine	99.07 $\pm$ 0.07	98.21 $\pm$ 0.09	98.71 $\pm$ 0.08
	$\|\mathrm{cosine}\|$	99.11 $\pm$ 0.06	98.24 $\pm$ 0.08	98.74 $\pm$ 0.07

TABLE II : Ablation and robustness analysis.

Explore similar work

A Synergistic CNN-Transformer Network with Pooling Attention Fusion for Hyperspectral Image Classification
Apr 26, 2026Peng Chen, Wenxuan He, Feng Qian +2Hybrid CNN-Transformer Architectures Transformer Attention
College of Engineering, Shantou University, Shantou, 515063, Guangdong, China · School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan, 430074, Hubei, China · Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun, 130033, Jilin, China +2
VP-Hype: A Hybrid Mamba-Transformer Framework with Visual-Textual Prompting for Hyperspectral Image Classification
Mar 1, 2026Abdellah Zakaria Sellam, Fadi Abdeladhim Zidi, Salah Eddine Bekhouche +4Hyperspectral Image Classification Hyperspectral Imaging
Institute of Applied Sciences and Intelligent Systems (ISASI), CNR, 73100 Lecce, Italy · Dept. of Engineering for Innovation, University of Salento, 73100 Lecce, Italy · VSC Laboratory, Department of Electronics and Automation, University of Biskra, Biskra, Algeria +4
SSFT: A Lightweight Spectral-Spatial Fusion Transformer for Generic Hyperspectral Classification
Apr 17, 2026Alexander Musiat, Nikolas Ebert, Oliver WasenmüllerRepresentation Learning Hyperspectral Imaging
Mannheim University of Applied Sciences, Germany

Apr 26, 2026J/K move · Enter open · S save

A Synergistic CNN-Transformer Network with Pooling Attention Fusion for Hyperspectral Image Classification

Peng Chen, Wenxuan He, Feng Qian, Guangyao Shi+1

College of Engineering, Shantou University, Shantou, 515063, Guangdong, China · School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan, 430074, Hubei, China · Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun, 130033, Jilin, China+2

Hybrid CNN-Transformer Architectures Transformer Attention Hyperspectral Image Classification

Open paper AskPDF arXiv

In the hyperspectral image (HSI) classification task, each pixel is categorized into a specific land-cover category or material. Convolutional neural networks (CNNs) and transformers have been widely used to extract local and non-local features in HSI classification. Recent works have utilized a multi-scale vision transformer (ViT) to enhance spectral feature capture and yield promising results. However, most existing methods still face challenges in the effective joint use of spatial-spectral information and in preserving information across layers during the propagation process. To address these issues, we propose a synergistic CNN-Transformer network with pooling attention fusion for HSI classification, which collaboratively utilizes CNNs and ViT to process spatial and spectral features separately. Specifically, we propose a Twin-Branch Feature Extraction (TBFE) module, which employs 3D and 2D convolution in parallel to comprehensively extract spectral and spatial features from HSI. A hybrid pooling attention (HPA) module is designed to aggregate spatial attention. Moreover, a cascade transformer encoder is employed for global spectral feature extraction, and a simple yet efficient cross-layer feature fusion (CFF) module is designed to reduce the loss of crucial information in the previous network layers. Extensive experiments are conducted on several representative datasets to demonstrate the superior performance of our proposed method compared to the state-of-the-art works. Code is available at https://github.com/chenpeng052/SCT-Net.git.

Abstract

Explore similar work

A Synergistic CNN-Transformer Network with Pooling Attention Fusion for Hyperspectral Image Classification

VP-Hype: A Hybrid Mamba-Transformer Framework with Visual-Textual Prompting for Hyperspectral Image Classification

SSFT: A Lightweight Spectral-Spatial Fusion Transformer for Generic Hyperspectral Classification