Enhancing Unregistered Hyperspectral Image Super-Resolution via Unmixing-based Abundance Fusion Learning

Yingkai Zhang; Tao Zhang; Jing Nie; Ying Fu

cs.CVMar 9, 2026

Enhancing Unregistered Hyperspectral Image Super-Resolution via Unmixing-based Abundance Fusion Learning

Authors: Yingkai Zhang, Tao Zhang, Jing Nie, Ying Fu

Organizations: Beijing Institute of Technology · Hangzhou Dianzi University

Abstract

Unregistered hyperspectral image (HSI) super-resolution (SR) typically aims to enhance a low-resolution HSI using an unregistered high-resolution reference image. In this paper, we propose an unmixing-based fusion framework that decouples spatial-spectral information to simultaneously mitigate the impact of unregistered fusion and enhance the learnability of SR models. Specifically, we first utilize singular value decomposition for initial spectral unmixing, preserving the original endmembers while dedicating the subsequent network to enhancing the initial abundance map. To leverage the spatial texture of the unregistered reference, we introduce a coarse-to-fine deformable aggregation module, which first estimates a pixel-level flow and a similarity map using a coarse pyramid predictor. It further performs fine sub-pixel refinement to achieve deformable aggregation of the reference features. The aggregative features are then refined via a series of spatial-channel abundance cross-attention blocks. Furthermore, a spatial-channel modulated fusion module is presented to merge encoder-decoder features using dynamic gating weights, yielding a high-quality, high-resolution HSI. Experimental results on simulated and real datasets confirm that our proposed method achieves state-of-the-art super-resolution performance. The code will be available at https://github.com/yingkai-zhang/UAFL.

Figures & tables

Figure 1: Comparison of performance (PSNR), computational cost (FLOPs), and memory cost (Params). The vertical axis represents PSNR (dB), while the horizontal axis indicates FLOPs. The size of each bubble corresponds to the number of parameters.

Figure 2: The motivation. (c) The explicit alignment within the SSCH [ 53 ] is prone to introducing spatial distortions and textural artifacts into the warped image. (d) The result validates the principle that leveraging a well-aligned, high-fidelity abundance enables the reconstruction of the HR HSI from its LR counterpart.

Figure 3: The overall architecture. We adopt a multi-scale encoder-decoder based on unmixing with several key modules: (a) Coarse-to-Fine Deformable Aggregation module, (b) Spatial-Channel Abundance Cross-Attention module, and (c) Spatial-Channel Modulation Fusion module to super-resolve high-quality HR HSI.

Panel of Figure 4 — Figure 4: Visual comparison on the simulated dataset, ICVL [ 3 ] . The ground truth and input HSIs are shown with band 20. The results of different methods under the scale factor ( $\times 4$ ) on the ICVL dataset with the error maps.

Method	Venue	PSNR $\uparrow$	SSIM $\uparrow$	SAM $\downarrow$
Bicubic	-	35.33	0.960	0.028
SSPSR [ 17 ]	TCI’20	40.19	0.982	0.033
MCNet [ 24 ]	RS’20	39.90	0.980	0.033
BiQRNN [ 7 ]	JSTAR’21	37.85	0.974	0.039
ESSA [ 48 ]	ICCV’23	38.41	0.977	0.057
Optimized [ 9 ]	CVPR’19	25.35	0.801	0.026

Table 1: Quantitative comparison under scale factor of

\times 4

on the simulated dataset, ICVL [ 3 ] . PSNR is in dB. The best and second best results are highlighted in bold and underline , respectively.

Method	Venue	Scale Factor $\times 4$			Scale Factor $\times 8$			Scale Factor $\times 16$			Params	FLOPs
Method	Venue	PSNR $\uparrow$	SSIM $\uparrow$	SAM $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	SAM $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	SAM $\downarrow$	(M)	(G)
Bicubic	-	34.07	0.941	0.042	28.43	0.870	0.058	24.84	0.815	0.086	-	-
SSPSR [ 17 ]	TCI’20	39.04	0.976	0.040	31.13	0.906	0.069	27.20	0.851	0.077	20.82	180.40
MCNet [ 24 ]	RS’20	39.07	0.974	0.038	31.31	0.904	0.063	27.04	0.845	0.088	2.20	280.30
BiQRNN [ 7 ]	JSTAR’21	37.80	0.969	0.044	31.66	0.908	0.060	26.53	0.832	0.111	1.30	468.70
ESSA [ 48 ]	ICCV’23	39.94	0.982	0.040	31.85	0.913	0.064	26.70	0.918	0.111	11.10	78.14

Table 2: Quantitative comparison of various methods under several scale factors on the real dataset, REAL [ 19 ] . PSNR is in dB. The best and second best results are highlighted in bold and underline , respectively.

Panel of Figure 5 — Figure 5: Visual comparison on the real dataset, REAL [ 19 ] . The ground truth and input HSIs are shown with band 20. The results of different methods under the scale factor ( $\times 4$ , $\times 8$ ) on the REAL dataset with the error maps.

Unmix	SCACA	CFDA	SCMF	PSNR $\uparrow$	SSIM $\uparrow$	SAM $\downarrow$	Params
✗	✗	✗	✗	41.26	0.984	0.036	5.08M
✓	✗	✗	✗	41.41	0.986	0.034	5.05M
✓	✓	✗	✗	41.66	0.987	0.034	4.85M
✓	✓	✓	✗	41.95	0.988	0.033	5.85M
✓	✓	✓	✓	42.05	0.988	0.033	5.94M

Table 3: Ablation study of the proposed components. Results on the REAL dataset at scale factor

\times 4

Method	PSNR $\uparrow$	SSIM $\uparrow$	SAM $\downarrow$	Params
w/o Aggregation	41.66	0.987	0.034	4.85M
w/ DCNv2 [ 58 ]	41.80	0.987	0.033	5.78M
w/ CFDA	41.95	0.988	0.033	5.85M

Table 4: Ablation study on the CFDA module. Results on the REAL dataset at scale factor

\times 4

Figure 6: Visual comparison of fused feature maps generated by different aggregation modules, followed by an identical attention block. Note the severe artifacts and blurred text produced by the standard DCNv2 [ 58 ] , in contrast to the superior clarity of the features from our CFDA.

Method	$\times 4$		$\times 8$		$\times 16$		Params
Method	PSNR $\uparrow$	SAM $\downarrow$	PSNR $\uparrow$	SAM $\downarrow$	PSNR $\uparrow$	SAM $\downarrow$	Params
w/o SCMF	41.95	0.033	37.02	0.047	31.90	0.068	5.85M
w/ SCMF	42.05	0.033	37.23	0.046	32.28	0.065	5.94M

Table 5: Ablation study on the SCMF module.

Explore similar work

Unregistered Spectral Image Fusion: Unmixing, Adversarial Learning, and Recoverability
Mar 23, 2026Jiahui Song, Sagar Shrestha, Xiao FuMulti-Source Image Fusion Hyperspectral Image Super-Resolution
The authors are with the School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, OR 97331, United States.
Lightweight Interpretable RGB-Guided Hyperspectral Super-Resolution under Real Cross-resolution Misalignment
Sep 1, 2026Mohamad Jouni, Aurélien Godet, Mauro Dalla MuraHyperspectral Image Super-Resolution Hyperspectral Imaging
Univ. Grenoble Alpes, CNRS Grenoble INP, GIPSA-Lab 38000 Grenoble, France · Institut Universitaire de France (IUF) Paris, France
Super-Resolving Unseen Hyperspectral Sensors at Any Scale via Spatial Operators
Sep 30, 2026Ji-Xuan He, Guohang Zhuang, Bo Junge +6Hyperspectral Image Super-Resolution Operator Learning
Xi’an Jiaotong University · Hefei University of Technology · National University of Singapore +1

Mar 23, 2026J/K move · Enter open · S save

Unregistered Spectral Image Fusion: Unmixing, Adversarial Learning, and Recoverability

Jiahui Song, Sagar Shrestha, Xiao Fu

The authors are with the School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, OR 97331, United States.

Multi-Source Image Fusion Hyperspectral Image Super-Resolution Hyperspectral Imaging Image Super-Resolution

Open paper AskPDF arXiv

This paper addresses the fusion of a pair of spatially unregistered hyperspectral image (HSI) and multispectral image (MSI) covering roughly overlapping regions. HSIs offer high spectral but low spatial resolution, while MSIs provide the opposite. The goal is to integrate their complementary information to enhance both HSI spatial resolution and MSI spectral resolution. While hyperspectral-multispectral fusion (HMF) has been widely studied, the unregistered setting remains challenging. Many existing methods focus solely on MSI super-resolution, leaving HSI unchanged. Supervised deep learning approaches were proposed for HSI super-resolution, but rely on accurate training data, which is often unavailable. Moreover, theoretical analyses largely address the co-registered case, leaving unregistered HMF poorly understood. In this work, an unsupervised framework is proposed to simultaneously super-resolve both MSI and HSI. The method integrates coupled spectral unmixing for MSI super-resolution with latent-space adversarial learning for HSI super-resolution. Theoretical guarantees on the recoverability of the super-resolution MSI and HSI are established under reasonable generative models -- providing, to our best knowledge, the first such insights for unregistered HMF. The approach is validated on semi-real and real HSI-MSI pairs across diverse conditions.

Abstract

Explore similar work

Unregistered Spectral Image Fusion: Unmixing, Adversarial Learning, and Recoverability

Lightweight Interpretable RGB-Guided Hyperspectral Super-Resolution under Real Cross-resolution Misalignment

Super-Resolving Unseen Hyperspectral Sensors at Any Scale via Spatial Operators