Learning Spatial-Spectral Refinement and Calibrating Complementary Observations for Hyperspectral Image Super-Resolution
Organizations: School of Management, Zhengzhou University, Zhengzhou, 450001, China · Center for Intelligent Decision-making and Machine Learning, Xi’an Jiaotong University, Xi’an, 710049, China · School of Computer Science and Technology, Xi’an Jiaotong University, Xi’an, 710049, China
Abstract
Hyperspectral and multispectral image fusion (HMIF) aims to reconstruct a high-resolution hyperspectral image (HR-HSI) by combining the fine spatial details of a high-resolution multispectral image (HR-MSI) with the rich spectral information of a low-resolution hyperspectral image (LR-HSI). Recent advances in implicit neural representations (INRs) have enabled flexible coordinate-based modeling for HMIF; however, existing INR-based approaches may not fully capture fine-grained spatial structures and rich spectral dependencies. Moreover, the LR-HSI and HR-MSI are primarily incorporated through degradation-consistency constraints, leaving their complementary information underexploited. To address these limitations, we propose Two-Stage Reconstruction with Implicit Tensor Neural Representation (TSR-ITNR), a unified self-supervised framework integrating representation refinement and observation-guided calibration. In Stage 1, TSR-ITNR learns an implicit Tucker representation and refines its low-rank spatial coefficient tensor and spectral basis to better capture fine spatial structures and interband correlations. A fixed pretrained denoiser further provides a deep prior for the preliminary reconstruction. In Stage 2, parameter-free calibration derives complementary and noninterfering corrections from both observations to recover information insufficiently captured in Stage 1. Theoretical analysis establishes the geometry-preserving property of spectral refinement and the orthogonal complementarity of calibration. Extensive experiments on multiple benchmark datasets demonstrate strong quantitative, visual, and spectral reconstruction performance without ground-truth HR-HSI supervision. Beyond conventional reconstruction metrics, we further assess the effectiveness of TSR-ITNR using downstream semantic segmentation accuracy.
Figures & tables
| Dataset | Index | Low Rank | Deep Learning | Hybrid | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CTDF | GTNN | CAFE+ | EDIP | CS2DIPs | DELTA | CNN-FUS | LRTFR | CLoRF | SSLRDN | TSR-ITNR | ||
| Pavia Centre | PSNR | 44.85 | 45.24 | 42.65 | 44.57 | 45.88 | 41.22 | 44.90 | 40.63 | 44.45 | 45.41 | 46.36 |
| SSIM | 0.983 | 0.984 | 0.975 | 0.982 | 0.985 | 0.971 | 0.984 | 0.969 | 0.983 | 0.985 | 0.985 | |
| SAM | 0.072 | 0.064 | 0.078 | 0.064 | 0.060 | 0.093 | 0.065 | 0.104 | 0.065 | 0.061 | 0.058 | |
| ERGAS | 38.25 | 34.63 | 42.94 | 36.07 | 32.81 | 51.08 | 35.86 | 52.08 | 36.18 | 33.94 | 32.55 | |
| Dataset | Index | Low Rank | Deep Learning | Hybrid | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CTDF | GTNN | CAFE+ | EDIP | CS2DIPs | DELTA | CNN-FUS | LRTFR | CLoRF | SSLRDN | TSR-ITNR | ||
| BGU | PSNR | 50.30 | 50.15 | 51.17 | 54.41 | 51.51 | 48.62 | 46.76 | 50.96 | 53.53 | 54.77 | 56.61 |
| SSIM | 0.996 | 0.997 | 0.996 | 0.998 | 0.998 | 0.993 | 0.987 | 0.997 | 0.998 | 0.999 | 0.999 | |
| SAM | 0.019 | 0.020 | 0.020 | 0.011 | 0.018 | 0.030 | 0.041 | 0.020 | 0.014 | 0.012 | 0.010 | |
| ERGAS | 21.29 | 22.10 | 13.97 | 9.65 | 16.16 | 24.02 | 26.20 | 14.17 | 10.62 | 8.90 | 7.66 | |
| Flower | PSNR | 48.09 | 48.46 | 46.40 | 48.88 | 49.02 | 44.89 | 45.31 | 44.69 | 49.38 | 48.81 | 50.23 |