Manifold Prior Guided Deep Unfolding for Hyperspectral Image Reconstruction
Organizations: Beijing Institute of Technology · Tohoku University · Hangzhou Dianzi University
Abstract
Hyperspectral Image (HSI) reconstruction has made substantial progress with the deep unfolding framework by decomposing the problem into a projection module and a denoiser module. Nevertheless, existing methods still exhibit limitations of insufficient matching with HSI data. The issues lie in two aspects: 1) the data-consistency projection module applies a fixed gradient descent step while ignoring the spectral content adaptivity of HSI; 2) the denoiser module either suffers from high computational complexity or is limited to the local receptive field, failing to efficiently capture the global spatial-spectral dependencies. In this paper, we propose a deep unfolding framework centered on a two-stage manifold learning strategy to obtain a degradation-free structural prior, which is then strategically injected into both the projection and denoiser modules. In the data-consistency projection, a prior-guided spectral weight mechanism is introduced to adaptively modulate the weight of spectral band. In the denoiser module, a manifold-guided Mamba is proposed to achieve the structure-aware extraction of spatial-spectral features through prior modulation. Quantitative and visual comparisons on synthetic and real-world datasets show that the proposed method significantly outperforms existing state-of-the-art approaches, while maintaining the structural interpretability of the unfolding framework. The code is available at https://github.com/SqLiu67/MPG-DU.
Figures & tables
| Method | Params | FLOPs | S1 | S2 | S3 | S4 | S5 | S6 | S7 | S8 | S9 | S10 | Avg |
| MST [ 29 ] | 2.03M | 28.15G | 36.55 | 36.29 | 33.46 | 39.78 | 35.40 | 36.10 | 34.53 | 33.08 | 34.38 | 35.19 | 35.48 |
| 0.963 | 0.953 | 0.904 | 0.951 | 0.964 | 0.963 | 0.924 | 0.945 | 0.926 | 0.966 | 0.946 | |||
| DAUHST-9stg [ 46 ] | 6.15M | 79.50G | 38.37 | 39.91 | 37.71 | 42.97 | 37.69 | 39.05 | 37.62 | 36.11 | 38.45 | 37.39 | 38.53 |
| 0.972 | 0.977 | 0.965 | 0.967 | 0.980 | 0.974 | 0.964 | 0.965 | 0.971 | 0.976 | 0.971 | |||
| PADUT-12stg [ 11 ] | 5.38M | 90.46G | 38.42 | 40.34 | 38.95 | 43.50 | 38.22 | 39.16 | 38.21 | 36.03 | 39.45 | 37.30 | 38.89 |
| 0.974 | 0.983 | 0.972 | 0.977 | 0.983 | 0.979 | 0.971 | 0.969 | 0.979 | 0.981 | 0.977 |
| Method | S1 | S2 | S3 | S4 | S5 | S6 | S7 | S8 | S9 | S10 | Avg |
| MST [ 29 ] | 35.40/0.941 | 35.87/0.944 | 36.51/0.953 | 42.27/0.973 | 32.77/0.947 | 34.80/0.955 | 33.66/0.925 | 32.67/0.948 | 35.39/0.949 | 32.50/0.941 | 35.18/0.948 |
| DAUHST-9stg [ 46 ] | 37.25/0.958 | 39.02/0.967 | 41.05/0.971 | 46.15/0.983 | 35.80/0.969 | 37.08/0.970 | 37.57/0.963 | 35.10/0.966 | 40.02/0.970 | 34.59/0.956 | 38.36/0.967 |
| PADUT-12stg [ 11 ] | 37.36/0.962 | 40.43/0.978 | 42.38/0.979 | 46.62/0.990 | 36.26/0.974 | 37.27/0.974 | 37.83/0.966 | 35.33/0.974 | 40.86/0.978 | 34.55/0.963 | 38.89/0.974 |
| SPECAT [ 47 ] | 38.12 / 0.977 | 40.79/ 0.984 | 38.59/0.973 | 43.39/0.985 | 38.02 / 0.984 | 38.41 / 0.984 | 37.82/0.965 | 34.80/0.971 | 38.75/0.980 | 37.04 / 0.986 | 38.57/0.979 |
| LADE-DUN-3stg [ 38 ] | 37.08/0.961 | 39.53/0.974 | 41.67/0.978 | 46.37/0.991 | 35.73/0.972 | 36.71/0.973 | 36.94/0.962 | 35.13/0.971 | 39.96/0.976 | 33.96/0.961 | 38.31/0.972 |
| LADE-DUN-9stg [ 38 ] | 38.07/0.969 | 41.16/0.982 | 43.70 / 0.983 | 48.01 / 0.993 | 37.76/0.980 | 37.65/ 0.980 | 38.58 / 0.973 | 36.31 / 0.979 | 42.66 / 0.984 | 35.18 /0.967 | 39.91 / 0.979 |
| Mamba | Cross-Prior Attn | Prior-Mamba | Prior-Spectral Weight | PSNR (dB) | SSIM | Params (M) |
| ✓ | 37.02 | 0.963 | 1.20 | |||
| ✓ | ✓ | 37.62 | 0.968 | 3.17 | ||
| ✓ | ✓ | ✓ | 38.25 | 0.969 | 3.60 | |
| ✓ | ✓ | ✓ | ✓ | 38.73 | 0.974 | 3.71 |