Organizations: College of Science, Northwest Agriculture & Forestry University, Yangling 712100, China · Key Laboratory of Photoelectronic Imaging Technology and System, School of Optics and Photonics, Beijing Institute of Technology, Beijing, China · State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China
Single-pixel imaging (SPI) offers a cost-effective route to hyperspectral acquisition but struggles to recover high-fidelity spatial and spectral details under extremely low sampling rates, a severely ill-posed inverse problem. While deep learning has shown potential, existing data-driven methods demand large-scale pretraining datasets that are often impractical in hyperspectral imaging. To overcome this limitation, we propose an end-to-end physics-informed framework that leverages untrained neural networks and RGB guidance for joint hyperspectral reconstruction and super-resolution without any external training data. The framework comprises three physically grounded stages: (1) a Regularized Least-Squares method with RGB-derived Grayscale Priors (LS-RGP) that initializes the solution by exploiting cross-modal structural correlations; (2) an Untrained Hyperspectral Recovery Network (UHRNet) that refines the reconstruction through measurement consistency and hybrid regularization; and (3) a Transformer-based Untrained Super-Resolution Network (USRNet) that upsamples the spatial resolution via cross-modal attention, transferring high-frequency details from the RGB guide. Extensive experiments on benchmark datasets demonstrate that our approach significantly surpasses state-of-the-art algorithms in both reconstruction accuracy and spectral fidelity. Moreover, a proof-of-concept experiment using a physical single-pixel imaging system validates the framework's practical applicability, successfully reconstructing a 144-band hyperspectral data cube at a mere 6.25% sampling rate. The proposed method thus provides a robust, data-efficient solution for computational hyperspectral imaging.
Figures & tables
Figure 1: The overall architecture of the proposed RGB-guided hyperspectral reconstruction framework. (a) End-to-end pipeline integrating SPI physics, RGB guidance, and untrained neural networks. (b) UHRNet: RGB-guided hyperspectral recovery network. (c) USRNet: transformer-based hyperspectral super-resolution network. (d) Head module for feature mapping. (e) Encoder with multi-head attention. (f) SEBlock for channel-wise attention. (g) ConvBlock with convolution and normalization layers.
Figure 2: Comparison of hyperspectral reconstruction quality across different methods for Bands 1, 16, and 21. Grayscale heatmaps visualize spatial fidelity. From left to right: Ground Truth (GT), DGI, GISC, TVAL3, GIDC, PYFINETUNE, MST++, and Ours. PSNR and SSIM values are displayed. Our method consistently achieves higher metrics and superior visual quality.
Figure 3: Performance under different SNR conditions at 6.25% sampling rate. (a) Average PSNR, (b) average SSIM, and (c) average SAM for each method. Our approach shows significantly better noise resilience.
Figure 4: Overall performance comparison in terms of PSNR and SAM. Our method achieves the best trade-off, indicating superior reconstruction fidelity and spectral preservation.
Figure 5: Effect of the number of measurement patterns on performance metrics (PSNR, SSIM, SAM) for our method and PYFINETUNE.
Figure 6: Visual comparison for Bands 9, 15, and 31 using only 4 patterns. Our method yields clearer textures and higher fidelity metrics.
Figure 7: Spectral recovery comparison for two pixels (marked in the RGB image) at 6.25% sampling. Our method (red) aligns best with the ground truth (blue), exhibiting the lowest SAM values.
Figure 8: Super-resolution reconstruction for Bands 10, 15, and 20. Our method delivers superior visual quality and objective metrics.
Method
PSNR (dB)
SSIM
SAM (°)
SAM (rad)
Ours
35.0147
0.9571
7.5949
0.1326
DGI
8.3223
0.0519
27.0787
0.4726
GIDC
22.7902
0.5758
22.9502
0.4006
GISC
9.9916
0.3472
27.2265
0.4752
TVAL3
13.2984
0.0991
89.9110
1.5692
PYFINETUNE
26.2854
0.6361
14.5766
0.2544
Table 1: Super-resolution performance comparison. Our method excels across all metrics.
Figure 9: ROI comparison: the RGB image with marked ROI (top left) and enlarged reconstructions from each method versus ground truth. Our result exhibits the finest detail preservation.
Configuration
PSNR (dB)
SSIM
SAM (°)
SAM (rad)
Full model
38.1406
0.9670
7.3064
0.1275
w/o measurement loss
25.5383
0.8616
21.2551
0.3710
w/o Fourier reg.
36.5002
0.9546
12.5095
0.2183
w/o sharpness loss
29.2406
0.8379
29.4664
0.5143
w/o smoothness loss
37.3611
0.9602
12.3805
0.2161
w/o perceptual loss
14.6182
0.5841
19.8794
0.3470
Table 2: Ablation study for RGB-guided hyperspectral recovery. Each loss term positively impacts performance.
Configuration
PSNR (dB)
SSIM
SAM (°)
SAM (rad)
Full model (Baseline)
35.01
0.9571
7.59
0.1326
Architecture:
no transformer blocks
32.43
0.9400
10.87
0.1896
shallow (depth=4)
34.64
0.9558
7.64
0.1334
fewer heads (heads=2)
33.64
0.9457
8.66
0.1512
narrow (dim=64)
34.77
0.9560
7.58
0.1323
Table 3: Ablation study for super-resolution network. The transformer architecture and RGB guidance are essential for high-quality upsampling.
Figure 10: Schematic of the real-world SPHI system. Broadband illumination from the DLP projector is structured by random patterns. Reflected light from the target enters a beam splitter; the left path is captured by the RGB camera (equipped with a 16 × lens), and the straight path is directed to the spectrometer via a 25 × lens and a 40 × microscope objective.
Figure 11: Photograph of the experimental setup showing key components: DLP projector, beam splitter (BS), RGB camera (RGB Cam), collection lens (Lens), microscope objective (Obj.), and fiber spectrometer (Spec.).
Parameter
Value
Spatial resolution
128×128 pixels
Number of patterns ( M )
1024
Sampling rate ( β )
6.25%
Spectral range
380–720 nm
Number of bands ( B )
144
Spectral interval
≈ 2.4 nm
Table 4: Key parameters of the real-world experimental system.
Figure 12: Spectral fidelity analysis. (a) RGB image with five target locations marked. (b)–(f) Ground truth (blue) vs. reconstructed spectra (red) for each target, with Pearson correlation r and SAM annotated.
Target
Pixel
r (raw)
r (SG)
SAM (SG)
Target 1
(51, 100)
0.899
0.913
17.6 ∘
Target 2
(73, 33)
0.946
0.967
16.3 ∘
Target 3
(104, 55)
0.887
0.909
21.9 ∘
Target 4
(27, 76)
0.813
0.871
28.8 ∘
Target 5
(50, 65)
0.869
0.885
25.6 ∘
Mean
—
0.883
0.909
22.0 ∘
Table 5: Quantitative spectral fidelity metrics for the five targets. SG smoothing enhances correlation and reduces SAM.
Figure 13: Visualization of the reconstructed 128×128×144 hyperspectral data cube from real measurements, covering 380–720 nm.
Department of Metabolism, Digestion and Reproduction, Imperial College London, London, UK · Department of Mathematics, Imperial College London, London, UK · Department of Clinical Neurosciences and NIHR Biomedical Research Centre, University of Cambridge, Cambridge, UK +4