Snapshot Spectral Imaging (SSI) provides high-dimensional temporal-spatial-spectral observation to uncover intrinsic physical characteristics. However, its complex system and repetitive calibration requirements hinder edge applications. Here, we propose a compact, cost-effective, calibration-free SSI method, Aperture Diffraction Imaging Spectrometer (ADIS), which consists only of a diffractive lens with a binary mask and a Bayer-filtered sensor, requiring no additional physical footprint compared to standard RGB cameras. ADIS disperses and multiplexes wavelengths, mapping energy to distinct sensor locations, enabling full-resolution recovery from superpixel-level encodings. ADIS directly leverages theoretically computed PSFs to enable calibration-free spectral reconstruction, while tolerating lens-dependent variations across different optical configurations and bridging the gap between simulation and reality. To achieve SSI by solving a sparsely-constrained inverse problem, we introduce the Orthogonal Diffraction-Aware Unfolding Framework (ODAUF) with Voxel Shift Transformer (VST) for improved orthogonal diffraction perception. Integrating VST into ODAUF forms the efficient Orthogonal Diffraction-Aware Unfolding Voxel Shift Transformer (ODAUVST), delivering excellent recovery and reduced parameters. By elaborating on theory, systematic and comprehensive comparing, and demonstrating real SSI results, we validate the superiority of ADIS, achieving calibration-free full-resolution SSI within a commercial camera footprint.
Figures & tables
Fig. 1: The imaging principle of ADIS. Light from the target scene is modulated by the Orthogonal Diffraction Lens (ODL) and captured by an array-filtered image sensor. The compressed measurements are reconstructed into HSIs using theoretically computed PSFs and filter functions (sensor parameters) as priors. The theoretically computed PSFs enable calibration-free reconstruction while tolerating lens-dependent variations across different optical configurations. The right side shows exemplar measurements and the corresponding reconstructed HSIs.
Fig. 2: (a) Calibration-free reconstruction of ADIS using theoretically computed PSFs, with robustness to lens-dependent convolutional degradation across different lens configurations; (b) ADIS forward propagation with collimated illumination; (c) illustrates the principles of CTIS and strategy of using long optical path by sacrificing spatial resolution, while ADIS reconstructs from aliasing; (d) depicts compact imaging methods with mosaic filter sensors.
Fig. 3: (a) demonstrates the application of the Babinet principle in the design of ADIS masks, which can be used to enhance light throughput; (b) depicts an alternative configuration of ADIS whereby the mask is located between two plano-convex lenses, producing a similar encoding process; (c) is the overview of proposed Aperture Diffraction Imaging Spectrometer (ADIS). The system consists of an optical encoder and a computational decoder, which encodes the HSIs into 2D images and reconstructs the underlying HSIs from the compressed measurements, respectively.
Fig. 4: Comparison of reconstruction fidelity for masks with different periods on the test set, evaluated using Spectral Angle Mapping (SAM) as the spectral accuracy metric. Fifteen diverse outdoor scenes from the NTIRE 2020 dataset are used as a test set to assess the suitability of the chosen mask parameters.
Fig. 5: Illustration of ODAUF with k stages. Theoretically, the denoisers in ODAUF as learnable regularization terms can be replaced with different designs.
Fig. 6: (a) Diagram of VST. VST adopts a three-layer U-shaped structure. VSAB consists of an FFN, a VS-MSA, and two layer normalizations. (b) VS-MSA employs parameters learned via LSM to perform feature shifting (CSM) and self-attention calculations on input features. (c) Illustration of LSM and CSM.
Algorithm
Inference Time
Params
GFLOPS
S1
S2
S3
S4
S5
S6
S7
S8
S9
S10
Avg
31.43
29.64
25.44
33.50
28.77
28.38
26.78
31.72
36.40
27.96
30.00
UNet [ 45 ]
1ms
23.32M
5.33
0.940
0.912
0.877
0.914
0.897
0.884
0.852
0.916
0.936
0.892
0.902
29.27
28.07
23.02
31.53
27.40
26.29
23.52
30.06
32.67
26.17
27.80
MIRNet [ 46 ]
1ms
2.05M
14.66
0.910
0.870
0.842
0.898
0.873
0.858
0.833
0.905
0.854
0.844
0.869
32.18
29.90
26.30
34.20
30.26
29.00
27.73
32.32
34.18
27.15
30.32
Lambda-Net [ 37 ]
1ms
32.73M
23.08
0.927
0.909
0.871
0.886
0.902
0.880
0.844
0.916
0.900
0.883
0.892
TABLE I: Quantitative comparison of reconstruction results of different algorithms, PSNR (dB) and SSIM are reported.
Fig. 7: Simulation HSI reconstruction comparisons of different scenes with 10 (out of 28) spectral channels on ADIS. The left side displays the RGB image of the scene alongside the simulated measurements, while the central side presents a comparison between the reconstructed intensity-spectral curves at marked points on the RGB image and the corresponding Ground Truth.
Fig. 8: Comparison of ADIS and two different existing compact computational spectral imaging methods, a DOE-based system and a prism-based system with ten ground-truth spectral images.
System
PSNR (dB)
SSIM
RMSE
SAM
Prism-based [ 29 ]
34.26
0.967
0.0199
4.45
DOE-based [ 11 ]
35.60
0.973
0.0173
4.34
ADIS (Ours)
35.97
0.978
0.0169
3.96
TABLE II: Quantitative comparison of ADIS with DOE-based system and prism-based system, PSNR (dB) and SSIM are reported.
System
PSNR (dB)
SSIM
Params
GFLOPS
Jeon2019
30.33
0.907
3.79M
44.01
COPF
31.92
0.937
3.85M
47.54
ODAUF
33.31
0.951
4.53M
53.64
TABLE III: Quantitative comparison of ODAUF with two alternative frameworks for ADIS reconstruction.
Fig. 9: ADIS-V1: Compact prototype based on a single-lens configuration and representative reconstruction results. The mask is directly on the front surface of the lens. Unlike prior methods relying on calibrated PSFs, we reconstruct these scenes directly using computed PSFs. The reference spectral curves (ground truth, GT) are measured with a point spectrometer.
Algorithm
ϖ
PSNR
SSIM
ϖ
PSNR
SSIM
ϖ
PSNR
SSIM
ODAUVST-5stg
0.01
32.38
0.928
0.02
31.62
0.908
0.05
30.11
0.873
CSST-9stg [ 44 ]
0.01
32.12
0.923
0.02
31.34
0.906
0.05
30.06
0.872
Restormer [ 49 ]
0.01
31.30
0.916
0.02
30.51
0.897
0.05
29.44
0.869
TABLE IV: Quantitative comparison of ODAUVST-5stg with Restormer and CSST-9stg under different noise levels, PSNR (dB), and SSIM are reported.
System
ϖ
PSNR
SSIM
SAM
ϖ
PSNR
SSIM
SAM
ADIS
0.00
35.97
0.978
3.96
0.01
32.81
0.939
5.90
DOE-based
0.00
35.60
0.973
4.34
0.01
30.35
0.864
6.36
TABLE V: Quantitative comparison of ADIS with the DOE-based system under noise conditions.
Fig. 10: ADIS-V2: We capture three natural scenes using our real prototype system as shown in (a). We directly reconstruct these scenes utilizing the computed PSFs corresponding to those of ADIS-V1. The reference spectral curves (ground truth, GT) are measured with a point spectrometer. Notably, our method eliminates artifacts caused by simulation-to-reality gaps inherent to DOE-based acquisition [ 11 , 12 ] , and the comparisons can be found in the Supp. Material.
Fig. 11: Quantitative evaluation of our ADIS prototype. (a) shows the reconstructed ColorChecker and reconstruction curves vs. ground truth. (b) demonstrates the spatial accuracy of our reconstruction. We compare the modulation transfer functions of the input image and the output reconstruction using the square region. (c) Metameric material discrimination using ADIS.
Fig. 12: The snapshot spectral imaging results of ADIS-V2 in an autonomous-driving scenario. Please refer to the supplementary materials for the reconstruction video.
Fig. 13: Performance comparison of ODAUVST-5stg with a fixed fractional shifts range ω and a learnable fractional shifts range ω on the test set.
Fixed integer shifts S
[1,1]
[1,1]
[1,1]
[1,1]
[2,2]
[3,3]
Fractional shifts range ω
3
2
1
learned
learned
learned
PSNR/dB
33.01
33.14
33.35
33.31
31.83
30.99
SSIM
0.944
0.947
0.949
0.951
0.929
0.912
TABLE VI: Ablation study on LSM with different fixed integer shifts S and different fractional shifts range ω .
padding modes
reflection
zero padding
border
PSNR
33.31
33.10
32.20
SSIM
0.951
0.948
0.945
TABLE VII: Ablation study on CSM with different padding methods.
Framework
Denoiser
K
PSNR/dB
SSIM
Time
Params
GFLOPs
COPF
SST
5
31.58
0.936
5ms
3.67M
40.49
ODAUF
SST
5
32.97
0.946
12ms
4.34M
46.59
COPF
VST
5
31.92
0.937
16ms
3.85M
47.54
ODAUF
VST
5
33.31
0.951
17ms
4.53M
53.64
COPF
SST
9
32.55
0.945
17ms
6.56M
70.50
TABLE VIII: Ablation study on ODAUF, VST with COPF, SST
Fig. 14: Comparison of simulation reconstruction with different mask forms.
Fig. 15: (a) The depth invariance of ADIS is observed when the depth is longer than 3 m; i.e., the SSIM of PSFs increases significantly (higher than 0.9). (b) The PSFs at different positions from the reference center to the end of the vertical FOV are compared within the valid field of view of 35 degrees. The position variance of the PSF is negligible with our optical configuration.
Fig. 16: ADIS with ODAUVST (learnable ω ) achieves high-quality real spectral acquisition in scenes with smooth content.
Fig. 17: (a) Comparison of simulated PSFs generated by an ADIS mask with 100 μm line width and 200 μm period under different random fabrication errors (0, 2, 5 μm ). (b) Comparison between theoretically calculated PSFs and experimentally measured ADIS PSFs under different lens configurations sharing the same effective focal length. For additional PSFs of ADIS under different random errors and different lenses, please refer to the supplementary materials.
Existing computational spectral imaging systems typically rely on coded aperture and beam splitters that block a substantial fraction of incident light, degrading reconstruction quality under light-starved conditions. To address this limitation, we develop the Oscillating Dispersion Imaging Spectrometer (ODIS), which for the first time achieves near-full light throughput by axially translating a disperser between the conjugate image plane and a defocused position, sequentially capturing a panchromatic (PAN) image and a dispersed measurement along a single optical path. We further propose a PAN-guided Dispersion-Aware Deep Unfolding Network (PDAUN) that recovers high-fidelity spectral information from maskless dispersion under PAN structural guidance. Its data-fidelity step derives an FFT-Woodbury preconditioned solver by exploiting the cyclic-convolution property of the ODIS forward model, while a Dispersion-Aware Deformable Convolution module (DADC) corrects sub-pixel spectral misalignment using PAN features. Experiments show state-of-the-art performance on standard benchmarks, and cross-system comparisons confirm that ODIS yields decisive gains under low illumination. High-fidelity reconstruction is validated on a physical prototype.
Jiuyun Zhang, Zhan Shi, Linsen Chen +1
Nanjing University, Nanjing, China · Key Laboratory of Optoelectronic Devices and Systems with Extreme Performances of MOE, Nanjing University, Nanjing, China
Enhancing perceptual dimensions while miniaturizing imaging systems presents significant challenges for high-dimensional visual sensing. Conventionally, the acquisition of the 5D (x,y,u,v,λ) spectral light field (5D-SLF) data cube relies on bulky and expensive camera arrays, which are impractical for widespread application. Existing single-detector systems are fundamentally limited by a trade-off between the resolutions of different dimensions owing to insufficient coding capabilities. Here we introduce an Aperture-aware Dispersion Light-field Imaging Spectrometer (ADLIS), that targets a synergy between compactness and resolution through aperture-multiplexed modulation, leveraging the inherent spectral-filtering properties of birefringent material. Using only a manufacturing-friendly and cost-effective phase plate made of birefringent quartz crystal, the aperture of the proposed ADLIS enables compact angular-spectral encoding that is highly sensitive to both the incident angle and spectrum of incoming light. In contrast to the viewpoint-separation approach of microlens arrays, ADLIS employs aperture encoding to superimpose all viewpoints onto each sensor pixel. This shifts the design paradigm from spatial division to encoding integration, aiming to achieve full-resolution light field recovery. Thus, we develop the Aperture-aware Dispersion Light-field Imaging (ADLI) framework, which optimizes the aperture design and 5D-SLF reconstruction in an end-to-end (E2E) manner. Trained by simulation data and validated through real-world experiments, our system achieves robust high-performance 5D-SLF imaging while maintaining full spatial resolution.
Chenglong Huang, Tao Lv, Jianing Yang +3
Nanjing University, Nanjing, 210023, China · Key Laboratory of Optoelectronic Devices and Systems with Extreme Performances of MOE, Nanjing University
Snapshot hyperspectral imaging avoids sequential scanning, but systems that jointly achieve stable reconstruction, low cost, and compact optics remain limited. We present a snapshot hyperspectral imaging system based on angular-to-spectral diversity conversion. A tapered kaleidoscope creates replicated views with distinct incidence directions, and a directly attached birefringent filter converts them into view-channel-dependent spectral transmittances, yielding complementary measurements that better condition the inverse problem for more stable single-shot spectral reconstruction. The system preserves a simple pixel-wise linear model for fast non-learning-based reconstruction and uses only off-the-shelf components without relay optics or cascaded modules. We select the birefringent filter configuration using a condition-number-based criterion and validate the system on both synthetic and real data.