Compact Snapshot Spectral Imaging with Calibration-Free Aperture Diffraction

Tao Lv; Quan Yuan; Shiqiao Li; Chenglong Huang; Linsen Chen; Chongde Zi; Shuming Wang; Xun Cao

Fig. 1: The imaging principle of ADIS. Light from the target scene is modulated by the Orthogonal Diffraction Lens (ODL) and captured by an array-filtered image sensor. The compressed measurements are reconstructed into HSIs using theoretically computed PSFs and filter functions (sensor parameters) as priors. The theoretically computed PSFs enable calibration-free reconstruction while tolerating lens-dependent variations across different optical configurations. The right side shows exemplar measurements and the corresponding reconstructed HSIs.

Fig. 2: (a) Calibration-free reconstruction of ADIS using theoretically computed PSFs, with robustness to lens-dependent convolutional degradation across different lens configurations; (b) ADIS forward propagation with collimated illumination; (c) illustrates the principles of CTIS and strategy of using long optical path by sacrificing spatial resolution, while ADIS reconstructs from aliasing; (d) depicts compact imaging methods with mosaic filter sensors.

Fig. 3: (a) demonstrates the application of the Babinet principle in the design of ADIS masks, which can be used to enhance light throughput; (b) depicts an alternative configuration of ADIS whereby the mask is located between two plano-convex lenses, producing a similar encoding process; (c) is the overview of proposed Aperture Diffraction Imaging Spectrometer (ADIS). The system consists of an optical encoder and a computational decoder, which encodes the HSIs into 2D images and reconstructs the underlying HSIs from the compressed measurements, respectively.

Fig. 4: Comparison of reconstruction fidelity for masks with different periods on the test set, evaluated using Spectral Angle Mapping (SAM) as the spectral accuracy metric. Fifteen diverse outdoor scenes from the NTIRE 2020 dataset are used as a test set to assess the suitability of the chosen mask parameters.

Fig. 5: Illustration of ODAUF with k stages. Theoretically, the denoisers in ODAUF as learnable regularization terms can be replaced with different designs.

Fig. 6: (a) Diagram of VST. VST adopts a three-layer U-shaped structure. VSAB consists of an FFN, a VS-MSA, and two layer normalizations. (b) VS-MSA employs parameters learned via LSM to perform feature shifting (CSM) and self-attention calculations on input features. (c) Illustration of LSM and CSM.

Algorithm	Inference Time	Params	GFLOPS	S1	S2	S3	S4	S5	S6	S7	S8	S9	S10	Avg
				31.43	29.64	25.44	33.50	28.77	28.38	26.78	31.72	36.40	27.96	30.00
UNet [ 45 ]	1ms	23.32M	5.33	0.940	0.912	0.877	0.914	0.897	0.884	0.852	0.916	0.936	0.892	0.902
				29.27	28.07	23.02	31.53	27.40	26.29	23.52	30.06	32.67	26.17	27.80
MIRNet [ 46 ]	1ms	2.05M	14.66	0.910	0.870	0.842	0.898	0.873	0.858	0.833	0.905	0.854	0.844	0.869
				32.18	29.90	26.30	34.20	30.26	29.00	27.73	32.32	34.18	27.15	30.32
Lambda-Net [ 37 ]	1ms	32.73M	23.08	0.927	0.909	0.871	0.886	0.902	0.880	0.844	0.916	0.900	0.883	0.892

TABLE I: Quantitative comparison of reconstruction results of different algorithms, PSNR (dB) and SSIM are reported.

Fig. 7: Simulation HSI reconstruction comparisons of different scenes with 10 (out of 28) spectral channels on ADIS. The left side displays the RGB image of the scene alongside the simulated measurements, while the central side presents a comparison between the reconstructed intensity-spectral curves at marked points on the RGB image and the corresponding Ground Truth.

Fig. 8: Comparison of ADIS and two different existing compact computational spectral imaging methods, a DOE-based system and a prism-based system with ten ground-truth spectral images.

System	PSNR (dB)	SSIM	RMSE	SAM
Prism-based [ 29 ]	34.26	0.967	0.0199	4.45
DOE-based [ 11 ]	35.60	0.973	0.0173	4.34
ADIS (Ours)	35.97	0.978	0.0169	3.96

TABLE II: Quantitative comparison of ADIS with DOE-based system and prism-based system, PSNR (dB) and SSIM are reported.

System	PSNR (dB)	SSIM	Params	GFLOPS
Jeon2019	30.33	0.907	3.79M	44.01
COPF	31.92	0.937	3.85M	47.54
ODAUF	33.31	0.951	4.53M	53.64

TABLE III: Quantitative comparison of ODAUF with two alternative frameworks for ADIS reconstruction.

Fig. 9: ADIS-V1: Compact prototype based on a single-lens configuration and representative reconstruction results. The mask is directly on the front surface of the lens. Unlike prior methods relying on calibrated PSFs, we reconstruct these scenes directly using computed PSFs. The reference spectral curves (ground truth, GT) are measured with a point spectrometer.

Algorithm	$\varpi$	PSNR	SSIM	$\varpi$	PSNR	SSIM	$\varpi$	PSNR	SSIM
ODAUVST-5stg	0.01	32.38	0.928	0.02	31.62	0.908	0.05	30.11	0.873
CSST-9stg [ 44 ]	0.01	32.12	0.923	0.02	31.34	0.906	0.05	30.06	0.872
Restormer [ 49 ]	0.01	31.30	0.916	0.02	30.51	0.897	0.05	29.44	0.869

TABLE IV: Quantitative comparison of ODAUVST-5stg with Restormer and CSST-9stg under different noise levels, PSNR (dB), and SSIM are reported.

System	$\varpi$	PSNR	SSIM	SAM	$\varpi$	PSNR	SSIM	SAM
ADIS	0.00	35.97	0.978	3.96	0.01	32.81	0.939	5.90
DOE-based	0.00	35.60	0.973	4.34	0.01	30.35	0.864	6.36

TABLE V: Quantitative comparison of ADIS with the DOE-based system under noise conditions.

Fig. 10: ADIS-V2: We capture three natural scenes using our real prototype system as shown in (a). We directly reconstruct these scenes utilizing the computed PSFs corresponding to those of ADIS-V1. The reference spectral curves (ground truth, GT) are measured with a point spectrometer. Notably, our method eliminates artifacts caused by simulation-to-reality gaps inherent to DOE-based acquisition [ 11 , 12 ] , and the comparisons can be found in the Supp. Material.

Fig. 11: Quantitative evaluation of our ADIS prototype. (a) shows the reconstructed ColorChecker and reconstruction curves vs. ground truth. (b) demonstrates the spatial accuracy of our reconstruction. We compare the modulation transfer functions of the input image and the output reconstruction using the square region. (c) Metameric material discrimination using ADIS.

Fig. 12: The snapshot spectral imaging results of ADIS-V2 in an autonomous-driving scenario. Please refer to the supplementary materials for the reconstruction video.

Fig. 13: Performance comparison of ODAUVST-5stg with a fixed fractional shifts range

\omega

and a learnable fractional shifts range

\omega

on the test set.

Fixed integer shifts $\mathcal{S}$	[1,1]	[1,1]	[1,1]	[1,1]	[2,2]	[3,3]
Fractional shifts range $\omega$	3	2	1	learned	learned	learned
PSNR/dB	33.01	33.14	33.35	33.31	31.83	30.99
SSIM	0.944	0.947	0.949	0.951	0.929	0.912

TABLE VI: Ablation study on LSM with different fixed integer shifts

\mathcal{S}

and different fractional shifts range

\omega

.

padding modes	reflection	zero padding	border
PSNR	33.31	33.10	32.20
SSIM	0.951	0.948	0.945

TABLE VII: Ablation study on CSM with different padding methods.

Framework	Denoiser	K	PSNR/dB	SSIM	Time	Params	GFLOPs
COPF	SST	5	31.58	0.936	5ms	3.67M	40.49
ODAUF	SST	5	32.97	0.946	12ms	4.34M	46.59
COPF	VST	5	31.92	0.937	16ms	3.85M	47.54
ODAUF	VST	5	33.31	0.951	17ms	4.53M	53.64
COPF	SST	9	32.55	0.945	17ms	6.56M	70.50

TABLE VIII: Ablation study on ODAUF, VST with COPF, SST

Fig. 14: Comparison of simulation reconstruction with different mask forms.

Fig. 15: (a) The depth invariance of ADIS is observed when the depth is longer than 3 m; i.e., the SSIM of PSFs increases significantly (higher than 0.9). (b) The PSFs at different positions from the reference center to the end of the vertical FOV are compared within the valid field of view of 35 degrees. The position variance of the PSF is negligible with our optical configuration.

Fig. 16: ADIS with ODAUVST (learnable

\omega

) achieves high-quality real spectral acquisition in scenes with smooth content.

Fig. 17: (a) Comparison of simulated PSFs generated by an ADIS mask with 100

\mu m

line width and 200

\mu m

period under different random fabrication errors (0, 2, 5

\mu m

). (b) Comparison between theoretically calculated PSFs and experimentally measured ADIS PSFs under different lens configurations sharing the same effective focal length. For additional PSFs of ADIS under different random errors and different lenses, please refer to the supplementary materials.

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Compact Snapshot Spectral Imaging with Calibration-Free Aperture Diffraction

Abstract

Figures & tables

Explore similar work

Oscillating Dispersion for Maximal Light-throughput Spectral Imaging

Aperture-aware Dispersion 5-D Light-field Imaging Spectrometer

Compact Low-Cost Hyperspectral Imaging via Angular-to-Spectral Diversity Conversion