Recently, Spectral Compressive Imaging (SCI) has achieved remarkable success, unlocking significant potential for dynamic spectral vision. However, existing reconstruction methods, primarily image-based, suffer from two limitations: (i) Encoding process masks spatial-spectral features, leading to uncertainty in reconstructing missing information from single compressed measurements, and (ii) The frame-by-frame reconstruction paradigm fails to ensure temporal consistency, which is crucial in the video perception. To address these challenges, this paper seeks to advance spectral reconstruction from the image level to the video level, leveraging the complementary features and temporal continuity across adjacent frames in dynamic scenes. Initially, we construct the first high-quality dynamic hyperspectral image dataset (DynaSpec), comprising 30 sequences obtained through frame-scanning acquisition. Subsequently, we propose the Propagation-Guided Spectral Video Reconstruction Transformer (PG-SVRT), which employs a spatial-then-temporal attention to effectively reconstruct spectral features from abundant video information, while using a bridged token to reduce computational complexity. Finally, we conduct simulation experiments to assess the performance of four SCI systems, and construct a DD-CASSI prototype for real-world data collection and benchmarking. Extensive experiments demonstrate that PG-SVRT achieves superior performance in reconstruction quality, spectral fidelity, and temporal consistency, while maintaining minimal FLOPs. Project page: https://github.com/nju-cite/DynaSpec
Figures & tables
Figure 1: Spectral compressive imaging and reconstruction. (a) SCI principle. (b) Image-based methods, with issues of uncertain reconstruction and temporal inconsistency (flickering intensity curves). (c) Video-based reconstruction, where information complementarity enhances completeness and temporal consistency (smooth intensity curves).
Figure 2: The proposed DynaSpec dataset. (a) Dynamic HSIs sequences acquired frame by frame to simulate the diverse motion of real-world scenarios. (b) A display of the 30 scenes.
Figure 3: Illustration of PG-SVRT. (a) and (c) The components of MGDP and CDBP. (b) PG-SVRT framework and key components.
Figure 4: Details of the CDPB, which consists primarily of CDPA and MDFFN. (a) CDPA is a spatial-then-temporal attention mechanism, where the blue line represents spatial feature processing and the red line indicates temporal feature processing. (b) Illustration of MDFFN.
PMVIS
SD-CASSI
NDSSI
DD-CASSI
PSNR ↑
28.45
37.78
37.84
41.52
SSIM ↑
0.8456
0.9700
0.9825
0.9893
SAM ↓
5.4162
4.0737
5.4091
3.9084
ST-RRED ↓
459.49
23.21
91.8
23.25
Table 1: Quantitative comparison of representative SCI architectures. All systems are solved via PG-SVRT under a unified mathematical framework to exclude algorithmic bias.
Figure 5: Measurements of different SCI systems
Method
MST-L CVPR 2022
CST-L ECCV 2022
DAUHST NeurIPS 2022
GAP-Net IJCV 2023
DADF-Plus-3 TMI 2023
RDLUF CVPR 2023
PADUT ICCV 2023
S2 -Transfor. TPAMI 2024
SSR CVPR 2024
DPU CVPR 2024
DPU ∗ CVPR 2024
PG-SVRT Ours
PSNR-K ↑
39.99
39.93
38.98
36.92
38.23
39.26
38.61
33.26
39.04
40.02
40.50
41.23
SSIM-K ↑
0.9881
0.9864
0.9832
0.9755
0.9832
0.9860
0.9828
0.9617
0.9842
0.9856
0.9853
0.9882
SAM-K ↓
3.8248
4.1342
5.4514
6.1204
4.7676
4.2932
4.7154
8.0837
5.2201
5.2250
5.1685
3.805
ST-RRED-K ↓
30.99
35.11
37.27
85.34
48.45
39.06
47.19
155.82
38.29
25.90
26.71
19.35
PSNR-D ↑
39.58
40.06
40.39
39.38
39.00
39.26
40.41
37.10
39.66
41.01
41.36
41.82
SSIM-D ↑
0.9873
0.9876
0.9883
0.9851
0.9861
0.9863
0.9881
0.9786
0.9873
0.9893
0.9889
0.9904
Table 2: Quantitative comparisons of several SOTA methods and PG-SVRT. The suffix -K denotes results on the KAIST, while -D represents evaluations on the DynaSpec testset. The best and second best results are highlighted in bold and underline, respectively.
Figure 6: Reconstruction results of PG-SVRT and comparison methods on the KAIST and DynaSpec test sets. The bottom-left corner of each subplot presents an enlarged detail view, while the bottom-right corner shows the difference with the GT. It is evident that, while all methods benefit from DD-CASSI and are able to recover structural details, our method achieves the superior fidelity.
Figure 7: Reconstruction of real measurements using comparison methods and PG-SVRT, with pseudo-RGB images generated from the reconstructed HSIs to assess reconstruction quality across all bands. Compared to other methods, PG-SVRT results exhibit fewer artifacts.
Figure 8: Complex real-world scenes reconstructed by PG-SVRT show clear structures and fine details in both the pseudo-RGB and HSIs.
Baseline
+ CDPA
+ MGDP
+ MDFFN
PSNR
39.97
41.30
41.41
41.52
SSIM
0.9827
0.9884
0.9886
0.9893
SAM
5.5312
4.3224
4.2513
3.9084
STRRED
43.90
25.44
24.63
23.25
Params
2.17 M
1.92 M
1.92 M
2.48M
GFLOPs
30.11
21.11
21.31
28.18
Table 3: Break-down ablation study of PG-SVRT, "+" indicates adding or replacing modules relative to the baseline.
Parallel
T-S
T-S w/ P
S-T
S-T w/ P
PSNR
41.35
41.04
41.47
41.08
41.52
SSIM
0.9886
0.9877
0.9892
0.9880
0.9893
SAM
4.1518
4.4995
3.9605
4.4128
3.9084
STRRED
26.00
28.67
26.60
26.40
23.25
Params
2.85 M
2.60 M
2.48 M
2.60 M
2.48 M
GFLOPs
33.23
30.16
28.18
30.16
28.18
Table 4: Comparison of spatiotemporal processing strategies.
Deep learning algorithms for video Snapshot Compressive Imaging (SCI) have achieved great success, yet they predominantly focus on reconstructing from clean measurements. This overlooks a critical real-world challenge: the captured signal itself is often severely degraded by motion blur and low light. Consequently, existing models falter in practical applications. To break this limitation, we pioneer the first study on robust video SCI restoration, shifting the goal from "reconstruction" to "restoration"--recovering the underlying pristine scene from a degraded measurement. To facilitate this new task, we first construct a large-scale benchmark by simulating realistic, continuous degradations on the DAVIS 2017 dataset. Second, we propose RobustSCI, a network that enhances a strong encoder-decoder backbone with a novel RobustCFormer block. This block introduces two parallel branches--a multi-scale deblur branch and a frequency enhancement branch--to explicitly disentangle and remove degradations during the recovery process. Furthermore, we introduce RobustSCI-C (RobustSCI-Cascade), which integrates a pre-trained Lightweight Post-processing Deblurring Network to significantly boost restoration performance with minimal overhead. Extensive experiments demonstrate that our methods outperform all SOTA models on the new degraded testbeds, with additional validation on real-world degraded SCI data confirming their practical effectiveness, elevating SCI from merely reconstructing what is captured to restoring what truly happened.
Hao Wang, Zhankuo Xu, Jiong Ni +3
Westlake University · Xi’an Jiaotong University · Dalian University of Technology +1
With the advent of snapshot cameras, hyperspectral video is becoming more readily available. In recent years, new applications have emerged which have led to increasingly larger datasets. However, hyperspectral video compression remains in the early stages. In this study, we explore the use of implicit neural representation as a candidate solution. We propose a novel extension of an existing RGB video compression model, achieving Bjøntegaard Delta PSNR gains of +4.99 dB and Bjøntegaard Delta rate of -88.88% compared to traditional hyperspectral image compression methods applied frame-by-frame. In addition to reconstruction quality, the effects on downstream task performance are measured in the form of object tracking success. Compared to video compressed with methods based on principal component analysis and JPEG2000 in low data regimes, our proposed method improves tracking area under the curve by up to 23.42% and distance precision by up to 35.56% on examples from the HOT2026 dataset.
Alfredo Scalera, Paul Murray, Jaime Zabalza
University of Strathclyde Department of Electronic & Electrical Engineering Glasgow, UK
While Deep Unfolding Networks (DUNs) dominate video Snapshot Compressive Imaging (SCI), they remain constrained by a uniform design philosophy. Existing methods repeatedly stack high-complexity priors with identical structures, ignoring the fact that optimization trajectories converge toward static states. This results in representation stagnation, where high-cost computations are wasted on minimal feature updates. To address this inefficiency, we present Differential Unfolding (DU), a heterogeneous framework that replaces uniform repetition with dynamic evolution. Central to DU is the Differential Evolutionary Framework (DEF), which partitions the unfolding process into two complementary roles: structural anchoring and differential evolution. In this scheme, high-parameter general stages are sparsely deployed to generate high-fidelity feature foundations. Complementing these, lightweight differential stages employ a Differential Representation Prior (DRP) to propagate and refine these foundational features through a differential mechanism. By integrating Differential Representation Attention (DRA) for evolving attention maps and a Differential Modulated FFN (DM-FFN) for feature rectification, DRP effectively models cross-stage variations with minimal overhead. By focusing computational resources on dynamic evolution rather than static redundancy, DU achieves a superior trade-off between accuracy and efficiency. Extensive experiments verify that our method establishes new state-of-the-art results while significantly slashing computational overhead. https://github.com/Muyuan-Zhang/DU
Muyuan Zhang, Jiancheng Zhang, Haijin Zeng +1
Northwestern Polytechnical University, Xi’an, China · Harbin Institute of Technology (Shenzhen), Shenzhen, China