eess.IVMay 20, 2022

Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging

Authors: Yuanhao Cai, Jing Lin, Haoqian Wang, Xin Yuan, Henghui Ding, Yulun Zhang, Radu Timofte, Luc Van Gool

Organizations: Shenzhen International Graduate School, Tsinghua University · Shenzhen Institute of Future Media Technology · Westlake University · ETH Zürich · 

Abstract

In coded aperture snapshot spectral compressive imaging (CASSI) systems, hyperspectral image (HSI) reconstruction methods are employed to recover the spatial-spectral signal from a compressed measurement. Among these algorithms, deep unfolding methods demonstrate promising performance but suffer from two issues. Firstly, they do not estimate the degradation patterns and ill-posedness degree from the highly related CASSI to guide the iterative learning. Secondly, they are mainly CNN-based, showing limitations in capturing long-range dependencies. In this paper, we propose a principled Degradation-Aware Unfolding Framework (DAUF) that estimates parameters from the compressed image and physical mask, and then uses these parameters to control each iteration. Moreover, we customize a novel Half-Shuffle Transformer (HST) that simultaneously captures local contents and non-local dependencies. By plugging HST into DAUF, we establish the first Transformer-based deep unfolding method, Degradation-Aware Unfolding Half-Shuffle Transformer (DAUHST), for HSI reconstruction. Experiments show that DAUHST significantly surpasses state-of-the-art methods while requiring cheaper computational and memory costs. Code and models will be released at https://github.com/caiyuanhao1998/MST . An online Gradio demo is available at https://huggingface.co/spaces/CaiYuanhao/MST

Figures & tables

Explore similar work

CardsList
  1. Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstruction

    Mar 9, 2022Yuanhao Cai, Jing Lin, Xiaowan Hu +5Hyperspectral Image ReconstructionImage Reconstruction

  2. Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction

    Nov 15, 2021Yuanhao Cai, Jing Lin, Xiaowan Hu +5Hyperspectral Image ReconstructionImage Reconstruction

  3. Manifold Prior Guided Deep Unfolding for Hyperspectral Image Reconstruction

    Oct 20, 2026Shuqi Liu, Yingkai Zhang, Lin Gu +2Hyperspectral Image ReconstructionDeep Unfolding