Hyperspectral Image Reconstruction

Follow this topic Ask about this topic

Latest papers

20 papers1 paper in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Oct 20, 2026

Manifold Prior Guided Deep Unfolding for Hyperspectral Image Reconstruction

Hyperspectral Image (HSI) reconstruction has made substantial progress with the deep unfolding framework by decomposing the problem into a projection module and a denoiser module. Nevertheless, existing methods still exhibit limitations of insufficient matching with HSI data. The issues lie in two aspects: 1) the data-consistency projection module applies a fixed gradient descent step while ignoring the spectral content adaptivity of HSI; 2) the denoiser module either suffers from high computational complexity or is limited to the local receptive field, failing to efficiently capture the global spatial-spectral dependencies. In this paper, we propose a deep unfolding framework centered on a two-stage manifold learning strategy to obtain a degradation-free structural prior, which is then strategically injected into both the projection and denoiser modules. In the data-consistency projection, a prior-guided spectral weight mechanism is introduced to adaptively modulate the weight of spectral band. In the denoiser module, a manifold-guided Mamba is proposed to achieve the structure-aware extraction of spatial-spectral features through prior modulation. Quantitative and visual comparisons on synthetic and real-world datasets show that the proposed method significantly outperforms existing state-of-the-art approaches, while maintaining the structural interpretability of the unfolding framework. The code is available at https://github.com/SqLiu67/MPG-DU.

Shuqi Liu, Yingkai Zhang, Lin Gu +2

Beijing Institute of Technology · Tohoku University · Hangzhou Dianzi University

Hyperspectral Image Reconstruction Deep Unfolding

Sep 30, 2026

Super-Resolving Unseen Hyperspectral Sensors at Any Scale via Spatial Operators

Achieving cross-sensor generalization and arbitrary-scale reconstruction with a single model remains challenging in hyperspectral super-resolution (HSR). Although recent methods support arbitrary-scale reconstruction, applying them to new sensors or scales beyond the training range often requires additional data and computation to maintain reconstruction quality. To address these challenges, we propose OmniHSR, which predicts band-shared spatial operators rather than spectral values. Cross-Spectral Mapping (CSM) resamples inputs with any number of bands to fixed reference positions and predicts local operators with Gaussian supports. Continuous Operator-Field Reconstruction (COFR) composes these operators into a continuous field and applies them to all original bands for arbitrary-scale reconstruction. Experiments demonstrate that operator prediction outperforms direct spectral-value prediction on all seven datasets. Trained solely on ARAD with only 0.538M parameters, OmniHSR outperforms all directly transferred baselines on six unseen datasets without target-domain training data or adaptation. Across twelve upsampling factors from

\times2

to

\times48

, it improves average PSNR on Pavia U and Chikusei by 0.55 dB over the strongest baseline. It also surpasses baselines trained from scratch or adapted on the target sensor and achieves up to

36\times

faster inference. Our code will be publicly released soon.

Ji-Xuan He, Guohang Zhuang, Bo Junge +6

Xi’an Jiaotong University · Hefei University of Technology · National University of Singapore +1

Hyperspectral Image Super-Resolution Operator Learning Image Super-Resolution Hyperspectral Image Reconstruction

Sep 20, 2026

Compact Low-Cost Hyperspectral Imaging via Angular-to-Spectral Diversity Conversion

Snapshot hyperspectral imaging avoids sequential scanning, but systems that jointly achieve stable reconstruction, low cost, and compact optics remain limited. We present a snapshot hyperspectral imaging system based on angular-to-spectral diversity conversion. A tapered kaleidoscope creates replicated views with distinct incidence directions, and a directly attached birefringent filter converts them into view-channel-dependent spectral transmittances, yielding complementary measurements that better condition the inverse problem for more stable single-shot spectral reconstruction. The system preserves a simple pixel-wise linear model for fast non-learning-based reconstruction and uses only off-the-shelf components without relay optics or cascaded modules. We select the birefringent filter configuration using a condition-number-based criterion and validate the system on both synthetic and real data.

Kazuma Fujiwara, Takuya Funatomi, Kazuya Kitano +2

Nara Institute of Science and Technology, Nara, Japan · Kyoto University, Kyoto, Japan

Hyperspectral Imaging Computational Imaging Hyperspectral Image Reconstruction Image Inverse Problems

Sep 4, 2026

Learning Spatial-Spectral Refinement and Calibrating Complementary Observations for Hyperspectral Image Super-Resolution

Hyperspectral and multispectral image fusion (HMIF) aims to reconstruct a high-resolution hyperspectral image (HR-HSI) by combining the fine spatial details of a high-resolution multispectral image (HR-MSI) with the rich spectral information of a low-resolution hyperspectral image (LR-HSI). Recent advances in implicit neural representations (INRs) have enabled flexible coordinate-based modeling for HMIF; however, existing INR-based approaches may not fully capture fine-grained spatial structures and rich spectral dependencies. Moreover, the LR-HSI and HR-MSI are primarily incorporated through degradation-consistency constraints, leaving their complementary information underexploited. To address these limitations, we propose Two-Stage Reconstruction with Implicit Tensor Neural Representation (TSR-ITNR), a unified self-supervised framework integrating representation refinement and observation-guided calibration. In Stage 1, TSR-ITNR learns an implicit Tucker representation and refines its low-rank spatial coefficient tensor and spectral basis to better capture fine spatial structures and interband correlations. A fixed pretrained denoiser further provides a deep prior for the preliminary reconstruction. In Stage 2, parameter-free calibration derives complementary and noninterfering corrections from both observations to recover information insufficiently captured in Stage 1. Theoretical analysis establishes the geometry-preserving property of spectral refinement and the orthogonal complementarity of calibration. Extensive experiments on multiple benchmark datasets demonstrate strong quantitative, visual, and spectral reconstruction performance without ground-truth HR-HSI supervision. Beyond conventional reconstruction metrics, we further assess the effectiveness of TSR-ITNR using downstream semantic segmentation accuracy.

Liqian Yang, Xingchi Chen, Xinfeng Gui +2

School of Management, Zhengzhou University, Zhengzhou, 450001, China · Center for Intelligent Decision-making and Machine Learning, Xi’an Jiaotong University, Xi’an, 710049, China · School of Computer Science and Technology, Xi’an Jiaotong University, Xi’an, 710049, China

Hyperspectral Image Super-Resolution Hyperspectral Image Reconstruction Image Reconstruction Multi-Source Image Fusion Implicit Neural Representations

Aug 15, 2026

Registration-Free Hyperspectral Reconstruction from RGB via a Permutation-Invariant Gram-Matrix Principle

Reconstructing a spatially and spectrally high-resolution hyperspectral image (HR-HSI) from a low-resolution HSI (LR-HSI) and a high-resolution RGB image (HR-RGB) usually assumes precise registration and a known camera response function (CRF). Both assumptions are difficult to satisfy with different sensors. We remove both through a permutation-invariant supervision principle: the Gram matrix of an unmixed abundance map depends on shared material composition but not on pixel ordering. Matching abundance Gram matrices therefore allows RGB-to-HSI mapping to be learned without spatial correspondence and without a predefined CRF. Under a full random permutation of HR-RGB pixels, a state-of-the-art fusion method collapses, whereas our reconstruction is unchanged after inverse reindexing for evaluation. Building on this principle, a residual spectral super-resolution function maps HR-RGB directly to HR-HSI without registration, known CRF, or paired supervision. Across indoor, natural-scene, and remote-sensing benchmarks, the method achieves accuracy comparable to approaches that require these assumptions while remaining robust when they are violated. Loss ablations further show that reconstruction accuracy is largely insensitive to the specific discrepancy used to match the Gram matrices, indicating that performance arises primarily from the permutation-invariant principle rather than loss tuning.

Jiangsan Zhao, Masayuki Hirafuji, Seishi Ninomiya +2

Department of Agricultural Technology, Norwegian Institute of Bioeconomy Research (NIBIO), Ås, Norway · Laboratory of Field Phenomics, Graduate School of Agriculture and Life Sciences, The University of Tokyo, Nishitokyo, Tokyo 188-0002, Japan · Plant Phenomics Research Center, Nanjing Agricultural University, Nanjing, China

Hyperspectral Image Reconstruction Hyperspectral Imaging Hyperspectral Image Super-Resolution Permutation-Invariant Learning

Jun 4, 2026

Physics-Guided Deep Unfolding for Blind Cross-Sensor Spectral Super-Resolution via Learning the Spectral Transformation Function

Hyperspectral imaging provides rich spectral information for quantitative remote sensing, yet hyperspectral sensors remain costly and thus unavailable in many UAV deployments. Spectral super-resolution (SSR) seeks to reconstruct hyperspectral images (HSIs) from multispectral images (MSIs). Most existing SSR methods assume a fixed and known spectral response function (SRF) and are therefore limited to single-sensor settings. In practical cross-sensor scenarios, the spectral degradation from HSI to MSI is unknown and varies with sensor characteristics and scene content, which renders HSI reconstruction ill-posed. This paper proposes a physics-guided deep unfolding network, termed PGU-Net, to address blind cross-sensor SSR by jointly estimating the HSI and a learnable spectral transformation function (STF). PGU-Net unrolls an alternating optimization procedure into an end-to-end trainable architecture with stages, where each stage sequentially updates the HSI and the STF. Both modules combine learnable proximal networks with differentiable closed-form solvers, enabling physical interpretability while retaining strong representation capacity. Experiments on benchmark datasets (CAVE and NTIRE 2022) with multiple SRFs demonstrate accurate recovery of the STF (degradation operator) and improved reconstruction performance over state-of-the-art SSR methods. Furthermore, evaluations on a real UAV cross-sensor dataset (Headwall Nano HSI and DJI P4 Multispectral MSI) verify the effectiveness and robustness of PGU-Net under truly blind conditions, and suggest that the estimated STF may exhibit land-cover-related differences.

Zhaolin Li, Jinsong Chen, Shanxin Guo +3

a Center for Geo-Spatial Information, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055, PR China · c Shenzhen Engineering Laboratory of Ocean Environmental Big Data Analysis and Application, Shenzhen, 518055, PR China · b University of Chinese Academy of Sciences, Beijing 100049, China

Hyperspectral Image Super-Resolution Hyperspectral Imaging Deep Unfolding Hyperspectral Image Reconstruction

May 23, 2026

Leveraging pretrained RGB denoisers for hyperspectral image restoration

Hyperspectral image restoration faces several challenges, including limited training data, strong sensor specificity, and high spectral dimensionality. These limitations hinder the learning of robust hyperspectral priors, motivating the reuse of priors learned from large-scale RGB data. In this work, we propose a minimally trained, lightweight adapter that repurposes frozen pretrained RGB denoisers for hyperspectral restoration through a projection mapping. The method denoises low-dimensional spectral projections and reconstructs the hyperspectral cube through constrained linear aggregation, while preserving plug-and-play compatibility and the stability properties of the underlying RGB denoiser. Experiments on denoising, deblurring, and super-resolution across multiple datasets demonstrate consistent improvements over hyperspectral-specific baselines, showing the strong transferability of large-scale RGB priors.

Daniele Picone, Mohamad Jouni, Mauro Dalla-Mura

Univ. Grenoble Alpes, CNRS, Grenoble INP, GIPSA-Lab, 38000, Grenoble, France

Hyperspectral Imaging Transfer Learning Image Restoration Hyperspectral Image Reconstruction Image Denoising

May 20, 2026

HyperBench: Standardizing and Scaling Synthetic Evaluation for Hyperspectral Super-Resolution

Hyperspectral super-resolution (HSR) reconstructs a high-spatial-resolution hyperspectral image by fusing a low-resolution hyperspectral image (LR-HSI) with a high-resolution multispectral image (HR-MSI). In the absence of real-world paired data, HSR methods are evaluated almost exclusively on synthetic experiments derived from hyperspectral datasets through Wald's protocol. Despite the protocol's widespread adoption, its practical implementation varies markedly across research works, typically relying on a single (usually Gaussian) or very few point spread functions (PSFs), one or two spectral response functions (SRFs), and a couple of spatial downsampling factors. As a result, reported performance figures are difficult to compare across the literature, in addition to being often difficult to reproduce; furthermore, they may not generalize across realistic sensing conditions. We introduce HyperBench, a unified and extensible framework that standardizes synthetic experimentation for HSR. HyperBench supports diverse degradation configurations spanning ten PSFs, four SRFs derived from operational multispectral sensors, configurable spatial downsampling factors, and matched additive white Gaussian noise; its goal is to automate large-scale evaluation and structured logging. By decoupling model development from experimental design, the framework enables reproducible, apples-to-apples cross-method comparison with minimal friction. We use HyperBench to evaluate six recently proposed HSR methods across a 70-configuration sweep on four widely used hyperspectral scenes and observe that the inter-method PSNR spread widens from approximately 5 dB on the easiest PSF to over 13 dB on the hardest - a fragility that is structurally invisible to the prevailing single-configuration evaluation protocol. HyperBench code is available at https://github.com/ritikgshah/HyperBench .

Ritik Shah, Marco F. Duarte

University of Massachusetts Amherst, MA 01003

Synthetic Data Evaluation Remote Sensing Image Super-Resolution Hyperspectral Image Super-Resolution Hyperspectral Imaging Synthetic Benchmark Generation Benchmark Design+2

May 13, 2026

HADAR-Based Thermal Infrared Hyperspectral Image Restoration

Thermal-infrared (TIR) hyperspectral imagery (HSI) provides critical scene information for various applications. However, its practical utility is severely limited by unique sensor degradations beyond the capabilities of existing restoration methods, which are ignorant of underlying thermal physics. Here, we propose HAIR (HADAR-based Image Restoration) as a physics-driven framework for ground-based TIR-HSI restoration. HAIR utilizes the HADAR rendering equation (HRE) and combines it with the atmospheric downwelling radiative transfer equation (RTE) to model TIR-HSI using temperature, emissivity, and texture (TeX) physical triplets. This physical model leads to a TeX decompose-synthesize strategy that guarantees physical consistency and spatio-spectral noise resilience, in stark contrast to existing approaches. Moreover, our framework uses a forward-modeled atmospheric downwelling reference, along with spectral smoothness of emissivity and blackbody radiation, to enable spectral calibration and generation that would otherwise be elusive. Our extensive experiments on the outdoor DARPA Invisible Headlights dataset and in-lab FTIR measurements show that HAIR consistently outperforms state-of-the-art methods across denoising, inpainting, spectral calibration, and spectral super-resolution, establishing a benchmark in objective accuracy and visual quality.

Cheng Dai, Jiale Lin, Bingxuan Song +4

School of Science, Westlake University, Hangzhou 310030, China

Thermal Imaging Physics-Guided Image Restoration Remote Sensing Hyperspectral Imaging Image Restoration Hyperspectral Image Reconstruction+2

May 13, 2026

Phy-CoSF: Physics-Guided Continuous Spectral Fields Reconstruction and Super-Resolution for Snapshot Compressive Imaging

Recent advances have demonstrated that coded aperture snapshot spectral imaging (CASSI) systems show great potential for capturing 3D hyperspectral images (HSIs) from a single 2D measurement. Despite the inherent spectral continuity of scenes captured by CASSI, most existing reconstruction methods are restricted to fixed, discrete spectral outputs, thereby precluding continuous spectral reconstruction or spectral super-resolution. To address this challenge, we propose Phy-CoSF, which synergizes deep unfolding networks with implicit neural representations, establishing a new paradigm for continuous spectral reconstruction and super-resolution in CASSI. Specifically, we propose a two-phase architecture that bridges discrete-wavelength training with continuous spectral rendering, enabling the synthesis of high-fidelity HSIs at arbitrary target wavelengths. At the core of our framework lies the continuous spectral fields (CoSF) module, embedded within each unfolding stage as a dynamic prior, which comprises a triple-branch cross-domain feature mixer for comprehensive spatial-frequency-channel feature fusion, alongside a spectral synthesis head that generates spectral intensities by querying continuous wavelength coordinates. Extensive experimental results demonstrate that Phy-CoSF not only achieves continuous modeling at arbitrary spectral resolutions but also outperforms many state-of-the-art methods in both reconstruction fidelity and spectral detail preservation. Our code and more results are available at: https://github.com/PaiDii/Phy-CoSF.git.

Wudi Chen, Zhiyuan Zha, Xin Yuan +6

College of Communication Engineering, Jilin University, Changchun 130012, China. · School of Engineering, Westlake University, Hangzhou, Zhejiang 310024, China. · School of Electrical & Electronic Engineering, Nanyang Technological University, Singapore 639798. +4

Image Reconstruction Hyperspectral Image Super-Resolution Hyperspectral Imaging Deep Unfolding Hyperspectral Image Reconstruction

May 13, 2026

HIR-ALIGN: Enhancing Hyperspectral Image Restoration via Diffusion-Based Data Generation

Hyperspectral image (HSI) restoration is crucial for reliable analysis, as real-world HSIs suffer from noise, blur, and resolution loss. However, existing models trained on source data often fail on target domains lacking clean references, a common real-world scenario. To address this, we present HIR-ALIGN, a plug-and-play target-adaptive augmentation framework that enhances HSI restoration by augmenting limited training images with synthetic data matching the target distribution, without extra clean target-domain HSI data. It has three stages: (i) proxy generation, where off-the-shelf restoration models are applied to degraded target observations to produce semantics-preserving proxy HSIs that approximate clean target-domain images; (ii) distribution-adaptive synthesis, where a blur-robust unCLIP diffusion model generates target-aligned RGBs from proxy RGBs with prompt conditioning and embedding-space noise initialization. The warp-based spectral transfer module then synthesizes HSIs by aligning each generated RGB with its proxy RGB, estimating soft patch-wise transport weights, and applying these weights and learnable local interpolation kernels to the proxy HSI; and (iii) aligned supervised finetuning, where restoration networks pretrained on the source distribution are finetuned with proxy HSIs and synthesized target-aligned HSIs, then deployed on degraded target images. We also provide theoretical analysis showing that, under stated assumptions, the proposed augmentation-based finetuning obtains a tighter target-domain restoration-risk upper bound by jointly improving target-distribution coverage and controlling spectral bias. Experiments on simulated and real datasets across denoising, super-resolution, and other restoration tasks demonstrate that HIR-ALIGN is superior to proxy-only target-adaptation baselines and outperforms representative unsupervised methods in most cases.

Li Pang, Heng Zhao, Yijia Zhang +2

Synthetic Data Augmentation Unsupervised Domain Adaptation Hyperspectral Imaging Hyperspectral Image Reconstruction

May 7, 2026

R2H-Diff: Guided Spectral Diffusion Model for RGB-to-Hyperspectral Reconstruction

RGB-to-hyperspectral image reconstruction is a highly ill-posed inverse problem, since multiple plausible spectral distributions may correspond to the same RGB observation. Existing regression-based methods usually learn a deterministic mapping, which limits their ability to model reconstruction uncertainty and often leads to over-smoothed spectral responses. Although diffusion models provide strong distribution modeling capability, their direct application to hyperspectral reconstruction remains challenging due to the high spectral dimensionality, strong inter-band correlations, and strict requirement for spectral fidelity. To this end, we propose R2H-Diff, an efficient diffusion-based framework tailored for RGB-to-HSI reconstruction. Specifically, R2H-Diff formulates spectral recovery as a conditional iterative refinement process, enabling progressive reconstruction under RGB guidance. We proposed a Guided Spectral Refinement Module for RGB-conditioned feature fusion and a Hyperspectral-Adaptive Transposed Attention module for efficient spatial--spectral dependency modeling. Furthermore, a normalization-free denoising backbone is adopted to preserve spectral amplitude consistency, while a task-adapted linear noise schedule enables high-quality reconstruction with only five denoising steps. Extensive experiments on NTIRE2022, CAVE, and Harvard demonstrate that R2H-Diff achieves a favorable balance between reconstruction quality and computational efficiency. Notably, on NTIRE2022, R2H-Diff obtains 35.37 dB PSNR with a sub-million-parameter model of 0.58M parameters and 12.25G FLOPs, achieving the lowest model complexity among the evaluated methods while maintaining strong reconstruction fidelity.

Songyu Ding, Ronggiang Zhao, Mingchun Sun +1

Faculty of Computing, Harbin Institute of Technology, Harbin 150001, China · State Key Laboratory of Smart Farm Technologies and Systems, Harbin 150001, China

Hyperspectral Imaging Hyperspectral Image Reconstruction Diffusion-Based Inverse Problems Conditional Diffusion Models

Apr 8, 2026

Accelerating 4D Hyperspectral Imaging through Physics-Informed Neural Representation and Adaptive Sampling

High-dimensional hyperspectral imaging (HSI) enables the visualization of ultrafast molecular dynamics and complex, heterogeneous spectra. However, applying this capability to resolve spatially varying vibrational couplings in two-dimensional infrared (2DIR) spectroscopy, a type of coherent multidimensional spectroscopy (CMDS), necessitates prohibitively long data acquisition, driven by dense Nyquist sampling requirements and the need for extensive signal accumulation. To address this challenge, we introduce a physics-informed neural representation approach that efficiently reconstructs dense spatially-resolved 2DIR hyperspectral images from sparse experimental measurements. In particular, we used a multilayer perceptron (MLP) to model the relationship between the sub-sampled 4D coordinates and their corresponding spectral intensities, and recover densely sampled 4D spectra from limited observations. The reconstruction results demonstrate that our method, using a fraction of the samples, faithfully recovers both oscillatory and non-oscillatory spectral dynamics in experimental measurement. Moreover, we develop a loss-aware adaptive sampling method to progressively introduce potentially informative samples for iterative data collection while conducting experiments. Experimental results show that the proposed approach achieves high-fidelity spectral recovery using only

1/32

of the sampling budget, as opposed to exhaustive sampling, effectively reducing total experiment time by up to 32-fold. This framework offers a scalable solution for accelerating any experiments with hypercube data, including multidimensional spectroscopy and hyperspectral imaging, paving the way for rapid chemical imaging of transient biological and material systems.

Chi-Jui Ho, Harsh Bhakta, Wei Xiong +1

Dept. of Electrical and Computer Engineering, UC San Diego, La Jolla, CA 92093, USA · Dept. of Chemistry, UC San Diego, La Jolla, CA 92093, USA · Materials Science and Engineering Program, UC San Diego, La Jolla, CA, 92093, USA

Hyperspectral Image Reconstruction Hyperspectral Imaging 4D Reconstruction Adaptive Sampling

Apr 4, 2026

Physics-Informed Untrained Learning for RGB-Guided Superresolution Single-Pixel Hyperspectral Imaging

Single-pixel imaging (SPI) offers a cost-effective route to hyperspectral acquisition but struggles to recover high-fidelity spatial and spectral details under extremely low sampling rates, a severely ill-posed inverse problem. While deep learning has shown potential, existing data-driven methods demand large-scale pretraining datasets that are often impractical in hyperspectral imaging. To overcome this limitation, we propose an end-to-end physics-informed framework that leverages untrained neural networks and RGB guidance for joint hyperspectral reconstruction and super-resolution without any external training data. The framework comprises three physically grounded stages: (1) a Regularized Least-Squares method with RGB-derived Grayscale Priors (LS-RGP) that initializes the solution by exploiting cross-modal structural correlations; (2) an Untrained Hyperspectral Recovery Network (UHRNet) that refines the reconstruction through measurement consistency and hybrid regularization; and (3) a Transformer-based Untrained Super-Resolution Network (USRNet) that upsamples the spatial resolution via cross-modal attention, transferring high-frequency details from the RGB guide. Extensive experiments on benchmark datasets demonstrate that our approach significantly surpasses state-of-the-art algorithms in both reconstruction accuracy and spectral fidelity. Moreover, a proof-of-concept experiment using a physical single-pixel imaging system validates the framework's practical applicability, successfully reconstructing a 144-band hyperspectral data cube at a mere 6.25% sampling rate. The proposed method thus provides a robust, data-efficient solution for computational hyperspectral imaging.

Hao Zhang, Bilige Xu, Lichen Wei +2

College of Science, Northwest Agriculture & Forestry University, Yangling 712100, China · Key Laboratory of Photoelectronic Imaging Technology and System, School of Optics and Photonics, Beijing Institute of Technology, Beijing, China · State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China

Hyperspectral Image Reconstruction Hyperspectral Imaging Hyperspectral Image Super-Resolution Computational Imaging Physics-Informed ML

Feb 28, 2026

Exploring Spatiotemporal Feature Propagation for Video-Level Compressive Spectral Reconstruction: Dataset, Model and Benchmark

Recently, Spectral Compressive Imaging (SCI) has achieved remarkable success, unlocking significant potential for dynamic spectral vision. However, existing reconstruction methods, primarily image-based, suffer from two limitations: (i) Encoding process masks spatial-spectral features, leading to uncertainty in reconstructing missing information from single compressed measurements, and (ii) The frame-by-frame reconstruction paradigm fails to ensure temporal consistency, which is crucial in the video perception. To address these challenges, this paper seeks to advance spectral reconstruction from the image level to the video level, leveraging the complementary features and temporal continuity across adjacent frames in dynamic scenes. Initially, we construct the first high-quality dynamic hyperspectral image dataset (DynaSpec), comprising 30 sequences obtained through frame-scanning acquisition. Subsequently, we propose the Propagation-Guided Spectral Video Reconstruction Transformer (PG-SVRT), which employs a spatial-then-temporal attention to effectively reconstruct spectral features from abundant video information, while using a bridged token to reduce computational complexity. Finally, we conduct simulation experiments to assess the performance of four SCI systems, and construct a DD-CASSI prototype for real-world data collection and benchmarking. Extensive experiments demonstrate that PG-SVRT achieves superior performance in reconstruction quality, spectral fidelity, and temporal consistency, while maintaining minimal FLOPs. Project page: https://github.com/nju-cite/DynaSpec

Lijing Cai, Zhan Shi, Chenglong Huang +6

Hyperspectral Image Reconstruction Image Reconstruction Hyperspectral Imaging Compressed Sensing Temporal Consistency

Feb 25, 2026

Lumosaic: Hyperspectral Video via Active Illumination and Coded-Exposure Pixels

We present Lumosaic, a compact active hyperspectral video system designed for real-time capture of dynamic scenes. Our approach combines a narrowband LED array with a coded-exposure-pixel (CEP) camera capable of high-speed, per-pixel exposure control, enabling joint encoding of scene information across space, time, and wavelength within each video frame. Unlike passive snapshot systems that divide light across multiple spectral channels simultaneously and assume no motion during a frame's exposure, Lumosaic actively synchronizes illumination and pixel-wise exposure, improving photon utilization and preserving spectral fidelity under motion. A learning-based reconstruction pipeline then recovers 31-channel hyperspectral (400-700 nm) video at 30 fps and VGA resolution, producing temporally coherent and spectrally accurate reconstructions. Experiments on synthetic and real data demonstrate that Lumosaic significantly improves reconstruction fidelity and temporal stability over existing snapshot hyperspectral imaging systems, enabling robust hyperspectral video across diverse materials and motion conditions.

Dhruv Verma, Andrew Qiu, Roberto Rangel +8

Department of Computer Science, University of Toronto, Canada · Department of Electrical & Computer Engineering, University of Toronto, Canada

Hyperspectral Image Reconstruction Hyperspectral Imaging Computational Imaging

Jan 29, 2026

Correcting Spectra Outside the Backbone: A Model-Agnostic Rectifier for Hyperspectral Image Super-Resolution

Hyperspectral image super-resolution (HSI-SR) aims to recover spatial detail while preserving the spectral shape on which quantitative analysis relies. Recent HSI-SR methods, from repurposed RGB super-resolution backbones to dedicated spectral-spatial architectures, have greatly improved spatial reconstruction. However, overlooking the compact spectral structure of hyperspectral data leaves residual spectral errors, while binding the spectral treatment to each architecture forces it to be rebuilt for every new backbone. Yet the low-dimensional structure of spectra belongs to the data, not to any backbone. Backbones differ in the errors they leave, but not in the structure of the true spectra. One rectifier design can therefore serve any backbone. Building on this insight, we propose the Spectral Rectification Super-Resolution Network (\textbf{SR

^{2}

-Net}), a model-agnostic rectifier that needs nothing from the backbone but its output, leaves its internal architecture untouched, and is trained per backbone. SR

^{2}

-Net follows an enhance-then-rectify pipeline in which Hierarchical Spectral-Spatial Synergy Attention (\textbf{H-S

^{3}

A}) reinforces cross-band interactions, while Mode-Constrained Rectification (MCR) confines the correction to a learned compact spectral subspace. A degradation-consistency constraint further ties the output to the observed low-resolution input. Experiments with five backbones spanning CNN, Transformer, and diffusion families show that one fixed configuration improves spectral fidelity in every reported setting while preserving or improving spatial quality. Averaged over thirty in-domain settings, SR

^{2}

-Net removes 22.6% of the residual spectral error at a backbone-independent cost of 0.048M parameters.

Ji-Xuan He, Guohang Zhuang, Bo Junge +5

School of Computer Science and Technology, Xi’an Jiaotong University · School of Computer Science and Information Engineering, Hefei University of Technology

Hyperspectral Image Super-Resolution Hyperspectral Imaging Spectral Methods Hyperspectral Image Reconstruction

Oct 2, 2025

Flow-Matching Guided Deep Unfolding for Hyperspectral Image Reconstruction

Reconstructing a three-dimensional hyperspectral cube from a two-dimensional compressed measurement is a severely ill-posed inverse problem. Existing deep unfolding networks (DUNs) retain fidelity to the imaging model, but regression-trained denoisers can suppress spatial detail and smooth spectral structure under strong modulation. This paper proposes FMU, a deep unfolding framework that couples a measurement-conditioned flow-matching prior with a sensing-model-guided measurement update. A two-phase scheme first learns a compact clean-HSI latent target and then trains a conditional velocity field to generate this target from Gaussian noise. A mean-velocity regularizer additionally penalizes the first-moment error of the learned field. On the KAIST 10-scene benchmark under the optical-filter setting, FMU obtains 42.13,dB PSNR and 0.9900 SSIM, outperforming LADE-DUN by 1.16,dB in PSNR under the same training data, sensing mask, and evaluation protocol. Under the same optical-filter operator, FMU is further evaluated on held-out KAIST and ICVL scenes without fine-tuning. We also report quantitative simulated-CASSI results and qualitative reconstructions of real CASSI measurements.

Yi Ai, Yuanhao Cai, Yulun Zhang +2

School of Computer Science, Shanghai Jiao Tong University, China · Department of Computer Science, Johns Hopkins University, USA · School of Engineering, Westlake University, China

Flow Matching Image Reconstruction Hyperspectral Imaging Hyperspectral Image Reconstruction Deep Unfolding

Mar 3, 2025

Hyperspectral Image Restoration and Super-resolution with Physics-Aware Deep Learning for Biomedical Applications

Hyperspectral imaging is a powerful bioimaging tool which can uncover novel insights, thanks to its sensitivity to the intrinsic properties of materials. However, this enhanced contrast comes at the cost of system complexity, constrained by an inherent trade-off between spatial, spectral, and temporal resolution. To overcome this limitation, we present a self-supervised deep learning-based approach that restores and enhances pixel resolution post-acquisition without requiring external training data beyond the images to be restored. Fine-tuned using metrics aligned with the imaging model, our physics-aware method achieves a 16

\times

pixel super-resolution enhancement and a 12

\times

imaging speedup without the need of additional training data for transfer learning. Applied to both synthetic and experimental data from five different sample types, including healthy and diseased tissues, we demonstrate that the model preserves biological integrity, as we did not detect systematic loss of biological features or biologically consequential hallucinations in tested datasets. We also concretely demonstrate the model's ability to reveal disease-associated metabolic changes that would otherwise remain undetectable. Furthermore, we provide physical insights into the model's inner workings, paving the way for future refinements that could potentially reveal novel high resolution features in an explainable manner. All methods are available as open-source software on GitHub.

Yuchen Xiang, Zhaolu Liu, Monica Emili Garcia-Segura +12

Department of Metabolism, Digestion and Reproduction, Imperial College London, London, UK · Department of Mathematics, Imperial College London, London, UK · Department of Clinical Neurosciences and NIHR Biomedical Research Centre, University of Cambridge, Cambridge, UK +4

Hyperspectral Imaging Physics-Guided Image Restoration Hyperspectral Image Super-Resolution Self-Supervised Learning Image Restoration Physics-Informed ML+1

Dec 3, 2024

Filterless Snapshot Hyperspectral Imaging using Guided Patch Diffusion

We consider the problem of reconstructing a HxWx31 hyperspectral image from a

H\times W

grayscale snapshot measurement that is captured using only a single diffractive lens and a filterless panchromatic photosensor. This problem is severely ill-posed, but we present a model that produces high-quality results in simulation and experiment. We make efficient use of limited training data by creating a conditional denoising diffusion model that operates on small patches in a shift-invariant manner. During inference, we synchronize per-patch hyperspectral predictions using guidance by physical consistency with the system's optical point spread function. Our experiments reveal that the patch size can be as small as the point spread function, with local optical cues being the main source of information about complete spectra. Also, by drawing multiple samples, our model provides per-pixel uncertainty estimates that strongly correlate with reconstruction error.

Dean Hazineh, Luca Sacchi, Davide Cassara +2

Harvard University

Diffusion Model Guidance Physics-Guided Image Restoration Hyperspectral Imaging Computational Imaging Image Inverse Problems Diffusion-Based Inverse Problems+2