Accelerating 4D Hyperspectral Imaging through Physics-Informed Neural Representation and Adaptive Sampling

Chi-Jui Ho; Harsh Bhakta; Wei Xiong; Nicholas Antipa

eess.IVApr 8, 2026

Accelerating 4D Hyperspectral Imaging through Physics-Informed Neural Representation and Adaptive Sampling

Authors: Chi-Jui Ho, Harsh Bhakta, Wei Xiong, Nicholas Antipa

Organizations: Dept. of Electrical and Computer Engineering, UC San Diego, La Jolla, CA 92093, USA · Dept. of Chemistry, UC San Diego, La Jolla, CA 92093, USA · Materials Science and Engineering Program, UC San Diego, La Jolla, CA, 92093, USA

Abstract

High-dimensional hyperspectral imaging (HSI) enables the visualization of ultrafast molecular dynamics and complex, heterogeneous spectra. However, applying this capability to resolve spatially varying vibrational couplings in two-dimensional infrared (2DIR) spectroscopy, a type of coherent multidimensional spectroscopy (CMDS), necessitates prohibitively long data acquisition, driven by dense Nyquist sampling requirements and the need for extensive signal accumulation. To address this challenge, we introduce a physics-informed neural representation approach that efficiently reconstructs dense spatially-resolved 2DIR hyperspectral images from sparse experimental measurements. In particular, we used a multilayer perceptron (MLP) to model the relationship between the sub-sampled 4D coordinates and their corresponding spectral intensities, and recover densely sampled 4D spectra from limited observations. The reconstruction results demonstrate that our method, using a fraction of the samples, faithfully recovers both oscillatory and non-oscillatory spectral dynamics in experimental measurement. Moreover, we develop a loss-aware adaptive sampling method to progressively introduce potentially informative samples for iterative data collection while conducting experiments. Experimental results show that the proposed approach achieves high-fidelity spectral recovery using only $1/32$ of the sampling budget, as opposed to exhaustive sampling, effectively reducing total experiment time by up to 32-fold. This framework offers a scalable solution for accelerating any experiments with hypercube data, including multidimensional spectroscopy and hyperspectral imaging, paving the way for rapid chemical imaging of transient biological and material systems.

Figures & tables

Figure 1: Hyper-spectrum reconstruction through neural rendering. We sparsely sample the hyper-spectrum collected from the interaction between ultrafast pulses and samples, and then use the proposed neural rendering framework to reconstruct the dense hyper-spectrum data in both oscillatory and non-oscillatory regimes.

Panel of Figure 2 — Figure 2: Dimensionality of Spatially-Resolved 2DIR. a) The system diagram. b) Illustration of the four coordinate axes defining the hyperspectral volume. Each 2DIR frame parameterized by $(\omega_{1},\omega_{3})$ , while the temporal and spatial variations are recorded in different waiting time $t_{2}$ and spatial position $x$ .

Figure 3: Our MLP-based neural representation model. We construct neural representations for signals subsampled along either the population (slow varying) or coherence (fast varying) time dimensions. Population: A coordinate-based MLP is trained to learn the temporal dynamics along population time from sparsely sampled data and to generalize this knowledge to unsampled points during inference. Coherence: The MLP is optimized in the frequency domain, after which an IRFT is applied to recover estimated raw signals for loss computation, excluding intensities at unsampled coordinates. During inference, the trained network reconstructs rapidly varying 4D signals across the full coordinate domain.

Figure 4: 2D representation of 4DIR data. A bounding box is defined around the dominant peak in each $(\omega_{1},\omega_{3})$ slice. The intensities enclosed within this region are averaged to reduce the four-dimensional data to a two-dimensional representation along the $(x,t_{2})$ axes, which is subsequently used for evaluation.

Figure 5: Our loss-driven adaptive sampling method. Beginning with the initially sampled coordinates (blue dots), the MLP is trained to model the spectral dynamics and the loss is evaluated along the population time $t_{2}$ . The interval exhibiting the highest average prediction error is identified, and its midpoint is selected as the next sample.

Figure 6: Summary of results under different numbers of repeated samplings. (a) Measurements averaged over varying data averaging counts ( $r$ ) and reconstructed spectra under the same count. (c) Using the least noisy case (highest $r$ ) as reference, we compute the MSE of other measurements and predictions against it. (d) Mean and (e) standard deviation profiles of reconstructions at different repeat counts, with the reference shown as a solid black line.

$r$	Intensity MSE	Mean profile MSE	Std profile MSE
2	1.320e-02	1.280e-06	3.820e-07
5	8.330e-03	6.600e-07	1.240e-07
10	8.250e-03	6.050e-07	9.990e-08
20	4.990e-03	4.360e-07	4.860e-08

Table 1: Quantitative evaluation under different data averaging counts (

r

Figure 7: Summary of results under different sampling intervals (SI) along the population time. (a) Reference and reconstructed spectra obtained at various sampling rates along the population time axis. (b–d) Peak intensity, mean and standard deviation profiles of reconstructions at different sampling intervals, with the reference shown as a solid black line.

SI	Intensity MSE	Mean profile MSE	Std profile MSE
1	4.990e-03	4.360e-07	4.860e-08
2	5.590e-03	5.120e-07	6.390e-08
4	1.390e-02	1.290e-06	1.480e-07
8	1.680e-02	2.860e-06	1.570e-07
16	1.570e-01	3.490e-06	3.000e-07

Table 2: Quantitative evaluation under different sampling intervals (SI) along the population time.

Figure 8: Reconstruction results from subsampled

t_{1}

domain data. (a) Reconstructed 2DIR spectrum compared against the ground truth. (b) Temporal raw measurements showing the fully sampled ground truth and the subsampled input. (c) A 1D intensity profile extracted from (b) demonstrating signal recovery. To visualize oscillations beyond the native samples, the ground truth is densified via sinc interpolation, while the dense prediction is obtained by evaluating the IRFT with a finer

t_{1}

sampling.

Figure 9: Reconstruction results from subsampled $t_{1}$ domain data using different algorithms. (a) The 2DIR spectrum and (b) temporal measurements generated from different reconstruction algorithms compared against the ground truth.

Algorithm	Spectrum MSE	Unsampled Temporal MSE
RFFT	1.333e-02	1.583e-04
GIRAF-2D [ 10 ]	8.607e-02	3.443e-03
Fourier-features [ 45 ]	1.615e-01	6.190e-03

Table 3: Reconstruction performance on undersampled

t_{1}

data across different algorithms.

Figure 10: Reconstruction performance at different $t_{1}$ sampling rates. We display the reconstructed 2DIR spectrum under different sampling rates and associated temporal data. SR: sampling rate.

Figure 11: Reconstruction 2DIR Spectrum under low accumulation and sampling rates. With $r=5$ and undersampled $t_{1},t_{2}$ measurement, we display an unsampled $t_{2}$ slice of 2DIR spectrum and a slice of $(t_{2},\omega_{3})$ projection.

$t_{1}$ Sampling Rate	Spectrum MSE	Unsampled Temporal MSE
80%	1.333e-02	1.583e-04
60%	1.663e-02	1.824e-04
40%	3.369e-02	3.825e-04
20%	2.503e-01	2.429e-03

Table 4: RFFT reconstruction error under different

t_{1}

sampling rates.

Figure 12: Comparison of 2DIR Spectrum Reconstruction Under Differing Sampling Strategies. (a)-(c) Fully sampled temporal data and representative sampling masks for the two evaluated strategies. (d) Temporal reconstruction statistics across 10 independent trials. (e)-(g) Corresponding reconstructed 2DIR spectra for the examples shown in (a)-(c). (h) Summary of spectral reconstruction performance. (i)-(j) 1D illustrations of the reconstruction performance under the (i) Keep and (j) Drop strategies. Omitting the early

t_{1}

samples leaves the reconstruction underconstrained, leading the model to hallucinate spurious initial oscillations.

Figure 13: Reconstruction results from different sampling strategies. We overlay (a) peak-intensity, (b) mean, and (c) standard-deviation profiles derived from spectra reconstructed using different sampling strategies.

Method	Intensity MSE	Mean profile MSE	Std profile MSE
Ours	6.190e-02	2.710e-06	2.320e-07
Random	7.010e-02	4.950e-06	4.870e-07
Uniform	3.430e-01	4.870e-06	3.590e-07

Table 5: Quantitative comparison of different adaptive sampling strategies.

Figure 14: Effect of statistical moment-matching. Spectral recovery results obtained with (orange) and without (blue) moment-matching constraints for (a) the mean and (b) the standard deviation profiles.

Explore similar work

Physics-Informed Untrained Learning for RGB-Guided Superresolution Single-Pixel Hyperspectral Imaging
Apr 4, 2026Hao Zhang, Bilige Xu, Lichen Wei +2Hyperspectral Imaging Hyperspectral Image Super-Resolution
College of Science, Northwest Agriculture & Forestry University, Yangling 712100, China · Key Laboratory of Photoelectronic Imaging Technology and System, School of Optics and Photonics, Beijing Institute of Technology, Beijing, China · State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China
Phy-CoSF: Physics-Guided Continuous Spectral Fields Reconstruction and Super-Resolution for Snapshot Compressive Imaging
May 13, 2026Wudi Chen, Zhiyuan Zha, Xin Yuan +6Image Reconstruction Hyperspectral Image Super-Resolution
College of Communication Engineering, Jilin University, Changchun 130012, China. · School of Engineering, Westlake University, Hangzhou, Zhejiang 310024, China. · School of Electrical & Electronic Engineering, Nanyang Technological University, Singapore 639798. +4
Spectral Dynamic Attention Network for Hyperspectral Image Super-Resolution
Apr 30, 2026Tengya Zhang, Feng Gao, Lin Qi +2Hyperspectral Image Super-Resolution Hyperspectral Imaging
Sanya Oceanographic Institution, Ocean University of China, Sanya, China · State Key Laboratory of Physical Oceanography, Ocean University of China, Qingdao 266100, China · Department of Electrical and Computer Engineering, Mississippi State University, Starkville, MS 39762 USA

Apr 4, 2026J/K move · Enter open · S save

Physics-Informed Untrained Learning for RGB-Guided Superresolution Single-Pixel Hyperspectral Imaging

Hao Zhang, Bilige Xu, Lichen Wei, Xu Ma+1

College of Science, Northwest Agriculture & Forestry University, Yangling 712100, China · Key Laboratory of Photoelectronic Imaging Technology and System, School of Optics and Photonics, Beijing Institute of Technology, Beijing, China · State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China

Hyperspectral Imaging Hyperspectral Image Super-Resolution Computational Imaging Physics-Informed ML

Open paper AskPDF arXiv

Single-pixel imaging (SPI) offers a cost-effective route to hyperspectral acquisition but struggles to recover high-fidelity spatial and spectral details under extremely low sampling rates, a severely ill-posed inverse problem. While deep learning has shown potential, existing data-driven methods demand large-scale pretraining datasets that are often impractical in hyperspectral imaging. To overcome this limitation, we propose an end-to-end physics-informed framework that leverages untrained neural networks and RGB guidance for joint hyperspectral reconstruction and super-resolution without any external training data. The framework comprises three physically grounded stages: (1) a Regularized Least-Squares method with RGB-derived Grayscale Priors (LS-RGP) that initializes the solution by exploiting cross-modal structural correlations; (2) an Untrained Hyperspectral Recovery Network (UHRNet) that refines the reconstruction through measurement consistency and hybrid regularization; and (3) a Transformer-based Untrained Super-Resolution Network (USRNet) that upsamples the spatial resolution via cross-modal attention, transferring high-frequency details from the RGB guide. Extensive experiments on benchmark datasets demonstrate that our approach significantly surpasses state-of-the-art algorithms in both reconstruction accuracy and spectral fidelity. Moreover, a proof-of-concept experiment using a physical single-pixel imaging system validates the framework's practical applicability, successfully reconstructing a 144-band hyperspectral data cube at a mere 6.25% sampling rate. The proposed method thus provides a robust, data-efficient solution for computational hyperspectral imaging.

Abstract

Explore similar work

Physics-Informed Untrained Learning for RGB-Guided Superresolution Single-Pixel Hyperspectral Imaging

Phy-CoSF: Physics-Guided Continuous Spectral Fields Reconstruction and Super-Resolution for Snapshot Compressive Imaging

Spectral Dynamic Attention Network for Hyperspectral Image Super-Resolution