Universal computational thermal imaging overcoming the ghosting effect

Hongyi Xu; Du Wang; Chenjun Zhao; Jiashuo Chen; Jiale Lin; Liqin Cao; Yanfei Zhong; Yiyuan She

cs.CVApr 2, 2026

Universal computational thermal imaging overcoming the ghosting effect

Authors: Hongyi Xu, Du Wang, Chenjun Zhao, Jiashuo Chen, Jiale Lin, Liqin Cao, Yanfei Zhong, Yiyuan She, +1 more

Organizations: Department of Electronic and Information Engineering, School of Engineering and Research Center for Industries of the Future, Westlake University, Hangzhou 310030, China · State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing, Wuhan University, Wuhan 430079, China · Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China · Department of Physics, School of Science, Westlake University, Hangzhou 310030, China · School of Resource and Environmental Sciences, Wuhan University, Wuhan 430079, China · Institute of Theoretical Science, Westlake University, Hangzhou 310030, China · Institute of Natural Sciences, Westlake Institute for Advanced Study, Hangzhou 310024, China

Abstract

Thermal imaging is crucial for night vision but fundamentally hampered by the ghosting effect, a loss of detailed texture in cluttered photon streams. While conventional ghosting mitigation has relied on data post-processing, the recent breakthrough in heat-assisted detection and ranging (HADAR) opens a promising frontier for hyperspectral computational thermal imaging that produces night vision with day-like visibility. However, universal anti-ghosting imaging remains elusive, as state-of-the-art HADAR applies only to limited scenes with uniform materials, whereas material non-uniformity is ubiquitous in the real world. Here, we propose a universal computational thermal imaging framework, TAG (thermal anti-ghosting), to address material non-uniformity and overcome ghosting for high-fidelity night vision. TAG takes hyperspectral photon streams for nonparametric texture recovery, enabling our experimental demonstration of unprecedented expression recovery in thus-far-elusive ghostly human faces -- the archetypal, long-recognized ghosting phenomenon. Strikingly, TAG not only universally outperforms HADAR across various scenes, but also reveals the influence of material non-uniformity, shedding light on HADAR's effectiveness boundary. We extensively test facial texture and expression recovery across day and night, and demonstrate, for the first time, thermal 3D topological alignment and mood detection. This work establishes a universal foundation for high-fidelity computational night vision, with potential applications in autonomous navigation, reconnaissance, healthcare, and wildlife monitoring.

Figures & tables

Figure 1: Universal thermal anti-ghosting (TAG) for high-fidelity night vision with day-like visibility. a, The TAG framework. TAG takes hyperspectral thermal imagery for nonparametric TeX decomposition and produces high-fidelity textures, along with non-uniform temperature and emissivity. b, The SLOT principle. To break TeX degeneracy, SLOT uses a B-spline basis expansion for continuous emissivity and imposes a smoothness constraint, universally applicable to non-uniform materials. In contrast, HADAR relies on rigid material categorization and suffers from classification errors when material non-uniformity is present. c, Traditional thermal imaging cannot separate ambient scattering from direct emission, yielding textureless silhouettes that resemble ghosts. d, TAG vividly recovers otherwise hidden geometric textures and facial expressions that are crucial for various perception tasks. From left to right: neutral (sad), frown with eyes open (angry), grin with eyes and mouth open (happy), and pout.

Figure 2: The challenge of TeX degeneracy for texture recovery. a, Illustration of TeX degeneracy, where multiple TeX triplets produce an identical thermal radiation spectrum (taken from the red cross in b). b, Texture comparison. Left: poor texture recovery by panchromatic thermal imaging and CLAHE failing to break the TeX degeneracy. Right: vivid texture recovery by TAG, which successfully breaks the TeX degeneracy.

Figure 3: Shedding light on the effectiveness boundary of HADAR. a, Comparable texture recovery between HADAR and TAG when intra-class material non-uniformity is smaller (negligible) than the inter-class emissivity contrast. b, Failure of HADAR TeX vision when intra-class material non-uniformity is larger (significant) than the inter-class emissivity contrast. c, HADAR is robust to emissivity/library inaccuracy when material non-uniformity is negligible. The leftmost panel uses the baseline candidate emissivity estimated by the HADAR approach. The middle and right panels use candidate curves from TAG controlled by the regularization parameter ( $\lambda=10^{-3},10^{6}$ ).

Figure 4: Zero-shot cross-modal machine perception enabled by TAG. a, Input data comparison showing ghosting traditional thermal imaging and TAG with vivid textures. b, Task I: AI colorization. With textures, TAG perfectly accommodates color mapping and preserves spatial patterns. Without textures, traditional thermal images produce blurry color artifacts. c, Task II: 3D topological alignment. TAG recovers textures to flawlessly anchor a 468-point facial mesh, whereas alignment completely fails or drifts on traditional thermal images. d, Task III: Semantic perception and recognition. Standard RGB-trained vision engines successfully localize faces and predict nuanced human emotions ( i.e. , Sad, Happy) on TAG images, but suffer catastrophic breakdowns on thermal images.

Figure 5: Passive recovery of facial texture and expression with high-fidelity night vision in pitch darkness. a, Ghosting thermal imaging at night. b, Nighttime facial expression recovery by TAG. Left: neutral expression with mouth and eyes closed. Right: Smiling with mouth and eyes open.

Figure 6: Robustness of TAG in complex open-world field tests. a, Traditional thermal imaging from the DARPA Invisible Headlights dataset, where the ghosting effect blurs critical details. b, TAG consistently recovers great textures of unknown materials across vast scenes without requiring prior material libraries (surpassing HADAR), both during the day and at night.

Metrics		EN $\uparrow$	AG $\uparrow$	SF $\uparrow$	SD $\uparrow$
Male	IR	7.349	0.476	1.331	51.491
Male	TAG (Ours)	7.491	1.916	5.933	53.079
Female	IR	6.761	1.329	3.408	50.523
Female	TAG (Ours)	7.412	4.157	8.363	52.004

Table 1: Texture quantification comparing two universal modalities, the traditional thermal imaging (IR) and our TAG, in the human region in Fig. 4 a.

Explore similar work

T $^2$ exture: Sparsely Perturbed Thermal-to-Texture Imaging
Aug 3, 2026Jiashuo Chen, Cheng Dai, Yanan Hu +1Thermal Imaging Image Reconstruction
School of Engineering, Westlake University · School of Science, Westlake University
HADAR-Based Thermal Infrared Hyperspectral Image Restoration
May 13, 2026Cheng Dai, Jiale Lin, Bingxuan Song +4Thermal Imaging Physics-Guided Image Restoration
School of Science, Westlake University, Hangzhou 310030, China
Thermal is Always Wild: Characterizing and Addressing Challenges in Thermal-Only Novel View Synthesis
Mar 20, 2026M. Kerem Aydin, Vishwanath Saragadam, Emma AlexanderNovel View Synthesis Thermal Imaging
Northwestern University · University of California, Riverside

Aug 3, 2026J/K move · Enter open · S save

T $^2$ exture: Sparsely Perturbed Thermal-to-Texture Imaging

Jiashuo Chen, Cheng Dai, Yanan Hu, Fanglin Bao

School of Engineering, Westlake University · School of Science, Westlake University

Thermal Imaging Image Reconstruction

Open paper AskPDF arXiv

Thermal imaging remains effective under adverse illumination, yet passive long-wave infrared (LWIR) measurements often lack fine texture. Existing thermal texture imaging approaches commonly rely on spectral sensing or registered auxiliary modalities, incurring substantial data throughput or vulnerability to cross-modal degradation. We introduce T

^2

exture, a sparsely perturbed thermal texture imaging framework that aims to reconstruct temporally dense thermal texture sequences from densely sampled passive frames and a few actively perturbed keyframes. We define thermal texture as the residual between a source-on observation and its corresponding source-off passive state. Under sparse LWIR illumination and rapid quasi-steady paired acquisition, this residual attenuates the passive-emission background and approximates a source-induced reflected response, exposing localized material- and geometry-dependent texture. T

^2

exture reconstructs a dense sequence of this source-conditioned response through two stages. Stage 1 estimates the unobserved source-off passive state at each active instant from neighboring passive frames to obtain reliable differential texture anchors. Stage 2 combines sparse anchors with passive structural context near each target time to reconstruct the dense sequence. On the simulated benchmark, T

^2

exture adds only 0.20M parameters to AMT-L while improving PSNR by 6.66 dB. Extensive evaluations on simulated and real acquisitions further show clearer texture recovery and stronger structural preservation than representative VFI baselines. These results establish T

^2

exture as a practical framework for thermal texture imaging under sparse active acquisition.

Abstract

Explore similar work

T2^22exture: Sparsely Perturbed Thermal-to-Texture Imaging

HADAR-Based Thermal Infrared Hyperspectral Image Restoration

Thermal is Always Wild: Characterizing and Addressing Challenges in Thermal-Only Novel View Synthesis

T $^2$ exture: Sparsely Perturbed Thermal-to-Texture Imaging