GTSM: Gaussian Splatting with Thermal State-Space Dynamics for Spatio-Temporal Thermal Field Reconstruction and Prediction

Yunpeng Zhao; Haoxiang Zhang; Chao Liu

cs.CVOct 17, 2026

GTSM: Gaussian Splatting with Thermal State-Space Dynamics for Spatio-Temporal Thermal Field Reconstruction and Prediction

Authors: Yunpeng Zhao, Haoxiang Zhang, Chao Liu

Organizations: Key Laboratory for Information Science of Electromagnetic Waves (MoE), Fudan University, Shanghai 200433, China · Department of Electronic Engineering, Tsinghua University, Beijing, China

Abstract

Temperature-field reconstruction and prediction are important for condition monitoring, process analysis, and state assessment in industrial and dynamic environments. This work aims to recover a four-dimensional spatio-temporal thermal state field directly from two-dimensional observations, without requiring a predefined 3D model, prescribed thermodynamic equation, or predefined heating/cooling trajectory. To this end, we propose the Gaussian Thermal State-Space Model (GTSM), which represents thermal evolution through a shared scene-level latent process and Gaussian-specific local thermal states. The resulting continuous dynamics are propagated using differentiable RK2 integration, allowing thermal evolution to be inferred from multi-view and multi-temporal observations and continuously extrapolated beyond the observed interval. GTSM is designed as a backbone-agnostic thermal dynamics module for Gaussian representations and can be integrated with both static 3D Gaussian Splatting and dynamic 4D Gaussian Splatting, supporting thermal reconstruction and prediction under both static and time-varying target geometry. The reconstructed representation further enables novel-view thermal rendering and future-time thermal-state prediction. Experiments on real infrared sequences and synthetic dynamic thermal data demonstrate improved reconstruction accuracy and substantially stronger temporal extrapolation than representative baselines, while also validating the applicability of GTSM to deforming targets.

Figures & tables

Fig. 1: GTSM reconstructs spatio-temporal thermal fields from two-dimensional observations and enables novel-view synthesis and future thermal-state prediction for both static and dynamic targets.

Fig. 2: Overview of the proposed GTSM framework. GTSM can be coupled with static 3DGS or dynamic 4DGS representations, and models Gaussian-level thermal evolution through a scene-level latent driver and Gaussian-specific local dynamics for spatio-temporal thermal field reconstruction, novel-view rendering, and future prediction.

Fig. 3: Simulated component-level temperature curves of the satellite target under five ODE-based thermal cases. Solid and dashed curves denote different target components. The vertical dashed line at $t=0.75$ separates the observed interval from the future-time prediction interval.

Fig. 4: Qualitative comparison of temporal extrapolation on the RHD dataset for the Cooling Dumbbells and Warming Bottles scenes. The first ground-truth image shows the beginning of the training sequence, while the remaining columns show future-time test frames in chronological order.

Target	Schedule	$T(0)\rightarrow T(0.75)\rightarrow T(1.0)$ $(^{\circ}\mathrm{C})$	Val MAE	Val RMSE	Test MAE	Test RMSE
Foldingchair Seat–Frame	Constant	15,20 $\rightarrow$ 15,20 $\rightarrow$ 15,20	0.0608	0.9748	0.2035	1.8513
	Case1:natural warming	10,5 $\rightarrow$ 12.89,17.26 $\rightarrow$ 13.70,19.20	0.0438	0.6806	0.1549	1.4938
	Case2:natural cooling	35,40 $\rightarrow$ 31.03,24.56 $\rightarrow$ 29.93,22.19	0.1126	1.8314	0.2522	2.4806
	Case3:constant heating	15,15 $\rightarrow$ 19.90,17.54 $\rightarrow$ 21.21,17.97	0.0584	0.9683	0.1747	1.7681
	Case4:ramp heating	15,15 $\rightarrow$ 18.95,20.48 $\rightarrow$ 21.65,23.66	0.0572	0.9635	0.1981	1.9796
	Case5:heating-cooling	20,20 $\rightarrow$ 21.05,16.97 $\rightarrow$ 20.32,15.27	0.0779	1.1729	0.2208	1.7418

TABLE II: Quantitative evaluation under different targets, temperature ranges, and temporal evolution laws. Validation metrics measure novel-view temperature-field reconstruction within the observed interval (

t\leq 0.75

), while test metrics measure future-time temperature prediction beyond the observed interval (

t>0.75

Target	Setting	Val-Interpolation			Test-Extrapolation
Target	Setting	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Foldingchair	TRACE	40.03	0.996	0.005	29.59	0.982	0.010
	Ours RGB	40.20	0.996	0.005	30.47	0.980	0.009
	Case 1	41.58	0.991	0.017	33.99	0.973	0.024
	Case 2	32.70	0.985	0.022	29.13	0.969	0.026
	Case 3	38.24	0.998	0.021	32.20	0.971	0.026

TABLE III: Image-level evaluation on the synthetic dynamic thermal dataset. TRACE and RGB row reports geometric rendering quality, while Cases 1–5 report pseudo-color temperature reconstruction and prediction results.

Fig. 5: Qualitative results of GTSM integrated with the 4DGS framework on the Satellite and Foldingchair scenes. Each visualization combines the rendered RGB appearance and pseudo-color temperature map. The validation results show novel-view reconstruction within the observed temporal interval, while the test results simultaneously evaluate novel-view synthesis and future thermal prediction beyond the observed interval.

Fig. 6: Visualization of signed temperature prediction errors for the dynamic Foldingchair scene (top) and the static cooling Ebike scene (bottom). Note that different color ranges are used for the two scenes to reveal their respective error structures.

Spatiotemporal Forecasting

Spatiotemporal forecasting predicts how quantities change across locations and time, from traffic and weather to disease and environmental systems.

See all 423 papers

Dynamic Thermal Gaussians: Multimodal 4D Gaussian Splatting
Sep 21, 2026Rongfeng Lu, Lifeng Lin, Xiaobao Wei +7Thermal Imaging 3D Gaussian Splatting
Lishui University Lishui, China Lishui Key Laboratory of Low-Altitude Multimodal Sensing and Intelligent Computing Lishui, China · Hangzhou Dianzi University Hangzhou, China · University of Chinese Academy of Sciences Beijing, China +2
Supercharging Thermal Gaussian Splatting with Depth Estimation
May 28, 2026Manoj Biswanath, Chenxin Cai, Hannah Schieber +2Thermal Imaging 3D Gaussian Splatting
Photogrammetry and Remote Sensing, Munich Center for Machine Learning (MCML), Technical University of Munich, Munich, Germany · Technical University of Munich, Munich, Germany · Human-Centered Computing and Extended Reality Lab, TUM University Hospital, Clinic for Orthopedics and Sports Orthopedics, Munich Institute of Robotics and Machine Intelligence (MIRMI), Technical University of Munich, Munich, Germany
Beyond Thermal Imaging: Inferring Thermophysical Properties from Time-Resolved Thermal Observations
Jul 8, 2026Chenghao Xu, Malcolm Mielle, Olga FinkThermal Imaging 3D Reconstruction
Intelligent Maintenance and Operations Systems, ´Ecole polytechnique f´ed´erale de Lausanne, Station 18, Lausanne, 1015, Vaud, Switzerland. · Schindler Lab EPFL, Schindler Holding AG, Station 18, Lausanne, 1015, Vaud, Switzerland.

Sep 21, 2026J/K move · Enter open · S save

Dynamic Thermal Gaussians: Multimodal 4D Gaussian Splatting

Rongfeng Lu, Lifeng Lin, Xiaobao Wei, Quan Chen+6

Lishui University Lishui, China Lishui Key Laboratory of Low-Altitude Multimodal Sensing and Intelligent Computing Lishui, China · Hangzhou Dianzi University Hangzhou, China · University of Chinese Academy of Sciences Beijing, China+2

Thermal Imaging 3D Gaussian Splatting 4D Reconstruction Dynamic Scene Reconstruction

Open paper AskPDF arXiv

Thermography plays a vital role in military and broader thermal analysis applications. Recent progress in 3D thermal reconstruction has extended temperature analysis from 2D to 3D space, yet most existing works assume static temperature distributions, neglecting the temporal dynamics of heat transfer in real-world environments. To address this limitation, we propose the first dynamic RGB-Thermal reconstruction framework for complex scenes. Our method jointly models RGB appearance, thermal observations, and scene geometry as they change over time. Specifically, we introduce a multimodal dynamic scene representation that anchors both the color and thermal modalities to a shared geometric substrate, ensuring their consistency under spatiotemporal deformations. We further design multimodal embeddings to enhance the motion expressiveness for each modality, and propose a multimodal routing mechanism that retains a unified set of shared multimodal Gaussians as the geometric backbone while adaptively spawning modality-specific Gaussians to strengthen the representational capacity in detail-rich regions of each individual modality. In addition, we contribute a novel benchmark dataset featuring high-frequency temperature variations to facilitate the evaluation of 4D reconstruction. Extensive experiments demonstrate that our method achieves high-fidelity spatiotemporal reconstruction of both appearance and temperature. Our code and dataset are available at: https://github.com/LinLif1869/DTG.

Abstract

Spatiotemporal Forecasting

Explore similar work

Dynamic Thermal Gaussians: Multimodal 4D Gaussian Splatting

Supercharging Thermal Gaussian Splatting with Depth Estimation

Beyond Thermal Imaging: Inferring Thermophysical Properties from Time-Resolved Thermal Observations