ThermalSplat: Generalizable Pose-Free Gaussian Splatting for Sparse-View Thermal 3D Reconstruction
Organizations: Indian Institute of Technology Madras
Abstract
Thermal infrared imaging is vital for applications like search-and-rescue and autonomous navigation, yet trans- lating it into 3D scene representations remains a critical bottleneck. The low-texture nature and view-dependent emissivity of thermal images cause standard pose estima- tors to fail catastrophically, collapsing standard 3D Gaus- sian Splatting pipelines under sparse-view constraints. Fur- thermore, existing thermal reconstruction methods rely heavily on dense capture and computationally expensive per-scene optimization. Critically, many of these ap- proaches depend on aligned RGB-thermal datasets to ob- tain reliable camera poses and initial point clouds, ren- dering them unusable in purely thermal, unposed envi- ronments. To break this bottleneck, we present Thermal- Splat, the first generalizable, feed-forward Gaussian splat- ting framework that reconstructs 3D scenes directly from sparse thermal images without requiring known camera poses, RGB guidance, or per-scene training. We over- come thermal-specific challenges using a soft census loss for brightness-invariant structural matching and a bound- ary structure loss to explicitly supervise temperature discon- tinuities. Trained on an aggregated multi-source dataset, ThermalSplat synthesizes sharp reconstructions of unseen scenes in a single feed-forward pass. Extensive experi- ments demonstrate that ThermalSplat holds a strict advan- tage over scene-specific thermal splatting methods in the sparse regime. Furthermore, even compared against these baselines operating under their ideal dense capture set- tings, ThermalSplat achieves superior perceptual fidelity. By eliminating the need for computationally expensive per scene optimization, ThermalSplat offers an efficient and generalizable approach to thermal 3D reconstruction.
Figures & tables
| Thermal Scenes | |||||||
| Hand | Dark | Fire3 | Tower | UAV6 | Bldg. | ||
| Total Images | 42 | 86 | 41 | 177 | 132 | 96 | |
| Method | Metric | Dense Context (All Views) | |||||
| COLMAP | Registered Images | 42 | 0 | 0 | 11 | 132 | 35 |
| 3D Points | 622 | 0 | 0 | 493 | 22,209 | 222 | |
| VGGT | Registered Images | 42 | 0 | 0 | 0 | 132 | 96 |
| Method | Hand | Dark Scenes | Fire3Raw | BuildingA Winter | Transmission Tower | Mean | ||||||||||||
| PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS | |
| Dense Multi-View Input (Scene-Specific Optimization Baseline) | ||||||||||||||||||
| Thermal3DGS | 33.010 | 0.9375 | 0.1558 | 17.370 | 0.7321 | 0.4568 | 29.870 | 0.9734 | 0.1503 | 32.420 | 0.9576 | 0.1759 | 17.790 | 0.6988 | 0.3311 | 25.140 | 0.8383 | 0.2522 |
| ThermalGaussian ∗ | 30.110 | 0.9210 | 0.1724 | 18.260 | 0.7798 | 0.4250 | 30.410 | 0.9789 | 0.1373 | 33.240 | 0.9659 | 0.1545 | 18.060 | 0.7013 | 0.3162 | 26.020 | 0.8694 | 0.2411 |
| PhysIR | 31.050 | 0.9290 | 0.1640 | 18.490 | 0.7600 | 0.4220 | 35.190 | 0.9800 | 0.1357 | 30.710 | 0.9400 | 0.2100 | 18.390 | 0.7095 | 0.3100 | 26.766 | 0.8622 | 0.2773 |
| Sparse Two-View Input (Constraint Regime) | ||||||||||||||||||
| Method | Hand | Dark Scenes | Fire3Raw | BuildingA Winter | Transmission Tower | UAV6 | Mean | ||||||||||||||
| PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS | |
| MVSplat | 10.88 | 0.150 | 0.785 | 3.65 | 0.006 | 0.860 | 6.60 | 0.009 | 0.731 | 6.36 | 0.137 | 0.882 | 6.22 | 0.153 | 0.713 | 15.21 | 0.432 | 0.510 | 8.15 | 0.148 | 0.747 |
| MVSplat + | 12.41 | 0.475 | 0.594 | 9.68 | 0.454 | 0.762 | 23.91 | 0.880 | 0.296 | 23.20 | 0.840 | 0.341 | 14.13 | 0.450 | 0.375 | 18.49 | 0.513 | 0.367 | 16.97 | 0.602 | 0.456 |
| SPFSplat | 18.03 | 0.681 | 0.313 | 18.46 | 0.718 | 0.434 | 32.26 | 0.953 | 0.216 | 13.16 | 0.677 | 0.525 | 16.72 | 0.599 | 0.299 | 12.13 | 0.278 | 0.645 | 18.46 | 0.651 | 0.405 |
| SPFSplat + | 18.38 | 0.691 | 0.374 | 19.40 | 0.718 | 0.467 | 29.42 | 0.962 | 0.201 | 20.66 | 0.815 | 0.464 | 13.30 | 0.677 | 0.534 | 14.64 | 0.337 | 0.517 | 19.30 | 0.700 | 0.426 |
| NAS3R | 29.33 | 0.918 | 0.085 | 18.20 | 0.702 | 0.402 | 31.06 | 0.958 | 0.200 | 24.78 | 0.882 | 0.221 | 18.01 | 0.626 | 0.249 | 19.26 | 0.523 | 0.281 | 23.44 | 0.768 | 0.240 |
| Method | Rotation | Translation | Overall | ||||||
| NAS3R | 43.2 | 55.2 | 70.0 | 10.5 | 23.2 | 37.7 | 9.5 | 21.3 | 35.8 |
| NAS3R + | 51.3 | 68.0 | 80.7 | 9.5 | 23.4 | 40.2 | 7.7 | 21.5 | 38.6 |
| ThermalSplat (Ours) | 51.5 | 67.9 | 80.5 | 11.9 | 25.7 | 41.2 | 10.4 | 23.8 | 39.7 |
| Method | Paradigm | Time |
| Thermal3DGS | Per-scene Opt. | 15 min |
| ThermalGaussian OMMG | Per-scene Opt. | 25 min |
| PhysIR | Per-scene Opt. | 15 min |
| ThermalSplat (Ours) | Feed-Forward | 80 ms |
| Variant | PSNR | SSIM | LPIPS |
| Reference: RGB-pretrained & Fine-tuned baselines | |||
| Naive transfer | 23.44 | 0.768 | 0.240 |
| Photometric | 24.34 | 0.785 | 0.220 |
| ThermalSplat ablations | |||
| + Census Loss | 25.15 | 0.810 | 0.192 |
| + Boundary Loss | 25.46 | 0.822 | 0.183 |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Source | Scenes used |
| ThermalGaussian (14) | building, RotaryKiln, ebike, parterre, Plant_Equipment, daily_stuff, Glass_Cup, Dimsum, RoadBlock, ironvogt, landscape, Truck, Dark_Scenes , Transmission_tower |
| Thermal3DGS (17) | heated, corridor, bridge, wall, chair, sitting, standing_tall, building, bicycle, soccer_goal, merge, UAV3, UAV4, apple, UAV5, UAV6 , UAV1 |
| ThermoNeRF (9) | exhibition_building, trees, melting_ice_cup, double_robot, heater_watercup, heater_water_kettle, BuildingA_spring, BuildingA_winter , raspberry |
| MVTV (18) | human0_raw, human1_raw, human2_raw, fire0_raw, fire1_raw, fire2_raw, car0_raw, car1_raw, bench_raw, tree_raw, corner_raw, road_raw, chair_raw, grillman_raw, mason_raw, forest_raw, fire3_raw , human_raw |
| ThermalMix (5) | face, laptop, pan, pan1, hand |
| Total | 63 scenes |
| # Views | PSNR | SSIM | LPIPS |
| 2 | 25.55 | 0.828 | 0.181 |
| 3 | 25.83 | 0.827 | 0.186 |
| 5 | 26.35 | 0.829 | 0.179 |
| Hand | Dark Scenes | Fire3Raw | BuildingA_winter | Transmission Tower | UAV6 | Mean | ||||||||||||||||
| PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS | ||
| 0.05 | 0.05 | 27.90 | 0.889 | 0.103 | 19.83 | 0.741 | 0.354 | 29.50 | 0.968 | 0.160 | 29.91 | 0.941 | 0.127 | 17.47 | 0.566 | 0.273 | 27.01 | 0.850 | 0.069 | 25.44 | 0.8258 | 0.1810 |
| 0.05 | 0.10 | 27.28 | 0.893 | 0.102 | 20.00 | 0.747 | 0.350 | 29.85 | 0.968 | 0.160 | 29.94 | 0.940 | 0.130 | 17.05 | 0.547 | 0.292 | 28.24 | 0.855 | 0.071 | 25.39 | 0.8250 | 0.1842 |
| 0.10 | 0.05 | 27.73 | 0.893 | 0.102 | 20.26 | 0.750 | 0.336 | 29.37 | 0.968 | 0.163 | 29.95 | 0.942 | 0.125 | 17.20 | 0.550 | 0.286 | 28.28 | 0.854 | 0.075 | 25.47 | 0.8262 | 0.1808 |
| 0.075 | 0.075 | 28.01 | 0.890 | 0.101 | 20.23 | 0.745 | 0.347 | 29.31 | 0.969 | 0.163 | 30.21 | 0.943 | 0.124 | 17.49 | 0.573 | 0.278 | 28.02 | 0.849 | 0.070 | 25.55 | 0.8282 | 0.1805 |
| Scene | PhysIR-Splat | ThermalSplat | ||||
| PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS | |
| Statue | 29.19 | 0.938 | 0.212 | 33.22 | 0.962 | 0.131 |
| Telescope | 31.68 | 0.940 | 0.088 | 32.12 | 0.936 | 0.097 |
| Old-drinking-fountain | 35.37 | 0.968 | 0.213 | 35.11 | 0.967 | 0.129 |
| Parking | 36.50 | 0.981 | 0.107 | 36.45 | 0.972 | 0.097 |
| Metallic-container | 7.61 | 0.370 | 0.370 | 33.18 | 0.969 | 0.137 |
3D Reconstruction
3D reconstruction recovers a scene’s shape, appearance, and camera motion from images, video, depth, or other sensor measurements.
See all 1,017 papers