cs.CVMar 20, 2026
SaveThermal is Always Wild: Characterizing and Addressing Challenges in Thermal-Only Novel View Synthesis
Organizations: Northwestern University · University of California, Riverside
Abstract
Thermal cameras provide reliable visibility in darkness and adverse conditions, but thermal imagery remains significantly harder to use for novel view synthesis (NVS) than visible-light images. This difficulty stems primarily from two characteristics of affordable thermal sensors. First, thermal images have extremely low dynamic range, which weakens appearance cues and limits the gradients available for optimization. Second, thermal data exhibit rapid frame-to-frame photometric fluctuations together with slow radiometric drift, both of which destabilize correspondence estimation and create high-frequency floater artifacts during view synthesis, particularly when no RGB guidance (beyond camera pose) is available. Guided by these observations, we introduce a lightweight preprocessing and splatting pipeline that expands usable dynamic range and stabilizes per-frame photometry. Our approach achieves state-of-the-art performance across thermal-only NVS benchmarks, without requiring any dataset-specific tuning.
Figures & tables
Figure 1: Our method overcomes significant challenges in thermal images. Thermal data contain limited texture and lack multispectral cues, making correspondence estimation harder than in RGB. They also exhibit sensor-specific degradations, including (a) frame-to-frame photometric inconsistency from sensor heating, (b) softened transitions between hot and cold regions characteristic of microbolometer sensors, (c) vignetting that produces viewpoint-dependent attenuation, and (d) fixed-pattern noise visible as structured artifacts. Our method explicitly stabilizes the photometry in (a), while the effects in (b–d) are mitigated in our SOTA reconstructions (e) through multiview consistency enabled by a novel embedding approach for learned appearance modeling.
Figure 2: Dataset-level radiometric and spatial-frequency characteristics. ( a ) Standard deviation of the relative mean-intensity change across scenes. Thermal sequences show substantially larger radiometric fluctuations than RGB, with variation levels differing noticeably across datasets. ( b ) Radially averaged power spectra, averaged per dataset. Thermal frames consistently exhibit reduced high-frequency energy compared to RGB; among thermal datasets, TI-NSD shows comparatively stronger high-frequency content.
Figure 3: Photometric stabilization and contrast enhancement (a) An input thermal frame, with less contrast and texture than the corresponding RGB image (b). Notice the photometric drift when compared to reference frame (c), which shows the same scene at a different time point. (d) Our invertible enhancement improves photometric inconsistency and image contrast. (e) Temporal mean intensity across frames showing reduced radiometric drift after stabilization. (f) Normalized spatial frequency spectrum with enhanced thermal data resembling RGB statistics. (g) Our preprocessing expands the effective dynamic range of thermal images, shown with pixel instensity distributions on a sample frame. (h) Improved structural consistency and stability, illustrated by an elevated number of tracked SIFT features per frame. See Sec. S1 for more examples.
Figure 4: Our pipeline. Given thermal frames and camera poses, we first stabilize the inputs to ensure consistent training data across views (bottom). The frames are modeled with a novel combination of per-Gaussian embeddings, which encode spatial appearance, per-frame embeddings, which capture residual temporal artifacts, and a physics-restricted parameter set (grayscale with no spherical harmonics) that stabilizes learning. These components jointly enable consistent thermal reconstruction while preserving fine geometric and intensity details.
Table 1: Thermal-only NVS comparison across six multiview datasets. We report mean PSNR and SSIM across all scenes for six publicly available datasets. Our analysis of dataset difficulty predicts systematic trends in performance across methods. By designing a pipeline that addresses thermal data challenges directly, we deliver SOTA performance on thermal-only NVS.
Figure 5: Comparison to all methods. Representative novel views, PSNR (bottom left, dB), and SSIM (bottom right) across datasets. Our method yields sharper boundaries and more stable background temperature on challenging scenes (top rows), while maintaining competitive quality on easier scenes (bottom rows).
Figure 6: Comparison to Thermal3D-GS. Despite competitive metrics (PSNR in dB left, SSIM right), Thermal3D-GS exhibits significant failures in geometry and texture, while our method shows texture like the Human0 window, tree, and smooth wall, and faithfully reconstructs the Lion paw shape.
Figure 7: Photometric consistency promotes high-quality reconstruction. We render a smooth camera path for our method (top) and Thermal3D-GS (bottom). Thermal3D-GS produces bright floating structures that drift across frames and then assemble into a copy of the photometrically inconsistent training frame (right) when the viewpoint aligns. Our preprocessing and embedding-conditioned emission model handle this outlier without introducing floaters, yielding stable geometry throughout the trajectory.
| Method | MSX- Ebike | T. Mix - Lion | MVTV - Human | Lin et al. - Sink | Ye et al. - Seq.1 | TINSD - Sitting | Avg. | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | |
| 3DGS (Baseline) | 20.45 | 0.86 | 19.25 | 0.71 | 21.21 | 0.81 | 20.81 | 0.74 | 28.24 | 0.83 | 29.51 | 0.88 | 22.25 | 0.81 |
| 3DGS + Preprocessing | 22.79 | 0.86 | 24.11 | 0.82 | 22.01 | 0.84 | 23.71 | 0.78 | 29.90 | 0.85 | 22.42 | 0.85 | 23.01 | 0.83 |
| 3DGS + Emission MLP | 25.73 | 0.89 | 24.17 | 0.82 | 24.22 | 0.89 | 24.57 | 0.83 | 32.12 | 0.89 | 25.98 | 0.87 | 24.93 | 0.87 |
| Ours | 25.97 | 0.92 | 24.25 | 0.81 | 26.18 | 0.90 | 24.27 | 0.88 | 33.32 | 0.90 | 30.01 | 0.87 | 26.14 | 0.88 |
Table 2: Ablation study. We demonstrate that our preprocessing algorithm and “in-the-wild” architecture (3DGS + Emission MLP) independently improve 3DGS performance, and combining them yields superior results. Additional analysis comparing our method to traditional histogram equalization and evaluating each preprocessing step is provided in Tab. S1
Figure S1 : Radiometric inconsistency across thermal datasets. Thermal sequences often exhibit frame-to-frame fluctuations, generally influenced by ISP operations, as well as slower radiometric drift that can arise as the sensor gradually warms during capture. Both effects may appear even in static scenes and produce unstable brightness trajectories, as illustrated across the three datasets. These inconsistencies pose a significant challenge for NeRF and 3D Gaussian Splatting methods, which assume photometric consistency across views; variations in thermal intensity can weaken multi-view matching and lead to distorted geometry. In examples from Lin et al. and MSX, we observe global intensity shifts affecting nearly all pixel values, while the ThermalMix example shows a localized change in the foreground object. Our algorithm provides a simple yet effective way to suppress these trends and stabilize temporal behavior.
Figure S2 : Spatial frequency characteristics of thermal versus RGB images. Thermal images generally exhibit weaker mid- and high-frequency components than RGB images. This is partly due to sensor-resolution limits, but also because heat diffuses across surfaces and through the surrounding air, producing the naturally smoother appearance typical of thermal scenes. As a result, fine texture and sharp edges are often diminished, reducing the high-frequency cues that NeRF and 3D Gaussian Splatting rely on for accurate geometry and appearance estimation. Our pipeline increases contrast and strengthens spatial gradients, making features more distinguishable while unavoidably amplifying noise and discretization artifacts. Even so, the enhanced gradients lead to more reliable COLMAP initializations (see Fig. 3 ) and provide stronger supervision for 3DGS, improving overall reconstruction quality.
Figure S3 : Intensity distributions of thermal and RGB images. Raw thermal images typically exhibit highly skewed histograms, where the majority of values accumulate near the lowest intensities. This narrow distribution limits contrast and reduces the visibility of meaningful variations in surface temperature, making it harder for NeRF and 3D Gaussian Splatting to extract useful spatial cues. Our pipeline expands the dynamic range and yields more spread-out intensity distributions, allowing important regions to stand out more clearly. Although this process can make noise more noticeable, the resulting images provide stronger supervision signals and improve reconstruction stability in 3DGS. Example input and processed frames are shown on the right.
Figure S4 : Revisit of Figure 3. Tracked object brightness and global mean before (left) and after (right) preprocessing; local fluctuations follow the global mean and are jointly attenuated.
| Method | MSX- Ebike | T. Mix - Lion | MVTV - Human | Lin et al. - Sink | Ye et al. - Seq.1 | TINSD - Sitting | Avg. | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | |
| 3DGS (Baseline) | 20.45 | 0.86 | 19.25 | 0.71 | 21.21 | 0.81 | 20.81 | 0.74 | 28.24 | 0.83 | 29.51 | 0.88 | 22.25 | 0.81 |
| 3DGS + Hist. Eq. | 12.54 | 0.48 | 20.29 | 0.79 | 16.85 | 0.68 | 21.02 | 0.74 | 28.62 | 0.81 | 21.12 | 0.82 | 18.86 | 0.72 |
| 3DGS + Contrast Enh. | 14.79 | 0.57 | 23.97 | 0.81 | 17.33 | 0.68 | 21.89 | 0.71 | 29.90 | 0.82 | 20.59 | 0.80 | 19.71 | 0.73 |
| 3DGS + Photo. Stab. | 23.72 | 0.87 | 19.24 | 0.71 | 21.65 | 0.81 | 21.82 | 0.76 | 28.44 | 0.82 | 24.50 | 0.86 | 22.19 | 0.81 |
| 3DGS + Stab + Enh | 22.79 | 0.86 | 24.11 | 0.82 | 22.01 | 0.84 | 23.71 | 0.78 | 29.90 | 0.85 | 22.42 | 0.85 | 23.01 | 0.83 |
Table S1 : Ablation study. We demonstrate that our preprocessing steps (constrast enhancement as an improvement over traditional histogram equalization, in combination with photometric stabilization) improve 3DGS performance, but not to the level of our method. We then demonstrate the effect of per-frame embedding ablation, with and without pre-processing.
Figure S4:
Figure S5:
Figure S6:
Figure S7:
Figure S8: Floaters
Figure S9:
Figure S10:
Figure S11:
Figure S12:
Figure S13:
Figure S14:
Figure S15:
Figure S16:
Figure S17:
Figure S18:
Figure S19:
Figure S20:
Figure S21:
Figure S22: