SEAR: Simple and Efficient Adaptation of Visual Geometric Transformers for Unpaired RGB+Thermal 3D Reconstruction

Vsevolod Skorokhodov; Chenghao Xu; Shuo Sun; Olga Fink; Malcolm Mielle

cs.CVMar 19, 2026

SEAR: Simple and Efficient Adaptation of Visual Geometric Transformers for Unpaired RGB+Thermal 3D Reconstruction

Authors: Vsevolod Skorokhodov, Chenghao Xu, Shuo Sun, Olga Fink, Malcolm Mielle

Organizations: Schindler EPFL Lab, Lausanne, Switzerland · EPFL, Lausanne, Switzerland · Örebro University, Örebro, Sweden

Abstract

Foundational feed-forward visual geometry models enable accurate and efficient camera pose estimation and scene reconstruction by learning strong scene priors from massive RGB datasets. However, their effectiveness drops when applied to mixed sensing modalities, such as RGB-thermal (RGB-T) images. We observe that while a visual geometry grounded transformer pretrained on RGB data generalizes well to thermal-only reconstruction, it struggles to align RGB and thermal modalities when processed jointly. To address this, we propose SEAR, a simple yet efficient fine-tuning strategy that adapts a pretrained geometry transformer to multimodal RGB-T inputs. Despite being trained on a relatively small RGB-T dataset, our approach significantly outperforms state-of-the-art methods for 3D reconstruction and camera pose estimation, achieving significant improvements over all metrics and delivering higher detail and consistency between modalities with negligible overhead in inference time compared to the original pretrained model. Notably, SEAR enables reliable multimodal pose estimation and reconstruction even under challenging conditions, such as low lighting and dense smoke. We validate our architecture through extensive ablation studies and demonstrate how the model aligns both modalities. Additionally, we introduce a new dataset featuring RGB and thermal sequences captured at different times, viewpoints, and illumination conditions, providing a robust benchmark for future work in multimodal 3D scene reconstruction. Code and models are publicly available at https://doi.org/10.5281/ZENODO.21077295.

Figures & tables

Panel of Figure 1 — Figure 1: SEAR architecture. RGB and thermal images are first tokenized using DINOv2. For each modality, camera-specific tokens are concatenated with the corresponding DINO tokens. The combined tokens are then processed by an Alternating-Attention (AA) module with LoRA adapters. Finally, the refined tokens are passed to separate prediction heads for camera parameter estimation and depth estimation. Trainable parameters are highlighted with a flame symbol.

Panel of Figure 2 — Figure 2: The SEAR dataset includes 9 scenes, each with paired RGB-thermal images captured along two distinct trajectories. Ground-truth poses (red/blue for each trajectory) are estimated via VGGT on all RGB images. The top 6 scenes feature trajectories under similar lighting, while the bottom 3 have large lighting variations (some RGB images are near fully black).

Figure 3: Qualitative results comparing RGB/thermal reconstructions (camera poses in red/blue); we show results for 4 methods at rows 1, 3, and 5, and zoom in on more interesting reconstruction details in rows 2, 4, and 6. Our method (SEAR) achieves higher accuracy, consistency, and level of detail than other methods.

	Public Datasets								SEAR Dataset
Method	AUC $\uparrow$	RRA $\uparrow$	RTA $\uparrow$	PCA $\downarrow$	PCC $\downarrow$	Chamfer $\downarrow$	Reg (%) $\uparrow$	FPS $\uparrow$	AUC $\uparrow$	RRA $\uparrow$	RTA $\uparrow$	Reg (%) $\uparrow$	FPS $\uparrow$
COLMAP +SPSG	57.6	82.5	74.6	1.64	1.20	1.42	44.7	0.44	74.4	99.9	95.9	27.9	0.66
$\text{MA}_{\text{ELoFTR}}$	13.1	70.1	41.2	0.49	5.16	2.82	21.8	0.21	15.9	73.7	53.1	37.3	0.17
$\text{MINIMA}_{\text{ROMA}}$	41.0	68.3	63.0	0.97	1.09	1.03	98.7	0.05	48.2	65.9	68.9	100.0	0.04
MP-SfM	31.4	78.0	52.5	1.59	0.58	1.08	100.0	0.04	-	-	-	-	-
DUSt3R	18.9	45.9	42.1	0.72	4.72	2.72	100.0	0.66	18.7	51.0	39.2	100.0	0.67

Table 1: Quantitative comparison of 3D reconstruction methods for all datasets. Metrics include AUC , RRA , and RTA @30 (higher is better,

\uparrow

), point cloud accuracy PCA , point cloud completeness PCC , Chamfer distance (lower is better,

\downarrow

), registration rate Reg. (higher is better,

\uparrow

), and frames per second FPS (higher is better,

\uparrow

). Best and second-best results are highlighted in blue and light blue , respectively.

Panel of Figure 4 — Figure 4: Reconstructions from the SmokeSeer dataset (dense smoke, top) and our new dataset’s scenes (lighting changes, middle and bottom rows). The first column shows cases where RGB images (right) are unreliable for localization, so thermal images (left) are used. Our method recovers the scene even with smoke (SmokeSeer) and aligns RGB and thermal camera poses in different lighting conditions (our dataset).

Panel of Figure 5 — Figure 5: The AUC, RRA, RTA (errors < $30^{\circ}$ , $15^{\circ}$ , $5^{\circ}$ ) across varying thermal-to-RGB image ratios. The filled area represents the boundary from $0.25-$ to $0.75-$ quantiles estimated by bootstrapping scenes $2000$ times.

Panel of Figure 6 — Figure 6: The blue line represents the median difference between RGB-to-RGB and RGB-to-thermal cosine similarity dependence across layers for SEAR method. The orange line represents the same dependency for the VGGT model. The filled area represents the boundary from $0.25-$ to $0.75-$ quantiles.

Method	AUC@30	RRA@30	RTA@30
No Camera Token	$67.8\pm 1.4$	$89.1\pm 1.0$	$86.6\pm 1.3$
Thermal Projector	$67.2\pm 1.1$	$88.3\pm 0.8$	$86.0\pm 0.8$
Thermal Embedding	$66.1\pm 1.8$	$87.6\pm 1.0$	$85.1\pm 1.2$
Global Only	$69.1\pm 1.5$	$89.4\pm 1.1$	$87.3\pm 1.6$
Frame Only	$62.4\pm 0.6$	$85.6\pm 0.9$	$83.0\pm 1.0$
SEAR	$68.4\pm 0.7$	$89.1\pm 0.5$	$86.9\pm 0.4$

Table 2: This table shows the camera pose estimation metrics average and 95% confidence interval for the different ablation studies of our paper.

	No Camera Token	Thermal Projector	Thermal Embedding	Global Only	Frame Only	Ours
No Camera Token	–	0.72	0.91	0.12	1.00	0.24
Thermal Projector	0.28	–	0.83	0.04	1.00	0.05
Thermal Embedding	0.09	0.17	–	0.02	1.00	0.02
Global Only	0.88	0.96	0.98	–	1.00	0.78
Frame Only	0.00	0.00	0.00	0.00	–	0.00
Ours	0.76	0.95	0.98	0.22	1.00	–

Table 3: Pairwise statistical significance (AUC@30). Each entry at position

(i,j)

reports the p-value from a one-sided Welch’s t-test for the null hypothesis that the row method outperforms the column method in AUC@30. Darker blue indicates smaller p-values (stronger evidence against the null).

Panel of Figure 7 — Figure 7: The AUC, RRA, RTA (errors < $30^{\circ}$ , $15^{\circ}$ , $5^{\circ}$ ) across varying LoRA $r$ . The filled area represents the boundary from $0.25-$ to $0.75-$ quantiles estimated by bootstrapping scenes $2000$ times.

Appendix figures & tables27 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	#Training Scenes	#Evaluation Scenes	Aligned modalities	Cross positional	Cross temporal	Ground Truth
For training and quantitative evaluation
ThermoScenes	17	5	True	True	False	VGGT
ThermalGaussian	11	3	True	True	False	VGGT
ThermalMix	4	2	True	True	False	VGGT
ThermalNeRF	7	2	False	True	False	VGGT
RF	9	6	True	True	False	Motion Capture

Appendix

Table 4: Detailed description of the datasets and split used for training and for evaluation.

Panel of Table 5 — Table 5: Summary of the SEAR dataset for well-lit scenes, where both trajectories are captured under the same bright illumination conditions. The figure includes RGB and thermal images, two camera trajectories obtained by running VGGT on the RGB images, the lengths of the two trajectories, and the minimum and maximum temperatures.

Panel of Table 6 — Table 6: Summary of the SEAR dataset for poorly lit scenes, where the trajectories are captured under different lighting conditions. The figure includes RGB and thermal images, the camera trajectory estimated by running VGGT on the thermal images (and is therefore less accurate), the lengths of the two trajectories, and the minimum and maximum temperatures for each trajectory.

	prpt-cup					INR-building
Method	AUC $\uparrow$	RRA $\uparrow$	RTA $\uparrow$	Reg (%) $\uparrow$	FPS $\uparrow$	AUC $\uparrow$	RRA $\uparrow$	RTA $\uparrow$	Reg (%) $\uparrow$	FPS $\uparrow$
COLMAP + SPSG	49.4	73.7	75.3	51.2	0.40	34.9	52.3	57.6	49.7	0.49
$\text{MA}_{\text{ELoFTR}}$	16.3	70.0	45.0	4.0	0.18	2.6	61.6	13.6	15.6	0.19
$\text{MINIMA}_{\text{ROMA}}$	8.4	26.6	31.3	100.0	0.02	15.3	48.2	42.0	100.0	0.03
MP-SfM	59.4	93.7	86.6	100.0	0.04	19.7	48.2	48.3	100.0	0.05
DUSt3R	19.9	29.2	32.9	100.0	0.43	6.5	44.6	33.9	100.0	0.66

Appendix

Table 7: Quantitative comparison of 3D reconstruction on ThermoScenes [ 9 ] if COLMAP is used to estimate ground truth camera poses.

	laptop					panel
Method	AUC $\uparrow$	RRA $\uparrow$	RTA $\uparrow$	Reg (%) $\uparrow$	FPS $\uparrow$	AUC $\uparrow$	RRA $\uparrow$	RTA $\uparrow$	Reg (%) $\uparrow$	FPS $\uparrow$
COLMAP + SPSG	56.6	100.0	90.9	10.8	0.28	17.6	32.2	48.3	28.0	0.32
$\text{MA}_{\text{ELoFTR}}$	23.2	51.1	61.5	48.9	0.15	46.0	73.6	98.1	35.4	0.36
$\text{MINIMA}_{\text{ROMA}}$	78.5	94.8	94.3	100.0	0.04	78.4	95.2	93.2	100.0	0.05
MP-SfM	47.3	64.3	65.8	100.0	0.04	73.5	100.0	98.0	100.0	0.06
DUSt3R	21.1	44.1	46.1	100.0	0.67	15.5	70.4	39.4	100.0	0.65

Appendix

Table 8: Quantitative comparison of 3D reconstruction on ThermalMix [ 29 ] if COLMAP is used to estimate ground truth camera poses.

	IronIngot					Parterre
Method	AUC $\uparrow$	RRA $\uparrow$	RTA $\uparrow$	Reg (%) $\uparrow$	FPS $\uparrow$	AUC $\uparrow$	RRA $\uparrow$	RTA $\uparrow$	Reg (%) $\uparrow$	FPS $\uparrow$
COLMAP + SPSG	96.0	100.0	99.7	100.0	0.54	46.0	100.0	100.0	3.0	0.53
$\text{MA}_{\text{ELoFTR}}$	13.7	56.0	58.5	61.5	0.12	32.0	66.8	85.5	24.2	0.23
$\text{MINIMA}_{\text{ROMA}}$	97.4	100.0	99.9	100.0	0.06	34.4	47.5	64.1	100.0	0.04
MP-SfM	95.2	100.0	99.6	100.0	0.06	26.8	44.1	64.0	100.0	0.02
DUSt3R	15.5	75.0	39.6	100.0	0.66	43.1	95.3	79.5	100.0	0.65

Appendix

Table 9: Quantitative comparison of 3D reconstruction on ThermalGaussian [ 23 ] if COLMAP on RGB is ground truth.

Figure 8: Qualitative results on SEAR dataset. Our method produces more accurate and structurally consistent point clouds than the baselines. All methods show limited performance on conference-room and old-drinking-fountain , likely because the thermal modality in these scenes provides weak localization cues.

Figure 9: Qualitative results on the RF dataset. Our method produces more accurate point clouds than the baselines. All methods show degraded performance on 01_Annexet_No_Radars_1 and 01_Annexet_No_Radars_2 , likely due to the complex camera motion in these sequences. From top to bottom the scenes are 01_Annexet_No_Radars_0 , 01_Annexet_No_Radars_1 , 01_Annexet_No_Radars_2 , 01_Annexet_No_Radars_3 , 04_Forest_pass_no_radars_0 , and 04_Forest_pass_no_radars_1 .

Figure 10: Qualitative results on the ThermalGaussian dataset. Our method produces more accurate point clouds than the baselines on Ebike and Parterre . For Iron Ingot , our reconstruction is visually highly accurate, although its quantitative metrics are lower than those of COLMAP and $\text{MINIMA}_{\text{ROMA}}$ .

Figure 11: Qualitative results on the ThermalMix dataset. Our method produces more accurate and complete point clouds than the baselines.

Figure 12: Qualitative results on the ThermalNeRF dataset. Our method produces more accurate and structurally coherent point clouds than the baselines.

Figure 13: Qualitative results on the ThermoScenes dataset. Our method produces more accurate point clouds than the baselines. All methods demonstrate limited performance on freezing_ice_cup , likely because of the inherent difficulty of this scene, discussed in Appendix 0.C .

Figure 14: Qualitative results on SEAR dataset, where the trajectories were captured under different lighting conditions. Our method produces more accurate point clouds than the baselines. Although $\text{MINIMA}_{\text{ROMA}}$ achieves competitive performance on house , it fails on the remaining scenes.

Figure 15: Qualitative results on the SmokeSeer dataset. Our method produces more accurate point clouds than the baselines. None of the methods reconstruct red-container , likely because the overlaid text in the thermal images introduces misleading localization cues.

	metallic-container					conference-room
Method	AUC $\uparrow$	RRA $\uparrow$	RTA $\uparrow$	Reg (%) $\uparrow$	FPS $\uparrow$	AUC $\uparrow$	RRA $\uparrow$	RTA $\uparrow$	Reg (%) $\uparrow$	FPS $\uparrow$
COLMAP + SPSG	75.8	100.0	96.8	61.5	0.59	78.1	99.6	94.0	25.4	0.84
$\text{MA}_{\text{ELoFTR}}$	14.1	97.8	43.2	50.0	0.17	2.4	56.0	9.0	15.2	0.34
$\text{MINIMA}_{\text{ROMA}}$	41.0	60.7	64.6	100.0	0.04	22.5	37.3	57.6	100.0	0.04
DUSt3R	23.7	75.4	54.3	100.0	0.67	15.9	34.0	43.1	100.0	0.66
MASt3R	41.2	60.9	57.3	100.0	0.27	25.6	38.0	56.8	100.0	0.26

Appendix

Table 10: Per-scene metrics on SEAR dataset. Compared with the other methods, our approach achieves both a higher registration rate and better pose estimation accuracy. All methods obtain low scores on conference-room and old-drinking-fountain , likely because the thermal modality in these scenes provides weak localization cues.

	01_Annexet_No_Radars_0								01_Annexet_No_Radars_1
Method	AUC $\uparrow$	RRA $\uparrow$	RTA $\uparrow$	PCA $\downarrow$	PCC $\downarrow$	Chamfer $\downarrow$	Reg (%) $\uparrow$	FPS $\uparrow$	AUC $\uparrow$	RRA $\uparrow$	RTA $\uparrow$	PCA $\downarrow$	PCC $\downarrow$	Chamfer $\downarrow$	Reg (%) $\uparrow$	FPS $\uparrow$
COLMAP + SPSG	58.7	100.0	76.3	3.17	1.56	2.36	49.6	0.24	86.1	100.0	99.9	1.44	1.03	1.24	24.8	0.34
$\text{MA}_{\text{ELoFTR}}$	15.4	96.6	31.5	4.27	8.47	6.37	10.3	0.12	3.8	79.1	10.4	0.68	3.05	1.87	13.9	0.16
$\text{MINIMA}_{\text{ROMA}}$	65.8	100.0	87.3	0.60	0.57	0.59	95.9	0.06	22.7	49.1	56.6	1.95	4.84	3.39	100.0	0.05
MP-SfM	7.4	89.2	26.0	1.2	0.5	0.8	100.0	0.03	2.0	70.2	14.9	5.0	1.4	3.2	100.0	0.03
DUSt3R	21.3	48.4	33.3	0.57	10.40	5.49	100.0	0.66	14.9	27.5	49.9	1.05	6.16	3.60	100.0	0.67

Appendix

Table 11: Per-scene metrics on the RF dataset. Our method achieves a superior registration rate and higher pose estimation accuracy than the competing methods. All methods obtain low scores on 01_Annexet_No_Radars_1 and 01_Annexet_No_Radars_2 , likely due to the complex camera motion in these sequences. In addition, the translation scores on 01_Annexet_No_Radars_3 are low because the trajectory contains frames with very small translational changes near the end, so even a slight deviation in translation can lead to large angular errors.

	Ebike					IronIngot
Method	AUC $\uparrow$	RRA $\uparrow$	RTA $\uparrow$	Reg (%) $\uparrow$	FPS $\uparrow$	AUC $\uparrow$	RRA $\uparrow$	RTA $\uparrow$	Reg (%) $\uparrow$	FPS $\uparrow$
COLMAP + SPSG	64.8	70.2	70.4	95.8	0.60	92.5	100.0	99.3	100.0	0.54
$\text{MA}_{\text{ELoFTR}}$	44.6	77.8	66.7	7.3	0.38	12.8	55.9	57.4	61.5	0.12
$\text{MINIMA}_{\text{ROMA}}$	93.9	100.0	100.0	100.0	0.07	94.2	100.0	99.6	100.0	0.06
MP-SfM	93.6	100.0	99.8	100.0	0.06	94.1	100.0	99.3	100.0	0.06
DUSt3R	20.0	29.7	35.9	100.0	0.65	15.6	74.9	39.9	100.0	0.66

Appendix

Table 12: Per-scene metrics on the ThermalGaussian dataset. Our method achieves the best pose estimation accuracy on Ebike and Parterre , while also delivering competitive performance on Iron Ingot .

	sink					generator
Method	AUC $\uparrow$	RRA $\uparrow$	RTA $\uparrow$	Reg (%) $\uparrow$	FPS $\uparrow$	AUC $\uparrow$	RRA $\uparrow$	RTA $\uparrow$	Reg (%) $\uparrow$	FPS $\uparrow$
COLMAP + SPSG	86.4	100.0	98.5	38.2	0.61	52.7	79.8	73.8	100.0	0.64
$\text{MA}_{\text{ELoFTR}}$	12.4	87.2	35.0	37.3	0.24	4.2	53.1	27.0	19.7	0.22
$\text{MINIMA}_{\text{ROMA}}$	48.8	100.0	70.5	100.0	0.07	62.1	100.0	84.5	100.0	0.06
MP-SfM	82.7	100.0	97.9	100.0	0.05	91.5	100.0	99.7	100.0	0.04
DUSt3R	33.6	100.0	61.6	100.0	0.67	7.4	41.4	32.9	100.0	0.66

Appendix

Table 13: Per-scene metrics on ThermalNeRF. Our method achieves a superior registration rate and higher pose estimation accuracy than the competing methods.

	freezing_ice_cup					prpt-cup
Method	AUC $\uparrow$	RRA $\uparrow$	RTA $\uparrow$	Reg (%) $\uparrow$	FPS $\uparrow$	AUC $\uparrow$	RRA $\uparrow$	RTA $\uparrow$	Reg (%) $\uparrow$	FPS $\uparrow$
COLMAP + SPSG	34.5	57.2	60.5	50.0	0.52	51.4	72.0	75.3	48.1	0.61
$\text{MA}_{\text{ELoFTR}}$	1.5	30.6	22.3	7.6	0.33	20.2	70.0	40.0	2.6	0.27
$\text{MINIMA}_{\text{ROMA}}$	4.3	12.0	25.4	100.0	0.04	7.1	20.9	31.0	100.0	0.04
MP-SfM	23.9	39.4	58.2	100.0	0.04	60.2	93.9	87.6	100.0	0.06
DUSt3R	14.6	30.1	42.4	100.0	0.66	19.9	28.2	33.6	100.0	0.65

Appendix

Table 14: Per-scene metrics on the ThermoScenes dataset. Our method achieves a higher registration rate and better pose estimation accuracy than the competing methods. All methods achieve low scores on freezing_ice_cup and melting_ice_cup , likely due to the inherent difficulty of these scenes.

	panel					laptop
Method	AUC $\uparrow$	RRA $\uparrow$	RTA $\uparrow$	Reg (%) $\uparrow$	FPS $\uparrow$	AUC $\uparrow$	RRA $\uparrow$	RTA $\uparrow$	Reg (%) $\uparrow$	FPS $\uparrow$
COLMAP + SPSG	15.8	31.8	46.4	28.0	0.32	56.1	100.0	91.6	10.8	0.28
$\text{MA}_{\text{ELoFTR}}$	43.7	76.4	98.6	35.4	0.36	20.6	51.0	56.4	48.9	0.15
$\text{MINIMA}_{\text{ROMA}}$	74.9	95.2	93.1	100.0	0.05	77.1	94.8	94.2	100.0	0.04
MP-SfM	79.3	100.0	98.1	100.0	0.06	47.2	64.3	65.7	100.0	0.04
DUSt3R	15.9	70.4	39.5	100.0	0.65	21.1	44.0	45.9	100.0	0.67

Appendix

Table 15: Per-scene metrics on the ThermalMix dataset. Our method achieves a higher registration rate and better pose estimation accuracy than the competing methods.

Method	AUC@5 $\uparrow$	AUC@10 $\uparrow$	AUC@20 $\uparrow$	RRA@5 $\uparrow$	RRA@10 $\uparrow$	RRA@20 $\uparrow$	RTA@5 $\uparrow$	RTA@10 $\uparrow$	RTA@20 $\uparrow$
$\text{MA}_{\text{ELoFTR}}$	0.3	1.0	4.5	7.1	17.8	37.3	3.0	8.1	23.6
$\text{MINIMA}_{\text{ROMA}}$	9.4	27.2	50.9	62.7	88.6	98.0	37.1	63.9	84.7
VGGT	1.8	8.2	21.9	60.4	76.2	84.9	7.4	24.6	47.6
SEAR	23.6	47.3	68.0	92.3	97.5	98.1	57.7	81.9	92.8

Appendix

Table 16: Relative camera pose estimation results on METU-VisTIR for multimodal methods. Our method outperforms all competing approaches by a large margin on nearly all metrics. The only exception is RRA@20, where it achieves a result slightly higher than that of

\text{MINIMA}_{\text{ROMA}}

Panel of Figure 16 — Figure 16: PCA-based analysis of RGB-thermal feature alignment. The figure shows three experiments: generator from ThermalNeRF (rows 1-2), prpt-cup from ThermoScenes (rows 3-4), and Ebike from ThermalGaussian (rows 5-6). For each experiment, the top row illustrates the feature evolution of our model on RGB-thermal inputs, while the bottom row shows the feature evolution of VGGT on the corresponding RGB-only inputs.

Panel of Figure 17 — Figure 17: The blue line represents the median symmetrized KL-divergence between RGB-thermal tokens dependency across layers for SEAR method. The orange line represents the same dependency for the VGGT model. The filled area represents the boundary from $0.25-$ to $0.75-$ quantiles.

Method	AUC@30 $\uparrow$	RRA@30 $\uparrow$	RTA@30 $\uparrow$	PCA $\downarrow$	PCC $\downarrow$	Chamfer $\downarrow$
VGGT	41.7	75.8	67.9	0.96	0.20	0.58
SEAR	71.8	93.7	90.8	0.49	0.07	0.28

Appendix

Table 17: Results on the Public Scenes dataset using only thermal modality. The table compares SEAR and VGGT when only thermal images are passed to the networks.

Panel of Figure 18 — Figure 18: The AUC, RRA, RTA (errors < $30^{\circ}$ , $15^{\circ}$ , $5^{\circ}$ ) across modalities and varying thermal-to-RGB image ratios. The accuracy of RGB images decreases when decreasing the amount of RGB images similarly to thermal images.

Method	AUC@30 $\uparrow$	RRA@30 $\uparrow$	RTA@30 $\uparrow$	PCA $\downarrow$	PCC $\downarrow$	Chamfer $\downarrow$	Reg., % $\uparrow$	FPS $\uparrow$
COLMAP + SPSG	60.2	85.3	78.0	1.64	1.20	1.42	40.5	0.5
$\text{MA}_{\text{ELoFTR}}$	14.7	72.1	47.8	0.49	5.16	2.82	25.7	0.2
$\text{MINIMA}_{\text{ROMA}}$	42.6	67.7	64.3	0.97	1.09	1.03	99.0	0.0
DUSt3R	18.9	47.0	41.4	0.72	4.72	2.72	100.0	0.7
MASt3R	32.7	66.4	55.9	0.66	0.27	0.46	100.0	0.2
VGGT	23.0	50.6	50.3	1.22	2.60	1.91	100.0	10.47

Appendix

Table 18: Quantitative comparison of 3D reconstruction methods for all datasets. The results are aggregated across Public Datasets and SEAR dataset. Our method yields better scores than the baselines.

Dataset	Train	Test
ThermalGaussian	RotaryKiln, DarkScenes Truck, PlantEquipment DailyStuff, LandScape RoadBlock, Dimsum Building, GlassCup TransmissionTower	Parterre, IronIngot Ebike
ThermalNeRF	engine, heater trace, sheet generators, charger pyrex	sink, generator
ThermoScenes	trees, heater_water_cup reflect-laptop, double_robot dorm1, raspberrypi buildingA_spring, MED-building buildingA_winter, reflect-cup exhibition_building, dorm2 reflect-kettle, building-sunrise BI-building, prpt-fridge heater_water_kettle	reflect-robot, freezing_ice_cup prpt-cup, melting_ice_cup INR-building
ThermalMix	hand, lion face, pan	panel, laptop
RF	02_Fox_area_no_radars_0 02_Fox_area_no_radars_1 02_Fox_area_no_radars_2 02_Fox_area_no_radars_3 05_Return_from_the_forest_no_radar_0 05_Return_from_the_forest_no_radar_1 06_Sunset_area_no_radars_0 06_Sunset_area_no_radars_1 06_Sunset_area_no_radars_2	01_Annexet_No_Radars_0 01_Annexet_No_Radars_1 01_Annexet_No_Radars_2 01_Annexet_No_Radars_3 04_Forest_pass_no_radars_0 04_Forest_pass_no_radars_1
SEAR dataset	–	conference-room, metallic-container old-drinking-fountain, parking statue, telescope

Appendix

Table 19: Train/test splits for each dataset. The Train column contains scenes used for model training, while the Test column contains held-out scenes used for evaluation.

3D Reconstruction

3D reconstruction recovers a scene’s shape, appearance, and camera motion from images, video, depth, or other sensor measurements.

See all 955 papers

Jun 3, 2026

Unpaired RGB-Thermal Gaussian-Splatting Using Visual Geometric Transformers

Multi-modal novel view synthesis (NVS) combining RGB and thermal imagery enables precise 3D scene reconstruction with visual and thermal information. However, existing methods typically rely on precisely calibrated RGB-thermal image pairs or stereo setups, limiting scalability and practical deployment. To address this, we introduce a framework for unpaired RGB-thermal NVS that leverages VGGT, a 3D feed-forward transformer architecture, to independently estimate camera poses for each modality. The pose sets are then aligned using the Procrustes algorithm with a cross-modal feature matcher, enabling joint registration without paired calibration. Building on this alignment, we further propose a multi-modal 3D Gaussian Splatting approach that learns directly from unpaired RGB and thermal images. Experiments on diverse scenes demonstrate that our method achieves competitive performance in thermal view synthesis while maintaining RGB fidelity. Moreover, we show that existing reconstruction approaches can produce modality-specific reconstructions that lack cross-modal consistency. We thus introduce a benchmarking framework to rigorously evaluate both per-modality image synthesis and the multi-modal coherence of reconstructed scenes.

Jean Cordonnier, Chenghao Xu, Olga Fink +1

Ecole Polytechnique Federale de Lausanne, Lausanne, Switzerland · Schindler EPFL Lab, Lausanne, Switzerland

Thermal Imaging 3D Gaussian Splatting Multi-View 3D Reconstruction Novel View Synthesis

Ask PDF

Jan 22, 2026

ThermoSplat: Cross-Modal 3D Gaussian Splatting with Feature Modulation and Geometry Decoupling

Multi-modal scene reconstruction integrating RGB and thermal infrared data is essential for robust environmental perception across diverse lighting and weather conditions. However, extending 3D Gaussian Splatting (3DGS) to multi-spectral scenarios remains challenging. Current approaches often struggle to fully leverage the complementary information of multi-modal data, typically relying on mechanisms that either tend to neglect cross-modal correlations or leverage shared representations that fail to adaptively handle the complex structural correlations and physical discrepancies between spectrums. To address these limitations, we propose ThermoSplat, a novel framework that enables deep spectral-aware reconstruction through active feature modulation and adaptive geometry decoupling. First, we introduce a Spectrum-Aware Adaptive Modulation that dynamically conditions shared latent features on thermal structural priors, effectively guiding visible texture synthesis with reliable cross-modal geometric cues. Second, to accommodate modality-specific geometric inconsistencies, we propose a Modality-Adaptive Geometric Decoupling scheme that learns independent opacity offsets and executes an independent rasterization pass for the thermal branch. Additionally, a hybrid rendering pipeline is employed to integrate explicit Spherical Harmonics with implicit neural decoding, ensuring both semantic consistency and high-frequency detail preservation. Extensive experiments on the RGBT-Scenes dataset demonstrate that ThermoSplat achieves state-of-the-art rendering quality across both visible and thermal spectrums.

Zhaoqi Su, Shihai Chen, Xinyan Lin +3

College of Physics and Information Engineering, Fuzhou University, Fuzhou 350108, China

Thermal Imaging 3D Gaussian Splatting 3D Reconstruction Multimodal Fusion

Ask PDF

Jun 9, 2026

DarkVGGT: Seeing Through Darkness Using Thermal Geometry

Recent feed-forward 3D reconstruction methods have demonstrated strong performance and flexibility in efficient end-to-end scene geometry estimation from image streams. However, their reliance on visible-light appearance makes them vulnerable in dark and low-visibility environments, where RGB cues are severely degraded and geometric evidence becomes ambiguous. To address this challenge, we propose DarkVGGT, a structured RGB-T adaptation of pretrained VGGT for robust 3D estimation in low-light scenes. DarkVGGT introduces two complementary modules. First, complementary thermal feature factorization processes thermal tokens through content-preserving and high-frequency residual pathways controlled by learned complementary allocation weights. Second, geometry-shared thermal routing aligns RGB and thermal representations in a shared latent space and selectively injects a gated feature correction into the RGB stream. Together, these components enable accurate thermal-informed geometry estimation under degraded RGB conditions while largely preserving performance in well-lit environments. Experiments on low-visibility RGB-T benchmarks demonstrate consistent improvements in both depth and camera pose estimation over existing feed-forward geometry baselines.

Minseong Kweon, Wenyuan Zhao, Nuo Chen +6

University of Minnesota · Texas A&M University · Stanford University

Thermal Imaging 3D Reconstruction Depth Estimation Camera Pose Estimation Feed-Forward 3D Reconstruction

Ask PDF

Abstract

3D Reconstruction

Explore similar work

Unpaired RGB-Thermal Gaussian-Splatting Using Visual Geometric Transformers

ThermoSplat: Cross-Modal 3D Gaussian Splatting with Feature Modulation and Geometry Decoupling

DarkVGGT: Seeing Through Darkness Using Thermal Geometry