BAT3R: Robust Online 3D Reconstruction with Bayesian Adaptive State Updates

Lijian Li; Yuanpeng He; Yudian Zheng; Chung-ju Huang; Linyu Li; Dongming Jin; Wenpin Jiao; Zhi Jin

cs.CVOct 12, 2026

BAT3R: Robust Online 3D Reconstruction with Bayesian Adaptive State Updates

Authors: Lijian Li, Yuanpeng He, Yudian Zheng, Chung-ju Huang, Linyu Li, Dongming Jin, Wenpin Jiao, Zhi Jin, +1 more

Organizations: University of Macau, Macau SAR, China · School of Computer Science, Peking University, Beijing, China · Key Laboratory of High Confidence Software Technologies (Peking University), Ministry of Education, Beijing, China

Abstract

Feed-forward transformer models have driven remarkable progress in 3D vision,yet their quadratic complexity renders them impractical for long video sequences. Streaming 3D reconstruction addresses this bottleneck by processing frames sequen?tially with constant memory. However, existing recurrent architectures inevitably suffer from progressive degradation over long sequences, as they fail to account for the intrinsic reliability of observations and the accumulated confidence of the historical state, leading to severe error accumulation. To address these issues, we introduce BAT3R, a training-free Bayesian adaptation framework comprising two synergistic modules: State Innovation Gating (SIG) and Bayesian-guided Adaptive State Updating (BASU). SIG acts as a distributional regularizer that anchors state evolution to its geometric initialization, effectively arresting representational drift during long-term inference. Building on this, BASU reformulates state update as a recursive Bayesian inference process within a Kalman filtering framework. By jointly modeling the temporal reliability of observations via information-theoretic gain and spatial relevance via cross-attention maps, BASU achieves an asymmetric spatio-temporal state fusion: well-consolidated historical regions are protected from redundant updates, while genuinely novel scene content is aggressively incor?porated. Extensive experiments on standard benchmarks (7-Scenes, TUM, ScanNet, KITTI, etc.) demonstrate that BAT3R consistently outperforms state-of-the-art online 3D reconstruction methods, with the most significant gains observed on long sequences and complex scenes.

Figures & tables

Figure 1: Robust Online 3D Reconstruction. Given a stream of RGB images as input, our BAT3R framework enables dense 3D reconstruction in an online, continuous manner, estimating both camera parameters and dense 3D geometry with each incoming frame.

Figure 2: Overview of the BAT3R framework consisting of two modules State Innovation Gating (SIG) and Bayesian-guided Adaptive State Updating (BASU). SIG prevents representational drift by anchoring updates to the initial geometric state, while BASU optimizes observation fusion through a recursive Kalman filtering framework. By balancing observation reliability with historical confidence, the framework achieves robust, uncertainty-aware 3D scene reconstruction.

Panel of Figure 3 — Figure 3: Evaluation of computational efficiency on ScanNet. (a) GPU memory and (b) inference speed across varying numbers of input views.

Figure 4: Comparison of Camera Pose Estimation. Results on ScanNet (left) and TUM-D (right). OOM denotes the method out-of-memory beyond this point.

Figure 5: Scale-invariant relative depth evaluation on Bonn dataset.

Figure 6: Comparison of 3D Reconstruction on 7-scene.

Setting	Camera Pose Estimation						Video Depth Estimation
	TUM-dynamics (90 frames)			ScanNet (90 frames)			Bonn (100 frames)		KITTI (100 frames)
	ATE $\downarrow$	RPE trans $\downarrow$	RPE rot $\downarrow$	ATE $\downarrow$	RPE trans $\downarrow$	RPE rot $\downarrow$	Abs Rel $\downarrow$	$\delta<1.25\uparrow$	Abs Rel $\downarrow$	$\delta<1.25\uparrow$
BAT3R (Full)	0.016	0.008	0.287	0.058	0.022	0.852	0.061	97.0	0.100	89.2
w/o BRU	0.024	0.008	0.288	0.099	0.021	0.562	0.062	96.5	0.106	87.9
w/o SIG	0.016	0.008	0.289	0.061	0.025	1.163	0.065	95.6	0.105	88.8
w/o Attention	0.017	0.008	0.260	0.060	0.024	0.988	0.063	96.8	0.105	88.1

Table 1: Comprehensive Ablation Study on Camera Pose and Video Depth Estimation. We evaluate the impact of different components across four standard benchmarks.

Panel of Figure 7 — Figure 7: Qualitative comparison of 3D reconstruction. We compare BAT3R with TTT3R and CUT3R across three challenging sequences in the 7-Scene dataset.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

		Sintel (50 frames)			TUM-dynamics (90 frames)			ScanNet (90 frames)
Method	Online	ATE $\downarrow$	RPE trans $\downarrow$	RPE rot $\downarrow$	ATE $\downarrow$	RPE trans $\downarrow$	RPE rot $\downarrow$	ATE $\downarrow$	RPE trans $\downarrow$	RPE rot $\downarrow$
Robust-CVD [ 26 ]	✗	0.360	0.154	3.443	0.153	0.026	3.528	0.227	0.064	7.374
CasualSAM [ 61 ]	✗	0.141	0.035	0.615	0.071	0.010	1.712	0.158	0.034	1.618
DUSt3R [ 45 ]	✗	0.417	0.250	5.796	0.083	0.017	3.567	0.081	0.028	0.784
MAST3R [ 27 ]	✗	0.185	0.060	1.496	0.038	0.012	0.448	0.078	0.020	0.475
MonST3R [ 58 ]	✗	0.111	0.044	0.869	0.098	0.019	0.935	0.077	0.018	0.529

Appendix

Table 2: Evaluation on Camera Pose Estimation- Short Sequence on Sintel [ 7 ] , TUM-dynamic [ 36 ] , and ScanNet [ 15 ] datasets. TTT3R achieves the best overall performance among online methods, while its accuracy has not yet matched strong offline methods (e.g., VGGT), where full attention, despite being slower and more memory-demanding, preserves the entire history context.

Figure 8: Qualitative comparison of eight estimated camera trajectories on the TUM RGB-D dataset.

Figure 9: Metric depth evaluation on KITTI, excluding VGGT-based methods that don’t support metric depth.

			Sintel (50 frames)		BONN (110 frames)		KITTI (110 frames)
Alignment	Method	Online	Abs Rel $\downarrow$	$\delta<1.25$ $\uparrow$	Abs Rel $\downarrow$	$\delta<1.25$ $\uparrow$	Abs Rel $\downarrow$	$\delta<1.25$ $\uparrow$
Per-sequence Scale	DUSt3R [ 45 ]	✗	0.656	45.2	0.155	83.3	0.144	81.3
	MAST3R [ 27 ]	✗	0.641	43.9	0.252	70.1	0.183	74.5
	MonST3R [ 58 ]	✗	0.378	55.8	0.067	96.3	0.168	74.4
	Easi3R [ 9 ]	✗	0.377	55.9	0.059	97.0	0.102	91.2
	AETHER [ 63 ]	✗	0.324	50.2	0.273	59.4	0.056	97.8

Appendix

Table 3: Evaluation of Video Depth Estimation - Short Sequence. We report scale-invariant relative depth (aligned by a per-sequence scale) and metric scale absolute depth accuracy on Sintel [ 7 ] , KITTI [ 19 ] and Bonn [ 31 ] datasets. BAT3R achieves competitive performance among online methods.

Model	Input	7-Scenes						NRGBD
		Acc. $\downarrow$		Comp. $\downarrow$		NC $\uparrow$		Acc. $\downarrow$		Comp. $\downarrow$		NC $\uparrow$
		Mean	Med.	Mean	Med.	Mean	Med.	Mean	Med.	Mean	Med.	Mean	Med.
VGGT (Offline) [ 42 ]	300	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM
	400	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM
	500	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM
StreamVGGT [ 64 ]	300	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM

Appendix

Table 4: 3D Reconstruction Results on 7-Scenes and NRGBD.

Panel of Figure 10 — Figure 10: Hyperparameter sensitivity analysis defined in Eq. 6 . We report the Average ATE to evaluate the impact of (a) the minimum update floor $\eta$ , (b) the scaling factor $\kappa$ controlling the transition slope, and (c) the shifting threshold $\tau$ .

Panel of Figure 11 — Figure 11: Ablation studies on additional hyperparameters. We report the Average ATE for (a) relative floor, (b) APN alpha, and (c) KG weight settings to evaluate their sensitivity.

Explore similar work

TTSA3R: Training-Free Temporal-Spatial Adaptive Persistent State for Streaming 3D Reconstruction
Jan 30, 2026Zhijie Zheng, Xinhao Xiang, Jiawei ZhangStreaming 3D Reconstruction 3D Reconstruction
University of California, Davis · CA 95616, USA
Info3R: Information-Adaptive Test-Time Training for 3D Reconstruction
Sep 18, 2026Sunghyun Baek, Hanna Bae, Minchan Kwon +1Streaming 3D Reconstruction 3D Reconstruction
Department of Electrical Engineering, Korea Advanced Institute of Science and Technology, South Korea
PAS3R: Pose-Adaptive Streaming 3D Reconstruction for Long Video Sequences
Mar 22, 2026Lanbo Xu, Liang Guo, Caigui Jiang +1Streaming 3D Reconstruction Monocular 3D Reconstruction
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, China · University of East Anglia, Norwich, NR47TJ, United Kingdom

Jan 30, 2026J/K move · Enter open · S save

TTSA3R: Training-Free Temporal-Spatial Adaptive Persistent State for Streaming 3D Reconstruction

Zhijie Zheng, Xinhao Xiang, Jiawei Zhang

University of California, Davis · CA 95616, USA

Streaming 3D Reconstruction 3D Reconstruction Catastrophic Forgetting

Open paper AskPDF arXiv

Streaming recurrent models enable efficient 3D reconstruction by maintaining persistent state representations. However, they suffer from catastrophic forgetting over long sequences due to balancing historical information with new observations. Recent methods alleviate this by deriving adaptive signals from the attention perspective, but they operate on single dimensions without considering temporal and spatial consistency. To this end, we propose a training-free framework termed TTSA3R that leverages both temporal state evolution and spatial observation quality for adaptive state updates in 3D reconstruction. In particular, we devise a Temporal Adaptive Update Module that regulates update magnitude by analyzing temporal state evolution patterns. Then, a Spatial Contextual Update Module is introduced to localize spatial regions that require updates through observation-state alignment and scene dynamics. These complementary signals are finally fused to determine the state updating strategies. Extensive experiments show that TTSA3R achieves competitive performance on standard short-sequence benchmarks and provides substantially stronger robustness on extended sequences. On NRGBD, as sequences extend from 50 to 250 frames, TTSA3R exhibits only a 1.33x error increase, compared with over 4x degradation for CUT3R. This highlights the practical value of temporal-spatial adaptive updates for long-term reconstruction stability. Our code is available at https://github.com/anonus2357/ttsa3r.

Abstract

Explore similar work

TTSA3R: Training-Free Temporal-Spatial Adaptive Persistent State for Streaming 3D Reconstruction

Info3R: Information-Adaptive Test-Time Training for 3D Reconstruction

PAS3R: Pose-Adaptive Streaming 3D Reconstruction for Long Video Sequences