PAS3R: Pose-Adaptive Streaming 3D Reconstruction for Long Video Sequences
Authors: Lanbo Xu, Liang Guo, Caigui Jiang, Cheng Wang
Organizations: State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, China · University of East Anglia, Norwich, NR47TJ, United Kingdom
Online monocular 3D reconstruction enables dense scene recovery from streaming video but remains fundamentally limited by the stability-adaptation dilemma: the reconstruction model must rapidly incorporate novel viewpoints while preserving previously accumulated scene structure. Existing streaming approaches rely on uniform or attention-based update mechanisms that often fail to account for abrupt viewpoint transitions, leading to trajectory drift and geometric inconsistencies over long sequences. We introduce PAS3R, a pose-adaptive streaming reconstruction framework that dynamically modulates state updates according to camera motion and scene structure. Our key insight is that frames contributing significant geometric novelty should exert stronger influence on the reconstruction state, while frames with minor viewpoint variation should prioritize preserving historical context. PAS3R operationalizes this principle through a motion-aware update mechanism that jointly leverages inter-frame pose variation and image frequency cues to estimate frame importance. To further stabilize long-horizon reconstruction, we introduce trajectory-consistent training objectives that incorporate relative pose constraints and acceleration regularization. A lightweight online stabilization module further suppresses high-frequency trajectory jitter and geometric artifacts without increasing memory consumption. Extensive experiments across multiple benchmarks demonstrate that PAS3R significantly improves trajectory accuracy, depth estimation, and point cloud reconstruction quality in long video sequences while maintaining competitive performance on shorter sequences.
Figures & tables
Figure 1: The overall pipeline. Given a streaming image sequence, the model maintains a persistent memory that is iteratively updated through cross-attention with incoming frames. PAS3R introduces pose-adaptive state update modulation, where the update intensity (learning rate) is dynamically adjusted based on camera motion and image structure (via Fourier analysis), allowing the model to balance adaptation to novel viewpoints and preservation of accumulated geometry. During inference, the model predicts camera poses and dense point clouds at each step. During training, we further enforce trajectory-consistent optimization using a novel pose loss with ATE, RPE, and acceleration constraints. A lightweight online spatiotemporal stabilization module is applied to reduce trajectory jitter and geometric artifacts. Together, these components enable stable and scalable long-horizon streaming 3D reconstruction.
Figure 2: Effect of adaptive state update weighting. We evaluate the impact of adaptive update weighting in a real-world indoor video sequence. The red bounding box highlights a keyframe region where the camera undergoes significant pose variation. Ideally, the trajectory following this region should remain approximately parallel. However, both CUT3R [ 69 ] and TTT3R [ 13 ] exhibit noticeable drift from the real trajectory, while the adaptive weighting strategy (TTTWL denotes PAS3R with adaptive update only) helps maintain trajectory continuity under abrupt viewpoint changes.
Figure 3: Effect of trajectory-consistent training. When large camera motion occurs after relatively stable frames, adaptive updates alone may produce abrupt trajectory variations. The proposed loss formulation reduces these discontinuities and produces smoother and more physically consistent camera trajectories. TTTWL denotes PAS3R with adaptive update only (Section 3.1 ), while TTTLM denotes PAS3R with adaptive update and trajectory-consistent training (Section 3.2 ).
Figure 4: Effect of online trajectory stabilization.
Figure 5: Effect of online spatial geometry refinement.
Figure 6: Comparison of camera pose estimation on Scannet [ 15 ]
Method
Sintel
Tum
ATE ↓
RPE trans ↓
RPE rot ↓
ATE ↓
RPE trans ↓
RPE rot ↓
Mem4D [ 10 ]
0.263
0.091
0.812
0.061
0.020
0.517
CUT3R [ 69 ]
0.209
0.071
0.632
0.047
0.015
0.448
TTT3R [ 13 ]
0.210
0.090
0.730
0.029
0.013
0.379
IVGGT [ 81 ]
0.237
0.096
0.806
0.027
0.012
0.313
Ours
0.211
0.053
0.688
0.027
0.011
0.475
Table 1: Quantitative comparison on Sintel [ 7 ] and Tum [ 54 ] datasets. Bold indicates the best performance, and underlined indicates the second best. ↓ indicates lower is better.
Figure 7: Comparison of depth on Bonn [ 39 ]
Method
Sintel
Bonn
Kitti
Abs Rel ↓
σ<1.25↑
Abs Rel ↓
σ<1.25↑
Abs Rel ↓
σ<1.25↑
Mem4D [ 10 ]
0.520
43.1
0.072
95.7
0.140
82.0
CUT3R [ 69 ]
0.432
47.0
0.077
93.8
0.122
87.6
TTT3R [ 13 ]
0.406
48.8
0.069
95.4
0.114
90.6
IVGGT [ 81 ]
0.329
66.0
0.059
97.4
0.185
69.8
Ours
0.407
48.7
0.064
96.6
0.115
91.4
Table 2: Quantitative comparison on Sintel, Bonn [ 39 ] , and Kitti [ 23 ] datasets. Bold indicates the best performance, and underlined indicates the second best. ↓ indicates lower is better, while ↑ indicates higher is better.
Figure 8: Comparison of 3D Reconstruction on 7scene [ 52 ]
Figure 9: Qualitative comparison of 3D reconstruction results. In the Stair scene, methods such as IVGGT and CUT3R suffer from large spatial drift. In the Office and Kitchen scene, IVGGT, TTT3R, and CUT3R all experience varying degrees of structural collapse at wall corners, whereas PAS3R remains well aligned with the GT.
Method
Tum1000
Bonn500
7Scene400
ATE ↓
RPE trans ↓
RPE rot ↓
Abs Rel ↓
δ<1.25↑
Acc ↓
Comp ↓
NC ↑
w/o pose-adaptive state update
0.10854
0.00374
0.46871
0.0757
95.24
0.048
0.028
0.563
w/o trajectory-consistent training
0.05616
0.00459
0.59215
0.0710
95.35
0.023
0.021
0.562
w/o spatiotemporal stabilization
0.04980
0.01138
0.61098
0.0710
95.36
0.019
0.019
0.561
Full Method
0.05214
0.00354
0.52595
0.0709
95.35
0.018
0.021
0.564
Table 3: Quantitative ablation study results on Tum1000 [ 54 ] , Bonn500 [ 39 ] , and 7Scene400 [ 52 ] datasets. The best results are highlighted in bold , and the second-best results are underlined . ↓ indicates lower is better, while ↑ indicates higher is better.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 10: Comparison of camera pose estimation on TUM [ 54 ]
Figure 11: Comparison of camera trajectories on Scannet [ 15 ]
Figure 12: Comparison of depth on Kitti [ 23 ]
Method
7-Scenes [ 52 ]
NRGBD [ 52 ]
Acc ↓
Comp ↓
Acc ↓
Comp ↓
Mem4D [ 10 ]
0.185
0.178
0.271
0.212
CUT3R [ 69 ]
0.120
0.156
0.100
0.078
TTT3R [ 13 ]
0.156
0.196
0.105
0.080
IVGGT [ 81 ]
0.124
0.106
0.110
0.109
Ours
0.151
0.182
0.103
0.087
Appendix
Table 4: Quantitative comparison on 7-Scenes and NRGBD datasets [ 52 ] . Bold indicates the best performance, and underlined indicates the second best. Acc (Accuracy) and Comp (Completeness) are error metrics, where ↓ indicates lower is better.
Figure 13: More qualitative comparison of 3D reconstruction results
Figure 14: Comparison of GPU usage and FPS on Scannet [ 15 ]
University of Macau, Macau SAR, China · School of Computer Science, Peking University, Beijing, China · Key Laboratory of High Confidence Software Technologies (Peking University), Ministry of Education, Beijing, China