cs.CVMar 22, 2026

PAS3R: Pose-Adaptive Streaming 3D Reconstruction for Long Video Sequences

Authors: Lanbo Xu, Liang Guo, Caigui Jiang, Cheng Wang

Organizations: State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, China · University of East Anglia, Norwich, NR47TJ, United Kingdom

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Online monocular 3D reconstruction enables dense scene recovery from streaming video but remains fundamentally limited by the stability-adaptation dilemma: the reconstruction model must rapidly incorporate novel viewpoints while preserving previously accumulated scene structure. Existing streaming approaches rely on uniform or attention-based update mechanisms that often fail to account for abrupt viewpoint transitions, leading to trajectory drift and geometric inconsistencies over long sequences. We introduce PAS3R, a pose-adaptive streaming reconstruction framework that dynamically modulates state updates according to camera motion and scene structure. Our key insight is that frames contributing significant geometric novelty should exert stronger influence on the reconstruction state, while frames with minor viewpoint variation should prioritize preserving historical context. PAS3R operationalizes this principle through a motion-aware update mechanism that jointly leverages inter-frame pose variation and image frequency cues to estimate frame importance. To further stabilize long-horizon reconstruction, we introduce trajectory-consistent training objectives that incorporate relative pose constraints and acceleration regularization. A lightweight online stabilization module further suppresses high-frequency trajectory jitter and geometric artifacts without increasing memory consumption. Extensive experiments across multiple benchmarks demonstrate that PAS3R significantly improves trajectory accuracy, depth estimation, and point cloud reconstruction quality in long video sequences while maintaining competitive performance on shorter sequences.

Figures & tables

Figure 1: The overall pipeline. Given a streaming image sequence, the model maintains a persistent memory that is iteratively updated through cross-attention with incoming frames. PAS3R introduces pose-adaptive state update modulation, where the update intensity (learning rate) is dynamically adjusted based on camera motion and image structure (via Fourier analysis), allowing the model to balance adaptation to novel viewpoints and preservation of accumulated geometry. During inference, the model predicts camera poses and dense point clouds at each step. During training, we further enforce trajectory-consistent optimization using a novel pose loss with ATE, RPE, and acceleration constraints. A lightweight online spatiotemporal stabilization module is applied to reduce trajectory jitter and geometric artifacts. Together, these components enable stable and scalable long-horizon streaming 3D reconstruction.
Figure 2: Effect of adaptive state update weighting. We evaluate the impact of adaptive update weighting in a real-world indoor video sequence. The red bounding box highlights a keyframe region where the camera undergoes significant pose variation. Ideally, the trajectory following this region should remain approximately parallel. However, both CUT3R [ 69 ] and TTT3R [ 13 ] exhibit noticeable drift from the real trajectory, while the adaptive weighting strategy (TTTWL denotes PAS3R with adaptive update only) helps maintain trajectory continuity under abrupt viewpoint changes.
Figure 3: Effect of trajectory-consistent training. When large camera motion occurs after relatively stable frames, adaptive updates alone may produce abrupt trajectory variations. The proposed loss formulation reduces these discontinuities and produces smoother and more physically consistent camera trajectories. TTTWL denotes PAS3R with adaptive update only (Section 3.1 ), while TTTLM denotes PAS3R with adaptive update and trajectory-consistent training (Section 3.2 ).
Figure 4: Effect of online trajectory stabilization.
Figure 5: Effect of online spatial geometry refinement.
Figure 6: Comparison of camera pose estimation on Scannet [ 15 ]
MethodSintelTum
ATE ↓\downarrowRPE trans ↓\downarrowRPE rot ↓\downarrowATE ↓\downarrowRPE trans ↓\downarrowRPE rot ↓\downarrow
Mem4D [ 10 ]0.2630.0910.8120.0610.0200.517
CUT3R [ 69 ]0.2090.0710.6320.0470.0150.448
TTT3R [ 13 ]0.2100.0900.7300.0290.0130.379
IVGGT [ 81 ]0.2370.0960.8060.0270.0120.313
Ours0.2110.0530.6880.0270.0110.475
Table 1: Quantitative comparison on Sintel [ 7 ] and Tum [ 54 ] datasets. Bold indicates the best performance, and underlined indicates the second best. ↓\downarrow indicates lower is better.
Figure 7: Comparison of depth on Bonn [ 39 ]
MethodSintelBonnKitti
Abs Rel ↓\downarrowσ<1.25↑\sigma<1.25\uparrowAbs Rel ↓\downarrowσ<1.25↑\sigma<1.25\uparrowAbs Rel ↓\downarrowσ<1.25↑\sigma<1.25\uparrow
Mem4D [ 10 ]0.52043.10.07295.70.14082.0
CUT3R [ 69 ]0.43247.00.07793.80.12287.6
TTT3R [ 13 ]0.40648.80.06995.40.11490.6
IVGGT [ 81 ]0.32966.00.05997.40.18569.8
Ours0.40748.70.06496.60.11591.4
Table 2: Quantitative comparison on Sintel, Bonn [ 39 ] , and Kitti [ 23 ] datasets. Bold indicates the best performance, and underlined indicates the second best. ↓\downarrow indicates lower is better, while ↑\uparrow indicates higher is better.
Figure 8: Comparison of 3D Reconstruction on 7scene [ 52 ]
Figure 9: Qualitative comparison of 3D reconstruction results. In the Stair scene, methods such as IVGGT and CUT3R suffer from large spatial drift. In the Office and Kitchen scene, IVGGT, TTT3R, and CUT3R all experience varying degrees of structural collapse at wall corners, whereas PAS3R remains well aligned with the GT.
MethodTum1000Bonn5007Scene400
ATE ↓\downarrowRPE trans ↓\downarrowRPE rot ↓\downarrowAbs Rel ↓\downarrowδ<1.25↑\delta<1.25\uparrowAcc ↓\downarrowComp ↓\downarrowNC ↑\uparrow
w/o pose-adaptive state update0.108540.003740.468710.075795.240.0480.0280.563
w/o trajectory-consistent training0.056160.004590.592150.071095.350.0230.0210.562
w/o spatiotemporal stabilization0.049800.011380.610980.071095.360.0190.0190.561
Full Method0.052140.003540.525950.070995.350.0180.0210.564
Table 3: Quantitative ablation study results on Tum1000 [ 54 ] , Bonn500 [ 39 ] , and 7Scene400 [ 52 ] datasets. The best results are highlighted in bold , and the second-best results are underlined . ↓\downarrow indicates lower is better, while ↑\uparrow indicates higher is better.
Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 10: Comparison of camera pose estimation on TUM [ 54 ]
Figure 11: Comparison of camera trajectories on Scannet [ 15 ]
Figure 12: Comparison of depth on Kitti [ 23 ]
Method7-Scenes [ 52 ]NRGBD [ 52 ]
Acc ↓\downarrowComp ↓\downarrowAcc ↓\downarrowComp ↓\downarrow
Mem4D [ 10 ]0.1850.1780.2710.212
CUT3R [ 69 ]0.1200.1560.1000.078
TTT3R [ 13 ]0.1560.1960.1050.080
IVGGT [ 81 ]0.1240.1060.1100.109
Ours0.1510.1820.1030.087
Appendix
Table 4: Quantitative comparison on 7-Scenes and NRGBD datasets [ 52 ] . Bold indicates the best performance, and underlined indicates the second best. Acc (Accuracy) and Comp (Completeness) are error metrics, where ↓\downarrow indicates lower is better.
Figure 13: More qualitative comparison of 3D reconstruction results
Figure 14: Comparison of GPU usage and FPS on Scannet [ 15 ]

Explore similar work

CardsList
Aug 27, 2026

Revisiting Local Context for Long-Horizon Streaming 3D Reconstruction

Streaming 3D reconstruction from extremely long videos requires estimating camera motion and scene geometry online under bounded memory and computation. Early streaming models achieve causal, bounded-cost inference using finite context buffers or compact recurrent states, yet their estimates often deteriorate as sequences grow. Recent methods improve long-horizon stability by coupling short-range context with persistent or multi-level long-range memory. We pursue a different route: we keep the learned temporal state strictly local and formulate predictions whose targets remain independent of sequence length. We present ABot-Recon, a simple streaming model that caches KV features from only the preceding 11 frames. It predicts a point map in the current camera coordinate system together with an adjacent-frame relative pose. These predictions remain equivariant under changes of reference frame, and global poses and geometry are recovered through sequential composition. To reduce accumulated drift, a lightweight temporal refiner improves relative rotations using recent visual and motion context, while a composition-aware pose loss supervises multi-step pose composition. Extensive evaluations on challenging long-sequence benchmarks demonstrate the superior long-horizon performance of our local-context approach. On Oxford Spires, ABot-Recon achieves an ATE of 4.35 m and an RPE-R of 0.12∘0.12^\circ, reducing both errors by approximately 40% relative to the best prior results.

Jiarong Han, Jincheng Xiong, Yuzhou Liu +6

AMAP CV Lab Alibaba Group

Streaming 3D Reconstruction3D ReconstructionCamera Pose EstimationVisual Odometry
AskPDF
May 22, 2026

HorizonStream: Long-Horizon Attention for Streaming 3D Reconstruction

Online 3D reconstruction requires estimating camera pose and scene geometry under strict causal and bounded-memory constraints. Existing methods often suffer from drift, jitter, or collapse on long sequences. We trace these failures to a fundamental mismatch. Streaming geometry is inherently temporally heterogeneous, with evidence ranging from short-lived correspondences to persistent global scale. However, current architectures impose uniform and pathological influence patterns. For example, sliding windows enforce hard cutoffs, while ungated recurrence and causal attention cause cache saturation and spike-like attention sinks. To resolve this, we formalize geometric propagation as an evidence influence kernel and propose HorizonStream, a long-horizon Transformer that explicitly factorizes this kernel. For the long-range temporal factor, Geometric Linear Attention learns channel-wise decay rates to enable bounded, multi-timescale propagation of geometric evidence. For the short-range spatial factor, Geometric Local Attention with Spatiotemporal RoPE performs reliable 3D matching while suppressing attention sinks. Finally, Metric Readout Tokens recover stable scale and rigid pose directly from the persistent geometric state. Extensive experiments show that HorizonStream, trained on only 48-frame clips, generalizes stably to sequences exceeding 10,000\ frames with constant memory and linear time, achieving state-of-the-art streaming 3D reconstruction performance. Project Page: https://3dagentworld.github.io/horizonstream/

Chong Cheng, Peilin Tao, Nanjie Yao +9

HKUST(GZ) · Horizon Robotics · CASIA +2

3D ViTsStreaming 3D Reconstruction3D ReconstructionCamera Pose EstimationLinear Attention
AskPDF
Oct 12, 2026

BAT3R: Robust Online 3D Reconstruction with Bayesian Adaptive State Updates

Feed-forward transformer models have driven remarkable progress in 3D vision,yet their quadratic complexity renders them impractical for long video sequences. Streaming 3D reconstruction addresses this bottleneck by processing frames sequen?tially with constant memory. However, existing recurrent architectures inevitably suffer from progressive degradation over long sequences, as they fail to account for the intrinsic reliability of observations and the accumulated confidence of the historical state, leading to severe error accumulation. To address these issues, we introduce BAT3R, a training-free Bayesian adaptation framework comprising two synergistic modules: State Innovation Gating (SIG) and Bayesian-guided Adaptive State Updating (BASU). SIG acts as a distributional regularizer that anchors state evolution to its geometric initialization, effectively arresting representational drift during long-term inference. Building on this, BASU reformulates state update as a recursive Bayesian inference process within a Kalman filtering framework. By jointly modeling the temporal reliability of observations via information-theoretic gain and spatial relevance via cross-attention maps, BASU achieves an asymmetric spatio-temporal state fusion: well-consolidated historical regions are protected from redundant updates, while genuinely novel scene content is aggressively incor?porated. Extensive experiments on standard benchmarks (7-Scenes, TUM, ScanNet, KITTI, etc.) demonstrate that BAT3R consistently outperforms state-of-the-art online 3D reconstruction methods, with the most significant gains observed on long sequences and complex scenes.

Lijian Li, Yuanpeng He, Yudian Zheng +6

University of Macau, Macau SAR, China · School of Computer Science, Peking University, Beijing, China · Key Laboratory of High Confidence Software Technologies (Peking University), Ministry of Education, Beijing, China

Streaming 3D ReconstructionBayesian Filtering3D ViTs
AskPDF