Reconstructing dynamic 4D scenes remains challenging due to the presence of moving objects that corrupt camera pose estimation. Existing optimization methods alleviate this issue with additional supervision, but they are mostly computationally expensive and impractical in real-time applications. To address these limitations, we propose MoRe, a feedforward 4D reconstruction network that efficiently recovers dynamic 3D scenes from monocular videos. Built upon a strong static reconstruction backbone, MoRe employs an attention-forcing strategy to disentangle dynamic motion from static structure. To further enhance robustness, we fine-tune the model on large-scale, diverse datasets encompassing both dynamic and static scenes. Moreover, our grouped causal attention captures temporal dependencies and adapts to varying token lengths across frames, ensuring temporally coherent geometry reconstruction. Extensive experiments on multiple benchmarks demonstrate that MoRe achieves high-quality dynamic reconstructions with exceptional efficiency.
Figures & tables
Figure 2: Method Overview. During training, an attention-forcing mechanism aligns the attention weights with ground-truth motion masks, enabling the model to effectively disentangle dynamic motion from static scene structure. For streaming reconstruction task, MoRe is based on a causal transformer where global attention is replaced by aggregated causal attention.
Figure 3: Attention Map Visualization. We visualize the attention map of the camera token within VGGT [ 39 ] and observe that the model tends to confuse moving objects with static background regions, which accounts for the degradation in prediction accuracy.
Figure 4: Grouped Causal Attention. Unlike traditional causal attention, our design allows image tokens within the same frame to attend to each other regardless of their ordering. This formulation enables the model to preserve causal temporal reasoning while maintaining spatial consistency within each frame.
Figure 5: Streaming Inference pipeline. Leveraging causal attention, our model can efficiently process streaming input in an online manner. To enhance camera pose accuracy, we apply a bundle-adjustment-like post-processing step after the entire sequence has been processed. Specifically, for each frame, we duplicate the camera token and perform inference again using the previously cached key-value pairs.
Figure 6: Qualititive Comparison of Our Full Attention Model with Other Methods. MoRe delivers outstanding performance in real-world scenes, outperforming other methods through its precise and robust geometry estimation.
Sintel
Bonn
TUM-dynamics
ScanNet (static)
Method
type
ATE ↓
RPE trans↓
RPE rot↓
ATE ↓
RPE trans↓
RPE rot↓
ATE ↓
RPE trans↓
RPE rot↓
ATE ↓
RPE trans↓
RPE rot↓
MapAnything [ 16 ]
FA
0.2104
0.0919
2.7396
0.0248
0.0132
0.6927
0.0244
0.0161
0.3871
0.0603
0.0280
0.8490
VGGT [ 39 ]
FA
0.1715
0.0617
0.4695
0.0141
0.0100
0.6323
0.0109
0.0092
0.2992
0.0347
0.0151
0.3758
MoRe(FA)
FA
0.0877
0.0580
0.3899
0.0138
0.0099
0.6267
0.0115
0.0088
0.2980
0.0375
0.0147
0.3847
Spann3R [ 38 ]
Streaming
0.3313
0.1111
4.4952
0.0344
0.0212
2.2539
0.0421
0.0333
0.8120
0.0651
0.0339
0.9348
CUT3R [ 40 ]
Streaming
0.2163
0.0756
0.6518
0.0420
0.0094
0.6825
0.0438
0.0134
0.4210
0.0929
0.0223
0.5811
Table 1: Camera Pose Estimation on Sintel [ 3 ] , TUM-dynamics [ 33 ] , Bonn [ 27 ] , and ScanNet [ 5 ] datasets. FA refers to full attention.
Figure 7: Qualitative Comparison of our Stream Model with Other Methods. MoRe strikes an optimal balance between reconstruction quality and computational efficiency, delivering high-fidelity results at competitive inference speeds.
Sintel
Bonn
TUM-dynamics
kitti
Method
type
Abs Rel ↓
δ<1.25↑
Abs Rel ↓
δ<1.25↑
Abs Rel ↓
δ<1.25↑
Abs Rel ↓
δ<1.25↑
Flare [ 51 ]
FA
0.729
0.336
0.116
0.897
0.152
0.790
0.356
0.570
MapAnything [ 16 ]
FA
0.632
0.461
0.125
0.859
0.241
0.813
0.277
0.742
VGGT [ 39 ]
FA
0.387
0.584
0.055
0.970
0.132
0.901
0.073
0.961
MoRe(FA)
FA
0.335
0.645
0.055
0.971
0.120
0.902
0.066
0.967
Spann3R [ 38 ]
Streaming
0.740
0.364
0.229
0.729
0.264
0.645
0.422
0.358
Table 2: Video Depth Estimation on Sintel [ 3 ] , TUM-dynamics [ 33 ] , Bonn [ 27 ] and kitti [ 12 ] .
Sintel
TUM-dynamics
Method
ATE ↓
RPE trans↓
RPE rot↓
ATE ↓
RPE trans↓
RPE rot↓
w/o attention forcing
0.163
0.092
0.660
0.028
0.014
0.329
w/o BA-like refinement
0.155
0.085
0.619
0.027
0.014
0.321
full
0.147
0.082
0.616
0.026
0.013
0.320
Table 3: Ablation on Camera Pose Estimation
Sintel
Bonn
kitti
Method
Abs Rel ↓
δ<1.25↑
Abs Rel ↓
δ<1.25↑
Abs Rel ↓
δ<1.25↑
w/o GCA
0.277
0.592
0.070
0.953
0.079
0.949
w/
0.254
0.637
0.068
0.961
0.072
0.966
Table 4: Ablation on Video Depth Estimation
Figure 8: Qualitative Results of Motion Mask Extraction. Our method robustly captures moving objects across diverse scenes and objects.
Figure 9: Qualitative Comparison of Our Model with Other Methods. We present an extensive set of visual results showcasing reconstruction quality, motion handling, and robustness under challenging dynamic scenarios.
Method
Co3Dv2
RRA@30 ↑
RTA@30 ↑
AUC@30 ↑
Fast3R [ 46 ]
97.49
91.11
73.43
CUT3R [ 40 ]
96.19
92.69
75.82
FLARE [ 51 ]
96.38
93.76
73.99
VGGT [ 39 ]
98.96
97.13
88.59
π3 [ 43 ]
99.05
97.33
88.41
Table 6: Camera Pose Estimation Comparison on Co3Dv2 [ 30 ] .
Figure 10: Attention Map Comparison. We visualize the attention map on Dynamic Replica [ 34 ] and DAVIS [ 28 ] dataset. Our motion-aligned training suppresses undesired attention from camera tokens to dynamic objects, yielding cleaner and more structured attention patterns.
Sintel
TUM-dynamics
Method
ATE ↓
RPE trans↓
RPE rot↓
ATE ↓
RPE trans↓
RPE rot↓
w/ KL loss
0.185
0.084
0.707
0.029
0.015
0.350
Ours
0.147
0.082
0.616
0.026
0.013
0.320
Table 7: Ablation on Loss Function for Motion Alignment.
Reconstructing dynamic 4D scenes from monocular videos is a fundamental yet challenging task. While recent 3D foundation models provide strong geometric priors, their performance significantly degrades in dynamic environments. This degradation stems from a fundamental tension: the inherent coupling of camera ego-motion and object motion within global attention mechanisms. In this paper, we propose a novel, training-free progressive decoupling framework that disentangles dynamics from statics in a principled, coarse-to-fine manner. Our core insight is to resolve the tension by first stabilizing the camera pose, followed by geometric refinement. Specifically, our approach consists of three synergistic components: (1) a Dynamic-Mask-Guided Pose Decoupling module that isolates pose estimation from dynamic interference, yielding a stable motion-free reference frame; (2) a Topological Subspace Surgery mechanism that orthogonally decomposes the depth manifold, safely preserving dynamic objects while injecting refined, mask-aware geometry into static regions; and (3) an Information-Theoretic Confidence-Aware Fusion strategy that formulates depth integration as a heteroscedastic Bayesian inference problem, adaptively blending multi-pass predictions via inverse-variance weighting. Extensive experiments on standard 4D reconstruction benchmarks demonstrate that our method achieves consistent and substantial improvements across principal point-cloud metrics. Notably, our approach shows competitive performance in robust 4D scene reconstruction without requiring fine-tuning, suggesting the potential of mathematically grounded dynamic-static disentanglement.
Ying Zang, Xuanyi Liu, Yidong Han +9
4Huzhou University · 2Peking University · 1KOKONI 3D, Moxin Technology +4
Realistic reconstruction of dynamic 4D scenes from monocular videos is essential for understanding the physical world. Despite recent progress in neural rendering, existing methods often struggle to recover accurate 3D geometry and temporally consistent motion in complex environments. To address these challenges, we propose MotionScale, a 4D Gaussian Splatting framework that scales efficiently to large scenes and extended sequences while maintaining high-fidelity structural and motion coherence. At the core of our approach is a scalable motion field parameterized by cluster-centric basis transformations that adaptively expand to capture diverse and evolving motion patterns. To ensure robust reconstruction over long durations, we introduce a progressive optimization strategy comprising two decoupled propagation stages: 1) A background extension stage that adapts to newly visible regions, refines camera poses, and explicitly models transient shadows; 2) A foreground propagation stage that enforces motion consistency through a specialized three-stage refinement process. Extensive experiments on challenging real-world benchmarks demonstrate that MotionScale significantly outperforms state-of-the-art methods in both reconstruction quality and temporal stability. Project page: https://hrzhou2.github.io/motion-scale-web/.
Haoran Zhou, Gim Hee Lee
Department of Computer Science, National University of Singapore
Reconstructing dynamic 4D scenes is an important yet challenging task. While 3D foundation models like VGGT excel in static settings, they often struggle with dynamic sequences where motion causes significant geometric ambiguity. To address this, we present a framework designed to disentangle dynamic and static components by modeling uncertainty across different stages of the reconstruction process. Our approach introduces three synergistic mechanisms: (1) Entropy-Guided Subspace Projection, which leverages information-theoretic weighting to adaptively aggregate multi-head attention distributions, effectively isolating dynamic motion cues from semantic noise; (2) Local-Consistency Driven Geometry Purification, which enforces spatial continuity via radius-based neighborhood constraints to eliminate structural outliers; and (3) Uncertainty-Aware Cross-View Consistency, which formulates multi-view projection refinement as a heteroscedastic maximum likelihood estimation problem, utilizing depth confidence as a probabilistic weight. Experiments on dynamic benchmarks show that our approach outperforms current state-of-the-art methods, reducing Mean Accuracy error by 13.43% and improving segmentation F-measure by 10.49%. Our framework maintains the efficiency of feed-forward inference and requires no task-specific fine-tuning or per-scene optimization.
Ying Zang, Yidong Han, Chaotao Ding +8
Huzhou University · KOKONI 3D, Moxin Technology · Univeristy of Science and Technology of China +3