While feed-forward models excel in static 3D geometry estimation, extending them to dynamic 4D scenes requires modeling visual geometry as a causally evolving state. Current methods typically rely on unordered temporal aggregation, which lacks the directional structure of physical time. Furthermore, existing supervision often treats geometry and motion as disjoint objectives, failing to exploit their inherent physical coupling. Consequently, they lack a unified constraint to effectively supervise temporal state transitions, leading to inconsistent 4D scene representations. To address this, we propose a feed-forward framework for causal 4D visual geometry learning from monocular videos. At its core, Causal Spatiotemporal Mamba augments temporal aggregation with directional state propagation. Through complementary Temporal-Spatial and Spatial-Temporal Scans, CSM continuously propagates geometry-aware states across frames to model spatiotemporal evolution. To align this causal state evolution with observable dynamics, we introduce Unified Motion Supervision. Rather than treating motion as an independent objective, UMS leverages geometry-aware predictions from the camera and depth heads to construct a unified forward motion field that is consistent with the underlying 3D structure. This geometry-grounded supervision explicitly constrains inter-frame state transitions, enabling stable states propagation across frames. Extensive evaluations on dynamic benchmarks demonstrate state-of-the-art performance across video depth, camera pose, and optical flow estimation. These results validate that the synergy between the directional state propagation of Causal Spatiotemporal Mamba and Unified Motion Supervision effectively captures continuous scene evolution, significantly reducing temporal drift and reinforcing spatial-temporal consistency in 4D representation learning.
Figures & tables
Figure 1: Overview of our causal spatiotemporal state evolution framework. Given an ordered monocular video, the model first extracts geometry-aware tokens with global context, and then applies Causal Spatiotemporal Mamba (CSM) to propagate these tokens through causal spatiotemporal state evolution. Complementary Temporal-Spatial Scan (TS Scan) and Spatial-Temporal Scan (ST Scan) update geometry-aware states across temporal and spatial dimensions while preserving the temporal order of scene evolution. The updated spatiotemporal representations parameterize camera extrinsics, intrinsics, pixel-aligned 3D point maps, and forward optical flow. Unified Motion Supervision (UMS) constructs a unified forward motion field with geometry-grounded supervision mechanism, enforcing geometry-aligned inter-frame state transitions.
Figure 2: Comparison of 4D modeling paradigms. Attention-based methods [ 25 , 37 ] aggregate context through pairwise token affinities, while Image/Video Mamba [ 15 , 14 ] applies generic bidirectional scans for visual representation learning. In contrast, CSM introduces geometry-aware causal scans that propagate states along the temporal order of the video. The complementary Temporal-Spatial Scan and Spatial-Temporal Scan jointly model temporal evolution and spatial structure, enabling ordered spatiotemporal state propagation for monocular 4D visual geometry.
Method
Sintel
Bonn
ATE ↓
RPE-T ↓
RPE-R ↓
ATE ↓
RPE-T ↓
RPE-R ↓
VGGT
0.214
0.079
0.643
0.021
0.023
0.937
StreamVGGT
0.216
0.106
0.779
0.045
0.010
0.662
MoRe
0.147
0.082
0.616
0.021
0.012
0.650
PAGE-4D
0.175
0.074
0.602
0.021
0.013
0.650
CUT3R
0.235
0.068
0.669
0.035
0.009
0.680
Table 2: Camera pose estimation on dynamic benchmarks. Best results are in bold , second-best are underlined .
Method
Sintel
Bonn
KITTI
Abs Rel ↓
δ<1.25↑
Abs Rel ↓
δ<1.25↑
Abs Rel ↓
δ<1.25↑
VGGT
0.471
0.486
0.103
0.903
0.143
0.804
StreamVGGT
0.354
0.591
OOM
OOM
0.176
0.718
MoRE
0.340
0.592
0.057
0.967
0.141
0.878
PAGE-4D
0.299
0.735
0.087
0.907
0.147
0.894
CUT3R
0.534
0.558
0.075
0.943
0.118
0.881
Table 3: Video depth estimation on dynamic benchmarks. Metric δ<1.25 is reported in %. Best results are in bold , and second-best results are underlined . “OOM” denotes out-of-memory under the same single-pass feed-forward inference setting.
Figure 3: Qualitative comparison of dynamic geometry reconstruction. Our method produces more accurate and temporally coherent geometry in dynamic regions, with sharper structures and fewer artifacts than existing feed-forward 4D methods.
Figure 4: Qualitative comparison of optical flow estimation. Benefiting from our unified motion supervision, our model yields highly accurate motion fields with sharp boundaries. Notably, in the middle columns where moving objects occupy a small proportion and share high visual similarity with the background, Flow3r struggles to effectively detect the foreground optical flow. In contrast, our approach successfully disentangles these challenging motions.
Method
Sintel EPE ↓
KITTI EPE ↓
Flow3r
45.146
38.649
Ours
16.108
20.161
Table 4: Comparison results of optical flow estimation.
Components
Video Depth
Pose
Modeling
UMS
Abs Rel ↓
δ<1.25↑
ATE ↓
RPE-T ↓
RPE-R ↓
Only Global-Attn
✘
0.481
0.500
0.321
0.213
0.814
Causal-Attn
✘
0.451
0.592
0.311
0.191
0.637
Vision Mamba
✘
0.309
0.564
0.329
0.177
0.732
CSM
✘
0.253
0.631
0.124
0.078
0.559
CSM
✔
0.243
0.689
0.116
0.050
0.552
Table 5: Ablation study on architectural components and training strategies. Evaluation is performed on Sintel. Best results are shown in bold.
Reconstructing dynamic 4D scenes from monocular videos is a fundamental yet challenging task. While recent 3D foundation models provide strong geometric priors, their performance significantly degrades in dynamic environments. This degradation stems from a fundamental tension: the inherent coupling of camera ego-motion and object motion within global attention mechanisms. In this paper, we propose a novel, training-free progressive decoupling framework that disentangles dynamics from statics in a principled, coarse-to-fine manner. Our core insight is to resolve the tension by first stabilizing the camera pose, followed by geometric refinement. Specifically, our approach consists of three synergistic components: (1) a Dynamic-Mask-Guided Pose Decoupling module that isolates pose estimation from dynamic interference, yielding a stable motion-free reference frame; (2) a Topological Subspace Surgery mechanism that orthogonally decomposes the depth manifold, safely preserving dynamic objects while injecting refined, mask-aware geometry into static regions; and (3) an Information-Theoretic Confidence-Aware Fusion strategy that formulates depth integration as a heteroscedastic Bayesian inference problem, adaptively blending multi-pass predictions via inverse-variance weighting. Extensive experiments on standard 4D reconstruction benchmarks demonstrate that our method achieves consistent and substantial improvements across principal point-cloud metrics. Notably, our approach shows competitive performance in robust 4D scene reconstruction without requiring fine-tuning, suggesting the potential of mathematically grounded dynamic-static disentanglement.
Ying Zang, Xuanyi Liu, Yidong Han +9
4Huzhou University · 2Peking University · 1KOKONI 3D, Moxin Technology +4
Reconstructing dynamic 4D scenes remains challenging due to the presence of moving objects that corrupt camera pose estimation. Existing optimization methods alleviate this issue with additional supervision, but they are mostly computationally expensive and impractical in real-time applications. To address these limitations, we propose MoRe, a feedforward 4D reconstruction network that efficiently recovers dynamic 3D scenes from monocular videos. Built upon a strong static reconstruction backbone, MoRe employs an attention-forcing strategy to disentangle dynamic motion from static structure. To further enhance robustness, we fine-tune the model on large-scale, diverse datasets encompassing both dynamic and static scenes. Moreover, our grouped causal attention captures temporal dependencies and adapts to varying token lengths across frames, ensuring temporally coherent geometry reconstruction. Extensive experiments on multiple benchmarks demonstrate that MoRe achieves high-quality dynamic reconstructions with exceptional efficiency.
Juntong Fang, Zequn Chen, Weiqi Zhang +4
School of Software, Tsinghua University, Beijing, China · Li Auto
Learning a 4D scene representation from a single monocular video that supports dynamic novel-view synthesis while maintaining faithful geometry over time remains challenging. Dynamic Gaussian Splatting achieves strong rendering performance through photometric optimization, yet does not explicitly enforce multi-view geometric consistency. In contrast, 3D foundation models recover coherent scene geometry and camera motion, but their point-based outputs are not designed for photorealistic rendering. We propose Ground4D, a geometry-grounded framework built on two stages. First, we perform geometry initialization via 3D foundation models, leveraging VGGT in a training-free manner to reconstruct multi-view-consistent 3D geometry and camera poses from monocular video. The recovered geometry provides a structured and reliable initialization for dynamic Gaussian representations. Second, we conduct geometry-consistency-aware refinement via dynamic Gaussian Splatting, optimizing the representation through differentiable rendering while maintaining multi-view geometric consistency across both observed and synthesized viewpoints. Furthermore, Ground4D inherently models the continuous 4D dynamics of the scene, naturally supporting rendering at arbitrary timestamps. By integrating foundation-level geometric priors into dynamic Gaussian optimization, Ground4D achieves stronger reconstruction fidelity and rendering performance, underscoring the role of geometry-grounded constraints in robust 4D scene modeling.
Qing Zhao, Weijian Deng, Pengxu Wei +1
School of Computer Science and Engineering, Sun Yat-sen University, China · Tsinghua Shenzhen International Graduate School, Tsinghua University, China