While feed-forward models excel in static 3D geometry estimation, extending them to dynamic 4D scenes requires modeling visual geometry as a causally evolving state. Current methods typically rely on unordered temporal aggregation, which lacks the directional structure of physical time. Furthermore, existing supervision often treats geometry and motion as disjoint objectives, failing to exploit their inherent physical coupling. Consequently, they lack a unified constraint to effectively supervise temporal state transitions, leading to inconsistent 4D scene representations. To address this, we propose a feed-forward framework for causal 4D visual geometry learning from monocular videos. At its core, Causal Spatiotemporal Mamba augments temporal aggregation with directional state propagation. Through complementary Temporal-Spatial and Spatial-Temporal Scans, CSM continuously propagates geometry-aware states across frames to model spatiotemporal evolution. To align this causal state evolution with observable dynamics, we introduce Unified Motion Supervision. Rather than treating motion as an independent objective, UMS leverages geometry-aware predictions from the camera and depth heads to construct a unified forward motion field that is consistent with the underlying 3D structure. This geometry-grounded supervision explicitly constrains inter-frame state transitions, enabling stable states propagation across frames. Extensive evaluations on dynamic benchmarks demonstrate state-of-the-art performance across video depth, camera pose, and optical flow estimation. These results validate that the synergy between the directional state propagation of Causal Spatiotemporal Mamba and Unified Motion Supervision effectively captures continuous scene evolution, significantly reducing temporal drift and reinforcing spatial-temporal consistency in 4D representation learning.
Figures & tables
Figure 1: Overview of our causal spatiotemporal state evolution framework. Given an ordered monocular video, the model first extracts geometry-aware tokens with global context, and then applies Causal Spatiotemporal Mamba (CSM) to propagate these tokens through causal spatiotemporal state evolution. Complementary Temporal-Spatial Scan (TS Scan) and Spatial-Temporal Scan (ST Scan) update geometry-aware states across temporal and spatial dimensions while preserving the temporal order of scene evolution. The updated spatiotemporal representations parameterize camera extrinsics, intrinsics, pixel-aligned 3D point maps, and forward optical flow. Unified Motion Supervision (UMS) constructs a unified forward motion field with geometry-grounded supervision mechanism, enforcing geometry-aligned inter-frame state transitions.
Figure 2: Comparison of 4D modeling paradigms. Attention-based methods [ 25 , 37 ] aggregate context through pairwise token affinities, while Image/Video Mamba [ 15 , 14 ] applies generic bidirectional scans for visual representation learning. In contrast, CSM introduces geometry-aware causal scans that propagate states along the temporal order of the video. The complementary Temporal-Spatial Scan and Spatial-Temporal Scan jointly model temporal evolution and spatial structure, enabling ordered spatiotemporal state propagation for monocular 4D visual geometry.
Method
Sintel
Bonn
ATE ↓
RPE-T ↓
RPE-R ↓
ATE ↓
RPE-T ↓
RPE-R ↓
VGGT
0.214
0.079
0.643
0.021
0.023
0.937
StreamVGGT
0.216
0.106
0.779
0.045
0.010
0.662
MoRe
0.147
0.082
0.616
0.021
0.012
0.650
PAGE-4D
0.175
0.074
0.602
0.021
0.013
0.650
CUT3R
0.235
0.068
0.669
0.035
0.009
0.680
Table 2: Camera pose estimation on dynamic benchmarks. Best results are in bold , second-best are underlined .
Method
Sintel
Bonn
KITTI
Abs Rel ↓
δ<1.25↑
Abs Rel ↓
δ<1.25↑
Abs Rel ↓
δ<1.25↑
VGGT
0.471
0.486
0.103
0.903
0.143
0.804
StreamVGGT
0.354
0.591
OOM
OOM
0.176
0.718
MoRE
0.340
0.592
0.057
0.967
0.141
0.878
PAGE-4D
0.299
0.735
0.087
0.907
0.147
0.894
CUT3R
0.534
0.558
0.075
0.943
0.118
0.881
Table 3: Video depth estimation on dynamic benchmarks. Metric δ<1.25 is reported in %. Best results are in bold , and second-best results are underlined . “OOM” denotes out-of-memory under the same single-pass feed-forward inference setting.
Figure 3: Qualitative comparison of dynamic geometry reconstruction. Our method produces more accurate and temporally coherent geometry in dynamic regions, with sharper structures and fewer artifacts than existing feed-forward 4D methods.
Figure 4: Qualitative comparison of optical flow estimation. Benefiting from our unified motion supervision, our model yields highly accurate motion fields with sharp boundaries. Notably, in the middle columns where moving objects occupy a small proportion and share high visual similarity with the background, Flow3r struggles to effectively detect the foreground optical flow. In contrast, our approach successfully disentangles these challenging motions.
Method
Sintel EPE ↓
KITTI EPE ↓
Flow3r
45.146
38.649
Ours
16.108
20.161
Table 4: Comparison results of optical flow estimation.
Components
Video Depth
Pose
Modeling
UMS
Abs Rel ↓
δ<1.25↑
ATE ↓
RPE-T ↓
RPE-R ↓
Only Global-Attn
✘
0.481
0.500
0.321
0.213
0.814
Causal-Attn
✘
0.451
0.592
0.311
0.191
0.637
Vision Mamba
✘
0.309
0.564
0.329
0.177
0.732
CSM
✘
0.253
0.631
0.124
0.078
0.559
CSM
✔
0.243
0.689
0.116
0.050
0.552
Table 5: Ablation study on architectural components and training strategies. Evaluation is performed on Sintel. Best results are shown in bold.
School of Computer Science and Engineering, Sun Yat-sen University, China · Tsinghua Shenzhen International Graduate School, Tsinghua University, China