Dynamic scene reconstruction in autonomous driving remains a fundamental challenge due to significant temporal variations, moving objects, and complex scene dynamics. Existing feed-forward 3D models have demonstrated strong performance in static reconstruction but still struggle to capture dynamic motion. To address these limitations, we propose DynamicVGGT, a unified feed-forward framework that extends VGGT from static 3D perception to dynamic 4D reconstruction. Our goal is to model point motion within feed-forward 3D models in a dynamic and temporally coherent manner. To this end, we jointly predict the current and future point maps within a shared reference coordinate system, allowing the model to implicitly learn dynamic point representations through temporal correspondence. To efficiently capture temporal dependencies, we introduce a Motion-aware Temporal Attention (MTA) module that learns motion continuity. Furthermore, we design a Dynamic 3D Gaussian Splatting Head that explicitly models point motion by predicting Gaussian velocities using learnable motion tokens under scene flow supervision. It refines dynamic geometry through continuous 3D Gaussian optimization. Extensive experiments on autonomous driving datasets demonstrate that DynamicVGGT significantly outperforms existing methods in reconstruction accuracy, achieving robust feed-forward 4D dynamic scene reconstruction under complex driving scenarios.
Figures & tables
Figure 1: DynamicVGGT extends static multi-view 3D perception to dynamic 4D reconstruction by enabling 3D Gaussian rendering and adaptively modeling motion across multiple temporal scales without explicit camera extrinsic alignment.
Figure 2: Proposed DynamicVGGT training framework. Given a sequence of multi-view images {V1,V2,V3} , the model first encodes them using a pretrained DINOv2 backbone to extract patch tokens and camera tokens for each view, while motion tokens are initialized as learnable parameters that encode temporal priors. The patch and camera tokens are processed by the Alternating-Attention (AA) blocks to model intra-frame spatial geometry, whereas the Motion-aware Temporal Attention (MTA) blocks operate in parallel to model inter-frame temporal dependencies using the motion tokens. The resulting temporal features TA are then fed into a Dynamic 3D Gaussian Head (DGSHead) for dynamic 3DGS reconstruction and a Future Point Head for future point prediction.
Figure 3: Dynamic task formulation. We formulate dynamic point maps by designing two complementary tasks that model point-wise motion over time. The Future Point Head learns implicit motion through inter-frame point consistency, while the Dynamic 3D Gaussian Splatting Head provides explicit motion supervision via scene flow to refine dynamic geometry.
Figure 4: Depth and Point Maps Comparison . The sparsity of LiDAR point clouds degrades the results, leading to less smooth depth maps and rougher point clouds.
Methods
KITTI (monocular)
Waymo (3 cameras)
Mean
Med.
Mean
Med.
Acc.↓
Comp.↓
NC↑
Acc.↓
Comp.↓
NC↑
Acc.↓
Comp.↓
NC↑
Acc.↓
Comp.↓
NC↑
VGGT [ 26 ]
1.489
0.690
0.918
1.329
0.535
0.971
4.635
2.667
0.561
2.634
1.734
0.590
StreamVGGT [ 38 ]
1.078
0.495
0.899
0.867
0.390
0.949
4.598
2.626
0.564
2.567
1.789
0.592
DynamicVGGT
0.901
0.584
0.939
0.733
0.464
0.963
4.021
2.390
0.562
1.971
1.564
0.603
Table 1: Point Map Reconstruction on KITTI and Waymo(val). KITTI uses monocular input with every 3 consecutive frames per camera. Waymo uses 3 frames (stride 4) from FRONT, SIDE_LEFT, and SIDE_RIGHT cameras, totaling 9 images per group.
Methods
Supervision
Dynamic-only
Full image
PSNR↑
SSIM↑
PSNR↑
SSIM↑
Per-scene optimization
3DGS [ 10 ]
Full
17.13
0.267
25.13
0.741
DeformableGS [ 34 ]
Full
17.10
0.266
25.29
0.761
Feed-forward model
GS-LRM [ 36 ]
Camera
20.02
0.520
25.18
0.753
Table 2: Comparison to state-of-the-art methods on Waymo (val). PSNR and SSIM are reported. Full: requires dense scene annotations. Camera: requires camera intrinsics and extrinsics.
Methods
KITTI(Mono)
NYU-v2(Mono)
KITTI(MVS)
Abs Rel↓
δ<1.25↑
Abs Rel↓
δ<1.25↑
Abs Rel↓
δ<1.25↑
DUSt3R [ 28 ]
0.109
0.873
0.081
0.909
0.143
0.814
MASt3R [ 11 ]
0.077
0.948
0.110
0.865
0.115
0.848
MonST3R [ 35 ]
0.098
0.895
0.094
0.887
0.107
0.884
VGGT [ 26 ]
0.082
0.938
0.059
0.951
0.062
0.969
StreamVGGT [ 38 ]
0.082
0.947
0.057
0.959
0.173
0.721
Table 3: Monocular and MVS depth estimation.
Methods
KITTI(Mono)
Waymo(3cam)
Mean
Med.
Mean
Med.
Acc.↓
Comp.↓
NC↑
Acc.↓
Comp.↓
NC↑
Acc.↓
Comp.↓
NC↑
Acc.↓
Comp.↓
NC↑
Baseline
1.489
0.690
0.918
1.329
0.535
0.971
4.635
2.667
0.561
2.634
1.734
0.590
+ TA & FPH(stage1)
0.927
0.600
0.915
0.857
0.474
0.932
4.330
2.939
0.561
2.224
1.649
0.593
+ DGSHead(stage2)
0.901
0.584
0.939
0.733
0.464
0.963
4.021
2.390
0.562
1.971
1.564
0.603
Table 4: Ablation study. We evaluate ablated variants of DynamicVGGT on point map estimation over KITTI and Waymo (val). KITTI uses monocular input with three consecutive frames, while Waymo uses 3 frames (stride 4) from the FRONT, SIDE LEFT, and SIDE RIGHT cameras, yielding 9 images per sample. Metrics include Accuracy (Acc.), Completeness (Comp.), Normal Consistency (NC).
Figure 5: Point map reconstruction. DynamicVGGT reconstructs denser, smoother, and more geometrically consistent point maps than VGGT, maintaining temporal coherence even under large viewpoint or scene changes. Zoom in for better view.
Figure 6: Scene Reconstruction and Novel View Synthesis. Given input frames 0, 2, and 4, DynamicVGGT reconstructs the corresponding scenes and synthesizes novel views for the next frame. For both KITTI and Waymo, multi-view inputs are used; for Waymo, we visualize the front-camera results due to limited view overlap. The model achieves high-quality reconstruction and realistic novel view generation across dynamic driving scenes.
High-fidelity reconstruction of driving scenes is crucial for autonomous driving. While recent feedforward 3D Gaussian Splatting (3DGS) methods enable fast reconstruction, their per-pixel Gaussian prediction paradigm often suffers from multi-view inconsistency and layering artifacts. Moreover, existing methods often model dynamic instances via dense flow prediction, which lacks explicit cross-view correspondence and instance-level consistency. In this paper, we propose PointForward, a feedforward driving reconstruction framework through point-aligned representations. Unlike pixel-aligned methods, we initialize sparse 3D queries in world space and aggregate multi-view image information via spatial-temporal fusion onto these queries, enforcing explicit cross-view consistency in a single feedforward pass. To handle scene dynamics, we introduce scene graphs that explicitly organize moving instances during reconstruction. By leveraging 3D bounding boxes, our method enables instance-level motion propagation and temporally consistent dynamic representations. Extensive experiments demonstrate that PointForward achieves state-of-the-art performance on large-scale driving benchmarks. The code will be available upon the publication of the paper.
Cheng Chi, Xianqi Wang, Hongcheng Luo +9
Xiaomi EV · Huazhong University of Science and Technology · Zhejiang University
High-fidelity visual reconstruction and novel-view synthesis are essential for realistic closed-loop evaluation in autonomous driving. While 4D Gaussian Splatting (4DGS) offers a promising balance of accuracy and efficiency, existing per-scene optimization methods require costly iterative refinement, rendering them unscalable for extensive urban environments. Conversely, current feed-forward approaches often suffer from degraded photometric quality. To address these limitations, we propose ReconDrive, a feed-forward framework that leverages and extends the 3D foundation model VGGT for rapid, high-fidelity 4DGS generation. Our architecture introduces two core adaptations to tailor the foundation model to dynamic driving scenes: (1) Hybrid Gaussian Prediction Heads, which decouple the regression of spatial coordinates and appearance attributes to overcome the photometric deficiencies inherent in generalized foundation features; and (2) a Static-Dynamic 4D Composition strategy that explicitly captures temporal motion via velocity modeling to represent complex dynamic environments. Benchmarked on nuScenes, ReconDrive significantly outperforms existing feed-forward baselines in reconstruction, novel-view synthesis, and 3D perception. It achieves performance competitive with per-scene optimization while being orders of magnitude faster, providing a scalable and practical solution for realistic driving simulation.
Haibao Yu, Kuntao Xiao, Jiahang Wang +7
The University of Hong Kong · Tuojing Intelligence · King’s College London +2
Reconstructing dynamic 4D scenes remains challenging due to the presence of moving objects that corrupt camera pose estimation. Existing optimization methods alleviate this issue with additional supervision, but they are mostly computationally expensive and impractical in real-time applications. To address these limitations, we propose MoRe, a feedforward 4D reconstruction network that efficiently recovers dynamic 3D scenes from monocular videos. Built upon a strong static reconstruction backbone, MoRe employs an attention-forcing strategy to disentangle dynamic motion from static structure. To further enhance robustness, we fine-tune the model on large-scale, diverse datasets encompassing both dynamic and static scenes. Moreover, our grouped causal attention captures temporal dependencies and adapts to varying token lengths across frames, ensuring temporally coherent geometry reconstruction. Extensive experiments on multiple benchmarks demonstrate that MoRe achieves high-quality dynamic reconstructions with exceptional efficiency.
Juntong Fang, Zequn Chen, Weiqi Zhang +4
School of Software, Tsinghua University, Beijing, China · Li Auto