Feed-forward 4D driving-scene reconstruction enables efficient modeling of dynamic autonomous-driving scenes, but current methods suffer from two complementary failure modes: semantic leakage of traffic participants into the static representation and geometric drift of Gaussian locations across viewpoints. We propose IGRecon4D, a feed-forward dynamic scene reconstruction framework that addresses these failures with training-time instance-prior distillation and LiDAR-anchored Gaussian positioning. Specifically, an instance-level semantic branch provides a potentially dynamic participant prior, which is distilled into dynamic-mask learning through the Instance-Dynamic Consistency Constraint to encourage high-recall localization of traffic-participant regions prone to dynamic-static leakage. In parallel, sparse LiDAR measurements guide dense depth estimation, and the resulting depths are back-projected to anchor Gaussian positions for improved multi-view geometric consistency. Experiments on the Waymo Open Dataset and zero-shot evaluation on nuScenes demonstrate improved full-image reconstruction on the main benchmark and stronger traffic-participant-region reconstruction under cross-dataset evaluation.
Figures & tables
Table 1
Figure 1: Overview of IGRecon4D. Given multi-camera videos, camera parameters, and sparse LiDAR depth maps, IGRecon4D predicts Gaussian attributes, LiDAR-anchored Gaussian positions, dynamic masks, and instance-level semantic masks. The instance-level semantic branch supplies a potentially dynamic participant prior for dynamic-mask supervision, while the dynamic masks decompose the scene into static and dynamic Gaussians for rendering.
Figure 2: Dynamic-static Gaussian rendering. Dense depth maps are back-projected with camera parameters to obtain Gaussian positions. The predicted dynamic masks allocate traffic-participant regions with high potentially dynamic prior to per-frame Gaussians and the remaining regions to the global static Gaussian set, while sky Gaussians model low-opacity sky regions. All Gaussian components are jointly rendered with a differentiable rasterizer.
Method
TPR PSNR ↑
TPR SSIM ↑
PSNR ↑
SSIM ↑
LPIPS ↓
D-RMSE ↓
Time (s) ↓
Memory (GB) ↓
AnySplat [ 15 ]
21.70
0.680
25.43
0.751
0.217
9.756
2.56
20.96
STORM [ 18 ]
19.25
0.504
25.37
0.776
0.315
18.849
1.74
85.35
DGGT [ 19 ]
24.82
0.751
26.93
0.868
0.116
7.470
10.88
47.36
NoPo4D [ 39 ]
21.70
0.640
25.27
0.776
0.200
8.872
3.61
24.00
DrivingForward [ 38 ]
19.55
0.751
17.61
0.802
0.222
12.007
2.73
2.78
Ours
26.76
0.858
29.51
0.869
0.102
4.610
16.20
69.27
Table 1: Quantitative comparison on the Waymo Open Dataset. Metrics are computed on the validation split under the same input-view and input-frame setting for all methods. Traffic-participant-region (TPR) metrics are computed on SAM3-generated traffic-participant masks, while full-image metrics are computed over all pixels.
Figure 3: Qualitative comparison on the Waymo Open Dataset. IGRecon4D better preserves traffic participants and static background details than feed-forward baselines, with fewer participant-region omissions and sharper object boundaries.
Figure 4: Novel-view synthesis from interpolated camera poses. We render from interpolated viewpoints between Camera 0 and Camera 1, and between Camera 0 and Camera 2, and compare IGRecon4D with DGGT [ 19 ] under identical camera poses. cam0_1 and cam0_2 denote the interpolated viewpoints between Camera 0 and Camera 1, and between Camera 0 and Camera 2, respectively. IGRecon4D produces fewer geometric shifts and stronger multi-view consistency.
Method
Left View FID ↓
Right View FID ↓
Ours
117.73
132.41
DGGT [ 19 ]
129.47
151.10
Table 2: Novel-view synthesis measured by FID. FID is used as a proxy metric between rendered interpolated views and the real-image distribution under the same evaluation protocol; lower scores indicate smaller distributional discrepancy.
Method
TPR PSNR ↑
TPR SSIM ↑
PSNR ↑
SSIM ↑
LPIPS ↓
D-RMSE ↓
Ours
26.76
0.858
29.51
0.869
0.102
4.610
w/o IDC
23.81
0.774
28.92
0.862
0.108
4.624
w/o LiDAR-guided depth
23.73
0.758
25.20
0.759
0.213
11.584
w/o IDC and w/o LiDAR-guided depth
21.02
0.626
24.77
0.749
0.222
11.606
Table 3: Ablation study on the Waymo Open Dataset. Traffic-participant-region (TPR) metrics are computed on SAM3-generated traffic-participant masks, while full-image metrics are computed over all pixels.
Figure 5: Qualitative ablation study. Replacing IDC-supervised dynamic-mask learning with a DGGT-initialized dynamic head reduces high-recall localization of traffic-participant regions, while replacing LiDAR-guided depth estimation with DPT-style depth prediction introduces geometric misalignment and degrades rendering quality.
Method
TPR PSNR ↑
TPR SSIM ↑
PSNR ↑
SSIM ↑
LPIPS ↓
AnySplat [ 15 ]
20.33
0.545
27.89
0.857
0.120
STORM [ 18 ]
15.57
0.255
16.90
0.472
0.533
DGGT [ 19 ]
23.43
0.722
20.73
0.807
0.173
NoPo4D [ 39 ]
20.22
0.598
23.10
0.655
0.282
DrivingForward [ 38 ]
18.88
0.764
18.89
0.775
0.191
Ours
25.05
0.809
25.93
0.752
0.172
Table 4: Zero-shot reconstruction comparison on nuScenes. Models are evaluated without fine-tuning on nuScenes; traffic-participant-region (TPR) metrics are computed on traffic-participant masks.
Figure 6: Qualitative zero-shot reconstruction on nuScenes. Without fine-tuning on nuScenes, IGRecon4D preserves traffic participants more reliably than feed-forward baselines.
Method
mAP ↑
mAP 50 ↑
mAP 25 ↑
PQ ↑
mIoU ↑
Ours
0.540
0.808
0.898
0.574
0.918
Mask2Former [ 58 ]
0.660
0.843
0.872
0.174
0.848
DVIS++ [ 59 ]
0.686
0.895
0.913
0.612
0.926
VidEoMT [ 60 ]
0.630
0.886
0.904
0.567
0.915
Table 5: Agreement with the Pseudo-label Teacher. All methods are evaluated against SAM3-generated pseudo instance annotations on Waymo; the branch is used to provide IDC supervision rather than to serve as a standalone segmentation model.
Figure 7: Instance-level semantic predictions. The instance-level semantic branch is evaluated as a source of a potentially dynamic participant prior for IDC rather than as a standalone segmentation model; the predicted masks provide usable instance-level supervision for dynamic-mask learning.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Method
TPR PSNR ↑
TPR SSIM ↑
PSNR ↑
SSIM ↑
LPIPS ↓
D-RMSE ↓
Time (s) ↓
Memory (GB) ↓
Ours
26.76
0.858
29.51
0.869
0.102
4.610
16.20
69.27
EVolSplat4D [ 61 ]
19.71
0.603
23.24
0.715
0.288
13.870
69.65
4.78
IGGT [ 20 ]
16.09
0.273
19.33
0.560
0.442
9.615
3.40
24.50
EPS3D [ 21 ]
18.46
0.533
23.60
0.725
0.213
9.812
3.48
41.81
Appendix
Table 6: Additional quantitative comparison on the Waymo Open Dataset. Metrics are computed on the validation split under the same input-view and input-frame setting for all methods. Traffic-participant-region (TPR) metrics are computed on SAM3-generated traffic-participant masks, while full-image metrics are computed over all pixels.
Figure 8: Qualitative comparison with additional methods on the Waymo Open Dataset. IGRecon4D better preserves traffic participants and static background details than feed-forward baselines, with fewer participant-region omissions and sharper object boundaries.
Method
TPR PSNR ↑
TPR SSIM ↑
PSNR ↑
SSIM ↑
LPIPS ↓
D-RMSE ↓
Ours
26.76
0.858
29.51
0.869
0.102
4.610
Directly using SAM3 [ 53 ]
26.52
0.846
29.14
0.860
0.106
4.487
RGB input only
22.06
0.778
29.22
0.858
0.103
6.896
w/o sky gaussians
25.72
0.848
9.31
0.603
0.302
4.599
Appendix
Table 7: Additional ablation study on the Waymo Open Dataset. Traffic-participant-region (TPR) metrics are computed on SAM3-generated traffic-participant masks, while full-image metrics are computed over all pixels.
Figure 9: Additional qualitative ablation results. The examples compare IGRecon4D with RGB-only input, removal of sky Gaussians, and direct use of SAM3 masks.
Metric
Ours
DGGT [ 19 ]
TPR PSNR ↑
20.57
18.95
TPR SSIM ↑
0.654
0.575
PSNR ↑
25.62
21.83
SSIM ↑
0.774
0.699
LPIPS ↓
0.163
0.212
D-RMSE ↓
5.058
6.718
Appendix
Table 8: Temporal interpolation comparison on the Waymo Open Dataset. Traffic-participant-region (TPR) metrics are computed on SAM3-generated traffic-participant masks, while full-image metrics are computed over all pixels.
Figure 10: Qualitative comparison of temporal interpolation results. Ground truth (GT), IGRecon4D, and DGGT are shown from left to right. Red boxes highlight regions where DGGT produces visible artifacts on traffic participants.
Method
Accuracy ↑
Recall ↑
Ours
90.26%
84.88%
DGGT [ 19 ]
50.47%
34.53%
Appendix
Table 9: Evaluation of predicted dynamic masks. Accuracy and recall are measured against SAM3-generated pseudo ground truth; higher is better for both metrics.
Figure 11: Qualitative evaluation of predicted dynamic masks. RGB images, SAM3-generated pseudo ground-truth masks, our predictions, and DGGT predictions are shown from left to right.
Figure 12: Failure case in a nighttime scene. Ground truth (GT) and our reconstruction are shown side by side. The red box highlights a visible reconstruction artifact.
Dynamic scene reconstruction in autonomous driving remains a fundamental challenge due to significant temporal variations, moving objects, and complex scene dynamics. Existing feed-forward 3D models have demonstrated strong performance in static reconstruction but still struggle to capture dynamic motion. To address these limitations, we propose DynamicVGGT, a unified feed-forward framework that extends VGGT from static 3D perception to dynamic 4D reconstruction. Our goal is to model point motion within feed-forward 3D models in a dynamic and temporally coherent manner. To this end, we jointly predict the current and future point maps within a shared reference coordinate system, allowing the model to implicitly learn dynamic point representations through temporal correspondence. To efficiently capture temporal dependencies, we introduce a Motion-aware Temporal Attention (MTA) module that learns motion continuity. Furthermore, we design a Dynamic 3D Gaussian Splatting Head that explicitly models point motion by predicting Gaussian velocities using learnable motion tokens under scene flow supervision. It refines dynamic geometry through continuous 3D Gaussian optimization. Extensive experiments on autonomous driving datasets demonstrate that DynamicVGGT significantly outperforms existing methods in reconstruction accuracy, achieving robust feed-forward 4D dynamic scene reconstruction under complex driving scenarios.
Zhuolin He, Jing Li, Guanghao Li +9
Fudan University · Yinwang Intelligent Technology · Huawei +2
High-fidelity reconstruction of driving scenes is crucial for autonomous driving. While recent feedforward 3D Gaussian Splatting (3DGS) methods enable fast reconstruction, their per-pixel Gaussian prediction paradigm often suffers from multi-view inconsistency and layering artifacts. Moreover, existing methods often model dynamic instances via dense flow prediction, which lacks explicit cross-view correspondence and instance-level consistency. In this paper, we propose PointForward, a feedforward driving reconstruction framework through point-aligned representations. Unlike pixel-aligned methods, we initialize sparse 3D queries in world space and aggregate multi-view image information via spatial-temporal fusion onto these queries, enforcing explicit cross-view consistency in a single feedforward pass. To handle scene dynamics, we introduce scene graphs that explicitly organize moving instances during reconstruction. By leveraging 3D bounding boxes, our method enables instance-level motion propagation and temporally consistent dynamic representations. Extensive experiments show that PointForward achieves state-of-the-art performance on large-scale driving benchmarks.
Cheng Chi, Xianqi Wang, Hongcheng Luo +9
Xiaomi EV · Huazhong University of Science and Technology · Zhejiang University
High-fidelity visual reconstruction and novel-view synthesis are essential for realistic closed-loop evaluation in autonomous driving. While 4D Gaussian Splatting (4DGS) offers a promising balance of accuracy and efficiency, existing per-scene optimization methods require costly iterative refinement, rendering them unscalable for extensive urban environments. Conversely, current feed-forward approaches often suffer from degraded photometric quality. To address these limitations, we propose ReconDrive, a feed-forward framework that leverages and extends the 3D foundation model VGGT for rapid, high-fidelity 4DGS generation. Our architecture introduces two core adaptations to tailor the foundation model to dynamic driving scenes: (1) Hybrid Gaussian Prediction Heads, which decouple the regression of spatial coordinates and appearance attributes to overcome the photometric deficiencies inherent in generalized foundation features; and (2) a Static-Dynamic 4D Composition strategy that explicitly captures temporal motion via velocity modeling to represent complex dynamic environments. Benchmarked on nuScenes, ReconDrive significantly outperforms existing feed-forward baselines in reconstruction, novel-view synthesis, and 3D perception. It achieves performance competitive with per-scene optimization while being orders of magnitude faster, providing a scalable and practical solution for realistic driving simulation.
Haibao Yu, Kuntao Xiao, Jiahang Wang +7
The University of Hong Kong · Tuojing Intelligence · King’s College London +2