Feed-forward 4D driving-scene reconstruction enables efficient modeling of dynamic autonomous-driving scenes, but current methods suffer from two complementary failure modes: semantic leakage of traffic participants into the static representation and geometric drift of Gaussian locations across viewpoints. We propose IGRecon4D, a feed-forward dynamic scene reconstruction framework that addresses these failures with training-time instance-prior distillation and LiDAR-anchored Gaussian positioning. Specifically, an instance-level semantic branch provides a potentially dynamic participant prior, which is distilled into dynamic-mask learning through the Instance-Dynamic Consistency Constraint to encourage high-recall localization of traffic-participant regions prone to dynamic-static leakage. In parallel, sparse LiDAR measurements guide dense depth estimation, and the resulting depths are back-projected to anchor Gaussian positions for improved multi-view geometric consistency. Experiments on the Waymo Open Dataset and zero-shot evaluation on nuScenes demonstrate improved full-image reconstruction on the main benchmark and stronger traffic-participant-region reconstruction under cross-dataset evaluation.
Figures & tables
Table 1
Figure 1: Overview of IGRecon4D. Given multi-camera videos, camera parameters, and sparse LiDAR depth maps, IGRecon4D predicts Gaussian attributes, LiDAR-anchored Gaussian positions, dynamic masks, and instance-level semantic masks. The instance-level semantic branch supplies a potentially dynamic participant prior for dynamic-mask supervision, while the dynamic masks decompose the scene into static and dynamic Gaussians for rendering.
Figure 2: Dynamic-static Gaussian rendering. Dense depth maps are back-projected with camera parameters to obtain Gaussian positions. The predicted dynamic masks allocate traffic-participant regions with high potentially dynamic prior to per-frame Gaussians and the remaining regions to the global static Gaussian set, while sky Gaussians model low-opacity sky regions. All Gaussian components are jointly rendered with a differentiable rasterizer.
Method
TPR PSNR ↑
TPR SSIM ↑
PSNR ↑
SSIM ↑
LPIPS ↓
D-RMSE ↓
Time (s) ↓
Memory (GB) ↓
AnySplat [ 15 ]
21.70
0.680
25.43
0.751
0.217
9.756
2.56
20.96
STORM [ 18 ]
19.25
0.504
25.37
0.776
0.315
18.849
1.74
85.35
DGGT [ 19 ]
24.82
0.751
26.93
0.868
0.116
7.470
10.88
47.36
NoPo4D [ 39 ]
21.70
0.640
25.27
0.776
0.200
8.872
3.61
24.00
DrivingForward [ 38 ]
19.55
0.751
17.61
0.802
0.222
12.007
2.73
2.78
Ours
26.76
0.858
29.51
0.869
0.102
4.610
16.20
69.27
Table 1: Quantitative comparison on the Waymo Open Dataset. Metrics are computed on the validation split under the same input-view and input-frame setting for all methods. Traffic-participant-region (TPR) metrics are computed on SAM3-generated traffic-participant masks, while full-image metrics are computed over all pixels.
Figure 3: Qualitative comparison on the Waymo Open Dataset. IGRecon4D better preserves traffic participants and static background details than feed-forward baselines, with fewer participant-region omissions and sharper object boundaries.
Figure 4: Novel-view synthesis from interpolated camera poses. We render from interpolated viewpoints between Camera 0 and Camera 1, and between Camera 0 and Camera 2, and compare IGRecon4D with DGGT [ 19 ] under identical camera poses. cam0_1 and cam0_2 denote the interpolated viewpoints between Camera 0 and Camera 1, and between Camera 0 and Camera 2, respectively. IGRecon4D produces fewer geometric shifts and stronger multi-view consistency.
Method
Left View FID ↓
Right View FID ↓
Ours
117.73
132.41
DGGT [ 19 ]
129.47
151.10
Table 2: Novel-view synthesis measured by FID. FID is used as a proxy metric between rendered interpolated views and the real-image distribution under the same evaluation protocol; lower scores indicate smaller distributional discrepancy.
Method
TPR PSNR ↑
TPR SSIM ↑
PSNR ↑
SSIM ↑
LPIPS ↓
D-RMSE ↓
Ours
26.76
0.858
29.51
0.869
0.102
4.610
w/o IDC
23.81
0.774
28.92
0.862
0.108
4.624
w/o LiDAR-guided depth
23.73
0.758
25.20
0.759
0.213
11.584
w/o IDC and w/o LiDAR-guided depth
21.02
0.626
24.77
0.749
0.222
11.606
Table 3: Ablation study on the Waymo Open Dataset. Traffic-participant-region (TPR) metrics are computed on SAM3-generated traffic-participant masks, while full-image metrics are computed over all pixels.
Figure 5: Qualitative ablation study. Replacing IDC-supervised dynamic-mask learning with a DGGT-initialized dynamic head reduces high-recall localization of traffic-participant regions, while replacing LiDAR-guided depth estimation with DPT-style depth prediction introduces geometric misalignment and degrades rendering quality.
Method
TPR PSNR ↑
TPR SSIM ↑
PSNR ↑
SSIM ↑
LPIPS ↓
AnySplat [ 15 ]
20.33
0.545
27.89
0.857
0.120
STORM [ 18 ]
15.57
0.255
16.90
0.472
0.533
DGGT [ 19 ]
23.43
0.722
20.73
0.807
0.173
NoPo4D [ 39 ]
20.22
0.598
23.10
0.655
0.282
DrivingForward [ 38 ]
18.88
0.764
18.89
0.775
0.191
Ours
25.05
0.809
25.93
0.752
0.172
Table 4: Zero-shot reconstruction comparison on nuScenes. Models are evaluated without fine-tuning on nuScenes; traffic-participant-region (TPR) metrics are computed on traffic-participant masks.
Figure 6: Qualitative zero-shot reconstruction on nuScenes. Without fine-tuning on nuScenes, IGRecon4D preserves traffic participants more reliably than feed-forward baselines.
Method
mAP ↑
mAP 50 ↑
mAP 25 ↑
PQ ↑
mIoU ↑
Ours
0.540
0.808
0.898
0.574
0.918
Mask2Former [ 58 ]
0.660
0.843
0.872
0.174
0.848
DVIS++ [ 59 ]
0.686
0.895
0.913
0.612
0.926
VidEoMT [ 60 ]
0.630
0.886
0.904
0.567
0.915
Table 5: Agreement with the Pseudo-label Teacher. All methods are evaluated against SAM3-generated pseudo instance annotations on Waymo; the branch is used to provide IDC supervision rather than to serve as a standalone segmentation model.
Figure 7: Instance-level semantic predictions. The instance-level semantic branch is evaluated as a source of a potentially dynamic participant prior for IDC rather than as a standalone segmentation model; the predicted masks provide usable instance-level supervision for dynamic-mask learning.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Method
TPR PSNR ↑
TPR SSIM ↑
PSNR ↑
SSIM ↑
LPIPS ↓
D-RMSE ↓
Time (s) ↓
Memory (GB) ↓
Ours
26.76
0.858
29.51
0.869
0.102
4.610
16.20
69.27
EVolSplat4D [ 61 ]
19.71
0.603
23.24
0.715
0.288
13.870
69.65
4.78
IGGT [ 20 ]
16.09
0.273
19.33
0.560
0.442
9.615
3.40
24.50
EPS3D [ 21 ]
18.46
0.533
23.60
0.725
0.213
9.812
3.48
41.81
Appendix
Table 6: Additional quantitative comparison on the Waymo Open Dataset. Metrics are computed on the validation split under the same input-view and input-frame setting for all methods. Traffic-participant-region (TPR) metrics are computed on SAM3-generated traffic-participant masks, while full-image metrics are computed over all pixels.
Figure 8: Qualitative comparison with additional methods on the Waymo Open Dataset. IGRecon4D better preserves traffic participants and static background details than feed-forward baselines, with fewer participant-region omissions and sharper object boundaries.
Method
TPR PSNR ↑
TPR SSIM ↑
PSNR ↑
SSIM ↑
LPIPS ↓
D-RMSE ↓
Ours
26.76
0.858
29.51
0.869
0.102
4.610
Directly using SAM3 [ 53 ]
26.52
0.846
29.14
0.860
0.106
4.487
RGB input only
22.06
0.778
29.22
0.858
0.103
6.896
w/o sky gaussians
25.72
0.848
9.31
0.603
0.302
4.599
Appendix
Table 7: Additional ablation study on the Waymo Open Dataset. Traffic-participant-region (TPR) metrics are computed on SAM3-generated traffic-participant masks, while full-image metrics are computed over all pixels.
Figure 9: Additional qualitative ablation results. The examples compare IGRecon4D with RGB-only input, removal of sky Gaussians, and direct use of SAM3 masks.
Metric
Ours
DGGT [ 19 ]
TPR PSNR ↑
20.57
18.95
TPR SSIM ↑
0.654
0.575
PSNR ↑
25.62
21.83
SSIM ↑
0.774
0.699
LPIPS ↓
0.163
0.212
D-RMSE ↓
5.058
6.718
Appendix
Table 8: Temporal interpolation comparison on the Waymo Open Dataset. Traffic-participant-region (TPR) metrics are computed on SAM3-generated traffic-participant masks, while full-image metrics are computed over all pixels.
Figure 10: Qualitative comparison of temporal interpolation results. Ground truth (GT), IGRecon4D, and DGGT are shown from left to right. Red boxes highlight regions where DGGT produces visible artifacts on traffic participants.
Method
Accuracy ↑
Recall ↑
Ours
90.26%
84.88%
DGGT [ 19 ]
50.47%
34.53%
Appendix
Table 9: Evaluation of predicted dynamic masks. Accuracy and recall are measured against SAM3-generated pseudo ground truth; higher is better for both metrics.
Figure 11: Qualitative evaluation of predicted dynamic masks. RGB images, SAM3-generated pseudo ground-truth masks, our predictions, and DGGT predictions are shown from left to right.
Figure 12: Failure case in a nighttime scene. Ground truth (GT) and our reconstruction are shown side by side. The red box highlights a visible reconstruction artifact.