Organizations: CAS Engineering Laboratory for Intelligent Industrial Vision, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · Department of Automation, Tsinghua University, Beijing 100084, China · Giga AI, Beijing, China
Dynamic driving scene reconstruction has made remarkable strides in recent years by leveraging generative priors, particularly in the quality of novel trajectory rendering. Nevertheless, a significant gap remains between the generated data and real-world sensor observations, severely hindering further improvements in reconstruction fidelity. In this paper, we present ReconDreamer#, a self-evolving framework that synergizes reconstruction and generation for dynamic driving scenes, which substantially bridges the gap between synthesized novel trajectory data and recorded real-world observations. Within this self-evolving closed-loop framework, the generative and reconstruction models mutually reinforce each other through iterative refinement, ultimately yielding a significant enhancement in the 3D representation capability of the reconstruction model. Additionally, we introduce a geometry-prior-based adaptive ground model to further enhance the rendering quality of the reconstruction model. To validate the effectiveness of the proposed method, we conduct comprehensive evaluations on Waymo, nuScenes, PandaSet, and EUVS benchmarks. Specifically on Waymo, at a 6-m lateral shift, ReconDreamer# improves NTA-IoU and NTL-IoU by 8.8% and 3.0%, respectively, and reduces FID by 15.1% relative to ReconDreamer++.
Figures & tables
Fig. 1: Overview of the self-evolving reconstruction–restoration loop. Starting from recorded video and an under-trained Gaussian scene, DriveRestorer improves novel-trajectory renderings used to train a new reconstruction. Rendering that reconstruction back at recorded poses yields new degraded–real pairs for the next restorer generation. The top rows compare renderings at 3-m and 6-m lane shifts across iterations.
Base components
Ground support
New mechanisms
New analyses
Method
Backbone / restoration
Ground decomposition
NTDNet
Policy
Round-trip self-evolution
Surface initialization
Soft planes / refinement
Generation / geometry
ReconDreamer [ 10 ]
✓
–
–
Backbone- dependent
–
–
–
–
ReconDreamer++ [ 16 ]
✓
✓
✓
Fixed
–
–
–
–
ReconDreamer#
✓
✓
✓
Adaptive
✓
✓
✓
✓
TABLE I: Comparison of the conference versions and the journal extension.
Fig. 2: Trajectory round-trip construction for Self-Evolving DriveRestorer. Novel-trajectory rendering and restoration feed an auxiliary reconstruction, which returns to recorded poses to construct inputs paired with real video targets.
Recorded trajectory
Lateral shift: 3 m
Lateral shift: 6 m
Method
PSNR ↑
SSIM ↑
LPIPS ↓
NTA ↑
NTL ↑
FID ↓
NTA ↑
NTL ↑
FID ↓
Street Gaussians [ 18 ]
36.50
0.9570
0.1149
0.4980
53.19
130.75
0.3740
49.27
213.04
FreeVS [ 92 ]
26.59
0.8057
0.1325
0.5050
56.84
104.23
0.4650
55.37
121.44
DriveDreamer4D [ 11 ]
34.37
0.9447
0.1321
0.4570
53.30
113.45
0.1590
50.09
261.81
ReconDreamer [ 10 ]
34.31
0.9431
0.1519
0.5390
54.58
93.56
0.4670
52.58
149.19
TABLE II: Quantitative comparison on Waymo. NTA and NTL denote NTA-IoU and NTL-IoU, respectively.
Fig. 3: Qualitative comparison on Waymo. Columns show FreeVS, Street Gaussians, ReconDreamer, ReconDreamer++, and ReconDreamer# ; rows show the recorded-view reference and renderings at 0-m, 3-m, and 6-m lateral shifts. Orange boxes enlarge vehicle and roadside details. Ground truth is shown only for the recorded view.
Recorded trajectory
Lateral shift: 3 m
Lateral shift: 6 m
Method
PSNR ↑
SSIM ↑
LPIPS ↓
NTA ↑
NTL ↑
FID ↓
NTA ↑
NTL ↑
FID ↓
Street Gaussians [ 18 ]
34.91
0.9528
0.1185
0.2190
48.72
149.50
0.1320
46.57
247.73
ReconDreamer [ 10 ]
33.85
0.9497
0.1235
0.3250
49.72
125.43
0.2460
47.08
197.52
ReconDreamer++ [ 16 ]
34.69
0.9533
0.1161
0.3650
51.33
118.27
0.2680
49.26
187.51
ReconDreamer#
35.62
0.9592
0.1111
0.3759
51.47
105.41
0.2772
49.53
133.99
TABLE III: Quantitative comparison on nuScenes.
Fig. 4: Qualitative comparison on nuScenes. Columns show Street Gaussians, ReconDreamer, ReconDreamer++, and ReconDreamer# ; rows show the recorded-view reference and renderings at 0-m, 3-m, and 6-m lateral shifts. Orange boxes enlarge vehicles and nearby scene details. Ground truth is shown only for the recorded view.
Method
No shift
Lane 2 m
Lane 3 m
Vert. 1 m
UniSim [ 42 ]
-
74.70
97.50
-
NeuRAD [ 44 ]
25.00
72.30
93.90
76.30
Street Gaussians [ 18 ]
14.60
66.30
80.70
78.40
ReconDreamer [ 10 ]
16.10
65.40
74.90
67.70
ReconDreamer++ [ 16 ]
13.80
61.90
71.70
62.40
TABLE IV: FID comparison on PandaSet under recorded, lateral-shift, and vertical-shift trajectories.
Fig. 5: Qualitative comparison on PandaSet under recorded, lateral-shift, and vertical-shift trajectories. Columns compare Street Gaussians, ReconDreamer, ReconDreamer++, and ReconDreamer# . Orange boxes highlight fine structures in the distant scene.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
Street Gaussians [ 18 ]
15.08
0.6043
0.3428
ReconDreamer [ 10 ]
20.14
0.6914
0.3286
ReconDreamer++ [ 16 ]
20.52
0.7033
0.3057
ReconDreamer#
21.59
0.7240
0.2995
TABLE V: Static-region comparison on EUVS Level 1.
Fig. 6: Qualitative comparison on EUVS Level 1. The upper row shows a training-trajectory view (Train Set), while the lower row shows an independently captured cross-lane test view (Test Set). Columns compare ground truth, Street Gaussians, ReconDreamer, ReconDreamer++, and ReconDreamer# . Dynamic foreground is masked in the rendered comparisons, and orange boxes enlarge facade signs.
SE
AGM
Recorded
3 m shift
6 m shift
Method
PSNR ↑
SSIM ↑
LPIPS ↓
NTA ↑
NTL ↑
FID ↓
NTA ↑
NTL ↑
FID ↓
ReconDreamer++
36.29
0.9573
0.1076
0.5720
57.06
72.02
0.4890
56.57
111.92
+ AGM
✓
36.72
0.9608
0.1015
0.5780
58.00
71.72
0.5070
57.40
108.10
+ Self-Evolution
✓
36.43
0.9568
0.1072
0.5830
57.75
70.45
0.5120
55.95
100.11
ReconDreamer#
✓
✓
36.89
0.9638
0.0953
0.5984
58.75
66.42
0.5320
58.29
94.99
TABLE VI: New-module ablation on Waymo. SE denotes Self-Evolving DriveRestorer. All variants retain the inherited depth and NTDNet components.
Fig. 7: Qualitative module ablation on Waymo. Columns compare ReconDreamer++, +AGM, +Self-Evolution, and the complete ReconDreamer# model. Rows show the reference view and renderings at 0 m, 3 m, and 6 m lateral shifts. Orange boxes highlight vehicle details and road-boundary artifacts.
Recorded
3 m shift
6 m shift
DriveRestorer
LPIPS ↓
NTA ↑
NTL ↑
FID ↓
NTA ↑
NTL ↑
FID ↓
R1
0.1015
0.5780
58.00
71.72
0.5070
57.40
108.10
R2
0.0985
0.5880
58.34
68.96
0.5180
57.84
101.94
R3
0.0974
0.5910
58.49
68.17
0.5240
57.99
99.45
R4
0.0953
0.5984
58.75
66.42
0.5320
58.29
94.99
TABLE VII: Multi-Round DriveRestorer comparison on Waymo. R1 is the initial static restorer; R2 – R4 are successive self-evolution generations.
Fig. 8: Ground-support comparison on Waymo scene 065. (a) The original ReconDreamer++ ground uses fixed support. (b) Complete AGM produces denser and more uniform support. Both top views share the same bounds, point size, and height colors; the cyan line marks the cross-section. (c) Signed residual ri=ni⊤(μi−pi) to the local LiDAR plane. The mean absolute residual e⊥ and P95 show that AGM expands road support while maintaining surface alignment.
Urban scene reconstruction from real-world observations has emerged as a powerful tool for self-driving development and testing. While current neural rendering approaches achieve high-fidelity rendering along the recorded trajectories, their quality degrades significantly under large viewpoint shifts, limiting the applicability for closed-loop simulation. Recent works have shown promising results in using diffusion models to enhance quality at these challenging viewpoints and distill improvements back into 3D representations. However, they often require costly per-scene optimization, and the distilled representations remain fragile and fail to generalize beyond limited synthesized views. To address these limitations, we propose GenRe, a novel diffusion-guided generalizable enhancer for urban scene reconstruction. GenRe takes as input any pretrained 3D Gaussian representation and fixes the deficiencies within a few minutes. By learning to distill generative priors across diverse scenes, GenRe produces robust and high-fidelity representation efficiently that generalizes reliably to challenging unseen viewpoints (e.g., lane change). Experiments show that GenRe outperforms existing methods in both quality and efficiency and benefits various downstream tasks, enabling robust and scalable sensor simulation for autonomous driving.
Henry Che, Jingkang Wang, Yun Chen +3
1Waabi · University of Illinois Urbana-Champaign · University of Toronto
High-fidelity reconstruction of driving scenes is crucial for autonomous driving. While recent feedforward 3D Gaussian Splatting (3DGS) methods enable fast reconstruction, their per-pixel Gaussian prediction paradigm often suffers from multi-view inconsistency and layering artifacts. Moreover, existing methods often model dynamic instances via dense flow prediction, which lacks explicit cross-view correspondence and instance-level consistency. In this paper, we propose PointForward, a feedforward driving reconstruction framework through point-aligned representations. Unlike pixel-aligned methods, we initialize sparse 3D queries in world space and aggregate multi-view image information via spatial-temporal fusion onto these queries, enforcing explicit cross-view consistency in a single feedforward pass. To handle scene dynamics, we introduce scene graphs that explicitly organize moving instances during reconstruction. By leveraging 3D bounding boxes, our method enables instance-level motion propagation and temporally consistent dynamic representations. Extensive experiments show that PointForward achieves state-of-the-art performance on large-scale driving benchmarks.
Cheng Chi, Xianqi Wang, Hongcheng Luo +9
Xiaomi EV · Huazhong University of Science and Technology · Zhejiang University
Feed-forward 4D driving-scene reconstruction enables efficient modeling of dynamic autonomous-driving scenes, but current methods suffer from two complementary failure modes: semantic leakage of traffic participants into the static representation and geometric drift of Gaussian locations across viewpoints. We propose IGRecon4D, a feed-forward dynamic scene reconstruction framework that addresses these failures with training-time instance-prior distillation and LiDAR-anchored Gaussian positioning. Specifically, an instance-level semantic branch provides a potentially dynamic participant prior, which is distilled into dynamic-mask learning through the Instance-Dynamic Consistency Constraint to encourage high-recall localization of traffic-participant regions prone to dynamic-static leakage. In parallel, sparse LiDAR measurements guide dense depth estimation, and the resulting depths are back-projected to anchor Gaussian positions for improved multi-view geometric consistency. Experiments on the Waymo Open Dataset and zero-shot evaluation on nuScenes demonstrate improved full-image reconstruction on the main benchmark and stronger traffic-participant-region reconstruction under cross-dataset evaluation.
Yifei Tong, Ziyang Huang, Yizhao Han +2
Department of Intelligent Science and Technology, Nanjing University, Suzhou 215163, China.