Organizations: CAS Engineering Laboratory for Intelligent Industrial Vision, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · Department of Automation, Tsinghua University, Beijing 100084, China · Giga AI, Beijing, China
Dynamic driving scene reconstruction has made remarkable strides in recent years by leveraging generative priors, particularly in the quality of novel trajectory rendering. Nevertheless, a significant gap remains between the generated data and real-world sensor observations, severely hindering further improvements in reconstruction fidelity. In this paper, we present ReconDreamer#, a self-evolving framework that synergizes reconstruction and generation for dynamic driving scenes, which substantially bridges the gap between synthesized novel trajectory data and recorded real-world observations. Within this self-evolving closed-loop framework, the generative and reconstruction models mutually reinforce each other through iterative refinement, ultimately yielding a significant enhancement in the 3D representation capability of the reconstruction model. Additionally, we introduce a geometry-prior-based adaptive ground model to further enhance the rendering quality of the reconstruction model. To validate the effectiveness of the proposed method, we conduct comprehensive evaluations on Waymo, nuScenes, PandaSet, and EUVS benchmarks. Specifically on Waymo, at a 6-m lateral shift, ReconDreamer# improves NTA-IoU and NTL-IoU by 8.8% and 3.0%, respectively, and reduces FID by 15.1% relative to ReconDreamer++.
Figures & tables
Fig. 1: Overview of the self-evolving reconstruction–restoration loop. Starting from recorded video and an under-trained Gaussian scene, DriveRestorer improves novel-trajectory renderings used to train a new reconstruction. Rendering that reconstruction back at recorded poses yields new degraded–real pairs for the next restorer generation. The top rows compare renderings at 3-m and 6-m lane shifts across iterations.
Base components
Ground support
New mechanisms
New analyses
Method
Backbone / restoration
Ground decomposition
NTDNet
Policy
Round-trip self-evolution
Surface initialization
Soft planes / refinement
Generation / geometry
ReconDreamer [ 10 ]
✓
–
–
Backbone- dependent
–
–
–
–
ReconDreamer++ [ 16 ]
✓
✓
✓
Fixed
–
–
–
–
ReconDreamer#
✓
✓
✓
Adaptive
✓
✓
✓
✓
TABLE I: Comparison of the conference versions and the journal extension.
Fig. 2: Trajectory round-trip construction for Self-Evolving DriveRestorer. Novel-trajectory rendering and restoration feed an auxiliary reconstruction, which returns to recorded poses to construct inputs paired with real video targets.
Recorded trajectory
Lateral shift: 3 m
Lateral shift: 6 m
Method
PSNR ↑
SSIM ↑
LPIPS ↓
NTA ↑
NTL ↑
FID ↓
NTA ↑
NTL ↑
FID ↓
Street Gaussians [ 18 ]
36.50
0.9570
0.1149
0.4980
53.19
130.75
0.3740
49.27
213.04
FreeVS [ 92 ]
26.59
0.8057
0.1325
0.5050
56.84
104.23
0.4650
55.37
121.44
DriveDreamer4D [ 11 ]
34.37
0.9447
0.1321
0.4570
53.30
113.45
0.1590
50.09
261.81
ReconDreamer [ 10 ]
34.31
0.9431
0.1519
0.5390
54.58
93.56
0.4670
52.58
149.19
TABLE II: Quantitative comparison on Waymo. NTA and NTL denote NTA-IoU and NTL-IoU, respectively.
Fig. 3: Qualitative comparison on Waymo. Columns show FreeVS, Street Gaussians, ReconDreamer, ReconDreamer++, and ReconDreamer# ; rows show the recorded-view reference and renderings at 0-m, 3-m, and 6-m lateral shifts. Orange boxes enlarge vehicle and roadside details. Ground truth is shown only for the recorded view.
Recorded trajectory
Lateral shift: 3 m
Lateral shift: 6 m
Method
PSNR ↑
SSIM ↑
LPIPS ↓
NTA ↑
NTL ↑
FID ↓
NTA ↑
NTL ↑
FID ↓
Street Gaussians [ 18 ]
34.91
0.9528
0.1185
0.2190
48.72
149.50
0.1320
46.57
247.73
ReconDreamer [ 10 ]
33.85
0.9497
0.1235
0.3250
49.72
125.43
0.2460
47.08
197.52
ReconDreamer++ [ 16 ]
34.69
0.9533
0.1161
0.3650
51.33
118.27
0.2680
49.26
187.51
ReconDreamer#
35.62
0.9592
0.1111
0.3759
51.47
105.41
0.2772
49.53
133.99
TABLE III: Quantitative comparison on nuScenes.
Fig. 4: Qualitative comparison on nuScenes. Columns show Street Gaussians, ReconDreamer, ReconDreamer++, and ReconDreamer# ; rows show the recorded-view reference and renderings at 0-m, 3-m, and 6-m lateral shifts. Orange boxes enlarge vehicles and nearby scene details. Ground truth is shown only for the recorded view.
Method
No shift
Lane 2 m
Lane 3 m
Vert. 1 m
UniSim [ 42 ]
-
74.70
97.50
-
NeuRAD [ 44 ]
25.00
72.30
93.90
76.30
Street Gaussians [ 18 ]
14.60
66.30
80.70
78.40
ReconDreamer [ 10 ]
16.10
65.40
74.90
67.70
ReconDreamer++ [ 16 ]
13.80
61.90
71.70
62.40
TABLE IV: FID comparison on PandaSet under recorded, lateral-shift, and vertical-shift trajectories.
Fig. 5: Qualitative comparison on PandaSet under recorded, lateral-shift, and vertical-shift trajectories. Columns compare Street Gaussians, ReconDreamer, ReconDreamer++, and ReconDreamer# . Orange boxes highlight fine structures in the distant scene.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
Street Gaussians [ 18 ]
15.08
0.6043
0.3428
ReconDreamer [ 10 ]
20.14
0.6914
0.3286
ReconDreamer++ [ 16 ]
20.52
0.7033
0.3057
ReconDreamer#
21.59
0.7240
0.2995
TABLE V: Static-region comparison on EUVS Level 1.
Fig. 6: Qualitative comparison on EUVS Level 1. The upper row shows a training-trajectory view (Train Set), while the lower row shows an independently captured cross-lane test view (Test Set). Columns compare ground truth, Street Gaussians, ReconDreamer, ReconDreamer++, and ReconDreamer# . Dynamic foreground is masked in the rendered comparisons, and orange boxes enlarge facade signs.
SE
AGM
Recorded
3 m shift
6 m shift
Method
PSNR ↑
SSIM ↑
LPIPS ↓
NTA ↑
NTL ↑
FID ↓
NTA ↑
NTL ↑
FID ↓
ReconDreamer++
36.29
0.9573
0.1076
0.5720
57.06
72.02
0.4890
56.57
111.92
+ AGM
✓
36.72
0.9608
0.1015
0.5780
58.00
71.72
0.5070
57.40
108.10
+ Self-Evolution
✓
36.43
0.9568
0.1072
0.5830
57.75
70.45
0.5120
55.95
100.11
ReconDreamer#
✓
✓
36.89
0.9638
0.0953
0.5984
58.75
66.42
0.5320
58.29
94.99
TABLE VI: New-module ablation on Waymo. SE denotes Self-Evolving DriveRestorer. All variants retain the inherited depth and NTDNet components.
Fig. 7: Qualitative module ablation on Waymo. Columns compare ReconDreamer++, +AGM, +Self-Evolution, and the complete ReconDreamer# model. Rows show the reference view and renderings at 0 m, 3 m, and 6 m lateral shifts. Orange boxes highlight vehicle details and road-boundary artifacts.
Recorded
3 m shift
6 m shift
DriveRestorer
LPIPS ↓
NTA ↑
NTL ↑
FID ↓
NTA ↑
NTL ↑
FID ↓
R1
0.1015
0.5780
58.00
71.72
0.5070
57.40
108.10
R2
0.0985
0.5880
58.34
68.96
0.5180
57.84
101.94
R3
0.0974
0.5910
58.49
68.17
0.5240
57.99
99.45
R4
0.0953
0.5984
58.75
66.42
0.5320
58.29
94.99
TABLE VII: Multi-Round DriveRestorer comparison on Waymo. R1 is the initial static restorer; R2 – R4 are successive self-evolution generations.
Fig. 8: Ground-support comparison on Waymo scene 065. (a) The original ReconDreamer++ ground uses fixed support. (b) Complete AGM produces denser and more uniform support. Both top views share the same bounds, point size, and height colors; the cyan line marks the cross-section. (c) Signed residual ri=ni⊤(μi−pi) to the local LiDAR plane. The mean absolute residual e⊥ and P95 show that AGM expands road support while maintaining surface alignment.