GeoV2V: Geometry-Grounded Video Diffusion Model for Driving Scene Generation
Authors: Yuchen Xi, Yuhan Guo, Chenwei Hou, Zixu Liu, Jin Fang, Jason Liu, Ruigang Yang
Organizations: Global Institute of Future Technology, Shanghai Jiao Tong University, China · Global College, Shanghai Jiao Tong University, China · School of Mechanical Engineering, Shanghai Jiao Tong University, China · Shanghai Pudong Development Bank, China
Novel view synthesis under trajectory changes is essential for autonomous driving simulation, yet existing methods struggle to generate consistent videos when extrapolating to unseen viewpoints such as lane shifts. Reconstruction-based approaches maintain geometric consistency but degrade in visual quality under large viewpoint changes, while reconstruct-then-restore methods often produce visually plausible frames yet suffer from geometric and temporal inconsistencies. We present GeoV2V, a geometry-grounded video-to-video diffusion framework that synthesizes temporally coherent driving videos along novel trajectories by integrating LiDAR and depth-derived geometric priors with full-video conditioning in the Wan 2.1 backbone. A key challenge in learning cross-lane transformations is the lack of synchronized multi-trajectory supervision in real-world datasets. To address this, we also introduce Para4D, a synthetic dataset capturing synchronized multi-lane driving videos within dynamic scenes, which provides explicit supervision for lane-shift generation and serves as a strong training prior for geometry-consistent view synthesis. Experiments on Waymo, nuScenes, and Para4D demonstrate state-of-the-art performance in visual quality, geometric consistency, and robustness to complex lighting dynamics under large trajectory shifts. Project page: https://xiyuche.github.io/GeoV2V/
Figures & tables
Figure 1: Overview and qualitative results of our method.
Figure 2: Motivation of our approach. Real-world videos arise from multiple factors including geometry, illumination dynamics, and optical media, making consistent synthesis difficult for existing methods (top). Our video-to-video model jointly preserves these factors and produces temporally consistent novel-view videos (bottom).
Figure 3: Overview of the Para4D dataset. We generate multi-lane synchronized driving videos in the CARLA simulator using diverse maps, randomized assets, and dynamic traffic agents. Parallel camera trajectories are placed across lanes to render synchronized lane-shifted videos for training.
Figure 4: Geometry prior preprocessing.
Figure 5: Architecture of GeoV2V, a geometry-guided video-to-video diffusion model. The model encodes source, pseudo, target (noisy), and LiDAR videos into latent representations. Through frame-wise conditioning with geometry signals and camera-pose embeddings, the DiT backbone performs geometry-consistent video-to-video synthesis for novel-view generation.
Figure 6: Qualitative new-view synthesis results on the Waymo dataset, where FreeVS [ 37 ] is evaluated using the original cropping strategy in their official implementation, resulting in synthesis restricted to only the lower portion of the scene.
Figure 7: Comparison of temporal dynamic lighting preservation. GeoV2V faithfully preserves the vehicle’s turn signal across time steps, while existing methods fail to maintain consistent dynamic lighting patterns.
Method
Annotation
Waymo Static
Waymo Dynamic
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
FreeVS
Bounding-box
22.77
0.670
0.350
21.78
0.596
0.362
OmniRe
Bounding-box
24.10
0.708
0.252
23.73
0.683
0.237
EmerNeRF
Bounding-box
24.92
0.701
0.245
23.91
0.698
0.304
StreetGS
Bounding-box
24.37
0.714
0.239
23.54
0.696
0.256
StreetCrafter-V
Bounding-box
19.46
0.611
0.353
18.92
0.597
0.353
Table 1: Comparison on real-world datasets (interpolation).
School of Vehicle and Mobility, Tsinghua University, Beijing, China · State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University, Beijing, China · School of Science, Minzu University, Beijing, China +1