Geo-EVS: Geometry-Conditioned Extrapolative View Synthesis for Autonomous Driving
Authors: Yatong Lan, Rongkui Tang, Lei He
Organizations: School of Vehicle and Mobility, Tsinghua University, Beijing, China · State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University, Beijing, China · School of Science, Minzu University, Beijing, China · Chongqing Changan Automobile Co., Ltd., Chongqing, China
Extrapolative novel view synthesis can reduce camera-rig dependency in autonomous driving by generating standardized virtual views from heterogeneous sensors. Existing methods degrade outside recorded trajectories because extrapolated poses provide weak geometric support and no dense target-view supervision. The key is to explicitly expose the model to out-of-trajectory condition defects during training. We propose Geo-EVS, a geometry-conditioned framework under sparse supervision. Geo-EVS has two components. Geometry-Aware Reprojection (GAR) uses fine-tuned VGGT to reconstruct colored point clouds and reproject them to observed and virtual target poses, producing geometric condition maps. This design unifies the reprojection path between training and inference. Artifact-Guided Latent Diffusion (AGLD) injects reprojection-derived artifact masks during training so the model learns to recover structure under missing support. For evaluation, we use a LiDAR-Projected Sparse-Reference (LPSR) protocol when dense extrapolated-view ground truth is unavailable. On Waymo, Geo-EVS improves sparse-view synthesis quality and geometric accuracy, especially in high-angle and low-coverage settings. It also improves downstream 3D detection.
Figures & tables
Figure 1. Overview of Geo-EVS. GAR constructs geometry-conditioned maps through reprojection with a unified projection interface for training and inference, and AGLD performs geometry-to-image generation with artifact-aware optimization. A block diagram showing the pipeline of Geo-EVS, including GAR and AGLD components and their interactions.
Method
Paradigm
Extra Prior
FID ↓
Street Gaussians
Reconstruction
None
20.5
BEVGen
Autoregression
None
31.27
Panacea
Diffusion
Stable Diffusion
19.63
MagicDrive
Diffusion
Stable Diffusion
18.45
DrivingDiffusion
Diffusion
Stable Diffusion
17.81
DriveWM
Diffusion
Stable Diffusion
14.62
Table 1. Perceptual comparison on Waymo with method taxonomy. We report each baseline’s paradigm and external prior to clarify comparison context; within this baseline subset satisfying the protocol requirements, Geo-EVS achieves the best FID.
Method
sparse-PSNR ↑
sparse-SSIM ↑
3DGS
19.840
0.824
EmerNeRF
21.850
0.876
Street Gaussians
23.000
0.932
FreeVS
22.920
0.895
Geo-EVS (Ours)
23.650
0.941
Table 2. Extrapolation robustness on Waymo. Top: Overall method comparison. Bottom: Geo-EVS detailed analysis by pose offset and geometric sparsity, showing consistent performance across extrapolation conditions.
Setting
LET-mAP L ↑
LET-mAPH ↑
L1/mAPH (Car) ↑
Original 5 Views
34.9
46.3
25.5
5 Native + 40 Generated Views (Ours)
35.9
47.1
26.8
Gain
+1.0
+0.8
+1.3
Table 3. Downstream 3D detection with BEVFormer on Waymo-mini-1/5. We compare the default 5-view setting and 45-view setting with 40 generated views. All metrics are evaluated on the Waymo validation split.
Variant
Artifact Lib.
Mask Type
p
Conditioning
S-PSNR ↑
Contribution
V1: Baseline (no artifacts)
✗
–
0.0
Concat
21.73
baseline
V2: + Random masks
✗
Random Box
0.4
Concat
22.34
+0.61 dB
V3: Full model (Reproj masks)
✓
Reproj Mask
0.4
Concat
23.65
+1.92 dB
Table 4. Core ablation study on extrapolative synthesis (evaluated on view extrapolation split). The three-variant design isolates artifact-prior quality under fixed concat conditioning.
Figure 2. Qualitative extrapolation comparison aligned with Table 2 . We first construct an intermediate target at 22.5∘ from the front ( 0∘ ) and front-side ( 45∘ ) views, then apply 1 m left/right lateral translations in the world frame to form out-of-manifold targets. Geo-EVS yields more coherent structures and cleaner local details than competing methods under this setting. A grid of street-view images comparing the extrapolation synthesis results of five methods: 3DGS, EmerNeRF, Street Gaussians, FreeVS, and the proposed Geo-EVS. The rows show different virtual views generated by applying 1-meter lateral shifts (left and right) from an intermediate 22.5-degree angle. Columns represent the different methods. The comparison shows that while baseline methods like 3DGS and EmerNeRF produce blurry or distorted structures, and FreeVS hallucinates incorrect details, the proposed Geo-EVS maintains clear edges, coherent geometry, and cleaner local details on elements like lane markings and vehicle shapes under large viewpoint shifts.
Methods
SSIM ↑
PSNR ↑
LPIPS ↓
3DGS
0.624
18.06
0.335
Street Gaussians
0.741
21.72
0.221
EmerNeRF
0.678
19.51
0.287
FreeVS
0.768
22.85
0.179
Geo-EVS (Ours)
0.736
23.42
0.154
Table 5. Comparison with NVS counterparts on novel camera synthesis. For all NVS methods, we use all front-side camera views as source views to synthesize the front camera views.
Figure 3. Qualitative interpolation comparison aligned with Table 5 . Following the same protocol, all methods take only front-side camera views as input and synthesize the front camera view. Geo-EVS preserves sharper structures and more consistent textures while remaining competitive with strong NVS baselines. A grid of street-view images comparing the interpolation synthesis results of the same five methods: 3DGS, EmerNeRF, Street Gaussians, FreeVS, and Geo-EVS. The task is to synthesize the front camera view using only front-side camera inputs. The columns compare the outputs of each method alongside the Ground Truth image. The visual results indicate that Geo-EVS produces sharper structural boundaries and more consistent textures compared to 3DGS and Street Gaussians, and is highly competitive with EmerNeRF and FreeVS in maintaining high-fidelity appearance in overlapping viewpoint regions.