LiDAR-EVS: Enhance Extrapolated View Synthesis for 3D Gaussian Splatting with Pseudo-LiDAR Supervision

Yiming Huang; Xin Kang; Sipeng Zhang; Hongliang Ren; Weihua Zhang; Junjie Lai

cs.ROMar 16, 2026

LiDAR-EVS: Enhance Extrapolated View Synthesis for 3D Gaussian Splatting with Pseudo-LiDAR Supervision

Authors: Yiming Huang, Xin Kang, Sipeng Zhang, Hongliang Ren, Weihua Zhang, Junjie Lai

Organizations: NVIDIA · CUHK

Abstract

3D Gaussian Splatting (3DGS) has emerged as a powerful technique for real-time LiDAR and camera synthesis in autonomous driving simulation. However, simulating LiDAR with 3DGS remains challenging for extrapolated views beyond the training trajectory, as existing methods are typically trained on single-traversal sensor scans, suffer from severe overfitting and poor generalization to novel ego-vehicle paths. To enable reliable simulation of LiDAR along unseen driving trajectories without external multi-pass data, we present LiDAR-EVS, a lightweight framework for robust extrapolated-view LiDAR simulation in autonomous driving. Designed to be plug-and-play, LiDAR-EVS readily extends to diverse LiDAR sensors and neural rendering baselines with minimal modification. Our framework comprises two key components: (1) pseudo extrapolated-view point cloud supervision with multi-frame LiDAR fusion, view transformation, occlusion curling, and intensity adjustment; (2) spatially-constrained dropout regularization that promotes robustness to diverse trajectory variations encountered in real-world driving. Extensive experiments demonstrate that LiDAR-EVS achieves SOTA performance on extrapolated-view LiDAR synthesis across three datasets, making it a promising tool for data-driven simulation, closed-loop evaluation, and synthetic data generation in autonomous driving systems.

Figures & tables

Figure 1: LiDAR-EVS enables robust LiDAR synthesis for extrapolated views beyond the training trajectory. Given single-traversal data, our framework generates pseudo supervision for novel viewpoints, achieving accurate LiDAR simulation on extrapolation view, outperforming existing methods that overfit to the training path.

Figure 2: LiDAR-EVS Pipeline. Our framework consists of two key modules: Pseudo LiDAR Curation and Spatially Constrained Dropout. Pseudo LiDAR curation include the following steps: (1) Multi-frame fusion, (2) Extrapolated view transformation, (3) Occlusion curling, (4) Intensity adjustment. With the proposed framework, we can optimize the Gaussian scene representation to achieve robust LiDAR synthesis for both interpolated and extrapolated view rendering.

Methods	Lane Right Extra. LiDAR			Lane Left Extra. LiDAR			Lane Inter. LiDAR			Lane Inter. Image
Methods	Depth $\downarrow$	CD $\downarrow$	Raydrop $\uparrow$	Depth $\downarrow$	CD $\downarrow$	Raydrop $\uparrow$	Depth $\downarrow$	CD $\downarrow$	Raydrop $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
NeuRAD [ 28 ]	0.006	0.48	79.8	0.004	0.48	79.5	0.0012	0.22	84.6	22.86	0.647	0.323
LiDAR-GS [ 3 ]	0.018	0.45	87.8	0.017	0.45	87.6	0.0026	0.29	92.8	-	-	-
SplatAD [ 9 ]	0.003	0.35	75.8	0.002	0.36	76.4	0.0005	0.19	80.6	22.37	0.692	0.192
Ours	0.001	0.18	84.3	0.001	0.19	84.0	0.0007	0.15	87.7	22.22	0.691	0.295

Table 1: Quantitative Results on the Para-Lane Dataset. Lane Right/Left Extra. indicates the extrapolated view is right/left-shifted with lane width. Inter. represents the interpolated view at original recorded trajectory.

Figure 3: Qualitative LiDAR Rendering Results on Para-Lane Dataset.

Methods	Lane Right Extra. LiDAR				Lane Left Extra. LiDAR				Lane Inter. LiDAR
Methods	Depth $\downarrow$	CD $\downarrow$	Intensity $\downarrow$	Raydrop $\uparrow$	Depth $\downarrow$	CD $\downarrow$	Intensity $\downarrow$	Raydrop $\uparrow$	Depth $\downarrow$	CD $\downarrow$	Intensity $\downarrow$	Raydrop $\uparrow$
NeuRAD [ 28 ]	1.611	1.23	0.087	51.9	3.567	1.44	0.094	52.1	0.018	0.42	0.042	92.7
OmniRE [ 5 ]	3.242	2.42	-	-	3.122	2.26	-	-	1.440	1.74	-	-
SplatAD [ 9 ]	1.399	2.07	0.084	57.8	1.314	1.34	0.091	57.9	0.009	0.41	0.038	93.7
Ours	0.072	0.55	0.064	74.4	0.306	0.54	0.068	73.3	0.011	0.41	0.041	92.7

Table 2: Quantitative LiDAR Rendering Results on nuScenes Dataset.

Methods	Lane Right Extra. LiDAR				Lane Left Extra. LiDAR				Lane Inter. LiDAR
Methods	Depth $\downarrow$	CD $\downarrow$	Intensity $\downarrow$	Raydrop $\uparrow$	Depth $\downarrow$	CD $\downarrow$	Intensity $\downarrow$	Raydrop $\uparrow$	Depth $\downarrow$	CD $\downarrow$	Intensity $\downarrow$	Raydrop $\uparrow$
NeuRAD [ 28 ]	0.011	0.90	0.097	40.9	0.009	0.88	0.098	39.8	0.005	0.34	0.061	95.5
OmniRE [ 5 ]	3.071	2.43	-	-	2.465	3.26	-	-	2.467	3.15	-	-
SplatAD [ 9 ]	0.014	0.94	0.097	41.3	0.011	0.99	0.098	40.4	0.008	0.31	0.058	97.2
Ours	0.011	0.36	0.067	79.1	0.011	0.38	0.069	79.0	0.008	0.31	0.061	96.1

Table 3: Quantitative LiDAR Rendering Results on Pandaset Dataset.

Figure 4: Qualitative LiDAR Rendering Results on nuScenes and Pandaset.

Methods	nuScenes			Pandaset
Methods	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
NeuRAD [ 28 ]	25.96	0.786	0.318	26.28	0.774	0.230
OmniRE [ 5 ]	22.77	0.706	0.210	23.63	0.706	0.223
SplatAD [ 9 ]	25.97	0.833	0.317	27.12	0.838	0.178
Ours	26.11	0.837	0.339	27.27	0.841	0.192

Table 4: Quantitative Image Rendering Results on nuScenes and Pandaset .

Figure 5: Qualitative Image Rendering Results on nuScenes, Pandaset and Para-Lane Dataset.

Methods	Lane Right Extra. LiDAR			Lane Left Extra. LiDAR			Lane Inter. LiDAR
Methods	Depth $\downarrow$	CD $\downarrow$	Raydrop $\uparrow$	Depth $\downarrow$	CD $\downarrow$	Raydrop $\uparrow$	Depth $\downarrow$	CD $\downarrow$	Raydrop $\uparrow$
w/o Pseudo	0.003	0.35	75.8	0.002	0.36	76.4	0.0005	0.19	80.6
Left Only	0.002	0.23	71.1	0.001	0.20	77.8	0.0007	0.16	83.5
Right Only	0.001	0.19	78.4	0.002	0.26	71.3	0.0007	0.16	83.5
Our Full	0.001	0.18	84.3	0.001	0.19	84.0	0.0007	0.15	87.7

Table 5: Pseudo LiDAR Ablation Results on Para-Lane Dataset.

Figure 6: Qualitative Results for Ablation Study on Para-Lane Dataset.

Dropout Rate	Lane Right Extra. LiDAR			Lane Left Extra. LiDAR			Lane Inter. LiDAR
Dropout Rate	Depth $\downarrow$	CD $\downarrow$	Raydrop $\uparrow$	Depth $\downarrow$	CD $\downarrow$	Raydrop $\uparrow$	Depth $\downarrow$	CD $\downarrow$	Raydrop $\uparrow$
0.0	0.0009	0.16	76.77	0.0010	0.18	75.77	0.0004	0.15	81.94
0.1	0.0009	0.16	78.26	0.0010	0.18	77.60	0.0004	0.15	83.33
0.2	0.0009	0.16	79.76	0.0011	0.18	79.13	0.0005	0.15	84.74
0.3	0.0010	0.16	81.70	0.0011	0.18	80.90	0.0005	0.15	86.07
0.4	0.0011	0.16	83.72	0.0011	0.18	82.76	0.0006	0.15	87.01

Table 6: Dropout Regularization Ablation Results on Paralane Dataset.

Methods	Lane Right Extra.	Lane Left Extra.	Lane Inter. LiDAR		Train Time (H) $\downarrow$
Methods	LiDAR MR/s $\uparrow$	LiDAR MR/s $\uparrow$	Camera MP/s $\uparrow$	LiDAR MR/s $\uparrow$	Train Time (H) $\downarrow$
NeuRAD [ 28 ]	3.25	3.25	17.2	2.43	1.78
LiDAR-GS [ 3 ]	1.42	1.42	-	1.39	0.64
SplatAD [ 9 ]	0.13	0.13	84.5	0.11	3.87
Ours	0.13	0.13	86.2	0.12	1.69

Table S1 : Computational Overhead on the Para-Lane Dataset. Lane Right/Left Extra. indicates the extrapolated view is right/left-shifted with lane width. Inter. represents the interpolated view at original recorded trajectory.

Methods	Lane Right Extra. LiDAR			Lane Left Extra. LiDAR			Lane Inter. LiDAR			Lane Inter. Image
Methods	Depth $\downarrow$	CD $\downarrow$	Raydrop $\uparrow$	Depth $\downarrow$	CD $\downarrow$	Raydrop $\uparrow$	Depth $\downarrow$	CD $\downarrow$	Raydrop $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
NeuRAD [ 28 ]	0.006	0.48	79.8	0.004	0.48	79.5	0.0012	0.22	84.6	22.86	0.647	0.323
LiDAR-GS [ 3 ]	0.018	0.45	87.8	0.017	0.45	87.6	0.0026	0.29	92.8	-	-	-
SplatAD [ 9 ]	0.003	0.35	75.8	0.002	0.36	76.4	0.0005	0.19	80.6	22.37	0.692	0.192
Ours+NeuRAD	0.002	0.25	85.2	0.002	0.24	85.2	0.0011	0.20	87.9	22.88	0.648	0.300
Ours+LiDAR-GS	0.003	0.31	89.8	0.012	0.43	87.3	0.0025	0.29	92.7	-	-	-

Table S2 : Quantitative Results on the Para-Lane Dataset. Lane Right/Left Extra. indicates the extrapolated view is right/left-shifted with lane width. Inter. represents the interpolated view at original recorded trajectory.

Methods	Lane Right Extra. LiDAR				Lane Left Extra. LiDAR				Lane Inter. LiDAR
Methods	Depth $\downarrow$	CD $\downarrow$	Intensity $\downarrow$	Raydrop $\uparrow$	Depth $\downarrow$	CD $\downarrow$	Intensity $\downarrow$	Raydrop $\uparrow$	Depth $\downarrow$	CD $\downarrow$	Intensity $\downarrow$	Raydrop $\uparrow$
w/o intensity adj.	0.015	0.42	0.074	75.7	0.014	0.34	0.071	76.8	0.009	0.27	0.068	96.3
Ours Full	0.006	0.29	0.059	76.9	0.006	0.29	0.061	76.5	0.005	0.26	0.055	96.6

Table S3 : Ablation Results on Pandaset Dataset for Intensity Adjustment.

Figure S1 : Qualitative Result for Intensity Adjustment

Figure S2 : Visualization of Intensity Difference After Adjustment Changes.

Fusion Number	Lane Right Extra. LiDAR			Lane Inter. LiDAR			Lane Inter. Image
Fusion Number	Depth $\downarrow$	CD $\downarrow$	Raydrop $\uparrow$	Depth $\downarrow$	CD $\downarrow$	Raydrop $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
5	0.001	0.20	78.9	0.0008	0.17	87.1	21.93	0.674	0.271
10	0.001	0.18	84.3	0.0007	0.17	87.1	21.94	0.672	0.287
20	0.001	0.18	84.3	0.0006	0.17	87.1	21.88	0.669	0.274
30	0.001	0.18	85.9	0.0006	0.17	87.3	21.92	0.667	0.285
40	0.001	0.18	85.2	0.0006	0.17	87.3	21.84	0.664	0.289

Table S4 : Ablation Results on Para-Lane Dataset for Multi-Frame Fusion.

Explore similar work

Geo-EVS: Geometry-Conditioned Extrapolative View Synthesis for Autonomous Driving
Apr 8, 2026Yatong Lan, Rongkui Tang, Lei HeNovel View Synthesis Autonomous Driving
School of Vehicle and Mobility, Tsinghua University, Beijing, China · State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University, Beijing, China · School of Science, Minzu University, Beijing, China +1
VisionNVS: Self-Supervised Inpainting for Novel View Synthesis under the Virtual-Shift Paradigm
Mar 18, 2026Hongbo Lu, Liang Yao, Chenghao He +4Novel View Synthesis Autonomous Driving Simulation
Shanghai Jiao Tong University · COWARobot Co. Ltd. · Hohai University
RS2AD-LiDAR: End-to-End Autonomous Driving LiDAR Data Generation from Roadside Sensor Observations
May 22, 2026Runyi Huang, Ni Ding, Ruidan Xing +3Synthetic Data Augmentation Autonomous Driving Perception
State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University, Beijing, 100084, China. · School of Vehicle and Mobility, Tsinghua University, Beijing, 100084, China. · College of Artificial Intelligence, Tsinghua University, Beijing, 100083, China. +2

Apr 8, 2026J/K move · Enter open · S save

Geo-EVS: Geometry-Conditioned Extrapolative View Synthesis for Autonomous Driving

Yatong Lan, Rongkui Tang, Lei He

School of Vehicle and Mobility, Tsinghua University, Beijing, China · State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University, Beijing, China · School of Science, Minzu University, Beijing, China+1

Novel View Synthesis Autonomous Driving Latent Diffusion Models

Open paper AskPDF arXiv

Extrapolative novel view synthesis can reduce camera-rig dependency in autonomous driving by generating standardized virtual views from heterogeneous sensors. Existing methods degrade outside recorded trajectories because extrapolated poses provide weak geometric support and no dense target-view supervision. The key is to explicitly expose the model to out-of-trajectory condition defects during training. We propose Geo-EVS, a geometry-conditioned framework under sparse supervision. Geo-EVS has two components. Geometry-Aware Reprojection (GAR) uses fine-tuned VGGT to reconstruct colored point clouds and reproject them to observed and virtual target poses, producing geometric condition maps. This design unifies the reprojection path between training and inference. Artifact-Guided Latent Diffusion (AGLD) injects reprojection-derived artifact masks during training so the model learns to recover structure under missing support. For evaluation, we use a LiDAR-Projected Sparse-Reference (LPSR) protocol when dense extrapolated-view ground truth is unavailable. On Waymo, Geo-EVS improves sparse-view synthesis quality and geometric accuracy, especially in high-angle and low-coverage settings. It also improves downstream 3D detection.

Abstract

Explore similar work

Geo-EVS: Geometry-Conditioned Extrapolative View Synthesis for Autonomous Driving

VisionNVS: Self-Supervised Inpainting for Novel View Synthesis under the Virtual-Shift Paradigm

RS2AD-LiDAR: End-to-End Autonomous Driving LiDAR Data Generation from Roadside Sensor Observations