Reconstructing high-fidelity LiDAR simulation scenes from real-world driving logs is essential for cost-effective safety validation of autonomous driving systems. While simulators based on Neural Radiance Fields (NeRF) achieve realistic scene reconstruction, their large-scale adoption is limited by slow rendering speeds. Recent 3D Gaussian splatting (3DGS) approaches have improved rendering efficiency through explicit primitive representation and differentiable rasterization. However, existing 3DGS-based methods remain largely restricted to rigid vehicles and produce redundant primitives through unconstrained densification. To address these limitations, we propose URL-Sim, a unified and regulated dynamic LiDAR simulation framework. URL-Sim constructs a dynamic Gaussian scene graph that decouples the modeling of traffic participants, including vehicles, pedestrians, and cyclists, enabling accurate reconstruction of heterogeneous dynamic objects within a single framework. It further introduces a regulated Gaussian densification strategy that ranks primitives with multi-dimensional importance scores and allocates Gaussians to the most critical regions while keeping the total primitive count bounded. Experiments on the Waymo and KITTI-360 datasets demonstrate that URL-Sim achieves higher quantitative accuracy and visual quality than existing baselines under comparable primitive budgets, and accurately recovers the geometry and poses of diverse dynamic traffic participants.
Figures & tables
Figure 1: Pipeline of URL-Sim. Raw data is parsed into a Dynamic Gaussian Scene Graph with specialized representations for static backgrounds and dynamic actors (rigid, SMPL human, deformable). These are processed through differentiable LiDAR ray tracing [ 25 ] , while the RGD strategy concurrently optimizes Gaussian distribution via multi-source importance scoring and budgeted sampling. A post-processing U-Net refines the final LiDAR output.
Figure 2: Comparison of LiDAR edge extraction strategies. (a) Conventional Laplacian edges. (b) Geometric depth smoothness. (c) Intensity gradient. (d) Proposed geo-semantic joint edge map ( τc=0.1 , τg=0.05 ), which fuses both cues to suppress clutter and isolate structurally meaningful boundaries.
Method
Efficiency
Depth
Intensity
Point Cloud
FPS ↑
GN ↓
RMSE ↓
MedAE ↓
LPIPS ↓
SSIM ↑
PSNR ↑
RMSE ↓
MedAE ↓
LPIPS ↓
SSIM ↑
PSNR ↑
CD ↓
F-score ↑
Rigid
20.1
3.17M
6.3597
0.0220
0.2697
0.7004
22.0404
0.0586
0.0061
0.1912
0.7827
25.0785
0.0863
0.9461
w/o NRM
19.7
3.00M
6.1119
0.0187
0.2564
0.7079
22.3780
0.0546
0.0055
0.1513
0.8071
26.0181
0.0710
0.9543
w/o RGD
18.2
3.09M
6.2186
0.0238
0.2630
0.7081
22.2407
0.0551
0.0058
0.1723
0.7861
25.6559
0.0717
0.9562
URL-Sim
17.6
3.00M
5.9844
0.0184
0.2042
0.7154
22.5603
0.0503
0.0052
0.1377
0.8259
26.5183
0.0468
0.9682
Table I: Ablation studies on dynamic scene of the Waymo Open Dataset [ 42 ] . Cells are highlighted as follows: BEST and SECOND BEST .“GN” in the table stands for number of Gaussian primitives
Method
Efficiency
Depth
Intensity
Point Cloud
FPS ↑
GN ↓
RMSE ↓
MedAE ↓
LPIPS ↓
SSIM ↑
PSNR ↑
RMSE ↓
MedAE ↓
LPIPS ↓
SSIM ↑
PSNR ↑
CD ↓
F-score ↑
Rigid
24.6
3.06M
4.0220
0.0341
0.0749
0.9010
26.1136
0.0940
0.0284
0.1879
0.6751
20.5697
0.1367
0.9201
w/o NRM
24.1
3.00M
3.4800
0.0276
0.0641
0.9156
27.3287
0.0846
0.0235
0.1420
0.7347
21.5308
0.0942
0.9441
w/o RGD
22.7
3.12M
3.8669
0.0340
0.0708
0.9041
26.3839
0.0915
0.0270
0.1766
0.6912
20.8249
0.1340
0.9235
URL-Sim
22.3
3.00M
3.4700
0.0277
0.0583
0.9162
27.3792
0.0844
0.0233
0.1404
0.7334
21.5475
0.0909
0.9447
Table II: Ablation studies on dynamic scene of the KITTI-360 Dataset [ 43 ]
Figure 3: Ablation of non-rigid modeling.
Figure 4: Ablation of SMPL human refinement. The red point clouds represent the ground truth LiDAR point clouds of human instances, while the blue and green point clouds denote the Gaussian means of the human models upon SMPL initialization and after optimization, respectively.
Method
Depth
Intensity
Point Cloud
RMSE ↓
SSIM ↑
PSNR ↑
RMSE ↓
SSIM ↑
PSNR ↑
CD ↓
F-score ↑
ADC
4.1685
0.8716
25.6781
0.1062
0.6614
19.4904
0.1243
0.9388
RS
4.1599
0.8700
25.6994
0.1076
0.6463
19.3746
0.1326
0.9352
w/o Sray
3.9513
0.8827
26.1966
0.0993
0.7005
19.9882
0.0871
0.9566
w/o edge
3.9102
0.8859
26.2679
0.0990
0.7085
20.0121
0.0858
0.9573
w/o opacity
3.9366
0.8826
26.2003
0.1013
0.7062
20.0309
0.0875
0.9566
Table III: Ablation studies on different components of RGD. ”RS” in the table stands for random sampling based on ADC
Method
Efficiency
Depth
Intensity
Point Cloud
FPS ↑
GN ↓
RMSE ↓
MedAE ↓
LPIPS ↓
SSIM ↑
PSNR ↑
RMSE ↓
MedAE ↓
LPIPS ↓
SSIM ↑
PSNR ↑
CD ↓
F-score ↑
DyNFL [ 17 ]
0.2
-
7.6841
0.0319
0.2764
0.6673
20.5682
0.0613
0.0066
0.1586
0.7538
24.6592
0.2037
0.6441
LiDAR-Nerf [ 13 ]
1.7
-
9.2613
0.2529
0.4283
0.5669
19.4217
0.0748
0.0169
0.2466
0.6349
22.7140
1.6044
0.7351
LiDAR4D [ 16 ]
0.2
-
6.2431
0.0262
0.1694
0.7018
21.8803
0.0536
0.0084
0.1500
0.7927
25.2151
0.0886
0.9201
LiDAR-GS [ 24 ]
9.7
3.13M
9.4453
0.2082
0.4132
0.6141
19.1070
0.0752
0.0113
0.2362
0.6543
22.6243
1.3034
0.7505
GS-LiDAR [ 23 ]
13.5
3.25M
6.5128
0.0622
0.2575
0.6982
21.8211
0.0663
0.0079
0.2363
0.7089
23.7152
0.4277
0.7986
Table IV: Quantitative comparison with SOTAs on the Waymo Open Dataset [ 42 ] . Metrics are averaged over selected four dynamic scenes.
Method
Efficiency
Depth
Intensity
Point Cloud
FPS ↑
GN ↓
RMSE ↓
MedAE ↓
LPIPS ↓
SSIM ↑
PSNR ↑
RMSE ↓
MedAE ↓
LPIPS ↓
SSIM ↑
PSNR ↑
CD ↓
F-score ↑
DyNFL [ 17 ]
0.4
-
6.5314
0.1038
0.2588
0.6557
21.9115
0.1149
0.0428
0.2859
0.3573
18.8066
0.3416
0.4776
LiDAR-Nerf [ 13 ]
5.3
-
4.8113
0.0917
0.1207
0.7429
24.4380
0.1169
0.0401
0.2397
0.5062
18.6800
0.2171
0.8314
LiDAR4D [ 16 ]
0.4
-
4.1722
0.0386
0.0780
0.8762
25.6880
0.0952
0.0306
0.1585
0.6685
20.4485
0.1930
0.8826
LiDAR-GS [ 24 ]
10.4
3.26M
7.1220
0.0903
0.3227
0.5397
21.0203
0.1569
0.0721
0.3076
0.2523
16.0927
0.2482
0.8432
GS-LiDAR [ 23 ]
13.5
3.19M
4.9931
0.0289
0.0819
0.9033
24.1976
0.0870
0.0216
0.1973
0.6701
21.2212
0.4029
0.8787
Table V: Quantitative comparison with SOTAs on the KITTI-360 dataset [ 43 ] . Metrics are averaged over selected four dynamic scenes.
Figure 5: Qualitative comparison for LiDAR depth simulation on Waymo Open Dataset [ 42 ] sequence seg-104859.
Figure 6: Qualitative comparison for LiDAR point cloud simulation on Waymo Open Dataset [ 42 ] .
Figure 7: LiDAR re-simulation and scene editing results.
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University, Beijing, 100871, China · Xiaomi EV, Beijing, 100085, China