cs.CVApr 1, 2026

ReinDriveGen: Reinforcement Post-Training for Out-of-Distribution Driving Scene Generation

Authors: Hao Zhang, Lue Fan, Weikang Bian, Zehuan Wu, Lewei Lu, Zhaoxiang Zhang, Hongsheng Li

Organizations: MMLab, CUHK · CASIA · SenseTime Research

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

We present ReinDriveGen, a framework that enables full controllability over dynamic driving scenes, allowing users to freely edit actor trajectories to simulate safety-critical corner cases such as front-vehicle collisions, drifting cars, vehicles spinning out of control, pedestrians jaywalking, and cyclists cutting across lanes. Our approach constructs a dynamic 3D point cloud scene from multi-frame LiDAR data, introduces a vehicle completion module to reconstruct full 360° geometry from partial observations, and renders the edited scene into 2D condition images that guide a video diffusion model to synthesize realistic driving videos. Since such edited scenarios inevitably fall outside the training distribution, we further propose an RL-based post-training strategy with a pairwise preference model and a pairwise reward mechanism, enabling robust quality improvement under out-of-distribution conditions without ground-truth supervision. Extensive experiments demonstrate that ReinDriveGen outperforms existing approaches on edited driving scenarios and achieves state-of-the-art results on novel ego viewpoint synthesis.

Figures & tables

Figure 1: ReinDriveGen enables photorealistic generation of OOD driving scenarios. ReinDriveGen can manipulate actor trajectories to synthesize diverse safety-critical corner cases such as vehicle in-place spinning, cyclist crossing, and left-turn collisions. Our RL-based post-training significantly improves generation quality for these OOD edits without requiring ground-truth supervision.
Figure 2: Overview of ReinDriveGen’s two core components. Left: Our simulator edits dynamic point clouds and completes vehicle geometry to render structural pseudo-images, which condition the video diffusion model for photorealistic synthesis. Right: To enhance quality in out-of-distribution (OOD) scenarios, we employ RL post-training. A pairwise reward mechanism ranks generated candidates into positive and negative sets, providing robust contrastive supervision.
Figure 3: Qualitative demonstrations of ReinDriveGen on OOD safety-critical corner cases. Every two consecutive rows show two frames from the same scene, with the edit type and edited region annotated in each image.
Figure 4: Qualitative comparison of novel trajectories in the lane-change scenario.
Figure 5: Qualitative comparison of vehicle trajectory editing. Top two rows: the target vehicle is shifted 6m left and 4m backward. Bottom two rows: the target vehicle is shifted 6m to the right and 1m backward.
ModelsPVGDD4D w/. PVGRecon. w/. PVGS3S^{3} Gauss.DD4D w/. S3S^{3} Gauss.Recon. w/. S3S^{3} Gauss.Deform.-GSDD4D w/. Deform.-GSRecon. w/. Deform.-GSOurs
NTA-IoU ↑\uparrow0.2560.4380.4640.1750.4950.4130.2400.3350.4430.549
NTL-IoU ↑\uparrow50.7053.0653.2149.0553.4251.6251.6252.9353.7856.13
FID ↓\downarrow105.2971.5274.32124.9066.93123.6192.2477.3276.2451.99
Table 1: Quantitative comparison in a lane-change scenario where the trajectory gradually shifts 4 m to the left.
Figure 6: Ablation study on vehicle completion. Vehicle completion fills in unobserved surfaces in the pseudo-images, effectively eliminating vehicle distortion and texture artifacts in the diffusion outputs.
MethodMotion smoothness ↑\uparrowBackground consistency ↑\uparrowImage quality ↑\uparrowFID ↓\downarrow
Street Gauss.0.9811110.9309750.63977584.7970
StreetCrafter0.9817150.9068720.65085683.6278
Ours0.9773550.9396970.69280980.6267
Table 2: Quantitative comparison on the vehicle trajectory editing scenario with a 5m lateral shift.
Figure 7: Ablation study on RL-based post-training. From top to bottom: doubled vehicle speed, in-place spinning, and vehicle rollover. As the RL training step increases, the generated vehicles exhibit progressively better geometry, texture fidelity, and lighting plausibility.
Figure 8: Evaluation score curves during RL training. (a) Overall evaluation score averaged across all OOD conditions. (b)–(d) Per-scenario scores for doubling speed, in-place spinning, and vehicle rollover, respectively.
Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Preference prediction accuracy of the pointwise reward model (green) and the pairwise preference model (orange) during training (top) and on the test set (bottom). The pairwise model achieves higher test-set accuracy (95% vs. 91%) with more stable training dynamics, demonstrating that jointly contrasting two samples within a shared forward pass yields more robust preference judgments than independent pointwise scoring.
Figure 10: Qualitative comparison of RL post-training with pointwise vs. pairwise reward. From left to right: video diffusion without RL, RL with pointwise reward, and RL with pairwise reward. Red dashed boxes highlight vehicle regions. While pointwise reward offers limited improvement and sometimes suffers from reward hacking artifacts, the pairwise reward mechanism consistently produces vehicles with better geometry, texture fidelity, and lighting plausibility, demonstrating its superiority as the reward signal for RL post-training.

Explore similar work

CardsList

  1. HorizonWeaver: Generalizable Multi-Level Semantic Editing for Driving Scenes

    Apr 6, 2026Mauricio Soroco, Francesco Pittaluga, Zaid Tasneem +5Text-Guided Image EditingAutonomous Driving Scenario Generation

    Simon Fraser University · NEC Labs America

  2. HorizonForge: Driving Scene Editing with Any Trajectories and Any Vehicles

    Feb 24, 2026Yifan Wang, Francesco Pittaluga, Zaid Tasneem +3Autonomous Driving Scenario Generation3D Scene Editing

    NEC Labs America · Stony Brook University · UC San Diego

  3. AnyScene: Towards Highly Controllable Driving Scene Generation at Anywhere and Beyond

    May 25, 2026Haiming Zhang, Junfei Zhou, Feng Jiang +6Long-Horizon Video Generation3D Scene Generation

    Li Auto · Southwest Jiaotong University · Tsinghua University