cs.CVApr 1, 2026

ReinDriveGen: Reinforcement Post-Training for Out-of-Distribution Driving Scene Generation

Authors: Hao Zhang, Lue Fan, Weikang Bian, Zehuan Wu, Lewei Lu, Zhaoxiang Zhang, Hongsheng Li

Organizations: MMLab, CUHK · CASIA · SenseTime Research

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

We present ReinDriveGen, a framework that enables full controllability over dynamic driving scenes, allowing users to freely edit actor trajectories to simulate safety-critical corner cases such as front-vehicle collisions, drifting cars, vehicles spinning out of control, pedestrians jaywalking, and cyclists cutting across lanes. Our approach constructs a dynamic 3D point cloud scene from multi-frame LiDAR data, introduces a vehicle completion module to reconstruct full 360° geometry from partial observations, and renders the edited scene into 2D condition images that guide a video diffusion model to synthesize realistic driving videos. Since such edited scenarios inevitably fall outside the training distribution, we further propose an RL-based post-training strategy with a pairwise preference model and a pairwise reward mechanism, enabling robust quality improvement under out-of-distribution conditions without ground-truth supervision. Extensive experiments demonstrate that ReinDriveGen outperforms existing approaches on edited driving scenarios and achieves state-of-the-art results on novel ego viewpoint synthesis.

Figures & tables

Figure 1: ReinDriveGen enables photorealistic generation of OOD driving scenarios. ReinDriveGen can manipulate actor trajectories to synthesize diverse safety-critical corner cases such as vehicle in-place spinning, cyclist crossing, and left-turn collisions. Our RL-based post-training significantly improves generation quality for these OOD edits without requiring ground-truth supervision.
Figure 2: Overview of ReinDriveGen’s two core components. Left: Our simulator edits dynamic point clouds and completes vehicle geometry to render structural pseudo-images, which condition the video diffusion model for photorealistic synthesis. Right: To enhance quality in out-of-distribution (OOD) scenarios, we employ RL post-training. A pairwise reward mechanism ranks generated candidates into positive and negative sets, providing robust contrastive supervision.
Figure 3: Qualitative demonstrations of ReinDriveGen on OOD safety-critical corner cases. Every two consecutive rows show two frames from the same scene, with the edit type and edited region annotated in each image.
Figure 4: Qualitative comparison of novel trajectories in the lane-change scenario.
Figure 5: Qualitative comparison of vehicle trajectory editing. Top two rows: the target vehicle is shifted 6m left and 4m backward. Bottom two rows: the target vehicle is shifted 6m to the right and 1m backward.
ModelsPVGDD4D w/. PVGRecon. w/. PVGS3S^{3} Gauss.DD4D w/. S3S^{3} Gauss.Recon. w/. S3S^{3} Gauss.Deform.-GSDD4D w/. Deform.-GSRecon. w/. Deform.-GSOurs
NTA-IoU ↑\uparrow0.2560.4380.4640.1750.4950.4130.2400.3350.4430.549
NTL-IoU ↑\uparrow50.7053.0653.2149.0553.4251.6251.6252.9353.7856.13
FID ↓\downarrow105.2971.5274.32124.9066.93123.6192.2477.3276.2451.99
Table 1: Quantitative comparison in a lane-change scenario where the trajectory gradually shifts 4 m to the left.
Figure 6: Ablation study on vehicle completion. Vehicle completion fills in unobserved surfaces in the pseudo-images, effectively eliminating vehicle distortion and texture artifacts in the diffusion outputs.
MethodMotion smoothness ↑\uparrowBackground consistency ↑\uparrowImage quality ↑\uparrowFID ↓\downarrow
Street Gauss.0.9811110.9309750.63977584.7970
StreetCrafter0.9817150.9068720.65085683.6278
Ours0.9773550.9396970.69280980.6267
Table 2: Quantitative comparison on the vehicle trajectory editing scenario with a 5m lateral shift.
Figure 7: Ablation study on RL-based post-training. From top to bottom: doubled vehicle speed, in-place spinning, and vehicle rollover. As the RL training step increases, the generated vehicles exhibit progressively better geometry, texture fidelity, and lighting plausibility.
Figure 8: Evaluation score curves during RL training. (a) Overall evaluation score averaged across all OOD conditions. (b)–(d) Per-scenario scores for doubling speed, in-place spinning, and vehicle rollover, respectively.
Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Preference prediction accuracy of the pointwise reward model (green) and the pairwise preference model (orange) during training (top) and on the test set (bottom). The pairwise model achieves higher test-set accuracy (95% vs. 91%) with more stable training dynamics, demonstrating that jointly contrasting two samples within a shared forward pass yields more robust preference judgments than independent pointwise scoring.
Figure 10: Qualitative comparison of RL post-training with pointwise vs. pairwise reward. From left to right: video diffusion without RL, RL with pointwise reward, and RL with pairwise reward. Red dashed boxes highlight vehicle regions. While pointwise reward offers limited improvement and sometimes suffers from reward hacking artifacts, the pairwise reward mechanism consistently produces vehicles with better geometry, texture fidelity, and lighting plausibility, demonstrating its superiority as the reward signal for RL post-training.

Explore similar work

CardsList
Apr 6, 2026

HorizonWeaver: Generalizable Multi-Level Semantic Editing for Driving Scenes

Ensuring safety in autonomous driving requires scalable generation of realistic, controllable driving scenes beyond what real-world testing provides. Yet existing instruction guided image editors, trained on object-centric or artistic data, struggle with dense, safety-critical driving layouts. We propose HorizonWeaver, which tackles three fundamental challenges in driving scene editing: (1) multi-level granularity, requiring coherent object- and scene-level edits in dense environments; (2) rich high-level semantics, preserving diverse objects while following detailed instructions; and (3) ubiquitous domain shifts, handling changes in climate, layout, and traffic across unseen environments. The core of HorizonWeaver is a set of complementary contributions across data, model, and training: (1) Data: Large-scale dataset generation, where we build a paired real/synthetic dataset from Boreas, nuScenes, and Argoverse2 to improve generalization; (2) Model: Language-Guided Masks for fine-grained editing, where semantics-enriched masks and prompts enable precise, language-guided edits; and (3) Training: Content preservation and instruction alignment, where joint losses enforce scene consistency and instruction fidelity. Together, HorizonWeaver provides a scalable framework for photorealistic, instruction-driven editing of complex driving scenes, collecting 255K images across 13 editing categories and outperforming prior methods in L1, CLIP, and DINO metrics, achieving +46.4% user preference and improving BEV segmentation IoU by +33%. Project page: https://msoroco.github.io/horizonweaver/

Mauricio Soroco, Francesco Pittaluga, Zaid Tasneem +5

Simon Fraser University · NEC Labs America

Text-Guided Image EditingAutonomous Driving Scenario GenerationDomain Generalization
AskPDF
Feb 24, 2026

HorizonForge: Driving Scene Editing with Any Trajectories and Any Vehicles

Controllable driving scene generation is critical for realistic and scalable autonomous driving simulation, yet existing approaches struggle to jointly achieve photorealism and precise control. We introduce HorizonForge, a unified framework that reconstructs scenes as editable Gaussian Splats and Meshes, enabling fine-grained 3D manipulation and language-driven vehicle insertion. Edits are rendered through a noise-aware video diffusion process that enforces spatial and temporal consistency, producing diverse scene variations in a single feed-forward pass without per-trajectory optimization. To standardize evaluation, we further propose HorizonSuite, a comprehensive benchmark spanning ego- and agent-level editing tasks such as trajectory modifications and object manipulation. Extensive experiments show that Gaussian-Mesh representation delivers substantially higher fidelity than alternative 3D representations, and that temporal priors from video diffusion are essential for coherent synthesis. Combining these findings, HorizonForge establishes a simple yet powerful paradigm for photorealistic, controllable driving simulation, achieving an 83.4% user-preference gain and a 25.19% FID improvement over the second best state-of-the-art method. Project page: https://horizonforge.github.io/ .

Yifan Wang, Francesco Pittaluga, Zaid Tasneem +3

NEC Labs America · Stony Brook University · UC San Diego

Autonomous Driving Scenario Generation3D Scene EditingAutonomous Driving BenchmarksGaussian Splatting
AskPDF
May 25, 2026

AnyScene: Towards Highly Controllable Driving Scene Generation at Anywhere and Beyond

Generating high-fidelity and controllable synthetic data is critical for advancing end-to-end autonomous driving, particularly for addressing the long tail of rare safety-critical scenarios. Existing occupancy-guided methods typically rely on shallow conditioning mechanisms and reference-frame-dependent video synthesis, which limits fine-grained controllability from arbitrary BEV layouts and restricts their applicability for scalable simulation. In this paper, we propose AnyScene, a unified occupancy-centric framework for driving scene generation. AnyScene generates semantic occupancy sequences from BEV layouts through a Spatial-Temporal Occupancy Diffusion Transformer that jointly tokenizes BEV and occupancy features in an autoregressive manner. This design enables precise controllability from cross-dataset and user-defined BEV inputs while naturally supporting long-horizon generation. Building upon the generated occupancy, a Geometry-Grounded View Expansion module treats occupancy as the canonical spatial representation and synthesizes temporally consistent multi-view driving videos in a reference-free and autoregressive fashion, supporting flexible camera configurations at inference time. Extensive experiments demonstrate that AnyScene achieves state-of-the-art performance in both occupancy and video generation. It exhibits strong generalization to unseen and customized layouts, and provides measurable benefits for downstream tasks such as sparse-view 3D reconstruction.

Haiming Zhang, Junfei Zhou, Feng Jiang +6

Li Auto · Southwest Jiaotong University · Tsinghua University

Long-Horizon Video Generation3D Scene GenerationControllable Video GenerationAutonomous Driving SimulationAutonomous Driving Scenario Generation
AskPDF