Diffusion probabilistic models can capture the multi-modal, interaction-rich distribution of joint future trajectories in driving scenes. We show that a single pretrained diffusion traffic model can serve two complementary roles in the autonomous driving development loop: as an ego motion planner, and as a controllable generator of safety-critical scenarios for stress-testing the planners. On the planning side, we introduce a Single-Stream Dual-Stream (SSDS) diffusion-transformer decoder that fuses scene context via joint attention rather than late cross-attention, improving closed-loop performance on nuPlan. We further propose Decoupled Annealing Posterior Sampling with Energy (DAPSE), a training-free guidance scheme that injects arbitrary energy functions at the clean-sample level, avoiding the first-order approximation errors while requiring no auxiliary networks. Beyond planning, we leverage the same diffusion model as a controllable scenario generator to create realistic long-tail driving interactions for closed-loop evaluation. Through inference-time guidance, selected agents are steered toward safety-critical behaviors, including aggressive cut-ins, lead-vehicle braking, and combined longitudinal-lateral interactions, while preserving realistic traffic behaviors. Evaluated in closed-loop nuPlan simulations with independent black-box planners, the generated scenarios expose failure modes that remain hidden under standard benchmarks. Although the SSDS-based planner achieves stronger nominal performance, it experiences larger degradation under these challenging scenarios, demonstrating that benchmark superiority does not necessarily translate to robustness. These results demonstrate that a single learned traffic prior can simultaneously improve motion planning and provide a realistic framework for systematic planner robustness evaluation.
Figures & tables
Figure 1: The dual planner and scenario generator framework. A shared scene encoder produces a scene condition used by two trajectory decoders: the ego decoder and the adversarial decoder are sampled independently, with the adversarial decoder being steered at each denoising step by the gradient of a scenario-specific energy (e.g. ETTC,Elc,Ebrake ). Both trajectories are tracked by an LQR controller, and the resulting histories are fed back for the next timestep.
Figure 2: The proposed Single-stream and Dual-stream based DiT architecture for planning of ego and prediction of neighbors .
Reactive
Non-Reactive
Data
Method
Val14 ↑
T14-hard ↑
Test14 ↑
Val14 ↑
T14-hard ↑
Test14 ↑
250K
DP
73.58
55.88
76.55
84.96
71.88
87.33
SSDS-DP
75.85
62.68
77.53
86.25
72.11
88.31
650K
DP
78.69
51.89
68.74
87.45
68.45
79.63
SSDS-DP
78.96
66.29
80.06
86.58
69.86
85.72
1M
DP
82.82
69.26
82.89
89.88
75.38
89.29
Table 1: Closed-loop nuPlan scores across training-set sizes.
Figure 3: Comparison between DP (baseline) and SSDS-DP (650K) in a Val14 Reactive simulation.
Figure 4: Speed maintenance with DAPSE.
Scenario
Planner
nuPlan Score ↑
TTC Bound (%) ↑
Ego-Fault Collision (%) ↓
Ego Comfort (%) ↑
Realism deviation ↓
No-Adv
DP
80.49
91
4
70
—
SSDS-DP
84.65
96
4
83
—
Adv
DP
67.29
87
23
61
0.38
SSDS-DP
53.96
70
26
78
0.36
Table 2: Closed-loop evaluation on 25 selected Val14 scenarios with combined cut-in and braking scenario. No-Adv: scenarios without any adversary; Adv: scenarios with one adversary.
Figure 5: Comparison between DP and SSDS-DP (1M) on generated adversarial scenarios at the same simulation timesteps.
Closed-loop traffic simulation remains challenging because it must generate interactive multi-agent behaviors that are scene-consistent and controllable throughout rollout. Prior diffusion-based approaches achieve strong realism, but their computational cost can hinder deployment in time-constrained replanning loops for autonomous vehicle planning and simulation. We present a diffusion-based scenario generation framework conditioned on instance-centric scene context and multimodal proposal priors, with optional test-time guidance for shaping safety-critical behaviors. A compact action-latent representation and proposal-based initialization improve sampling efficiency and reduce per-step runtime without retraining. Experiments on the Waymo Open Motion Dataset demonstrate a favorable balance among realism, safety, and controllability across diverse interactive scenarios, while showing that test-time guidance enables systematic trade-offs among competing objectives.
Shubham Vaijanath Phoolari, Aleyna Kara, Christoph Lauer +1
Authors are with CARIAD SE, Germany. · Author is with the Institute of Automotive Engineering (FZD), TU Darmstadt, Germany. · Author is with the Technical University of Munich, Germany.
Generating closed-loop traffic scenarios that are both realistic and controllable is crucial for evaluating autonomous driving systems, especially under rare safety-critical interactions. Existing learning-based methods often struggle to balance controllability and realism, offering either limited fine-grained control over traffic behavior or controllable scenarios at the expense of behavioral plausibility. This paper presents E2E-CDiff, an end-to-end conditional diffusion framework for controllable and realistic scenario generation. Conditioned on front-view visual observations, E2E-CDiff jointly denoises future motion states and executable low-level controls for route-interacting background vehicles. This unified state-action generation mitigates the planning-control mismatch in conventional two-stage trajectory-then-controller pipelines. Differentiable guidance further regulates speed, enforces drivable-area compliance, and supports collision-avoidance or collision-seeking behaviors, enabling both naturalistic and safety-critical scenario generation. Experiments on Bench2Drive show that E2E-CDiff achieves a favorable controllability-realism trade-off compared with representative reinforcement- and imitation-learning baselines, while its collision-guided variant induces challenging interactions across multiple autonomous driving systems. E2E-CDiff also performs competitively as a learning-based ego planner, demonstrating the generality of end-to-end state-action diffusion.
Jingzheng Li, Yufei Ge, Zhijun Chen +8
Zhongguancun Laboratory, Beijing 100094, China · Tianjin University, Tianjin 300073, China · Beihang University, Beijing 100091, China +4
Learning-based motion planners, despite recent progress, often suffer from temporal inconsistency. Small perturbations across frames can accumulate into unstable trajectories, degrading comfort and safety in closed-loop driving. Several methods attempt to inject history as a static conditioning signal to stabilize outputs, only to induce the planner to copy historical patterns instead of adapting to environment contexts. To address this limitation, we propose Diffusion Forcing Planner (DFP), a diffusion-based planning framework driven by history-guided control. Specifically, DFP decomposes the full trajectory into history, current and future segments, and assign independent noise levels to each segment. The model jointly denoises the historical and the future segments, enforcing a heterogeneous joint diffusion process. At inference, classifier-free guidance (CFG) is applied to steer future sampling using annealed history in a controllable manner. Closed-loop evaluation and comprehensive ablations on nuPlan show that DFP achieves competitive performance while producing continuous, stable, and controllable motion plans in complex driving scenarios.
Zehan Zhang, Neng Zhang, Yaoyi Li +2
University of Science and Technology of China · 3Hefei Institutes of Physical Science, Chinese Academy of Sciences · 2Yinwang Intelligent Technology Co., Ltd