Diffusion models have become a popular choice for decision-making tasks in robotics, and more recently, are also being considered for solving autonomous driving tasks. However, their applications and evaluations in autonomous driving remain limited to simulation-based or laboratory settings. The full strength of diffusion models for large-scale, complex real-world settings, such as End-to-End Autonomous Driving (E2E AD), remains underexplored. In this study, we conducted a systematic and large-scale investigation to unleash the potential of the diffusion models as planners for E2E AD, based on a tremendous amount of real-vehicle data and road testing. Through comprehensive and carefully controlled studies, we identify key insights into the diffusion loss space, trajectory representation, and data scaling that significantly impact E2E planning performance. Moreover, we also provide an effective reinforcement learning post-training strategy to further enhance the safety and robustness of the learned planner. The resulting diffusion-based learning framework, Hyper Diffusion Planner (HDP), is deployed on a real-vehicle platform and evaluated across 6 urban driving scenarios and 200 km of real-world testing, achieving a notable 10x performance improvement over the base model. Our work demonstrates that diffusion models, when properly designed and trained, can serve as effective and scalable E2E AD planners for complex, real-world autonomous driving tasks.
Figures & tables
Figure 1: Simulation-based benchmarks differ substantially from real-world deployment, leaving the true capability of diffusion-based planners largely unverified. Through a detailed investigation of the IL/RL training recipe, we unleash the potential of diffusion-based planning, achieving a 10x improvement in real world.
Figure 2: Model architecture of the base model.
Table 3
Figure 5: Comparison of prediction quality with waypoint and velocity representations.
Figure 6: Comparison of waypoint, velocity representations and velocity with hybrid loss, which yields both steadiness and smoothness.
Figure 6
Figure 9: Data scaling experiments. Both open- and closed-loop performance improve as data scales up.
Model Name
Data Size
Open-Loop Score
Closed-Loop Score
Success Rate
Stability Score
Overall Score
Base Model
M
51.07
15.67
0.00
7.83
with τ0 -loss & τ0 -pred
M
75.27
22.84
0.00
11.42
+ Velocity Supervision
M
84.38
34.72
9.24
21.98
+ Hybrid Loss
M
85.05
61.88
53.88
57.88
+ Data Scaling
L
86.07
70.59
59.00
64.79
Table 2: Main results. a represents the highest score. The open-loop score is evaluated through data replay on test datasets, while the closed-loop score is obtained from real-world road testing on a real-vehicle platform.
Figure 10: (a) Relative success rate vs. dataset volume in frequently-occurring scenarios; (b) stability score under the same scaling; (c) relative success rate before and after RL fine-tuning in safety-critical scenarios.
Figure 11: Closed-loop real-vehicle testing, illustrated with two representative frames.
Figure 12: Replay of bad cases before and after post-training. HDP-RL † in blue , HDP in red .
Figure 13: Reward versus training steps: HDP-RL with vs. without the RL-hybrid loss in Eq. ( 9 ). The subgraph is the image after rescaling the coordinates.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 14: Closed-loop real-vehicle testing results. Each row contains representative frames from the scenario.
τ0 -pred.
vt -pred.
ϵ -pred.
τ0 -loss: E∥τ^0−τ0∥2
τ^0=τθ
τ^0=αtτt−σtvθ;t
τ^0=(τt−σtϵθ)/αt
vt -loss: E∥v^t−vt∥2
v^t=(αtτt−τθ)/σt
v^t=vθ;t
v^t=(ϵθ−σtτt)/αt
ϵ -loss: E∥ϵ^−ϵ∥2
ϵ^=(τt−αtτθ)/σt
ϵ^=σtτt+αtvθ;t
ϵ^=ϵθ
Appendix
Table 3: Mutual conversions among the three diffusion quantities. Columns index the model parameterization and rows index the loss space. Predicted quantities are denoted with ⋅^ and the network is parameterized by θ . Diagonal entries are the canonical objectives; off-diagonal entries first transform the network output into the target space and then compute the squared error.
Figure 15
Type
Parameter
Symbol
Value
IL
Num. block
-
6
Dim. hidden layer
-
256
Num. multi-head
-
8
Hybrid loss weight
ω
0.1
RL
Group size
-
32
Temperature
β
1.0
Appendix
Table 6: Hyperparameters of HDP / HDP-RL
Model Name
Data Size
Open-Loop Score
Closed-Loop Score
Success Rate
Stability Score
Overall Score
Regression Model
XL
73.98
21.81
0.00
10.91
Base Model
M
51.07
15.67
0.00
7.83
with τ0 -loss & τ0 -pred
M
75.27
22.84
0.00
11.42
HDP
XL
88.94
71.24
79.53
75.38
HDP-RL
-
-
83.49
84.65
84.07
Appendix
Table 7: Compare the open-loop and closed-loop scores with baselines.
Figure 16: Visualization of anchor-based truncated diffusion using (a) offset prediction and (b) direct prediction. The ground truth is in red and model predictions in purple. The deeper the trajectory is, the higher the predicted score.
Model Name
Closed-Loop Score
Success Rate
Stability Score
Overall Score
HDP-RL † (only safety reward)
72.89
79.53
76.20
DPPO-style methods (multi-rewards)
65.50
89.76
77.63
HDP-RL (multi-rewards)
83.49
84.65
84.07
Appendix
Table 8: Compare the closed-loop scores with DPPO-style RL methods ( Black et al., 2024b ; Ren et al., 2025 ) .
Diffusion-based planning generates diverse action sequences or trajectories by progressively denoising candidates conditioned on goals, scenes, and constraints.
College of Automotive and Energy Engineering, Tongji University, Shanghai, 201804, China · College of Transportation Engineering, Tongji University, Shanghai, 201804, China · School of Mechanical and Aerospace Engineering, Nanyang Technological University, 639798, Singapore