Organizations: School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore · Collaborative Initiative, Interdisciplinary Graduate Programme, Nanyang Technological University, Singapore · College of Computing and Data Science, Nanyang Technological University, Singapore
End-to-end autonomous driving has achieved remarkable progress by planning trajectories directly from raw sensor data. Yet the step from planning to control still relies on hand-tuned tracking controllers that block the gradient path from vehicle motion back to the network. This gap stems from non-differentiable vehicle dynamics, aliased supervision from coarsely sampled expert labels, rate mismatch between high-rate control and low-rate re-planning, and vehicle model discrepancies between training and deployment. This paper proposes DIVA (Differentiable Identified Vehicle dynamics for end-to-end Autonomous driving) to close this gap. An identified actuator-level vehicle model is embedded in the training loop of the end-to-end driving network, forming a differentiable chain from perception to vehicle states. A learning-based control decoder replaces the external controller and is supervised on the rolled-out vehicle states, bypassing the aliased control-input labels. A rate-consistent control-input parameterization reconciles the heterogeneous sampling rates and enables asynchronous execution with a 3.8 ms control-only cycle. On-policy re-identification and an online disturbance observer address training-time distribution shift and deployment-time model mismatch respectively. On the closed-loop Bench2Drive benchmark, DIVA achieves a driving score of 86.7, surpassing the widely adopted state-of-the-art approach by 3.3 points while nearly halving mean jerk and reducing brake switching by 64%, and maintains superior path tracking accuracy under significant vehicle model mismatch at deployment.
Figures & tables
Fig. 1: Overview of DIVA: the control decoder cross-attends to memory tokens that combine the scene and planning features, the explicit route and target-speed reference, the state feedback and the disturbance estimate, and its control-input knots pass through a first-order hold into the identified differentiable vehicle dynamics, whose longitudinal and lateral actuator channels drive a single-track model integrated at 20 Hz. Under imitation learning in state space, the state loss propagates along the gradient path back through the vehicle dynamics into the control and planning decoders, and at deployment the control decoder runs on a fast path over cached perception, executing receding-horizon control inputs conditioned on the disturbance estimate of the online observer.
Fig. 2: On-policy re-identification: (a) coverage of the identification excitations (gray) and of the base controller (red, 50% and 85% contours); (b) map at throttle 0.7; (c) 141 on-policy upshifts; (d) 49 on-policy launches; (e) speed bias of both models under the controller’s commands; (f) position error on held-out drives; kinematic bicycle: the constant-parameter model of [ 31 ] shipped with TF++.
Official metrics
Ability (%) ↑
Execution metrics ↓
Method
Planner
DS ↑
SR (%) ↑
Effi. ↑
Comf. ↑
Over- taking
Merg- ing
Emerg. brake
Give way
Traffic signs
Jerk (m/s 3 )
∣ax∣ (m/s 2 )
WOT frac.
Brake sw. (1/min)
Steering rate (1/s)
Lat. off. p95 (m)
Speed err. (m/s)
Speed (m/s)
TF++ [ 30 ]
baseline
83.34
67.6
250.97
33.9
55.56
58.75
78.33
50.00
86.84
16.42
4.54
0.823
42.7
0.173
0.159
3.36
8.67
MPC+DOB
baseline
79.59
61.2
259.22
38.5
37.78
53.75
76.67
50.00
82.63
10.92
3.74
0.113
15.8
0.097
0.220
3.33
8.67
DIVA+PID
joint
83.80
67.1
248.31
31.4
60.00
56.25
78.33
50.00
82.63
16.72
4.57
0.821
42.8
0.169
0.126
3.04
8.72
DIVA (ours)
joint
86.73
71.7
256.29
36.6
55.56
65.82
90.00
40.00
87.89
8.86
3.19
0.203
15.4
0.098
0.100
2.96
8.63
TABLE I: Closed-Loop Driving and Execution Results on Bench2Drive-220
Fig. 3: Execution metrics on Bench2Drive-220: (a) jerk RMS and deceleration p99, (b) lateral acceleration and mean sideslip angle, (c) comfort-limit exceedance fraction for ax and jerk, (d) throttle total variation and steering-rate. TF++ and MPC+DOB: baseline planner; DIVA+PID and DIVA: joint planner.
Closed loop
Offline, 2 s horizon
WOT
Lat.
Brake
Speed
Lat.
us,0
Δ DS ↑
SR ↑
Jerk ↓
frac.
offset ↓
sw. ↓
err. ↓
p90 ↓
Speed ↓
error
Variant
DS ↑
[95% CI]
(%)
RC ↑
(m/s 3 )
↓
p95 (m)
(1/min)
(m/s)
(m)
(m/s)
↓
DIVA (ours)
69.3
–
36.5
91.69
10.84
0.21
0.109
17.8
4.65
0.306
1.110
0.0145
DIVA+PID
60.1
−9.27 [ −16.6 , −2.0 ] †
17.3
90.78
15.87
0.82
0.143
33.5
4.64
–
–
–
w/o vehicle model, control-input BC
62.0
−7.36 [ −14.3 , −0.5 ] †
21.2
91.78
12.78
0.11
0.159
24.2
4.54
0.543
1.345
0.0094
TABLE II: Ablation on Hard-52: Driving Score, Closed-Loop and Offline Metrics
Fig. 4: Lateral offset (p95, top) and mean jerk (bottom) under sustained vehicle-side mismatch on the traffic-free maneuver suite (16 routes). Columns: perturbed quantity with nominal at factor 1, bias 0 or dead time 0. All controllers on the joint planner.
Fig. 5: DS and tracking under vehicle-side mismatch, all controllers on the joint planner. (a) Maneuver suite: DS (top) and speed error (bottom) vs. perturbed quantity. (b) Hard-52 with traffic: DS, lateral offset p95, speed error and jerk for the nominal vehicle (shaded) and one strength of four perturbation families.
Fig. 6: Steering bias +0.10 on suite route 2084 (junction left turn at night). (a) Paths of the five controllers. (b) Front camera of DIVA+PID at 5.8 s, 0.5 s before its collision. (c) Observer estimate b^s against the injected bias. (d) Steering command; dotted: exact compensation cs=−bs . (e) Lateral offset from the lane center; dotted: lane edge. (f) Speed. DIVA+PID and DIVA w/o observer use no estimate.
Fig. 7: DS (a) and lateral offset p95 (b) on Hard-52 at reduced perception rates with 0.1 s latency; at 20 Hz, the synchronous runs without latency, where hold and fast path coincide. Hold: command held at the perception rate; fast path: asynchronous DIVA; DIVA+PID on held path: its PID at 20 Hz on the last perception result transformed to the current pose.
Fig. 8: Case studies on six routes where all controllers score 100; TF++ and MPC+DOB run on the baseline planner. (a) T-junction stop and turn. (b) Hard-braking lead vehicle. (c) Control-loss disturbance. (d) Pedestrian stepping out. (e) Parking exit into traffic. (f) Vehicle cut-in. Bottom row: front camera of DIVA on each route.
Configuration
Mean (ms)
p95 (ms)
Rate (Hz)
DIVA+PID
17.28
17.97
58
DOB-PID
18.08
20.60
55
MPC+DOB, CPU
45.08
45.92
22
MPC+DOB, GPU
81.47
83.79
12
DIVA, synchronous
21.30
23.93
47
DIVA, async., 20 Hz perception
18.44
19.19
54
TABLE III: Latency of the Complete Control Cycle
Fig. 9: Identification experiments: measured vehicle against model roll-outs under the same commands. (a) Acceleration map; (b) launch from rest; (c) full-brake step; (d) steering slew-rate limiter; (e) yaw-rate steps up to a lateral acceleration ay≈6 m/s 2 ; (f) steady-state yaw-rate gain in all 86 step-steer tests (dots) and of the model at 5, 12 and 20 m/s (lines; shaded: benchmark range).
Symbol
Quantity
Value
Specification
m
Mass
1696 kg
lf , lr
Center of mass to front, rear axle
1.172, 1.689 m
W
Track width
1.593 m
xo
Center of mass ahead of reference point
0.30 m
δmax(vx)
At 0, 20, 60, 120 km/h
70, 63, 56, 49 ∘
TABLE IV: Parameters of the Identified Vehicle Model
Item
Value
Token width, embedding MLP width
256, 128
Planning decoder: heads, FFN, GRU width
8, 2048, 64
Control decoder: heads, FFN, dropout
8, 1024, 0.1
wy , wx , wψ , wv , wR
1.0, 0.5, 2.0, 0.5, 0.5
wa , wh , wp
0.5, 0.2, 1.0
Anchor Ωj : decay per knot, factor at standstill
0.8, 4
TABLE V: Implementation Details and Hyperparameters
Most end-to-end autonomous driving methods rely on imitation learning from single expert demonstrations, often leading to conservative and homogeneous behaviors that limit generalization in complex real-world scenarios. In this work, we propose DIVER, an end-to-end driving framework that integrates reinforcement learning with diffusion-based generation to produce diverse and feasible trajectories. At the core of DIVER lies a reinforced diffusion-based generation mechanism. First, the model conditions on map elements and surrounding agents to generate multiple reference trajectories from a single ground-truth trajectory, alleviating the limitations of imitation learning that arise from relying solely on single expert demonstrations. Second, reinforcement learning is employed to guide the diffusion process, where reward-based supervision enforces safety and diversity constraints on the generated trajectories, thereby enhancing their practicality and generalization capability. Furthermore, to address the limitations of L2-based open-loop metrics in capturing trajectory diversity, we propose a novel Diversity metric to evaluate the diversity of multi-mode predictions.Extensive experiments on the closed-loop NAVSIM and Bench2Drive benchmarks, as well as the open-loop nuScenes dataset, demonstrate that DIVER significantly improves trajectory diversity, effectively addressing the mode collapse problem inherent in imitation learning.
Ziying Song, Lin Liu, Hongyu Pan +7
Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, School of Computer Science and Technology, Beijing Jiaotong University, Beijing, China · School of Artificial Intelligence (School of Software), Yanshan University, Qinhuangdao, China · Horizon Robotics +3
We present a flow-matching planner for autonomous driving that directly outputs actionable control trajectories defined by acceleration and curvature profiles. The model is conditioned on a bird's-eye-view (BEV) raster of the surrounding scene and generates control sequences in a small number of Ordinary Differential Equations (ODE) integration steps, enabling low-latency inference suitable for real-time closed-loop re-planning. We train exclusively on urban scenarios (real urban city streets, intersections and roundabouts of the city of Parma, Italy) collected from a 2D traffic simulator with reactive agents, and evaluate in closed-loop on both in-distribution and markedly out-of-distribution environments, including multi-lane highways and unseen urban scenarios. Our results show that the model generalizes reliably to these unseen conditions, maintaining stable closed-loop control and successfully completing scenarios that differ substantially from the training distribution. We attribute this to the BEV representation, which provides a geometry-centric view of the scene that is inherently less sensitive to distributional shifts, and to the flow-matching formulation, which learns a smooth vector field that degrades gracefully under distribution shift. We provide video demonstrations of closed-loop behavior at https://marcelloceresini.github.io/DirectControlFlowMatching.
Marcello Ceresini, Federico Pirazzoli, Andrea Bertogalli +5
1Università degli Studi di Parma · 2Alma Mater Studiorum - Università di Bologna · 3VisLab (an Ambarella Inc. company)
Diffusion models have become a popular choice for decision-making tasks in robotics, and more recently, are also being considered for solving autonomous driving tasks. However, their applications and evaluations in autonomous driving remain limited to simulation-based or laboratory settings. The full strength of diffusion models for large-scale, complex real-world settings, such as End-to-End Autonomous Driving (E2E AD), remains underexplored. In this study, we conducted a systematic and large-scale investigation to unleash the potential of the diffusion models as planners for E2E AD, based on a tremendous amount of real-vehicle data and road testing. Through comprehensive and carefully controlled studies, we identify key insights into the diffusion loss space, trajectory representation, and data scaling that significantly impact E2E planning performance. Moreover, we also provide an effective reinforcement learning post-training strategy to further enhance the safety and robustness of the learned planner. The resulting diffusion-based learning framework, Hyper Diffusion Planner (HDP), is deployed on a real-vehicle platform and evaluated across 6 urban driving scenarios and 200 km of real-world testing, achieving a notable 10x performance improvement over the base model. Our work demonstrates that diffusion models, when properly designed and trained, can serve as effective and scalable E2E AD planners for complex, real-world autonomous driving tasks.
Yinan Zheng, Tianyi Tan, Bin Huang +11
Institute for AI Industry Research (AIR), Tsinghua University · Xiaomi EV