Most end-to-end autonomous-driving systems learn by imitating human driving logs, leaving their learned behavior constrained by the quality and behavioral coverage of the recorded trajectories. This report presents DriveZero, an end-to-end system that learns driving behavior beyond human demonstrations. It decomposes driving into a perception model and an action model, pretrains each in the regime best suited to it, and combines them into one end-to-end planner. The two models call for different learning recipes: perception must understand the world, and benefits from massive and diverse visual data; action must interact with it, and requires closed-loop feedback. On the action side, we introduce DriveRL, a mixed-agent closed-loop reinforcement-learning framework. It converts real driving logs into interactive worlds, where a privileged teacher policy is trained with PPO through closed-loop rollouts. For the perception model, DriveVFM consolidates multiple frozen vision foundation models, including DINOv3, SigLIP2, SAM and Depth Anything V2, into a single backbone from raw images alone, requiring no task-specific annotations. DriveZero then unifies the two: a camera-only planner that distills the frozen DriveRL teacher through its rolled-out trajectories. The goal-conditioned teacher can moreover be queried under augmented driving intents, yielding diverse, goal-consistent supervision that logged data cannot provide. On nuPlan, DriveRL with value-guided test-time action search achieves a mean score of 93.57 across the Val14, Test14-hard, and Test14-random community splits in both non-reactive and reactive modes, exceeding the Log-Replay expert on all three splits. DriveZero achieves state-of-the-art performance on NAVSIMv1, NAVSIMv2 and the closed-loop HUGSIM benchmark without any human trajectory supervision.
Figures & tables
Figure 1: Overview of our end-to-end driving system. (1) Decoupled pretraining. The action model, DriveRL, is a privileged policy trained from scratch with closed-loop RL in mixed-agent interactive worlds built from real nuPlan logs. At inference, a value-guided test-time search rolls out several sampled actions, scores them using short-horizon rewards and the critic, and conditionally replaces the modal action with a higher-value candidate. The perception model, DriveVFM, distills frozen vision foundation models into a single driving backbone from raw images, without task-specific labels. (2) Unification by distillation. DriveZero initializes its encoder from DriveVFM and learns from DriveRL rollouts instead of human trajectories, yielding a camera-only end-to-end planner. (3) Performance. DriveRL exceeds the log-replay expert and prior methods on all six nuPlan closed-loop settings; DriveZero surpasses the human driver on NAVSIMv1 and sets the state of the art on NAVSIMv2 and HUGSIM without any human trajectory supervision.
Figure 2: Overview of the end-to-end student policy distillation framework. The privileged DriveRL teacher rolls out a 20-step trajectory from structured scene observations. The camera-only student DriveZero uses DriveVFM features and a command-conditioned Transformer planner to generate multiple 20-step trajectory proposals. Winner-takes-all (WTA) trajectory supervision transfers the teacher behavior, while the proposal-scoring branch is trained against PDM targets. The teacher is used only during training and is removed at inference time.
Component
Teacher configuration
Token width / attention heads
256 / 4
Ego-to-agent / ego-to-map attention
2 / 1 layers
Agent history
5 frames at 5 Hz
Agent / map capacity
96 / 256 tokens
Continuous action
Longitudinal jerk and tire steering-angle rate
Normalized action support
(0,1)2
Table 1: DriveRL training configuration. Core architecture and control settings. Complete implementation details are provided in Appendix.
Method
Human
Val14 ↑
Test14-hard ↑
Test14-random ↑
NR
R
NR
R
NR
R
Log-Replay
✗
93.53
80.32
85.96
68.80
94.03
75.86
IDM [ 77 ]
✗
75.60
77.33
56.15
62.26
70.39
74.42
PDM-Closed [ 17 ]
✓
92.84
92.12
65.08
75.19
90.05
91.63
UrbanDriver [ 65 ]
✓
68.57
64.11
50.40
49.95
51.83
67.15
PlanTF [ 8 ]
✓
84.27
76.95
69.70
61.61
85.62
79.58
Table 2: Closed-Loop Performance on the nuPlan [ 2 ] Benchmark. “Human” indicates whether human demonstrations are used as training supervision. DriveRL-TTS applies value-guided test-time action search to the same checkpoint as DriveRL. Best scores are bolded ; second-best scores are underlined .
Setting
Val14 ↑
Test14-hard ↑
Test14-random ↑
Mean ↑
Δ↑
NR
R
NR
R
NR
R
DriveRL
95.16
94.25
89.97
89.18
94.50
95.00
93.01
0.00
DriveRL-TTS ( N=8 )
95.17
94.49
89.97
89.05
94.92
95.11
93.12
+0.11
DriveRL-TTS ( N=16 )
95.22
94.38
89.98
88.97
95.29
94.98
93.13
+0.13
DriveRL-TTS ( N=32 )
95.49
94.43
90.95
88.85
95.83
94.84
93.40
+0.39
DriveRL-TTS ( N=64 )
95.54
94.53
91.13
89.15
95.93
95.12
93.57
+0.56
Table 3: Test-time Scaling of a Fixed DriveRL Checkpoint . DriveRL directly executes the Beta mode, whereas DriveRL-TTS evaluates policy-supported action candidates using a five-step rollout and a switching margin. Mean is the unweighted average over the six evaluations, and Δ is measured relative to DriveRL.
Method
Input
Human
NC ↑
DAC ↑
TTC ↑
C ↑
EP ↑
PDMS ↑
PDM-Closed
GT
✓
94.6
99.8
89.9
86.9
99.9
89.1
Human Driver
GT
✓
100
100
100
99.9
87.5
94.8
TransFuser [ 9 ]
C & L
✓
97.7
92.8
92.8
100
79.2
84.0
DRAMA [ 95 ]
C & L
✓
98.0
93.1
94.8
100
80.1
85.5
VAD-v2 [ 31 ]
C & L
✓
98.1
94.8
94.3
100
80.6
86.2
DiffusionDrive [ 45 ]
C & L
✓
98.2
96.2
94.7
100
82.2
88.1
Table 4: Pseudo Closed-Loop Performance on the NAVSIMv1 navtest [ 18 ] benchmark. Input: GT = ground-truth symbolic inputs; C = camera; L = LiDAR. “Human” indicates whether human demonstrations are used as training supervision. “-Scale” denotes scaling up the training set with simulation data from SimScale [ 76 ] . Best scores are bolded ; second-best scores are underlined .
Method
Human
Stage
NC ↑
DAC ↑
DDC ↑
TLC ↑
EP ↑
TTC ↑
LK ↑
HC ↑
EC ↑
S. ↑
EPDMS ↑
S1
94.4
98.8
100
99.5
100
93.5
99.3
87.7
36.0
-
PDM-Closed ∗ [ 17 ]
✓
S2
90.5
90.6
95.4
98.4
100
86.6
74.2
91.9
29.7
-
56.6
S1
96.2
79.6
99.1
99.6
84.1
95.1
94.2
97.6
79.1
-
LTF [ 9 ]
✓
S2
77.8
70.2
84.3
98.1
85.1
75.7
45.4
95.7
76.0
-
25.1
S1
96.2
92.4
95.7
99.6
83.8
96.0
94.7
96.4
60.9
-
NavFormer [ 39 ]
✓
S2
85.7
81.0
83.5
97.6
90.1
82.4
48.2
94.9
48.4
-
34.1
Table 5: Pseudo Closed-Loop Performance on the NAVSIMv2 navhard [ 3 ] Benchmark. ∗ marks methods using ground-truth symbolic inputs; all others use camera inputs. “Human” indicates whether human demonstrations are used as training supervision, and “S.” indicates per-stage EPDM score. “-Scale” denotes scaling up the training set with simulation data from SimScale [ 76 ] . Best scores are bolded ; second-best scores are underlined .
Method
Human
RC ↑
HD-Score ↑
E
M
H
X
Avg.
E
M
H
X
Avg.
LTF [ 9 ]
✓
67.8
35.1
26.2
40.5
38.9
58.9
18.0
9.8
25.9
23.7
VAD [ 32 ]
✓
51.3
31.1
25.3
26.5
31.4
36.3
9.5
8.0
11.5
13.4
UniAD [ 28 ]
✓
78.4
60.5
33.6
17.8
45.9
64.9
45.8
20.6
6.6
32.7
DrivoR-Scale [ 34 ]
✓
70.4
54.4
28.2
39.2
46.4
64.8
50.3
16.8
25.8
38.1
GigaPixel [ 62 ]
✗
78.6
60.7
32.7
35.4
50.1
67.4
51.9
19.1
21.6
38.5
Table 6: True Closed-Loop Performance on the HUGSIM [ 103 ] Benchmark. Zero-shot generalization using the DriveZero model from the NAVSIMv1 evaluation. “Human” indicates whether human demonstrations are used as training supervision. “-Scale” denotes scaling up the training set with simulation data from SimScale [ 76 ] . Best scores are bolded ; second-best scores are underlined .
Study
Configuration
NC ↑
DAC ↑
EP ↑
TTC ↑
Comfort ↑
PDMS ↑
Visual Backbone
DA3 Small [ 47 ]
99.10
98.84
89.68
96.21
99.93
93.31
DA2 ViT-S [ 90 ]
98.57
99.21
91.44
95.13
99.96
93.72
DINOv2 ViT-S [ 57 ]
98.86
99.07
91.63
95.46
99.98
93.88
DINOv3 ViT-S [ 71 ]
98.93
99.01
91.31
95.77
99.97
93.88
DriveVFM ViT-S
99.17
99.14
91.79
96.29
99.98
94.41
DriveVFM Teachers
DINOv3 + SigLIP2 [ 71 , 78 ]
98.67
98.98
91.55
95.27
99.98
93.69
Table 7: DriveVFM and DriveZero ablations on the NAVSIMv1 navtest [ 18 ] benchmark.
Figure 3: Feature Representation Visualization. From left to right: input images, PCA visualizations of DINOv3 and DriveVFM patch features, and their cosine-similarity heatmaps relative to the ground prototype. Blue indicates higher similarity to the ground and red indicates lower similarity. Boxes mark the long-tail obstacles of interest.
Figure 4: Visualization of DriveZero-Scale vs. Human Driver in NAVSIM navtest [ 18 ] Benchmark.
Figure 5: Visualization of DriveZero-Scale vs. Previous SOTA Model in NAVSIM navtest [ 18 ] Benchmark.
Figure 6: Visualization of DriveZero-Scale vs. Previous SOTA Model in the HUGSIM [ 103 ] Benchmark.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Value
Setting
Value
World initialization
11 frames / 2 s
Rollout horizon
110 steps at 5 Hz
Worlds per rank
2,048
Traffic quota / rank
1,024 reactive + 1,024 replay
Distributed scale
12 nodes / 96 GPUs
Global rollout batch
196,608 worlds
Nominal transitions / rank
225,280
Optimizer minibatch / rank
20,480
Minibatches / epoch
11
Optimization epochs
4
Optimizer steps / update
44
Training updates
2,400
Appendix
Table A1 : Training and rollout configuration. Runtime and optimization settings for DriveRL teacher.
Input
Per-item shape
History or capacity
Frame
Goal anchors
2-D point
2 anchors per policy input
Ego
Ego state
9-D
Current frame
Ego
Actor history
15-D / frame
5 frames; ≤96 actors including ego
Ego
Vector-map segment
10-D / token
≤256 tokens, including ≤64 lane centers
Ego
Traffic-light state
4-D / segment
Attached to the corresponding map segment
Map segment
Appendix
Table A2 : Structured Teacher input schema. Listed capacities are maximum counts after policy filtering.
Aspect
PPO training
Formal nuPlan deployment
Source
Future-log positions
Current ego state + route polyline
Update
Once at reset; fixed per episode
Every planner step
Slots
One or two; duplicate a single goal
Two; near and far route goals
Snapping
50% logged / 50% lane projection
Causal route polyline
Order semantics
Shared encoder + mean pool
Same unordered-set semantics
Appendix
Table A3 : Training and deployment goal protocols. Goal sources, update schedules, and slot semantics used by the Teacher.
Parameter
Value / range
Unit
Longitudinal jerk
[−8.00,5.00]
m/s3
Tire steering rate
[−0.80,0.80]
rad/s
Positive jerk cap ( a≥0 )
1.00
m/s3
Wheelbase
3.09
m
Rear axle to center
1.46
m
Maximum acceleration
4.00
m/s2
Appendix
Table A4 : Action and vehicle-dynamics parameters. Physical command ranges and constants used by the kinematic bicycle model.
Split / mode
Score
Collision
Drivable
Direction
Making progress
TTC
Progress
Speed
Comfort
Val14 NR
95.16
98.75
100.00
99.60
98.93
95.97
96.30
98.80
98.84
Val14 R
94.25
98.57
100.00
99.24
98.75
95.80
93.83
99.21
98.66
Test14-hard NR
89.97
98.90
98.90
98.16
98.16
93.38
91.74
97.67
95.96
Test14-hard R
89.18
97.43
99.26
98.53
96.69
94.49
88.73
98.69
96.32
Test14-random NR
94.50
98.08
99.23
99.81
100.00
95.40
95.48
98.33
98.85
Test14-random R
95.00
98.85
99.62
100.00
99.62
96.55
93.34
98.85
98.85
Appendix
Table A5 : Detailed nuPlan metrics for DriveRL. Results across six evaluation settings; higher is better for every column.
Setting
Single-Ego Teacher
Self-Play Teacher
Policy-controlled actors
Ego only
Ego + ≤10 vehicle NPCs
Traffic composition
Log/IDM mixed50
Policy NPCs + log/IDM mixed50
Training samples
922,703
909,645 (subset of Single-Ego data)
Worlds per rank
2,048
512
Global rollout worlds
196,608
49,152
Appendix
Table A6 : Configuration differences between DriveRL and DriveRL-SelfPlay. The two policies use the same mixed-agent simulator and PPO objective.
Model
Val14 ↑
Test14-hard ↑
Test14-random ↑
Mean ↑
NR
R
NR
R
NR
R
DriveRL
95.16
94.25
89.97
89.18
94.50
95.00
93.01
Self-Play, 3-perturb + augmentation
95.08
94.48
88.30
87.27
94.60
94.68
92.40
Self-Play, 3-perturb
95.15
93.74
88.90
87.47
93.70
94.77
92.29
Self-Play, augmentation
94.91
94.38
88.56
87.92
93.43
94.22
92.24
Self-Play
95.09
93.92
90.04
88.96
95.04
95.63
93.11
Appendix
Table A7 : Self-Play Comparison on the Six Closed-Loop Evaluations. All Self-Play variants are trained for 4,800 updates; the Single-Ego baseline uses its reported 2,400-update checkpoint. The best checkpoint is reported for each run, and Mean is the unweighted average over the six evaluations.
TL input
Val14 ↑
Test14-hard ↑
Test14-random ↑
NR
R
NR
R
NR
R
Off
93.35
93.19
86.49
85.41
93.48
92.24
On
94.76
93.62
88.35
85.70
95.21
93.89
Difference
+1.41
+0.43
+1.86
+0.29
+1.73
+1.65
Appendix
Table A8 : Traffic-Light Input Ablation. The red-light detector is diagnostic only in both settings.
Objective
Policy-only violation
Val14 ↑
Test14-hard ↑
Test14-random ↑
NR
R
NR
R
NR
R
TL input only
3.88%
94.29
93.19
84.85
85.65
93.79
93.09
+ TL reward
2.03%
93.08
91.35
81.71
82.96
92.36
91.79
Difference
−1.85 pp
−1.21
−1.84
−3.14
−2.69
−1.43
−1.30
Appendix
Table A9 : Traffic-Light Reward and Violation-Termination Ablation. All runs use 32 GPUs; “pp” denotes percentage points.
Split / mode
Default [gnear,gfar]
Near [gnear,gnear]
Far [gfar,gfar]
Val14 NR
95.16
94.97
94.36
Val14 R
94.25
94.00
93.22
Test14-hard NR
89.97
90.74
88.01
Test14-hard R
89.18
88.82
87.13
Test14-random NR
94.50
94.57
93.18
Test14-random R
95.00
95.23
92.84
Appendix
Table A10 : Goal-Anchor Layout Evaluation for the Fixed Single-Ego Teacher. The highest score in each row is bold.
Training distribution
Evaluation
Val14 ↑
Test14-hard ↑
Test14-random ↑
Mean ↑
NR
R
NR
R
NR
R
Single / dual (1:1, default)
Near + far
94.76
93.62
88.35
85.70
95.21
93.89
91.92
Earlier of two duplicated
Near + near
94.23
93.35
86.78
87.38
94.73
93.23
91.62
Later of two duplicated
Far + far
93.04
92.43
83.10
82.31
93.27
90.95
89.18
Appendix
Table A11 : Goal-Anchor Training Distribution Ablation. Models are trained from scratch with 32-GPU Adam, and evaluation layouts are matched to the corresponding training distributions. Mean is the unweighted average over the six evaluations.
Hyperparameter
Value
Model Configuration
Sensors
CAM_F0 , CAM_B0 , CAM_L0 , CAM_R0
Input resolution
960×512 (W × H)
Prediction horizon / frequency
20 steps at 5Hz
Trajectory proposals
64
Planning representation
256 dimensions
Appendix
Table A12 : Model and Training Configuration for DriveZero-Scale.
Backbone
NC ↑
DAC ↑
TTC ↑
Comfort ↑
EP ↑
DDC ↑
PDMS ↑
DINOv3 ViT-L
98.80
99.25
95.74
99.94
92.72
95.87
94.55
DriveVFM ViT-S
99.17
99.14
96.29
99.98
91.79
95.97
94.41
DriveVFM ViT-B
98.89
99.24
95.93
99.97
92.44
95.79
94.54
DriveVFM ViT-L
99.00
99.24
95.97
99.99
93.10
95.89
94.83
Appendix
Table A13 : Visual-backbone scaling for DriveZero on the NAVSIMv1 navtest benchmark. The backbones use the same data sources and identical mixture ratios. All downstream DriveZero variants use NAVSIM navtrain without SimScale and fine-tune rank-32 Q/V LoRA adapters.
Figure 27
Figure A2 : More Visualization Results of DriveZero-Scale vs. Previous SOTA Model in NAVSIM navtest [ 18 ] Benchmark.
Human demonstrations are widely considered the cornerstone of end-to-end (E2E) autonomous driving despite human demonstration's scarcity for long-tail and safety-critical scenarios. Nonetheless, current E2E autonomous driving (AD) training paradigms continue to rely on human demonstrations. Imitation learning (IL) requires human demonstrations for training, whereas reinforcement learning (RL) has emerged as a promising alternative to reduce this dependency. However, most existing RL methods for E2E AD still rely implicitly on human demonstrations. A pure rewards-based RL method can overcome the need for human demonstrations, but general RL policy gradient methods suffer from the cold-start problem. In this paper, we propose ZTRS (Zero-human demonstration end-to-end autonomous driving with TRajectory Scorer) - a complete RL-based E2E planning paradigm trained solely on real-world images and rule-based rewards, entirely without human demonstration. Through our proposed Exhaustive Policy Optimization (EPO), a policy gradient variant tailored for enumerable trajectory actions and dense supervision, ZTRS enables the model to generalize better to long-tail driving scenarios. We demonstrate this generalization through our SOTA performance against IL approaches on both long-tail Navhard and closed-loop HUGSIM datasets. Project page: https://zhenxinli.net/ZTRS/.
Zhenxin Li, Nadine Chang, Wenhao Yao +9
Institute of Trustworthy Embodied AI, Fudan University · *Work done during an internship at NVIDIA. · NVIDIA +3
End-to-end autonomous driving has achieved state-of-the-art performance on benchmarks and real-world deployments. Its standard training recipe, however, is expensive across all stages: collecting and labeling millions of driving frames is costly, and closed-loop RL on images is bottlenecked by the per-step cost of photorealistic rendering plus a forward pass through a large vision backbone. Self-play in vectorized simulators changes the economics: millions of rollout steps per second, and a state distribution naturally rich in collisions, near-misses, and recoveries that no driving log contains. Our approach exploits this asymmetry by decoupling learning to drive from learning to see. We pretrain a single policy by self-play, then align its latent space with a pretrained vision backbone, through the action KL divergence and a batch-relational low-rank structural loss. The action target comes from the self-play policy, so alignment never supervises against a logged trajectory: a paired dataset of (image, scene-state) frames suffices, with no need for the curated expert demonstrations that imitation pretraining is built on. On photorealistic 3D Gaussian splatting closed-loop scenarios, the resulting end-to-end policy matches or exceeds prior end-to-end methods.
End-to-end autonomous driving (E2E-AD) aims to directly map raw sensor information to driving actions. Recently, with the rapid advancement of multi-modal large language models (MLLMs), researchers have proposed the paradigm of Vision-Language-Action (VLA) models for E2E-AD, where it seeks to integrate visual perception, language understanding and action prediction within a single policy. However, existing VLA-based policies largely adopts imitation learning, where it only learns to drive by optimizing distance-based metrics w.r.t. logged expert trajectories. Such distribution shift between open-loop training and closed-loop inference leads to suboptimal performance in closed-loop planning. To close this gap, we present CLEAR, a system that enables closed-loop training using Reinforcement Learning (RL) at scale for E2E-AD. We propose to learn a novel residual waypoint policy around the waypoint prior from pretrained VLA policies, effectively harnessing the knowledge within. On another front, one of the key challenges to scale up RL for vision-based policies is the number of parallel simulation environments since RL is data hungry. To that end, we design a heterogeneous pipeline that places the simulator and the VLA learner on distinct compute groups, which allows us to dramatically increase the number of simulation environments running in parallel while avoiding resource contention and maintaining training stability. We show that with a simple reward, CLEAR significantly outperforms previous methods and sets new state-of-the-art performance on the challenging benchmarks of CARLA longest6 v2 and Bench2Drive.