DriveMLP: When Training Strategy Outperforms Model Complexity in Autonomous Driving

Alessandro Canevaro; Julian Schmidt; Hang Yu; Ye Jin; Sajad Marvi; Georg Martius; Julian Jordan

cs.LGOct 21, 2026

DriveMLP: When Training Strategy Outperforms Model Complexity in Autonomous Driving

Authors: Alessandro Canevaro, Julian Schmidt, Hang Yu, Ye Jin, Sajad Marvi, Georg Martius, Julian Jordan

Organizations: Mercedes-Benz AG, Research & Development, Sindelfingen, Germany · University of Tübingen, Tübingen, Germany · Karlsruhe Institute of Technology, ITIV, Karlsruhe, Germany · Technical University of Munich, Munich, Germany ·

Abstract

Learning motion planning policies that remain reliable under closed-loop execution is a central challenge in Autonomous Driving (AD). Imitation Learning (IL) scales well to large AD datasets but suffers from distribution shift, causal confusion, and compounding errors under closed-loop execution. In contrast, Reinforcement Learning (RL) typically requires handcrafted rule-based rewards and massive simulation. We study a simple closed-loop IL framework that uses RL purely as an optimization tool, guided by a trajectory-distance reward derived from logged expert data. We refer to this training strategy as Expert-Guided RL (EGRL). Through a controlled empirical study we find evidence that the training procedure, rather than model size and complexity, accounts for a substantial part of the performance gains. We then introduce DriveMLP, a lightweight MLP policy trained with EGRL, as a simple instantiation of these findings. It achieves a score of 86.51 on the nuPlan Val14 Reactive benchmark, a +3.2 point improvement over the previous best IL planner (Flow Planner), with >15x faster inference. Code is available at: https://github.com/alessandro-canevaro/DriveMLP .

Figures & tables

Figure 1: DriveMLP combines strong reactive performance with low inference cost. The plot shows nuPlan Val14-Reactive scores against inference frequency, with circle size indicating parameter count. DriveMLP, a lightweight MLP trained with EGRL, achieves higher scores and faster inference than the compared SOTA planners.

Figure 2: Working principle of the non-reactive simulator. The teal rectangles represent the logged expert ego trajectory. The gray rectangles represent other agents or obstacles moving according to recorded data, while the dark blue rectangle shows the ego rollout following the actions of the learned policy. Orange arrows show the position distance between the ground truth and the ego rollout which is then used in the reward computation.

	Highway-env			Val14		Test14-Hard		Test14
Method	Reward $\uparrow$	Ep. Len $\uparrow$	Term. $\downarrow$	NR	R	NR	R	NR	R
BC	17.20 $\pm$ 3.26	23.56 $\pm$ 4.28	48.7% $\pm$ 21.48	53.64	32.36	17.06	19.50	30.36	33.01
AIRL	20.37 $\pm$ 0.40	28.48 $\pm$ 0.46	9.3% $\pm$ 2.87	51.74	58.74	43.00	52.09	48.88	61.15
BC-RL	20.28 $\pm$ 0.92	28.70 $\pm$ 1.15	20.0% $\pm$ 17.38	77.19	78.05	66.05	65.35	77.17	77.23
EGRL	21.02 $\pm$ 0.79	29.17 $\pm$ 1.01	7.3% $\pm$ 3.55	90.38	86.51	76.68	74.50	86.38	87.06

Table 1: Controlled training strategy comparison on Highway-Env (left) and nuPlan (right). Highway-Env results averaged over 3 seeds. NR = Non-Reactive, R = Reactive. All training methods use the same DriveMLP policy architecture.

Model	Original (Val14-R)	+EGRL	$\Delta\%$	Params
PlanTF	76.95	78.26	+1.70	2.0M
PLUTO w/o ref.	78.11	83.41	+6.78	2.4M
Diff. Planner*	82.80	86.05	+3.92	2.3M

Table 2: EGRL training strategy applied to other SOTA backbones. *Flow Planner uses a similar encoder structure as Diffusion Planner.

	Val14		Test14-Hard		Test14
Method	NR	R	NR	R	NR	R
PlanTF	84.27	76.95	69.70	61.61	85.62	79.58
PLUTO w/o refine	88.89	78.11	70.03	59.74	89.90	78.62
Diffusion planner	89.87	82.80	75.99	69.22	89.19	82.93
Flow Planner	90.43	83.31	76.47	70.42	89.88	82.93
DriveMLP	90.38	86.51	76.68	74.50	86.38	87.06

Table 3: Overall scores on nuPlan benchmark. NR = Non-Reactive, R = Reactive simulation.

Reward	NR	R
Base	90.38	86.51
w/o speed reward	87.44	84.92
w/o exp.	84.87	82.74
w/o normalization	50.49	47.58

Table 4: Exponential reward ablation: NuPlan Val14.

Panel of Figure 3 — Figure 3: Training dynamics and validation performance over training steps. Left: Episode return mean. Right: Val14 R Score. Horizontal dashed lines indicate the BC, AIRL, and BC-RL baselines.

Figure 4: NuPlan score on Val14 Non-Reactive (solid bars) and Reactive (striped bars) benchmark for DriveMLP (light blue) and Diffusion Planner (dark blue) with Gaussian noise on acceleration (

\sigma_{\text{acc}}

) and on steering (

\mu_{\text{steer}}

\sigma_{\text{steer}}

Figure 5: NuPlan score on Val14 Non-Reactive (solid bars) and Reactive (striped bars) benchmark for DriveMLP (light blue) and Diffusion Planner (dark blue) with reduced training dataset sizes.

Method	Time (ms) $\downarrow$	Params (M) $\downarrow$	Val14-R $\uparrow$
PlanTF	8.30	2.02	76.95
PLUTO w/o refine	17.33	4.06	78.11
Diffusion Planner	72.34	6.04	82.80
Flow Planner	42.31	14.27	83.31
DriveMLP	2.76	2.33	86.51

Table 5: Inference speed (NVIDIA P6000), model size, and nuPlan Val14-R performance.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Method	CL Train	Demo Sup.	No Adv.	Lightweight	No Post-Proc.	Single Stage	nuPlan
PDM-Open [ 10 ] / PlanTF [ 7 ] / PLUTO [ 6 ] w/o refine	×	✓	✓	✓	✓	✓	✓
Diffusion [ 65 ] / Flow Planner [ 50 ]	×	✓	✓	×	✓	✓	✓
AIRL [ 12 ] / GAIL [ 22 ]	✓	✓	×	✓	✓	✓	×
CaRL [ 23 ] / GIGAFLOW [ 9 ]	✓	×	✓	✓	✓	✓	✓
BC-RL [ 33 ]	✓	✓	✓	✓	✓	✓	✓
CarPlanner [ 62 ]	✓	✓	✓	×	×	✓	✓

Appendix

Table 6: Comparison of key design components across related methods. CL Train = closed-loop training; Demo Sup. = Supervision primarily provided by expert demonstrations (no traffic rules); No Adv. = no adversarial/iterative reward learning; Lightweight =

\leq

5M params, no generative or autoregressive architecture; No Post-Proc. = no rule-based post-processing at train or test time; Single Stage = Single training stage; nuPlan = evaluated on nuPlan or similar large-scale ego-planning benchmarks. ✓ = yes, × = no.

	Highway-Env				nuPlan
Name	BC	AIRL	BC-RL	DriveMLP	BC	AIRL	BC-RL	DriveMLP
Algorithm	BC	AIRL (SAC)	BC+SAC	SAC	BC	AIRL (PPO)	BC+PPO	PPO
Training steps	-	300K	100K	100K	-	100M	100M	100M
Learning rate	0.001	0.0003	0.0003	0.0003	0.0001	0.00005	0.00005	0.00005
Learning rate schedule	exponential decay	-	-	-	exponential decay	cosine decay	cosine decay	cosine decay
Rollout buffer size	-	1M	1M	1M	-	98K	98K	98K

Appendix

Table 7: Hyperparameters used in Nuplan and Highway env experiments

Split	PlanTF	PLUTO w/o ref. †	Diff. Planner	Flow Planner	DriveMLP
80 scenarios	30.53	49.27	24.71	36.59	37.07 $\pm$ 2.36
335 scenarios	47.70	58.47	52.90	61.82	58.43 $\pm$ 2.74

Appendix

Table 8: Interplan benchmark results. † PLUTO incorporates centerline reference paths as an additional input.

Size	112K	247K	590K	1.1M	2.3M	4.6M	8.7M
Val14-R	82.22	84.39	84.96	85.74	86.51	86.18	86.33
$\Delta$ vs. BC	+53.31	+54.54	+53.20	+53.70	+54.15	+53.85	+54.22
Infer. (ms)	1.86	1.95	2.27	2.39	2.76	2.99	3.47
Train (L4-h)	56	60	74	87	112	155	185

Appendix

Table 9: MLP capacity scaling under EGRL. Improvements over BC are reported in score points.

Method	NR	R
BC	61.25	32.44
EGRL	82.01	79.68

Appendix

Table 10: Reduced transformer policy (500K parameters) on nuPlan Val14. NR = Non-Reactive, R = Reactive simulation.

Component	NR	R
Base	90.38	86.51
w/o off-road	80.76	78.67
w/o collision	80.08	79.14
w/o deviation	86.92	84.11

Appendix

Table 11: Episode termination ablations for DriveMLP on nuPlan Val14.

Table 17

Frequency	NR	R
0.5 Hz	75.23	77.61
1 Hz	90.38	86.51
2 Hz	85.13	82.28

Appendix

Table 15: Action frequency ablation: nuPlan Val14.

Architecture	Training Method	w/o Ego State	w/ Ego State
DriveMLP	BC	54.90	53.64
Transformer	BC	61.25	54.76
DriveMLP	EGRL	78.79	90.38

Appendix

Table 16: Ego-state ablation: nuPlan Val14 NR.

Figure 6: Noise brings the Diffusion Planner close to a neighboring agent, with no corrective response to avoid the collision (a), whereas DriveMLP remains close to the expert trajectory (b). Similarly, noise can cause the Diffusion Planner to miss a turn (c), while our planner remains largely unaffected (d).

Env	Reset	Step
Naive stack (ms)	559.4	366.27
Ours (ms)	3.9	1.59
Speedup	143x	230x

Appendix

Table 17: Single env. reset/step functions performance

Figure 7: Representative failure cases of DriveMLP. Expert trajectory is shown in orange, while the trajectory executed by ego is in blue. Failure modes include (a) overshooting turns and invading the opposite lane, (b) failure to progress due to a limited time window in which the intersection is clear, (c) side collisions with other vehicles, and (d) late braking for obstacles in the lane. All scenarios are taken from the Val14-Reactive evaluation split.

	DriveMLP				Diffusion Planner
	Val14		Test-Hard		Val14		Test-Hard
Metrics	NR	R	NR	R	NR	R	NR	R
Overall Score	90.39	86.51	76.69	74.51	89.86	82.95	75.73	68.94
Drivable Area Compliance	99.82	99.82	97.43	97.79	97.94	97.85	95.59	94.85
Driving Direction Comp.	99.37	99.42	97.79	97.98	99.51	99.82	98.90	98.90
Ego is Comfortable	87.39	88.10	86.03	87.13	92.49	89.36	91.91	86.40

Appendix

Table 18: Performance Comparison: DriveMLP vs. Diffusion Planner across nuPlan Splits

Autonomous Driving Planning

Autonomous driving planning selects safe, feasible vehicle actions and trajectories amid traffic, uncertainty, road rules, and changing goals.

See all 382 papers

PerlAD: Towards Enhanced Closed-loop End-to-end Autonomous Driving with Pseudo-simulation-based Reinforcement Learning
Mar 16, 2026Yinfeng Gao, Qichao Zhang, Deqing Liu +8End-to-End Autonomous Driving RL for Autonomous Driving
School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing 100083, China. · Xiaomi EV. · State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China, and also with the School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China.
CarPLAN: Context-Adaptive and Robust Planning with Dynamic Scene Awareness for Autonomous Driving
Mar 13, 2026Junyong Yun, Jungho Kim, ByungHyun Lee +5Motion Planning Autonomous Driving Planning
Department of Artificial Intelligence, Hanyang University, 04763, Republic of Korea. · Interdisciplinary Program in Artificial Intelligence, Seoul National University, 08826, Seoul, Republic of Korea · Department of Electrical and Computer Engineering, Seoul National University, 08826, Seoul, Republic of Korea. +1
PlanRL: A Trajectory Planning Architecture for Reinforcement Learning-based Driving Experts
Jun 25, 2026Joonhee Lim, Yongjae Lee, Jangho Shin +1Motion Planning Autonomous Driving Planning
Robotics Program, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, South Korea · Cho Chun Shik Graduate School of Mobility, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, South Korea · Hyundai Motor Company, Hwaseong 18280, South Korea

Mar 16, 2026J/K move · Enter open · S save

PerlAD: Towards Enhanced Closed-loop End-to-end Autonomous Driving with Pseudo-simulation-based Reinforcement Learning

Yinfeng Gao, Qichao Zhang, Deqing Liu, Zhongpu Xia+7

School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing 100083, China. · Xiaomi EV. · State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China, and also with the School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China.

End-to-End Autonomous Driving RL for Autonomous Driving Autonomous Driving Planning Autonomous Driving Simulation World Models for Autonomous Driving

Open paper AskPDF arXiv

End-to-end autonomous driving policies based on Imitation Learning (IL) often struggle in closed-loop execution due to the misalignment between inadequate open-loop training objectives and real driving requirements. While Reinforcement Learning (RL) offers a solution by directly optimizing driving goals via reward signals, the rendering-based training environments introduce the rendering gap and are inefficient due to high computational costs. To overcome these challenges, we present a novel Pseudo-simulation-based RL method for closed-loop end-to-end autonomous driving, PerlAD. Based on offline datasets, PerlAD constructs a pseudo-simulation that operates in vector space, enabling efficient, rendering-free trial-and-error training. To bridge the gap between static datasets and dynamic closed-loop environments, PerlAD introduces a prediction world model that generates reactive agent trajectories conditioned on the ego vehicle's plan. Furthermore, to facilitate efficient planning, PerlAD utilizes a hierarchical decoupled planner that combines IL for lateral path generation and RL for longitudinal speed optimization. Comprehensive experimental results demonstrate that PerlAD achieves state-of-the-art performance on the Bench2Drive benchmark, surpassing the previous E2E RL method by 10.29% in Driving Score without requiring expensive online interactions. Additional evaluations on the DOS benchmark further confirm its reliability in handling safety-critical occlusion scenarios.

Abstract

Autonomous Driving Planning

Explore similar work

PerlAD: Towards Enhanced Closed-loop End-to-end Autonomous Driving with Pseudo-simulation-based Reinforcement Learning

CarPLAN: Context-Adaptive and Robust Planning with Dynamic Scene Awareness for Autonomous Driving

PlanRL: A Trajectory Planning Architecture for Reinforcement Learning-based Driving Experts