Can Users Specify Driving Speed? Bench2Drive-Speed: Benchmark and Baselines for Desired-Speed Conditioned Autonomous Driving

Yuqian Shao; Xiaosong Jia; Langechuan Liu; Junchi Yan

cs.ROMar 26, 2026

Can Users Specify Driving Speed? Bench2Drive-Speed: Benchmark and Baselines for Desired-Speed Conditioned Autonomous Driving

Authors: Yuqian Shao, Xiaosong Jia, Langechuan Liu, Junchi Yan

Organizations: Sch. of Computer Science & Sch. of Artificial Intelligence, Shanghai Jiao Tong University · Institute of Trustworthy Embodied AI (TEAI), Fudan University · NVIDIA

Abstract

End-to-end autonomous driving (E2E-AD) has achieved remarkable progress. However, one practical and useful function has been long overlooked: users may wish to customize the desired speed of the policy or specify whether to allow the autonomous vehicle to overtake. To bridge this gap, we present Bench2Drive-Speed, a benchmark with metrics, dataset, and baselines for desired-speed conditioned autonomous driving. We introduce explicit inputs of users' desired target-speed and overtake/follow instructions to driving policy models. We design quantitative metrics, including Speed-Adherence Score and Overtake Score, to measure how faithfully policies follow user specifications, while remaining compatible with standard autonomous driving metrics. To enable training of speed-conditioned policies, one approach is to collect expert demonstrations that strictly follow speed requirements, an expensive and unscalable process in the real world. An alternative is to adapt existing regular driving data by treating the speed observed in future frames as the target speed for training. To investigate this, we construct CustomizedSpeedDataset, composed of 2,100 clips annotated with experts demonstrations, enabling systematic investigation of supervision strategies. Our experiments show that, under proper re-annotation, models trained on regular driving data perform comparably to on expert demonstrations, suggesting that speed supervision can be introduced without additional complex real-world data collection. Furthermore, we find that while target-speed following can be achieved without degrading regular driving performance, executing overtaking commands remains challenging due to the inherent difficulty of interactive behaviors. All code, datasets and baselines are available at https://github.com/Thinklab-SJTU/Bench2Drive-Speed

Figures & tables

Figure 1: Bench2Drive-Speed introduces target-speed commands and overtake/follow instructions, establishing the first benchmark for desired-speed conditioned autonomous driving. We quantitatively evaluate model controllability across multiple dimensions: speed tracking, interaction strategy, comfort, traffic compliance, etc .

Figure 2: We present Bench2Drive-Speed , including desired-speed conditioned task - with target speed and overtake/follow commands for speed control; dataset — 2,100 scenarios with extra commands annotated by expert demonstration and virtual target speed strategies; benchmark — controllability metrics (speed adherence, overtake/follow) jointly evaluated with safety, comfort, traffic compliance, etc . ; and baseline — the model takes visual and speed command inputs, capable of following target speed commands while attempting to execute overtake/follow behaviors.

Benchmark	Source	Closed-Loop	E2E	Style-Aware	Speed-Conditioned
nuScenes [ 3 ]	Real	✗	✓	✗	✗
CARLA [ 11 , 57 ]	Simulator	✓	✓	✗	✗
MetaDrive [ 28 ]	Simulator	✓	✓	✗	✗
Bench2Drive [ 22 ]	Simulator	✓	✓	✗	✗
NAVSIM [ 10 ]	Real	✗	✓	✗	✗
StyleDrive [ 16 ]	Real	✗	✓	✓	✗

Table 1: Comparison of Driving Benchmarks . Bench2Drive-Speed is the first benchmark to support closed-loop evaluation, end-to-end learning, and explicit speed-conditioned control for autonomous driving.

Figure 3: Three Difficulty Levels in Bench2Drive-Speed . The difficulty of adhering to the desired speed increases from easy to hard. Overtaking and following adherence are evaluated only in the medium and hard scenarios.

Figure 4: Illustration of CustomizedSpeedDataset . The dataset includes visual sensor inputs, ego-state information, bounding box annotations, overtaking commands, and target-speed commands from different sources.

Figure 5: Illustration of different target-speed annotation methods. Expert demonstrations are precise but rely on the data collection model’s internal hyperparameters—which are unavailable in practice—whereas re-annotation is more feasible.

Difficulty	Overtake	Follow
Medium	570	570
Hard	480	480
Total	1,050	1,050

Table 2: Composition of CustomizedSpeedDataset across difficulty levels and overtake/follow options.

Figure 6: Diverse scenarios in CustomizedSpeedDataset. The ego vehicle adheres to target-speed and overtaking commands across various situations, with ample coverage of complex scenarios.

Panel of Figure 7 — Figure 7: Target speed distribution in CustomizedSpeedDataset (Left) and Distribution of difficult scenarios included in CustomizedSpeedDataset, following CARLA Leaderboard v2 [ 57 ] (Right).

Difficulty	Route Layout	Special Incidents
Easy	Rural curving road	None
Easy	Left turn at urban intersection	None
Easy	Straight through urban intersection	None
Easy	Wide street	None
Medium	Rural curving road	Overtake or follow one vehicle
Medium	Wide street	Overtake or follow one vehicle

Table 3: Scenarios of Different Difficulty Levels .

Figure 8: Overview of TCP-Speed. The encoded ego state and driving commands are concatenated with features extracted from RGB inputs. The fused representation is then fed into both the trajectory and control branches. Additionally, the target waypoint(goal) and target-speed command are used to guide trajectory generation. In practice, only the output of the trajectory branch is utilized.

Data Source	From Expert	From Re-annotation (Virtual Target Speed)
Data Source	(Hyper-parameter)	Long Extension	Short Extension
Bench2Drive1K	–	Bench2Drive1K-Long	Bench2Drive1K-Short
CustomizedSpeedDataset	Expert2.1k	Virtual2.1k-Long	Virtual2.1k-Short

Table 4: Different Training Data Configurations .

Figure 9: Visualization of Speed Profile . The target speed adherence of TCP-Speed under the same route with different target speed specifications.

Figure 10: Heatmap of TCP-Speed speed-adherence scores for models trained on different datasets, evaluated on Easy, Medium, Hard, and All routes. Models trained with virtual target-speed supervision achieve performance comparable to those trained on expert demonstrations. Speed adherence is generally higher under the virtual-short setting than virtual-long.

Model	Dataset	Speed-Adherence Score $\uparrow$				Overtake Score $\uparrow$
Model	Dataset	A	E	M	H	A	E	M	H
$\text{TCP}_{\text{w/o Speed Command}}$	CustomizedSpeedDataset	41.54	42.00	40.95	41.67	18.75	-	37.50	0.00
TCP-Speed	Expert2.1k	68.79	76.80	65.38	64.20	21.88	-	37.50	6.25
TCP-Speed	Virtual2.1k-Short	69.23	76.18	68.71	62.81	40.63	-	56.25	25.00
TCP-Speed	Virtual2.1k-Long	68.36	73.17	67.93	63.98	40.63	-	56.25	25.00

Table 5: Speed-Adherence Score and Overtake Score on 48 evaluation routes of Bench2Drive-Speed. Metrics are reported for All(A), Easy (E), Medium (M), and Hard (H). The best result of each score is highlighted in bold.

Figure 11: Visualization of TCP-Speed Overtaking and Following. A successful overtaking case (upper) in which the ego vehicle passed the slow white vehicle ahead, and a successful following case (lower), in which the ego followed the vehicle till the end of the route.

Method	Driving Score $\uparrow$	Success Rate(%) $\uparrow$	Efficiency $\uparrow$	Comfortness $\uparrow$
AD-MLP [ 53 ]	18.05	0.00	48.45	22.63
UniAD-Tiny [ 18 ]	40.73	13.18	123.92	47.04
UniAD-Base [ 18 ]	45.81	16.36	129.21	43.58
VAD [ 24 ]	42.35	15.00	157.94	46.01
DriveTransformer-Large [ 23 ]	63.46	35.01	100.64	20.78
ThinkTwice* [ 21 ]	62.44	31.23	69.33	16.22

Table 6: Closed-loop Planning Performance on 220 Route of Bench2Drive . All models are trained under Bench2Drive1K. * denotes expert feature distillation. Non-TCP methods and distilled methods are shown in gray for reference.

Figure 12: Example of a configuration XML file. The configuration specifies route waypoints, traffic scenarios, target speed settings, and weather conditions, etc .

Panel of Figure 13 — Figure 13: (a) Distribution of weather conditions in CustomizedSpeedDataset. It covers all predefined weathers in the simulator, uniformly distributed. (b) Distribution of towns in CustomizedSpeedDataset. Route counts roughly reflect each town’s size and number of routes.

Panel of Figure 14 — Figure 14: (a) Scenario-wise distribution of hard routes in CustomizedSpeedDataset. Covering the most challenging scenarios in the simulator. (b) Distribution of leading vehicle speeds in overtake/follow scenarios within CustomizedSpeedDataset. This speed is sampled from a small range below the ego vehicle speed, resulting in a distribution biased toward lower speeds.

Model	Dataset	Speed Adherence Score $\uparrow$				Overtake Score $\uparrow$
Model	Dataset	A	E	M	H	A	E	M	H
$\text{TCP}_{\text{w/o Speed Command}}$	Bench2Drive1K	41.36	42.03	40.72	41.33	37.50	-	50.00	25.00
TCP-Speed	Bench2Drive1K-Short	67.95	71.01	69.63	63.21	31.25	-	50.00	12.50
TCP-Speed	Bench2Drive1K-Long	65.61	67.80	65.69	63.34	31.25	-	43.75	18.75
$\text{TCP}_{\text{w/o Speed Command}}$	Bench2Drive1K+CustomizedSpeedDataset	41.60	42.11	40.98	41.70	25.00	-	50.00	0.00
TCP-Speed	Bench2Drive1K-Short+Virtual2.1k-Short	71.65	77.33	72.63	64.99	34.38	-	50.00	18.75

Table 7: Speed-Adherence Score and Overtake Score on 48 evaluation routes of Bench2Drive-Speed. Metrics are reported for All(A), Easy (E), Medium (M), and Hard (H). The best result of each score is highlighted in bold.

Model	Dataset	Driving Score $\uparrow$	Success Rate (%) $\uparrow$	Efficiency $\uparrow$	Comfortness $\uparrow$
$\text{TCP}_{\text{w/o Speed Command}}$	Bench2Drive1K	56.52	20.83	44.39	24.79
TCP-Speed	Bench2Drive1K-Short	40.23	20.83	158.32	30.27
TCP-Speed	Bench2Drive1K-Long	45.18	27.08	145.22	29.67
$\text{TCP}_{\text{w/o Speed Command}}$	Bench2Drive1K+CustomizedSpeedDataset	52.71	27.08	52.17	33.00
TCP-Speed	Bench2Drive1K-Short+Virtual2.1k-Short	30.19	16.67	158.32	30.27
TCP-Speed	Bench2Drive1K-Long+Virtual2.1k-Long	48.40	31.25	145.22	29.67

Table 8: Overall closed-loop performance on traditional metrics (Driving Score, Success Rate, Efficiency, Comfortness) on 48 evaluation routes of Bench2Drive-Speed. The best overall result of each score is highlighted in bold.

Method	Driving Score $\uparrow$	Success Rate(%) $\uparrow$	Efficiency $\uparrow$	Comfortness $\uparrow$
AD-MLP	18.05	0.00	48.45	22.63
UniAD-Tiny	40.73	13.18	123.92	47.04
UniAD-Base	45.81	16.36	129.21	43.58
VAD	42.35	15.00	157.94	46.01
DriveTransformer-Large	63.46	35.01	100.64	20.78
ThinkTwice*	62.44	31.23	69.33	16.22

Table 9: Closed-loop Planning Performance on 220 routes of Bench2Drive . * denotes expert feature distillation. Non-TCP methods and expert-feature-distilled methods are shown in gray for reference.

Method	Merging	Overtaking	Emergency Brake	Give Way	Traffic Sign	Mean
AD-MLP	0.00	0.00	0.00	0.00	0.00	0.00
UniAD-Tiny	7.04	10.00	21.82	20.00	14.61	14.69
UniAD-Base	12.16	20.00	23.64	10.00	13.89	15.94
VAD	7.14	20.00	16.36	20.00	20.22	16.75
DriveTransformer-Large	17.57	35.00	48.36	40.00	52.10	38.60
DriveAdapter*	14.55	22.61	54.04	50.00	50.45	38.33

Table 10: Multi-Ability Results of E2E-AD Methods on Bench2Drive220 routes. * denotes expert feature distillation. Non-TCP methods and expert-feature-distilled methods are shown in gray for reference.

Autonomous Driving

Autonomous driving develops vehicles that perceive road scenes, predict interactions, and plan and control safe movement with limited or no human input.

See all 536 papers

Closed-Loop Aware Planning via Imitation-Free Speed Scoring for End-to-End Autonomous Driving
Oct 26, 2026Emad Aghajanzadeh, Thomas Schwarzfischer, Marc Kaufeld +8Autonomous Driving Planning End-to-End Autonomous Driving
Technical University of Munich, Germany · BMW Group, Germany · TU Wien, Austria
Safe2Drive: Evaluating Safe Driving Behaviors of E2E Autonomous Driving Models
May 29, 2026Nishad Sahu, Kalpana Panda, Congyuan Yu +3Autonomous Driving Benchmarks End-to-End Autonomous Driving
1Carnegie Mellon University · 2Birla Institute of Technology and Science Pilani
HiDrive: A Closed-Loop Benchmark for High-Level Autonomous Driving
May 11, 2026Zhongyu Xia, Guanyu Zhu, Guo Tang +2Autonomous Driving Benchmarks End-to-End Autonomous Driving
Wangxuan Institute of Computer Technology, Peking University

Oct 26, 2026J/K move · Enter open · S save

Closed-Loop Aware Planning via Imitation-Free Speed Scoring for End-to-End Autonomous Driving

Emad Aghajanzadeh, Thomas Schwarzfischer, Marc Kaufeld, Moritz Thoma+7

Technical University of Munich, Germany · BMW Group, Germany · TU Wien, Austria

Autonomous Driving Planning End-to-End Autonomous Driving Imitation Learning

Open paper AskPDF arXiv

End-to-end autonomous driving has predominantly relied on open-loop imitation learning for training, which presents two limitations. First, expert demonstrations show only one behavior per scenario, so the model cannot learn alternative valid behaviors and, in turn, cannot adapt to user-specified preferences at test time. Second, noise or ambiguity in these demonstrations can produce inconsistent, unreliable supervision for the end-to-end models. Driving is inherently multi-modal: multiple different behaviors can be valid responses to the same scene, and this diversity arises largely from speed rather than path. We introduce a planning paradigm that learns the path through imitation while scoring a set of candidate speeds across multiple criteria with a learned function supervised by physically grounded safety constraints. We realize this paradigm as Closed-Loop Aware Planning for Autonomous Driving (CLAP-AD), a plug-in planning head that integrates with existing end-to-end driving models and, to our knowledge, is the first to enable explicit test-time control over their speed selection without speed-conditioned training. On Bench2Drive-Speed, CLAP-AD sets a new state of the art Speed-Adherence Score of 82.53, topping TFv6, the strongest baseline on this metric, by 12.9 points while retaining strong closed-loop driving performance. On Bench2Drive, it improves driving score by +28.5 and +32.7 over UniAD and VAD, and on NAVSIM v2, built on real-world driving data, EPDMS by +6.1 over its backbone. Project page: https://clap-ad.github.io

Abstract

Autonomous Driving

Explore similar work

Closed-Loop Aware Planning via Imitation-Free Speed Scoring for End-to-End Autonomous Driving

Safe2Drive: Evaluating Safe Driving Behaviors of E2E Autonomous Driving Models

HiDrive: A Closed-Loop Benchmark for High-Level Autonomous Driving