Can Users Specify Driving Speed? Bench2Drive-Speed: Benchmark and Baselines for Desired-Speed Conditioned Autonomous Driving

Yuqian Shao; Xiaosong Jia; Langechuan Liu; Junchi Yan

cs.ROMar 26, 2026

Can Users Specify Driving Speed? Bench2Drive-Speed: Benchmark and Baselines for Desired-Speed Conditioned Autonomous Driving

Authors: Yuqian Shao, Xiaosong Jia, Langechuan Liu, Junchi Yan

Organizations: Sch. of Computer Science & Sch. of Artificial Intelligence, Shanghai Jiao Tong University · Institute of Trustworthy Embodied AI (TEAI), Fudan University · NVIDIA

Abstract

End-to-end autonomous driving (E2E-AD) has achieved remarkable progress. However, one practical and useful function has been long overlooked: users may wish to customize the desired speed of the policy or specify whether to allow the autonomous vehicle to overtake. To bridge this gap, we present Bench2Drive-Speed, a benchmark with metrics, dataset, and baselines for desired-speed conditioned autonomous driving. We introduce explicit inputs of users' desired target-speed and overtake/follow instructions to driving policy models. We design quantitative metrics, including Speed-Adherence Score and Overtake Score, to measure how faithfully policies follow user specifications, while remaining compatible with standard autonomous driving metrics. To enable training of speed-conditioned policies, one approach is to collect expert demonstrations that strictly follow speed requirements, an expensive and unscalable process in the real world. An alternative is to adapt existing regular driving data by treating the speed observed in future frames as the target speed for training. To investigate this, we construct CustomizedSpeedDataset, composed of 2,100 clips annotated with experts demonstrations, enabling systematic investigation of supervision strategies. Our experiments show that, under proper re-annotation, models trained on regular driving data perform comparably to on expert demonstrations, suggesting that speed supervision can be introduced without additional complex real-world data collection. Furthermore, we find that while target-speed following can be achieved without degrading regular driving performance, executing overtaking commands remains challenging due to the inherent difficulty of interactive behaviors. All code, datasets and baselines are available at https://github.com/Thinklab-SJTU/Bench2Drive-Speed

Figures & tables

Figure 1: Bench2Drive-Speed introduces target-speed commands and overtake/follow instructions, establishing the first benchmark for desired-speed conditioned autonomous driving. We quantitatively evaluate model controllability across multiple dimensions: speed tracking, interaction strategy, comfort, traffic compliance, etc .

Figure 2: We present Bench2Drive-Speed , including desired-speed conditioned task - with target speed and overtake/follow commands for speed control; dataset — 2,100 scenarios with extra commands annotated by expert demonstration and virtual target speed strategies; benchmark — controllability metrics (speed adherence, overtake/follow) jointly evaluated with safety, comfort, traffic compliance, etc . ; and baseline — the model takes visual and speed command inputs, capable of following target speed commands while attempting to execute overtake/follow behaviors.

Benchmark	Source	Closed-Loop	E2E	Style-Aware	Speed-Conditioned
nuScenes [ 3 ]	Real	✗	✓	✗	✗
CARLA [ 11 , 57 ]	Simulator	✓	✓	✗	✗
MetaDrive [ 28 ]	Simulator	✓	✓	✗	✗
Bench2Drive [ 22 ]	Simulator	✓	✓	✗	✗
NAVSIM [ 10 ]	Real	✗	✓	✗	✗
StyleDrive [ 16 ]	Real	✗	✓	✓	✗

Table 1: Comparison of Driving Benchmarks . Bench2Drive-Speed is the first benchmark to support closed-loop evaluation, end-to-end learning, and explicit speed-conditioned control for autonomous driving.

Figure 3: Three Difficulty Levels in Bench2Drive-Speed . The difficulty of adhering to the desired speed increases from easy to hard. Overtaking and following adherence are evaluated only in the medium and hard scenarios.

Figure 4: Illustration of CustomizedSpeedDataset . The dataset includes visual sensor inputs, ego-state information, bounding box annotations, overtaking commands, and target-speed commands from different sources.

Figure 5: Illustration of different target-speed annotation methods. Expert demonstrations are precise but rely on the data collection model’s internal hyperparameters—which are unavailable in practice—whereas re-annotation is more feasible.

Difficulty	Overtake	Follow
Medium	570	570
Hard	480	480
Total	1,050	1,050

Table 2: Composition of CustomizedSpeedDataset across difficulty levels and overtake/follow options.

Figure 6: Diverse scenarios in CustomizedSpeedDataset. The ego vehicle adheres to target-speed and overtaking commands across various situations, with ample coverage of complex scenarios.

Panel of Figure 7 — Figure 7: Target speed distribution in CustomizedSpeedDataset (Left) and Distribution of difficult scenarios included in CustomizedSpeedDataset, following CARLA Leaderboard v2 [ 57 ] (Right).

Difficulty	Route Layout	Special Incidents
Easy	Rural curving road	None
Easy	Left turn at urban intersection	None
Easy	Straight through urban intersection	None
Easy	Wide street	None
Medium	Rural curving road	Overtake or follow one vehicle
Medium	Wide street	Overtake or follow one vehicle

Table 3: Scenarios of Different Difficulty Levels .

Figure 8: Overview of TCP-Speed. The encoded ego state and driving commands are concatenated with features extracted from RGB inputs. The fused representation is then fed into both the trajectory and control branches. Additionally, the target waypoint(goal) and target-speed command are used to guide trajectory generation. In practice, only the output of the trajectory branch is utilized.

Data Source	From Expert	From Re-annotation (Virtual Target Speed)
Data Source	(Hyper-parameter)	Long Extension	Short Extension
Bench2Drive1K	–	Bench2Drive1K-Long	Bench2Drive1K-Short
CustomizedSpeedDataset	Expert2.1k	Virtual2.1k-Long	Virtual2.1k-Short

Table 4: Different Training Data Configurations .

Figure 9: Visualization of Speed Profile . The target speed adherence of TCP-Speed under the same route with different target speed specifications.

Figure 10: Heatmap of TCP-Speed speed-adherence scores for models trained on different datasets, evaluated on Easy, Medium, Hard, and All routes. Models trained with virtual target-speed supervision achieve performance comparable to those trained on expert demonstrations. Speed adherence is generally higher under the virtual-short setting than virtual-long.

Model	Dataset	Speed-Adherence Score $\uparrow$				Overtake Score $\uparrow$
Model	Dataset	A	E	M	H	A	E	M	H
$\text{TCP}_{\text{w/o Speed Command}}$	CustomizedSpeedDataset	41.54	42.00	40.95	41.67	18.75	-	37.50	0.00
TCP-Speed	Expert2.1k	68.79	76.80	65.38	64.20	21.88	-	37.50	6.25
TCP-Speed	Virtual2.1k-Short	69.23	76.18	68.71	62.81	40.63	-	56.25	25.00
TCP-Speed	Virtual2.1k-Long	68.36	73.17	67.93	63.98	40.63	-	56.25	25.00

Table 5: Speed-Adherence Score and Overtake Score on 48 evaluation routes of Bench2Drive-Speed. Metrics are reported for All(A), Easy (E), Medium (M), and Hard (H). The best result of each score is highlighted in bold.

Figure 11: Visualization of TCP-Speed Overtaking and Following. A successful overtaking case (upper) in which the ego vehicle passed the slow white vehicle ahead, and a successful following case (lower), in which the ego followed the vehicle till the end of the route.

Method	Driving Score $\uparrow$	Success Rate(%) $\uparrow$	Efficiency $\uparrow$	Comfortness $\uparrow$
AD-MLP [ 53 ]	18.05	0.00	48.45	22.63
UniAD-Tiny [ 18 ]	40.73	13.18	123.92	47.04
UniAD-Base [ 18 ]	45.81	16.36	129.21	43.58
VAD [ 24 ]	42.35	15.00	157.94	46.01
DriveTransformer-Large [ 23 ]	63.46	35.01	100.64	20.78
ThinkTwice* [ 21 ]	62.44	31.23	69.33	16.22

Table 6: Closed-loop Planning Performance on 220 Route of Bench2Drive . All models are trained under Bench2Drive1K. * denotes expert feature distillation. Non-TCP methods and distilled methods are shown in gray for reference.

Figure 12: Example of a configuration XML file. The configuration specifies route waypoints, traffic scenarios, target speed settings, and weather conditions, etc .

Panel of Figure 13 — Figure 13: (a) Distribution of weather conditions in CustomizedSpeedDataset. It covers all predefined weathers in the simulator, uniformly distributed. (b) Distribution of towns in CustomizedSpeedDataset. Route counts roughly reflect each town’s size and number of routes.

Panel of Figure 14 — Figure 14: (a) Scenario-wise distribution of hard routes in CustomizedSpeedDataset. Covering the most challenging scenarios in the simulator. (b) Distribution of leading vehicle speeds in overtake/follow scenarios within CustomizedSpeedDataset. This speed is sampled from a small range below the ego vehicle speed, resulting in a distribution biased toward lower speeds.

Model	Dataset	Speed Adherence Score $\uparrow$				Overtake Score $\uparrow$
Model	Dataset	A	E	M	H	A	E	M	H
$\text{TCP}_{\text{w/o Speed Command}}$	Bench2Drive1K	41.36	42.03	40.72	41.33	37.50	-	50.00	25.00
TCP-Speed	Bench2Drive1K-Short	67.95	71.01	69.63	63.21	31.25	-	50.00	12.50
TCP-Speed	Bench2Drive1K-Long	65.61	67.80	65.69	63.34	31.25	-	43.75	18.75
$\text{TCP}_{\text{w/o Speed Command}}$	Bench2Drive1K+CustomizedSpeedDataset	41.60	42.11	40.98	41.70	25.00	-	50.00	0.00
TCP-Speed	Bench2Drive1K-Short+Virtual2.1k-Short	71.65	77.33	72.63	64.99	34.38	-	50.00	18.75

Table 7: Speed-Adherence Score and Overtake Score on 48 evaluation routes of Bench2Drive-Speed. Metrics are reported for All(A), Easy (E), Medium (M), and Hard (H). The best result of each score is highlighted in bold.

Model	Dataset	Driving Score $\uparrow$	Success Rate (%) $\uparrow$	Efficiency $\uparrow$	Comfortness $\uparrow$
$\text{TCP}_{\text{w/o Speed Command}}$	Bench2Drive1K	56.52	20.83	44.39	24.79
TCP-Speed	Bench2Drive1K-Short	40.23	20.83	158.32	30.27
TCP-Speed	Bench2Drive1K-Long	45.18	27.08	145.22	29.67
$\text{TCP}_{\text{w/o Speed Command}}$	Bench2Drive1K+CustomizedSpeedDataset	52.71	27.08	52.17	33.00
TCP-Speed	Bench2Drive1K-Short+Virtual2.1k-Short	30.19	16.67	158.32	30.27
TCP-Speed	Bench2Drive1K-Long+Virtual2.1k-Long	48.40	31.25	145.22	29.67

Table 8: Overall closed-loop performance on traditional metrics (Driving Score, Success Rate, Efficiency, Comfortness) on 48 evaluation routes of Bench2Drive-Speed. The best overall result of each score is highlighted in bold.

Method	Driving Score $\uparrow$	Success Rate(%) $\uparrow$	Efficiency $\uparrow$	Comfortness $\uparrow$
AD-MLP	18.05	0.00	48.45	22.63
UniAD-Tiny	40.73	13.18	123.92	47.04
UniAD-Base	45.81	16.36	129.21	43.58
VAD	42.35	15.00	157.94	46.01
DriveTransformer-Large	63.46	35.01	100.64	20.78
ThinkTwice*	62.44	31.23	69.33	16.22

Table 9: Closed-loop Planning Performance on 220 routes of Bench2Drive . * denotes expert feature distillation. Non-TCP methods and expert-feature-distilled methods are shown in gray for reference.

Method	Merging	Overtaking	Emergency Brake	Give Way	Traffic Sign	Mean
AD-MLP	0.00	0.00	0.00	0.00	0.00	0.00
UniAD-Tiny	7.04	10.00	21.82	20.00	14.61	14.69
UniAD-Base	12.16	20.00	23.64	10.00	13.89	15.94
VAD	7.14	20.00	16.36	20.00	20.22	16.75
DriveTransformer-Large	17.57	35.00	48.36	40.00	52.10	38.60
DriveAdapter*	14.55	22.61	54.04	50.00	50.45	38.33

Table 10: Multi-Ability Results of E2E-AD Methods on Bench2Drive220 routes. * denotes expert feature distillation. Non-TCP methods and expert-feature-distilled methods are shown in gray for reference.

Autonomous Driving

Autonomous driving develops vehicles that perceive road scenes, predict interactions, and plan and control safe movement with limited or no human input.

See all 536 papers

Oct 26, 2026

Closed-Loop Aware Planning via Imitation-Free Speed Scoring for End-to-End Autonomous Driving

End-to-end autonomous driving has predominantly relied on open-loop imitation learning for training, which presents two limitations. First, expert demonstrations show only one behavior per scenario, so the model cannot learn alternative valid behaviors and, in turn, cannot adapt to user-specified preferences at test time. Second, noise or ambiguity in these demonstrations can produce inconsistent, unreliable supervision for the end-to-end models. Driving is inherently multi-modal: multiple different behaviors can be valid responses to the same scene, and this diversity arises largely from speed rather than path. We introduce a planning paradigm that learns the path through imitation while scoring a set of candidate speeds across multiple criteria with a learned function supervised by physically grounded safety constraints. We realize this paradigm as Closed-Loop Aware Planning for Autonomous Driving (CLAP-AD), a plug-in planning head that integrates with existing end-to-end driving models and, to our knowledge, is the first to enable explicit test-time control over their speed selection without speed-conditioned training. On Bench2Drive-Speed, CLAP-AD sets a new state of the art Speed-Adherence Score of 82.53, topping TFv6, the strongest baseline on this metric, by 12.9 points while retaining strong closed-loop driving performance. On Bench2Drive, it improves driving score by +28.5 and +32.7 over UniAD and VAD, and on NAVSIM v2, built on real-world driving data, EPDMS by +6.1 over its backbone. Project page: https://clap-ad.github.io

Emad Aghajanzadeh, Thomas Schwarzfischer, Marc Kaufeld +8

Technical University of Munich, Germany · BMW Group, Germany · TU Wien, Austria

Autonomous Driving Planning End-to-End Autonomous Driving Imitation Learning

Ask PDF

May 29, 2026

Safe2Drive: Evaluating Safe Driving Behaviors of E2E Autonomous Driving Models

Recent end-to-end (E2E) autonomous driving policies achieve high driving scores in closed-loop simulations. Yet it remains unclear whether these policies handle common safety-critical scenarios. We present Safe2Drive (S2D), a set of Bench2Drive-aligned scenario extensions focused on three frequent families of road hazards: work zones, pedestrian jaywalking, and occluded vulnerable road users (VRUs). Safe2Drive adds 100 common but challenging scenarios and introduces SafeDriving Score (SDS), a safety-centric metric that augments prior evaluators with pre-crash braking, work zone-object contact, lane centering, and smoothness checks. Evaluating two state-of-the-art policies (LEAD and SimLingo) on S2D, we find that their driving scores drop sharply relative to their reported Bench2Drive baselines (LEAD: from 94.70 DS on Bench2Drive to 39.95 DS on S2D; SimLingo: from 85.07 DS on Bench2Drive to 41.00 DS on S2D) and that SDS on S2D is low (11.85 for LEAD and 15.27 for Sim-Lingo). These results are consistent with brittle safe-driving behaviors such as poor work-zone understanding, red-light violations, and late or absent braking for pedestrians. This study highlights a lack of safe behavioral reasoning in E2E models even when tested on CARLA towns that are part of the training set. We plan to release the code and videos for all 100 S2D scenarios.

Nishad Sahu, Kalpana Panda, Congyuan Yu +3

1Carnegie Mellon University · 2Birla Institute of Technology and Science Pilani

Autonomous Driving Benchmarks End-to-End Autonomous Driving Autonomous Driving Safety Evaluation Autonomous Driving AI Safety Evaluation Autonomous Driving Simulation

Ask PDF

May 11, 2026

HiDrive: A Closed-Loop Benchmark for High-Level Autonomous Driving

End-to-end autonomous driving has witnessed rapid progress, yet existing benchmarks are increasingly saturated, with state-of-the-art models achieving near-perfect scores on widely used open-loop and closed-loop benchmarks. This saturation does not mean that the problem has been solved; instead, it reveals that current benchmarks remain limited in scenario diversity, object variety, and the breadth of driving capabilities they evaluate. In particular, they lack sufficient long-tail scenarios involving rare but safety-critical objects and fail to assess advanced decision-making such as legal compliance, ethical reasoning, and emergency response. To address these gaps, we propose HiDrive, a new closed-loop benchmark for end-to-end autonomous driving that emphasizes long-tail scenarios and a richer evaluation of driving capabilities. HiDrive introduces a diverse set of rare objects and uncommon traffic situations, and expands evaluation from basic driving skills to more advanced capabilities, including rule compliance, moral reasoning, and context-dependent emergency maneuvers. Correspondingly, we extend previous collision-avoidance-centered metrics into a comprehensive evaluation system that encompasses collision and braking, traffic-rule compliance, and moral-reasoning indicators. Built on a more advanced physics engine, HiDrive provides physically realistic lighting and high-fidelity visual rendering, offering a more challenging and realistic testbed for assessing whether autonomous driving systems can handle the complexity of real-world deployment. The HiDrive software, source code, digital assets, and documentation are available at https://github.com/VDIGPKU/HiDrive.

Zhongyu Xia, Guanyu Zhu, Guo Tang +2

Wangxuan Institute of Computer Technology, Peking University

Autonomous Driving Benchmarks End-to-End Autonomous Driving Autonomous Driving Safety Evaluation Autonomous Driving Autonomous Driving Planning Autonomous Driving Simulation

Ask PDF

Abstract

Autonomous Driving

Explore similar work

Closed-Loop Aware Planning via Imitation-Free Speed Scoring for End-to-End Autonomous Driving

Safe2Drive: Evaluating Safe Driving Behaviors of E2E Autonomous Driving Models

HiDrive: A Closed-Loop Benchmark for High-Level Autonomous Driving