TrafficSignBench: Rule-Centric Closed-Loop Evaluation of Traffic-Sign Compliance in Autonomous Driving

Victoria Smirnova; Viktoriia Zinkovich; Gregorii Bukhtuev; Artem Belyaev; Andrey Kuznetsov; Denis Shepelev; Vlad Shakhuro

cs.ROSep 29, 2026

TrafficSignBench: Rule-Centric Closed-Loop Evaluation of Traffic-Sign Compliance in Autonomous Driving

Authors: Victoria Smirnova, Viktoriia Zinkovich, Gregorii Bukhtuev, Artem Belyaev, Andrey Kuznetsov, Denis Shepelev, Vlad Shakhuro

Organizations: Lomonosov Moscow State University · FusionBrain Lab · Moscow Institute of Physics and Technology · HSE University · Innopolis University · NUST MISIS

Abstract

Autonomous driving planners are typically evaluated using aggregate metrics such as driving score, destination rate, and collision rate, which do not explicitly measure compliance with traffic rules. As a result, planners can achieve high benchmark scores while still exhibiting unsafe or illegal behaviors, limiting their applicability to real-world deployment. To address this gap, we introduce TrafficSignBench, a large-scale, traffic sign-centric benchmark for systematic and interpretable evaluation of traffic-rule compliance in autonomous driving. Our framework combines real-map-based simulation for realistic road layouts with rule-targeted procedural scenario generation for scalable and balanced coverage of underrepresented rules. We implement traffic rules corresponding to 34 traffic signs, each equipped with an automatic rule checker for detecting violations during closed-loop execution. This design yields 29,000 diverse road scenes and 29 distinct testing scenario types, enabling controlled evaluation of rule-specific planner behavior. We construct 5,800 testing scenes and demonstrate that current autonomous driving planners can exhibit poor traffic-rule compliance despite strong performance on standard evaluation metrics. To address this limitation, we transform existing planners into rule-compliant trajectory experts via explicit traffic-sign constraints, enabling scalable generation of high-quality oracle trajectories for fine-tuning.

Figures & tables

	Vienna Signatories				Non-Vienna
	Germany [ 29 ]	France [ 30 ]	Italy [ 31 ]	Turkey [ 32 ]	UK [ 33 ]	China [ 34 ]
Mean	91%	84%	94%	100%	97%	91%

TABLE I : Semantic equivalence of TrafficSignBench signs across countries. Values report the percentage of the 34 implemented signs that have a semantic equivalent with Russian traffic signs [ 35 ] .

Fig. 2 : Calibration of background traffic dynamics. Empirical CDFs compare real-world data from the nuPlan dataset [ 11 ] with values recorded in TrafficSignBench; dashed guides mark the three controlled density and speed probes, and

D

is the two-sample KS statistic.

Fig. 3 : Composition of the Oracle Expert dataset. Proportion of high-quality trajectories contributed by each of the 8 rule-compliant experts across the 4 semantic sign groups. The diverse distribution shows that no single planner dominates; rather, the experts exhibit complementary strengths, providing robust and varied demonstrations.

	Conventional driving				Group SCD (%)
Planner	DS $\uparrow$	Dest. (%) $\uparrow$	Eff. $\uparrow$	Coll. (%) $\downarrow$	Priority $\uparrow$	Speed $\uparrow$	Obstacle $\uparrow$	Routing $\uparrow$	Overall SCD $\uparrow$
Standard baselines
IDM	35.7	66.4	115.1	21.0	14.6	26.6	1.7	0.0	7.2
IDM- $s_{1}$	27.6	51.8	84.3	19.2	17.4	34.6	2.2	0.0	9.0
IDM- $s_{2}$	28.0	52.3	84.0	19.3	17.9	33.6	2.3	0.0	9.0
IDM- $s_{3}$	27.1	51.0	84.7	20.2	18.6	32.0	1.9	0.0	8.8

TABLE II: Closed-loop evaluation on all 29 scenario types. DS: Driving Score; Dest.: Destination Rate; Eff.: Efficiency; Coll.: Collision Rate. Conventional metrics are episode-weighted; SCD is macro-averaged over scenario types, giving every rule equal weight. Bold denotes the best result among standard baselines and our method. Superscript

e

denotes privileged rule-compliant policies with explicit access to the target rule; they are reported as privileged reference policies. Confidence intervals are shown in Fig. 4 and omitted here for readability.

Fig. 4 : Overall rule-conditioned performance across 29 scenario types. Points show individual SCD; whiskers show the sample confidence interval across scenario-level rates. Superscript $e$ denotes policies with privileged access to the target rule.

		SCD (%) $\uparrow$		Sign Compliance (%) $\uparrow$
Sign group	$n$	full	no sign	full	no sign
Priority (6)	120	55.5	12.6	71.4	18.5
Speed (4)	80	96.3	17.5	98.8	18.8
Obstacle (8)	160	83.8	5.6	89.4	11.9
Routing (11)	220	65.6	23.4	98.2	50.9
All scenario types	580	72.8	15.4	90.3	28.9

TABLE III : Sign-channel ablation. PlanT-2-FT evaluated on paired scenes with sign identity removed from the input; plates remain as anonymous static objects. Parentheses indicate the number of sign families;

n

is the number of paired episodes.

Fig. 5 : Alignment between conventional metrics and rule-conditioned success. Spearman rank correlations are computed over the 8 baseline planners and PlanT-2-FT; privileged experts are excluded.

Explore similar work

Beyond Self-Play and Scale: A Behavior Benchmark for Generalization in Autonomous Driving
May 11, 2026Aron Distelzweig, Faris Janjoš, Andreas Look +7Autonomous Driving Self-Play
University of Freiburg · Bosch Center for Artificial Intelligence · Coburg University of Applied Sciences +1
DriveVer: Lightweight Trajectory Evaluator as Test-Time Verifier for Autonomous Driving
Jul 1, 2026Chong He, Yuechen Luo, Fang Li +2Autonomous Driving Drives
1Tsinghua University · University of Macau
HiDrive: A Closed-Loop Benchmark for High-Level Autonomous Driving
May 11, 2026Zhongyu Xia, Guanyu Zhu, Guo Tang +2Causality-Aware End-To-End Autonomous Driving Autonomous Driving
Wangxuan Institute of Computer Technology, Peking University

May 11, 2026, cs.ROJ/K move · Enter open · S save

Beyond Self-Play and Scale: A Behavior Benchmark for Generalization in Autonomous Driving

Aron Distelzweig, Faris Janjoš, Andreas Look, Anna Rothenhäusler+6

University of Freiburg · Bosch Center for Artificial Intelligence · Coburg University of Applied Sciences+1

Autonomous Driving Self-Play

Open paper AskPDF arXiv

Recent Autonomous Driving (AD) works such as GigaFlow and PufferDrive have unlocked Reinforcement Learning (RL) at scale as a training strategy for driving policies. Yet such policies remain disconnected from established benchmarks, leaving the performance of large-scale RL for driving on standardized evaluations unknown. We present BehaviorBench -- a comprehensive test suite that closes this gap along three axes: Evaluation, Complexity, and Behavior Diversity. In terms of Evaluation, we provide an interface connecting PufferDrive to nuPlan, which, for the first time, enables policies trained via RL at scale to be evaluated on an established planning benchmark for autonomous driving. Complementarily, we offer an evaluation framework that allows planners to be benchmarked directly inside the PufferDrive simulation, at a fraction of the time. Regarding Complexity, we observe that today's standardized benchmarks are so simple that near-perfect scores are achievable by straight lane following with collision checking. We extract a meaningful, interaction-rich split from the Waymo Open Motion Dataset (WOMD) on which strong performance is impossible without multi-agent reasoning. Lastly, we address Behavior Diversity. Existing benchmarks commonly evaluate planners against a single rule-based traffic model, the Intelligent Driver Model (IDM). We provide a diverse suite of interactive traffic agents to stress-test policies under heterogeneous behaviors, beyond just using IDM. Overall, our benchmarking analysis uncovers the following insight: despite learning interactive behaviors in an emergent manner, policies trained via pure self-play under standard reward functions overfit to their training opponents and fail to generalize to other traffic agent behaviors. Building on this observation, we propose a hybrid planner that combines a PPO policy with a rule-based planner.

Abstract

Explore similar work

Beyond Self-Play and Scale: A Behavior Benchmark for Generalization in Autonomous Driving

DriveVer: Lightweight Trajectory Evaluator as Test-Time Verifier for Autonomous Driving

HiDrive: A Closed-Loop Benchmark for High-Level Autonomous Driving