Traffic Sign Recognition in Autonomous Driving: Dataset, Benchmark, and Field Experiment

Guoyang Zhao; Weiqing Qi; Kai Zhang; Chenguang Zhang; Zeying Gong; Zhihai Bi; Kai Chen; Benshan Ma

cs.CVMar 24, 2026

Traffic Sign Recognition in Autonomous Driving: Dataset, Benchmark, and Field Experiment

Authors: Guoyang Zhao, Weiqing Qi, Kai Zhang, Chenguang Zhang, Zeying Gong, Zhihai Bi, Kai Chen, Benshan Ma, +2 more

Organizations: The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511453, China · Lingnan University, Hong Kong SAR, China · Shenzhen Unity Drive Innovation Technology Co., Ltd., Shenzhen 518063, China · The Hong Kong University of Science and Technology, Hong Kong SAR, China

Abstract

Traffic Sign Recognition (TSR) is a core perception capability for autonomous driving, where robustness to cross-region variation, long-tailed categories, and semantic ambiguity is essential for reliable real-world deployment. Despite steady progress in recognition accuracy, existing traffic sign datasets and benchmarks offer limited diagnostic insight into how different modeling paradigms behave under these practical challenges. We present TS-1M, a large-scale and globally diverse traffic sign dataset comprising over one million real-world images across 454 standardized categories, together with a diagnostic benchmark designed to analyze model capability boundaries. Beyond standard train-test evaluation, we provide a suite of challenge-oriented settings, including cross-region recognition, rare-class identification, low-clarity robustness, and semantic text understanding, enabling systematic and fine-grained assessment of modern TSR models. Using TS-1M, we conduct a unified benchmark across three representative learning paradigms: classical supervised models, self-supervised pretrained models, and multimodal vision-language models (VLMs). Our analysis reveals consistent paradigm-dependent behaviors, showing that semantic alignment is a key factor for cross-region generalization and rare-category recognition, while purely visual models remain sensitive to appearance shift and data imbalance. Finally, we validate the practical relevance of TS-1M through real-scene autonomous driving experiments, where traffic sign recognition is integrated with semantic reasoning and spatial localization to support map-level decision constraints. Overall, TS-1M establishes a reference-level diagnostic benchmark for TSR and provides principled insights into robust and semantic-aware traffic sign perception. Project page: https://guoyangzhao.github.io/projects/ts1m.

Figures & tables

No.	Name	Region	Class Num.	Semantic Name	Examples	Year
1	STS [ 29 ]	Sverige	20	✓	5582	2011
2	LISA [ 1 ]	America	47	✓	7855	2012
3	GTSRB [ 20 ]	Germany	43	✗	51839	2013
4	BelgiumTS [ 15 ]	Belgium	62	✗	7125	2014
5	TT100K [ 21 ]	China	127	✗	30000	2016
6	RTSD [ 16 ]	Russia	106	✗	105509	2016

TABLE I: Comparison of Mainstream TSR Datasets.

Fig. 2: Overview of the TS-1M dataset construction process. The pipeline covers multi-source data collection, unified image preprocessing, semantic label normalization across heterogeneous datasets, and data augmentation to support large-scale, consistent, and robust benchmarking for TSR.

Fig. 3: Statistical overview of the TS-1M dataset. The statistics summarize the spatial geographic distribution, category-wise sample distribution, and pixel-resolution distribution of images in TS-1M.

Fig. 4: Examples of challenge-oriented sets in TS-1M. The figure illustrates representative samples from the cross-region, rare-class, low-clarity, and semantic description sets designed for systematic TSR evaluation.

Fig. 5: Modeling frameworks of three representative learning paradigms in TS-1M. The figure compares classic supervised models, self-supervised pretrained models, and VLMs from the perspectives of training pipelines and inference mechanisms.

Panel of Table 6 — TABLE II: Theoretical Capability Analysis of Different Model Categories Across Critical TSR Dimensions. This table reflects modeling hypotheses based on architecture design and prior literature. The empirical evaluation is provided in Section V .

Model	Model Variant	Pretraining Type / Structure	Key Characteristics for TSR Task
ResNet50 [ 6 ]	ResNet50	Standard CNN	Robust baseline with fast inference and stable performance on large, clean signs
ResNet101 [ 6 ]	ResNet101	Standard CNN	Higher-capacity backbone; improved precision under well-lit, structured scenes
ResNeXt50 [ 59 ]	ResNext50	Standard CNN	Flexible feature encoding for fine-grained visual distinctions
ShuffleNetV2 [ 62 ]	ShuffleNetV2	Lightweight CNN	Extremely efficient, ideal for embedded or low-power roadside devices
MobileNetV3 [ 61 ]	MobileNetV3 Small	Lightweight CNN	Well-balanced speed-accuracy design; robust for on-vehicle deployment
EfficientNetV2 [ 75 ]	EfficientNetV2 Small	Scalable CNN	High efficiency under varying sign sizes via compound scaling

TABLE III: Unified Comparison of Benchmarked Models for TSR. The descriptions emphasize structural and learning characteristics rather than task-specific performance categories.

Method	Model	Params (M) $\downarrow$	FLOPs (G) $\downarrow$	Acc.@1 $\uparrow$	Pre. $\uparrow$	Recall $\uparrow$	F-1 $\uparrow$	$\Delta$ (%) $\uparrow$
Classic Model	ResNet50 [ 6 ]	25.6	4.09	0.9260	0.8724	0.8633	0.8570	-
	ResNet101 [ 6 ]	44.5	7.80	0.9315	0.8899	0.8979	0.8844	+0.55
	ResNext50 [ 59 ]	25.0	4.23	0.9403	0.9005	0.9007	0.8920	+1.43
	ShufflenetV2 [ 62 ]	2.28	0.15	0.9234	0.8583	0.8811	0.8592	-0.26
	MobileNetV3 [ 61 ]	2.54	0.06	0.9265	0.8876	0.8653	0.8628	+0.05
	EfficientNetV2 [ 75 ]	21.46	9.72	0.9372	0.9135	0.8954	0.8977	+1.12

TABLE IV: Overall Benchmark Results on the TS-1M Dataset. The table compares model performance and computational cost across classic supervised models, self-supervised pretrained models, and VLMs.

\Delta

indicates the Acc.@1 relative improvement with respect to the ResNet50 baseline. and highlight the best and second-best results, respectively.

Method	Model	TT100K	GTSRB	PTSD	TurkeyTS	PolishTS	BelgiumTS	RTSD	MTSD	DFG	ARTS	Avg.	$\Delta$ (%)
Method	Model	China	Germany	Iran	Turkey	Polish	Belgium	Russia	World	Slovenia	America	Avg.	$\Delta$ (%)
Classic Model	ResNet50 [ 6 ]	0.9679	0.9994	0.9576	0.9283	0.8757	0.9847	0.9809	0.8554	0.8610	0.9255	0.9150	-
	ResNet101 [ 6 ]	0.9813	0.9998	0.9576	0.9350	0.8834	0.9861	0.9899	0.8552	0.8864	0.9492	0.9208	+0.58
	ResNext50 [ 59 ]	0.9836	0.9995	0.9686	0.9541	0.8776	0.9889	0.9941	0.8711	0.9284	0.9642	0.9318	+1.68
	ShufflenetV2 [ 62 ]	0.9784	0.9996	0.9588	0.9390	0.8914	0.9833	0.9908	0.8367	0.8446	0.9397	0.9117	-0.34
	MobileNetV3 [ 61 ]	0.9763	0.9984	0.9619	0.9351	0.8768	0.9796	0.9904	0.8555	0.8560	0.9273	0.9177	+0.27

TABLE V: Results of Cross-Region Recognition. Accuracy @1 is reported on multiple regional datasets, where Avg. denotes the mean performance and

\Delta

denotes the relative improvement over the ResNet50 baseline. and highlight the best and second-best results, respectively.

Method	Model	Rare $<$ 40					Rare $<$ 50
Method	Model	Acc.@1	Pre.	Recall	F-1	$\Delta$ (%)	Acc.@1	Pre.	Recall	F-1	$\Delta$ (%)
Classic Model	ResNet50 [ 6 ]	0.8381	0.9460	0.8373	0.8636	-	0.8438	0.9469	0.8424	0.8683	-
	ResNet101 [ 6 ]	0.8610	0.9532	0.8597	0.8767	+2.29	0.8639	0.9521	0.8625	0.8792	+2.01
	ResNext50 [ 59 ]	0.8530	0.9518	0.8525	0.8745	+1.49	0.8601	0.9519	0.8568	0.8794	+1.63
	ShufflenetV2 [ 62 ]	0.8334	0.9303	0.8307	0.8509	-0.47	0.8395	0.9281	0.8363	0.8546	-0.43
	MobileNetV3 [ 61 ]	0.8090	0.9326	0.8071	0.8306	-2.91	0.8179	0.9314	0.8148	0.8366	-2.59

TABLE VI: Performance on Rare-Class Recognition in the TS-1M Benchmark. Results are reported on long-tailed traffic sign categories with fewer than 40 samples ( Rare

<

40 ) and fewer than 50 samples ( Rare

<

50 ) per class, evaluating model robustness under data-scarce conditions.

\Delta

denotes the Acc.@1 relative improvement over the ResNet50 baseline. and highlight the best and second-best results, respectively.

Model	Acc.@1	Pre.	Recall	F-1	$\Delta$ (%)
ResNet50 [ 6 ]	0.8768	0.7707	0.7768	0.7553	-
ResNet101 [ 6 ]	0.8876	0.7824	0.8414	0.7948	+1.08
ResNext50 [ 59 ]	0.9055	0.8005	0.8512	0.8123	+2.87
ShufflenetV2 [ 62 ]	0.8691	0.7284	0.8082	0.7468	-0.77
MobileNetV3 [ 61 ]	0.8789	0.7882	0.7925	0.7729	+0.21
EfficientNetV2 [ 75 ]	0.9014	0.8284	0.8325	0.8164	+2.46

TABLE VII: Performance on Low-Clarity Samples in the TS-1M Benchmark. Results are reported on images with resolutions smaller than

30\times 30

pixels, evaluating model robustness under severe visual degradation.

\Delta

denotes the Acc.@1 relative improvement over the ResNet50 baseline. and highlight the best and second-best results, respectively.

Model	Semantic Description			Zero-shot				Finetuned w/o Description				Finetuned w/ Description
Model	Class	Scenario	Rules	Acc.@1	Pre.	Recall	F-1	Acc.@1	Pre.	Recall	F-1	Acc.@1	Pre.	Recall	F-1
CLIP [ 71 ]	✓	✗	✗	0.0664	0.1227	0.0918	0.0730	0.9554	0.9359	0.9040	0.9105	0.9426	0.9094	0.8895	0.8849
	✓	✓	✗	0.0805	0.1067	0.1051	0.0782	0.8149	0.7621	0.8394	0.7436	0.9514	0.9043	0.9009	0.8928
	✓	✗	✓	0.0851	0.1103	0.1057	0.0794	0.7450	0.6827	0.8032	0.6695	0.9530	0.9179	0.9116	0.9014
	✓	✓	✓	0.0971	0.1166	0.1068	0.0822	0.6739	0.6612	0.7753	0.6281	0.9581	0.9345	0.9161	0.9160
BLIP [ 72 ]	✓	✗	✗	0.1432	0.2060	0.1969	0.1426	0.9448	0.8872	0.8949	0.8770	0.8958	0.8111	0.8574	0.8158

TABLE VIII: Effect of Semantic Description Enhancement on VLMs. The table compares zero-shot inference, fine-tuning without semantic descriptions, and fine-tuning with semantic descriptions under different prompt compositions (class name, scenario, and rules). and highlight the best and second-best results, respectively.

Fig. 6: t-SNE visualization of feature representations learned by different models on TS-1M. Classic supervised models exhibit scattered and overlapping clusters, while self-supervised models achieve improved compactness. VLMs form the most structured and semantically coherent clusters, highlighting the benefit of image-text alignment for TSR.

Fig. 7: Grad-CAM visualization of model attention on traffic sign. The result illustrates differences in spatial attention patterns across model categories, with stronger CNNs and VLMs focusing on more informative and semantically relevant regions, compared with lightweight and reconstruction-based models.

Model	Training (h/epoch) $\downarrow$ 4 × A100 GPU	Testing (FPS) $\uparrow$ 1 × 5090 GPU
ResNet50 [ 6 ]	0.09	434
ResNet101 [ 6 ]	0.12	218
ResNext50 [ 59 ]	0.11	393
ShufflenetV2 [ 62 ]	0.03	411
MobileNetV3 [ 61 ]	0.03	446
EfficientNetV2 [ 75 ]	0.13	162

TABLE IX: Time Consumption on the TS-1M Dataset. Training time per epoch and inference FPS are reported for all evaluated models. and highlight the best and second-best results, respectively.

Fig. 8: Real-scene autonomous driving experiment for semantic traffic sign integration. (a) Autonomous vehicle with multi-sensor setup for map construction and traffic sign perception. (b) Driving route and scene settings with diverse road structures. (c) Visualization of TSR and semantic registration in the LiDAR point cloud map, where each sign is associated with its category, driving rules, and spatial location for downstream planning and decision-making.

Explore similar work

Distilling Vision-Language Models for Robust Traffic Sign Perception in Autonomous Vehicles
Aug 9, 2026Pedram MohajerAnsari, Amir Salarpour, Mert D. PeséAdversarial Robustness VLM Distillation
Clemson University, Clemson, SC, USA
Multi-Modal Traffic Sign Detection with Semantic Attributes for Autonomous Driving
Aug 21, 2026Meda Lazar, Sourab Sridhar, Shashwata Gupta +3Autonomous Driving Perception Object Detection
Arriver System Software S.r.l., Romania · Qualcomm Auto Ltd Sweden Filial · Qualcomm Auto Ltd., UK +1
Traffic Sign Recognition for Autonomous Driving Using Branched YOLOv2 and Geometric Features
Sep 18, 2026Arefeh RezaeiAutonomous Driving Perception Efficient Object Detection
Department of Computer Engineering, K. N. Toosi University of Technology

Aug 9, 2026J/K move · Enter open · S save

Distilling Vision-Language Models for Robust Traffic Sign Perception in Autonomous Vehicles

Pedram MohajerAnsari, Amir Salarpour, Mert D. Pesé

Clemson University, Clemson, SC, USA

Adversarial Robustness VLM Distillation Autonomous Driving Perception Physical Adversarial Attacks

Open paper AskPDF arXiv

Traffic sign recognition (TSR) models based on deep neural networks achieve strong clean-data performance but remain vulnerable to physically realizable adversarial attacks, including shadow perturbations, natural-light interference, and printed patches. Existing defenses often improve robustness against one attack type while degrading performance on others, and can reduce clean accuracy. We propose LAMDA (Language-Anchored Model for Direction Alignment), a training framework that transfers language-grounded structure into TSR models without using adversarial examples or adding inference-time overhead. LAMDA builds two fixed prototype banks from VLM-generated sign descriptions and class names using a frozen OpenCLIP text encoder, and uses them to supervise visual features through two complementary auxiliary losses during training. At inference, the adapter and prototype banks are discarded, leaving a standard backbone and classifier. Evaluated on GTSRB and LISA across four backbones and three physical attack types, LAMDA is the only method among ten evaluated that consistently improves robustness across all attack-backbone-dataset combinations, with gains of up to +12.5 pp under shadow attacks and +13.2 pp under natural-light attacks, while preserving or improving clean accuracy in nearly all cases.

Abstract

Explore similar work

Distilling Vision-Language Models for Robust Traffic Sign Perception in Autonomous Vehicles

Multi-Modal Traffic Sign Detection with Semantic Attributes for Autonomous Driving

Traffic Sign Recognition for Autonomous Driving Using Branched YOLOv2 and Geometric Features