Traffic Sign Recognition in Autonomous Driving: Dataset, Benchmark, and Field Experiment
Authors: Guoyang Zhao, Weiqing Qi, Kai Zhang, Chenguang Zhang, Zeying Gong, Zhihai Bi, Kai Chen, Benshan Ma, +2 more
Organizations: The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511453, China · Lingnan University, Hong Kong SAR, China · Shenzhen Unity Drive Innovation Technology Co., Ltd., Shenzhen 518063, China · The Hong Kong University of Science and Technology, Hong Kong SAR, China
Traffic Sign Recognition (TSR) is a core perception capability for autonomous driving, where robustness to cross-region variation, long-tailed categories, and semantic ambiguity is essential for reliable real-world deployment. Despite steady progress in recognition accuracy, existing traffic sign datasets and benchmarks offer limited diagnostic insight into how different modeling paradigms behave under these practical challenges. We present TS-1M, a large-scale and globally diverse traffic sign dataset comprising over one million real-world images across 454 standardized categories, together with a diagnostic benchmark designed to analyze model capability boundaries. Beyond standard train-test evaluation, we provide a suite of challenge-oriented settings, including cross-region recognition, rare-class identification, low-clarity robustness, and semantic text understanding, enabling systematic and fine-grained assessment of modern TSR models. Using TS-1M, we conduct a unified benchmark across three representative learning paradigms: classical supervised models, self-supervised pretrained models, and multimodal vision-language models (VLMs). Our analysis reveals consistent paradigm-dependent behaviors, showing that semantic alignment is a key factor for cross-region generalization and rare-category recognition, while purely visual models remain sensitive to appearance shift and data imbalance. Finally, we validate the practical relevance of TS-1M through real-scene autonomous driving experiments, where traffic sign recognition is integrated with semantic reasoning and spatial localization to support map-level decision constraints. Overall, TS-1M establishes a reference-level diagnostic benchmark for TSR and provides principled insights into robust and semantic-aware traffic sign perception. Project page: https://guoyangzhao.github.io/projects/ts1m.
Figures & tables
No.
Name
Region
Class Num.
Semantic Name
Examples
Year
1
STS [ 29 ]
Sverige
20
✓
5582
2011
2
LISA [ 1 ]
America
47
✓
7855
2012
3
GTSRB [ 20 ]
Germany
43
✗
51839
2013
4
BelgiumTS [ 15 ]
Belgium
62
✗
7125
2014
5
TT100K [ 21 ]
China
127
✗
30000
2016
6
RTSD [ 16 ]
Russia
106
✗
105509
2016
TABLE I: Comparison of Mainstream TSR Datasets.
Fig. 2: Overview of the TS-1M dataset construction process. The pipeline covers multi-source data collection, unified image preprocessing, semantic label normalization across heterogeneous datasets, and data augmentation to support large-scale, consistent, and robust benchmarking for TSR.
Fig. 3: Statistical overview of the TS-1M dataset. The statistics summarize the spatial geographic distribution, category-wise sample distribution, and pixel-resolution distribution of images in TS-1M.
Fig. 4: Examples of challenge-oriented sets in TS-1M. The figure illustrates representative samples from the cross-region, rare-class, low-clarity, and semantic description sets designed for systematic TSR evaluation.
Fig. 5: Modeling frameworks of three representative learning paradigms in TS-1M. The figure compares classic supervised models, self-supervised pretrained models, and VLMs from the perspectives of training pipelines and inference mechanisms.
TABLE II: Theoretical Capability Analysis of Different Model Categories Across Critical TSR Dimensions. This table reflects modeling hypotheses based on architecture design and prior literature. The empirical evaluation is provided in Section V .
Model
Model Variant
Pretraining Type / Structure
Key Characteristics for TSR Task
ResNet50 [ 6 ]
ResNet50
Standard CNN
Robust baseline with fast inference and stable performance on large, clean signs
ResNet101 [ 6 ]
ResNet101
Standard CNN
Higher-capacity backbone; improved precision under well-lit, structured scenes
ResNeXt50 [ 59 ]
ResNext50
Standard CNN
Flexible feature encoding for fine-grained visual distinctions
ShuffleNetV2 [ 62 ]
ShuffleNetV2
Lightweight CNN
Extremely efficient, ideal for embedded or low-power roadside devices
MobileNetV3 [ 61 ]
MobileNetV3 Small
Lightweight CNN
Well-balanced speed-accuracy design; robust for on-vehicle deployment
EfficientNetV2 [ 75 ]
EfficientNetV2 Small
Scalable CNN
High efficiency under varying sign sizes via compound scaling
TABLE III: Unified Comparison of Benchmarked Models for TSR. The descriptions emphasize structural and learning characteristics rather than task-specific performance categories.
Method
Model
Params (M) ↓
FLOPs (G) ↓
Acc.@1 ↑
Pre. ↑
Recall ↑
F-1 ↑
Δ (%) ↑
Classic Model
ResNet50 [ 6 ]
25.6
4.09
0.9260
0.8724
0.8633
0.8570
-
ResNet101 [ 6 ]
44.5
7.80
0.9315
0.8899
0.8979
0.8844
+0.55
ResNext50 [ 59 ]
25.0
4.23
0.9403
0.9005
0.9007
0.8920
+1.43
ShufflenetV2 [ 62 ]
2.28
0.15
0.9234
0.8583
0.8811
0.8592
-0.26
MobileNetV3 [ 61 ]
2.54
0.06
0.9265
0.8876
0.8653
0.8628
+0.05
EfficientNetV2 [ 75 ]
21.46
9.72
0.9372
0.9135
0.8954
0.8977
+1.12
TABLE IV: Overall Benchmark Results on the TS-1M Dataset. The table compares model performance and computational cost across classic supervised models, self-supervised pretrained models, and VLMs. Δ indicates the Acc.@1 relative improvement with respect to the ResNet50 baseline. and highlight the best and second-best results, respectively.
Method
Model
TT100K
GTSRB
PTSD
TurkeyTS
PolishTS
BelgiumTS
RTSD
MTSD
DFG
ARTS
Avg.
Δ (%)
China
Germany
Iran
Turkey
Polish
Belgium
Russia
World
Slovenia
America
Classic Model
ResNet50 [ 6 ]
0.9679
0.9994
0.9576
0.9283
0.8757
0.9847
0.9809
0.8554
0.8610
0.9255
0.9150
-
ResNet101 [ 6 ]
0.9813
0.9998
0.9576
0.9350
0.8834
0.9861
0.9899
0.8552
0.8864
0.9492
0.9208
+0.58
ResNext50 [ 59 ]
0.9836
0.9995
0.9686
0.9541
0.8776
0.9889
0.9941
0.8711
0.9284
0.9642
0.9318
+1.68
ShufflenetV2 [ 62 ]
0.9784
0.9996
0.9588
0.9390
0.8914
0.9833
0.9908
0.8367
0.8446
0.9397
0.9117
-0.34
MobileNetV3 [ 61 ]
0.9763
0.9984
0.9619
0.9351
0.8768
0.9796
0.9904
0.8555
0.8560
0.9273
0.9177
+0.27
TABLE V: Results of Cross-Region Recognition. Accuracy @1 is reported on multiple regional datasets, where Avg. denotes the mean performance and Δ denotes the relative improvement over the ResNet50 baseline. and highlight the best and second-best results, respectively.
Method
Model
Rare < 40
Rare < 50
Acc.@1
Pre.
Recall
F-1
Δ (%)
Acc.@1
Pre.
Recall
F-1
Δ (%)
Classic Model
ResNet50 [ 6 ]
0.8381
0.9460
0.8373
0.8636
-
0.8438
0.9469
0.8424
0.8683
-
ResNet101 [ 6 ]
0.8610
0.9532
0.8597
0.8767
+2.29
0.8639
0.9521
0.8625
0.8792
+2.01
ResNext50 [ 59 ]
0.8530
0.9518
0.8525
0.8745
+1.49
0.8601
0.9519
0.8568
0.8794
+1.63
ShufflenetV2 [ 62 ]
0.8334
0.9303
0.8307
0.8509
-0.47
0.8395
0.9281
0.8363
0.8546
-0.43
MobileNetV3 [ 61 ]
0.8090
0.9326
0.8071
0.8306
-2.91
0.8179
0.9314
0.8148
0.8366
-2.59
TABLE VI: Performance on Rare-Class Recognition in the TS-1M Benchmark. Results are reported on long-tailed traffic sign categories with fewer than 40 samples ( Rare < 40 ) and fewer than 50 samples ( Rare < 50 ) per class, evaluating model robustness under data-scarce conditions. Δ denotes the Acc.@1 relative improvement over the ResNet50 baseline. and highlight the best and second-best results, respectively.
Model
Acc.@1
Pre.
Recall
F-1
Δ (%)
ResNet50 [ 6 ]
0.8768
0.7707
0.7768
0.7553
-
ResNet101 [ 6 ]
0.8876
0.7824
0.8414
0.7948
+1.08
ResNext50 [ 59 ]
0.9055
0.8005
0.8512
0.8123
+2.87
ShufflenetV2 [ 62 ]
0.8691
0.7284
0.8082
0.7468
-0.77
MobileNetV3 [ 61 ]
0.8789
0.7882
0.7925
0.7729
+0.21
EfficientNetV2 [ 75 ]
0.9014
0.8284
0.8325
0.8164
+2.46
TABLE VII: Performance on Low-Clarity Samples in the TS-1M Benchmark. Results are reported on images with resolutions smaller than 30×30 pixels, evaluating model robustness under severe visual degradation. Δ denotes the Acc.@1 relative improvement over the ResNet50 baseline. and highlight the best and second-best results, respectively.
Model
Semantic Description
Zero-shot
Finetuned w/o Description
Finetuned w/ Description
Class
Scenario
Rules
Acc.@1
Pre.
Recall
F-1
Acc.@1
Pre.
Recall
F-1
Acc.@1
Pre.
Recall
F-1
CLIP [ 71 ]
✓
✗
✗
0.0664
0.1227
0.0918
0.0730
0.9554
0.9359
0.9040
0.9105
0.9426
0.9094
0.8895
0.8849
✓
✓
✗
0.0805
0.1067
0.1051
0.0782
0.8149
0.7621
0.8394
0.7436
0.9514
0.9043
0.9009
0.8928
✓
✗
✓
0.0851
0.1103
0.1057
0.0794
0.7450
0.6827
0.8032
0.6695
0.9530
0.9179
0.9116
0.9014
✓
✓
✓
0.0971
0.1166
0.1068
0.0822
0.6739
0.6612
0.7753
0.6281
0.9581
0.9345
0.9161
0.9160
BLIP [ 72 ]
✓
✗
✗
0.1432
0.2060
0.1969
0.1426
0.9448
0.8872
0.8949
0.8770
0.8958
0.8111
0.8574
0.8158
TABLE VIII: Effect of Semantic Description Enhancement on VLMs. The table compares zero-shot inference, fine-tuning without semantic descriptions, and fine-tuning with semantic descriptions under different prompt compositions (class name, scenario, and rules). and highlight the best and second-best results, respectively.
Fig. 6: t-SNE visualization of feature representations learned by different models on TS-1M. Classic supervised models exhibit scattered and overlapping clusters, while self-supervised models achieve improved compactness. VLMs form the most structured and semantically coherent clusters, highlighting the benefit of image-text alignment for TSR.
Fig. 7: Grad-CAM visualization of model attention on traffic sign. The result illustrates differences in spatial attention patterns across model categories, with stronger CNNs and VLMs focusing on more informative and semantically relevant regions, compared with lightweight and reconstruction-based models.
Model
Training (h/epoch) ↓ 4 × A100 GPU
Testing (FPS) ↑ 1 × 5090 GPU
ResNet50 [ 6 ]
0.09
434
ResNet101 [ 6 ]
0.12
218
ResNext50 [ 59 ]
0.11
393
ShufflenetV2 [ 62 ]
0.03
411
MobileNetV3 [ 61 ]
0.03
446
EfficientNetV2 [ 75 ]
0.13
162
TABLE IX: Time Consumption on the TS-1M Dataset. Training time per epoch and inference FPS are reported for all evaluated models. and highlight the best and second-best results, respectively.
Fig. 8: Real-scene autonomous driving experiment for semantic traffic sign integration. (a) Autonomous vehicle with multi-sensor setup for map construction and traffic sign perception. (b) Driving route and scene settings with diverse road structures. (c) Visualization of TSR and semantic registration in the LiDAR point cloud map, where each sign is associated with its category, driving rules, and spatial location for downstream planning and decision-making.
Traffic sign recognition (TSR) models based on deep neural networks achieve strong clean-data performance but remain vulnerable to physically realizable adversarial attacks, including shadow perturbations, natural-light interference, and printed patches. Existing defenses often improve robustness against one attack type while degrading performance on others, and can reduce clean accuracy. We propose LAMDA (Language-Anchored Model for Direction Alignment), a training framework that transfers language-grounded structure into TSR models without using adversarial examples or adding inference-time overhead. LAMDA builds two fixed prototype banks from VLM-generated sign descriptions and class names using a frozen OpenCLIP text encoder, and uses them to supervise visual features through two complementary auxiliary losses during training. At inference, the adapter and prototype banks are discarded, leaving a standard backbone and classifier. Evaluated on GTSRB and LISA across four backbones and three physical attack types, LAMDA is the only method among ten evaluated that consistently improves robustness across all attack-backbone-dataset combinations, with gains of up to +12.5 pp under shadow attacks and +13.2 pp under natural-light attacks, while preserving or improving clean accuracy in nearly all cases.
Pedram MohajerAnsari, Amir Salarpour, Mert D. Pesé
Reliable traffic sign detection is a prerequisite for the global deployment of autonomous driving systems, where regulatory compliance and road safety depend on perceiving signs correctly across regions, ranges, and weather conditions. Despite recent progress, vision-based methods continue to face three fundamental limitations: poor cross-regional generalization due to high diversity across countries, degraded performance on small-object detection at long ranges (traffic signs occupy as little as 10×10 pixels at 200m), and fragile temporal tracking under the strongly non-linear perspective distortion that occurs as a vehicle approaches a sign. In this paper, we address the problem of robust, long-range, region-agnostic traffic sign perception by combining camera and Light Detection and Ranging (LiDAR) sensing. We present a multi-modal detection framework whose Intensity-Aware Deformable Fusion module aligns retro-reflective LiDAR cues with camera features, anchoring detection on geometric invariants rather than region-specific visual appearance. We further introduce a dual motion-model tracker that explicitly accounts for non-linear perspective transformations during vehicle approach, substantially improving temporal consistency over linear motion assumptions. Additionally, we develop a semantic attribute classification pipeline that estimates occlusion level, readability, sign embeddedness, and road relevance, providing actionable context to downstream planning. Extensive evaluation on our dataset, spanning 60+ countries and 2,500+ hours of driving data, shows that the proposed pipeline achieves an Object Miss Ratio (OMR) of 0.49% across 221,068 evaluation sequences, demonstrating globally generalizable traffic sign perception in commercial-grade autonomous driving systems.
Meda Lazar, Sourab Sridhar, Shashwata Gupta +3
Arriver System Software S.r.l., Romania · Qualcomm Auto Ltd Sweden Filial · Qualcomm Auto Ltd., UK +1
Traffic sign recognition (TSR) is an important perception task for autonomous driving and advanced driver-assistance systems, where a system must both localize traffic signs and determine their semantic classes efficiently. This work presents a TSR system based on YOLOv2 for simultaneous detection and classification. Two complementary modifications are studied. First, YOLOv2 is extended with intermediate prediction layers, forming a branched architecture that can terminate inference early for easy cases and reduce computation time. Both whole-image and cell-wise branching strategies are investigated. Second, geometric information is introduced to reduce classification errors between visually similar signs. An unsupervised Bayesian image-segmentation method produces binary representations that are compared with class-specific geometric templates inside YOLOv2 bounding boxes. This information is used either during inference or as an additional signal during training. A dedicated dataset is constructed by combining GTSDB and GTSRB samples using seamless cloning and controlled image transformations. Experiments cover ten traffic-sign classes, with 3,000 training and 300 test samples. The selected branched architecture reports 0.647 s runtime and 0.680 mAP, compared with 0.6607 s and 0.680 mAP for baseline YOLOv2. Geometric verification during inference increases mAP to 0.713, while the geometric-feature training variant achieves 0.697 mAP with a reported runtime of 0.6608 s.
Arefeh Rezaei
Department of Computer Engineering, K. N. Toosi University of Technology