cs.ROOct 9, 2026

Towards Safer Cities: A Design Process for Perception Datasets in Urban Driving Scenarios

Authors: Alexander Kroth, Cristian Martinez, Steven Peters

Organizations: Institute of Automotive Engineering Darmstadt (FZD), TU Darmstadt, Otto-Berndt-Str. 2, 64287 Darmstadt, Germany

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Urban environments present significant challenges for the perception systems of automated driving due to their high complexity, characterized by dense traffic, diverse obstacles, or a wide range of possible scenarios. While numerous automotive datasets exist, few focus on urban settings, and the majority are acquired exclusively on passenger cars and overlook other transport modalities such as heavy-duty trucks, busses and trams. In contrast to passenger vehicles, these platforms operate under different geographical and operational constraints; including traversing pedestrian zones or platform-specific tasks such as movement of trailer or winter service. In this work a method is proposed for a guided dataset creation based on an analysis of existing datasets using user-generated map data. Furthermore, representative truck-specific use cases are identified and exemplified to demonstrate the applicability of the proposed methodology and to support targeted acquisition planning.

Figures & tables

Figure 1: Acquisition strategy is informed by missing ODD entities acquired by comparing defined ODD and recorded ODD
LayerAccessibility in DatasetsPlanning for RecordingExternal Resources
Digital Information–+-
Environmental ConditionsooOpen-Meteo
Dynamic Objects++–-
Temporary Modifications of L1 and L2––-
Roadside Structureso+OpenStreetMap
Road Network and Traffic Guidance Objects+++OpenStreetMap
Table 1: Evaluation of the information layers of the six layer model (top) and the additional categories proposed by Mendiboure et al. (bottom) with respect to dataset acquisition planning. Symbols denote low (–), medium (o), high (+), and very high (++) availability in existing datasets and ease of targeted acquisition. External data sources that can support the recording process are listed in the third column.
Figure 2: Relationship between the coverage of existing automotive and rail-specific datasets, the urban ODD, and the proposed truck and tram ODDs. The green region highlights scenarios that are relevant to both truck and tram operation and therefore represent the most beneficial targets for future data acquisition.
 nuScenes [ 3 ]NVIDIA Physical AI [ 15 ]TruckScenes [ 5 ]TruckDrive [ 6 ]
Scenes1000300k7473828
ScopeUrbanUrbanHighway, Urban, TerminalHighway
RegionUS, AsiaEurope, USEuropeUS
Camera67411
LiDAR11610
Radar510610
Table 2: Comparison of the most relevant automotive perception datasets considered for this work. With respect to the NVIDIA dataset, zero classes were considered since these are not available at the time this paper is being written.
 RailSem19 [ 24 ]OSDaR23 [ 20 ]RailGoerl24 [ 21 ]Hovanotayan et al. [ 7 ]
Scenes85004512.2 kNA
ScopeUrban, regular operationRegular operationClosed-loopUrban
RegionWorldwideEuropeEuropeAsia
Camera1911
LiDAR0601
Radar0110
Table 3: Comparison of publicly available rail-specific perception datasets.
Figure 3: Pipeline for extracting road infrastructure metadata from driving datasets. GPS coordinates are extracted from the dataset recordings, normalized to a common format, queried against OpenStreetMap via Overpass, and converted into structured JSON metadata.
Figure 4: Pipeline for extracting infrastructure metadata from the Darmstadt road network. Roads that are inaccessible to trucks and road types outside the scope of the proposed datasets are filtered out before querying OpenStreetMap. Relevant infrastructure attributes are subsequently extracted
Figure 5: Distribution of intersection road widths in the analyzed area. Each point represents a pair of intersecting roads, grouped according to their representative widths.
Figure 6: Spatial distribution of intersection candidates within the analyzed area around Darmstadt. The red marker indicates the center point at Luisenplatz, and the red circle denotes the search radius. Yellow points mark the identified intersection locations.
More in this topic

Autonomous Driving Datasets

Autonomous driving datasets collect and organize driving-scene data to train and evaluate perception, prediction, planning, and safety systems.

See all 141 papers

Explore similar work

CardsList

  1. VRUD: A Drone Dataset for Complex Vehicle-VRU Interactions within Mixed Traffic

    Apr 1, 2026Ziyu Wang, Hongrui Kou, Cheng Wang +4Autonomous Driving DatasetsAutonomous Driving

    National Key Laboratory of Automotive Chassis Integration and Bionics, Jilin University, Changchun 130025, China · DRIVEResearch, Changchun, China · School of Engineering and Physical Sciences, Heriot-Watt University, Edinburgh EH1 4AS, U.K. +1

  2. Creating Impactful Autonomous Driving Datasets: A Strategic Guide from Research Gap to Benchmark

    Jul 1, 2026Richard Schwarzkopf, Jonas Merkert, Frank Bieder +22Autonomous Driving DatasetsAutonomous Driving Benchmarks

    Karlsruhe Institute of Technology (KIT) · FZI Research Center for Information Technology, Karlsruhe

  3. TaCarla: A comprehensive benchmarking dataset for end-to-end autonomous driving

    Feb 26, 2026Tugrul Gorgulu, Atakan Dag, M. Esat Kalfaoglu +4Autonomous Driving DatasetsAutonomous Driving Benchmarks

    The authors are currently with Trutek AI. · This work was conducted at Trutek AI. The author is currently with Ultralytics. · This work was conducted at Trutek AI. The author is currently with Amazon.