WorkIng: A Real-World and Synthetic Highway Work-Zone Benchmark Dataset for Autonomous Navigation

Xujun Xie; Dominik Rößle; Adithya Mohan; Daniel Cremers; Torsten Schön

cs.CVOct 30, 2026

WorkIng: A Real-World and Synthetic Highway Work-Zone Benchmark Dataset for Autonomous Navigation

Authors: Xujun Xie, Dominik Rößle, Adithya Mohan, Daniel Cremers, Torsten Schön

Organizations: AImotion Bavaria, Technische Hochschule Ingolstadt, Germany · Technische Universität München, Germany

Abstract

Highway work zones are critical edge cases for autonomous vehicles (AVs), combining atypical geometries and jurisdiction-specific regulations that existing benchmarks fail to capture: real-world datasets lack 360° multi-modal, full-scene 3D work zone annotation, and no single recorded site can span the layouts and regulatory variants needed for generalizable models. We present WorkIng, a dual-domain benchmark and training resource. WorkIng-Highway provides a 360° multi-modal work zone benchmark with synchronized 128-channel LiDAR, six RGB cameras, and RTK-GPS across 32 km, yielding 12,000+ annotated frames and 227,399 tracked 3D bounding boxes. WorkIng-CARLA complements this with procedural simulation spanning 24 regulation-aware scenarios across six canonical topologies and six international traffic standards, populated by 100+ custom 3D assets. Benchmarking representative 3D detection and end-to-end planning architectures shows that real-world data remains the primary performance driver, and that scenario-stratified synthetic augmentation improves architectures across 2D recognition, 3D detection, and trajectory planning. Resources: https://github.com/cvims/WorkIng

Figures & tables

Dataset	WZ	Sens.	Cam.	Ego GT	3D	Lbl.
KITTI [ 17 ]	Low	C+L	$90^{\circ}$	RTK-GPS	Yes	H
nuScenes [ 6 ]	Low	C+L	$360^{\circ}$	RTK-GPS	Yes	H
Waymo [ 40 ]	Low	C+L	$360^{\circ}$	RTK-GPS	Yes	H
DrivIng [ 33 ]	Low	C+L	$360^{\circ}$	RTK-GPS	Yes	H
BDD100K [ 48 ]	Low	C	Front	No	No	H
ROADWork [ 18 ]	High	C	Front	Est. (Vis)	No	H+A

Table 1: Comparison of autonomous driving datasets. WZ: Work Zone coverage; Sens.: Sensor modalities (C: Camera, L: LiDAR); Cam.: Camera Coverage (FoV); Ego GT: Ego-Motion Ground Truth; WZ only: static work-zone objects; Lbl.: Label source (H: Human, A: Automatic).

Class	Scenario	BBoxes	Tracks	Frames	Points
Car	Day	66,197	1,887	34.9	481
	Night	4,161	127	32.8	447
	CARLA	277,874	7,887	35.2	572
Van	Day	20,751	327	63.5	1,117
	Night	715	23	31.1	1,135
	CARLA	50,296	1,400	35.9	433

Table 2: Statistical overview of the WorkIng datasets. Total bounding boxes (BBoxes), unique track IDs (Tracks), average instance duration (Frames), and LiDAR point density (Points) per class. n.a. = not available.

Category	Asset Type	Visual Variations (Shape & Color)	Content / Count
Traffic Signs	Speed Limit	Rect: White, Yellow (Small/Large) Circle: White, Yellow	Range: 30–120 km/h
	Warning & Guide	Rect: White, Yellow, Blue Diamond: Yellow Triangle: White, Yellow	Directives: Left, Right, Forward, Middle, Merge, Turn, Bend
Safety Furniture	Barriers	Plastic (Red/White), Concrete, Steel, Wood	44 Variations
	Cones	Traffic Cones (Orange/Red), Cylinders	14 Variations
	Blocks	Roadblocks, Temporary Fencing	15 Variations
Markings	Lane Paint	Lines, Directional Arrows (Yellow, Orange, White)	3 Types + Directional

Table 3: WorkIng-CARLA Asset Catalog. Summary of the available 3D assets by category, visual variation (shape and color), and functional content. The library includes 100+ unique meshes and texture variations to support diverse regional standards.

Scene	Model	ATE $\downarrow$	ASE $\downarrow$	AOE $\downarrow$	mAP $\uparrow$	NDS $\uparrow$
Day	CenterPoint [ 47 ]	0.21	0.18	0.07	0.76	79.40
	PETR [ 28 ]	0.87	0.27	0.24	0.37	43.52
	BEVFusion [ 30 ]	0.23	0.23	0.14	0.75	76.59
Night	CenterPoint [ 47 ]	0.14	0.15	0.07	0.78	81.55
	PETR [ 28 ]	0.87	0.18	0.06	0.33	44.04
	BEVFusion [ 30 ]	0.22	0.25	0.20	0.78	77.71

Table 4: 3D object detection macro performance and error summary. Evaluation of baseline architectures across Day, Night, and CARLA scenarios using standard nuScenes metrics (mAP) and true-positive error metrics (ATE, ASE, AOE). Arrows (

\uparrow/\downarrow

) indicate the direction of better performance, and bold text highlights the top results per scene. Class-specific mAP score matrices are detailed in the Supplemental Material. Real-world (Day/Night) results correspond to models trained on WorkIng-Highway only; CARLA results correspond to models trained on WorkIng-CARLA.

		Day					Night
Model	Train	ATE	ASE	AOE	mAP	NDS	ATE	ASE	AOE	mAP	NDS
CenterPoint	W	0.21	0.18	0.07	0.76	79.40	0.14	0.15	0.07	0.78	81.55
	D	0.22	0.24	0.31	0.45	55.81	0.60	0.43	0.30	0.45	48.90
	C	0.50	0.61	1.07	0.13	18.37	0.69	0.67	0.45	0.25	30.36
	W+D	0.14	0.18	0.18	0.78	79.70	0.14	0.14	0.05	0.83	85.53
	W+D+C	0.14	0.18	0.17	0.79	80.75	0.13	0.12	0.04	0.86	87.63

Table 5: Ablation: training data composition for 3D detection. All models evaluated on the WorkIng-Highway test set. W = WorkIng-Highway; D = DrivIng; C = WorkIng-CARLA. ATE/ASE/AOE: lower is better; mAP/NDS: higher is better. Bold : best performance per model.

Zero-shot foundation models [ 18 ]
Model	AP	AP 50	AP 75	AP S	AP M	AP L
OpenSeeD [ 49 ]	2.9	5.8	2.5	1.3	2.8	4.0
Detic [ 52 ]	4.2	6.3	4.6	2.1	5.5	6.0
OMG-LLaVA [ 51 ]	0.2	0.3	0.2	0.0	0.1	0.8
Train: C (WorkIng-CARLA only)
Mask R-CNN [ 19 ]	4.13	5.07	4.23	2.74	3.48	5.12

Table 6: 2D asset-recognition ablation. Mask AP (%) on ROADWork’s real test set, COCO protocol. Top block: zero-shot foundation models, values as reported by ROADWork [ 18 ] (not reproduced here). Remaining blocks: this work, all three configurations trained under a single identical recipe per architecture. C = WorkIng-CARLA only; R = ROADWork only; C+R = combined. Bold : best per architecture.

Scene	Model	L2 Error (m) $\downarrow$				Collision Rate (%) $\downarrow$
Scene		0.5s	1.0s	2.0s	Avg.	0.5s	1.0s	2.0s	Avg.
Day	UniAD_tiny [ 20 ]	0.23	0.55	1.45	0.74	0.11	0.33	5.97	2.14
Day	VAD_tiny [ 23 ]	0.18	0.43	1.25	0.62	0.14	0.34	5.13	1.87
Night	UniAD_tiny [ 20 ]	0.37	0.70	1.63	0.90	0.85	0.88	4.84	2.19
Night	VAD_tiny [ 23 ]	0.35	0.86	1.60	0.94	0.85	0.88	4.76	2.16

Table 7: Quantitative comparison of planning performance across different scenes. The L2 displacement error (in meters) and the collision rate (in %) are evaluated at future time horizons of 0.5s, 1.0s, and 2.0s, along with their average. For both metrics, lower values are better (

\downarrow

). Bold values denote the best performance in each scenario.

	L2 Error (m) $\downarrow$				Collision Rate (%) $\downarrow$				LB2.0 Score $\uparrow$
Model	0.5s	1.0s	2.0s	Avg.	0.5s	1.0s	2.0s	Avg.	DS	RC	IP
UniAD_tiny [ 20 ]	0.23	0.48	1.34	0.68	0.23	0.75	2.84	1.27	0.63	0.80	0.79
VAD_tiny [ 23 ]	0.25	0.54	1.19	0.66	0.08	0.54	1.96	0.86	0.65	0.76	0.85

Table 8: Closed-loop CARLA Leaderboard evaluation.

L_{2}

error (m) and Collision Rate (%) measured under reactive closed-loop rollout, alongside Driving Score (DS, composed), Route Completion (RC), and Infraction Penalty (IP). Higher is better for DS/RC/IP; lower is better for

L_{2}

/Collision Rate. Bold : best per column.

	UniAD_tiny				VAD_tiny
	$L_{2}$		Coll.%		$L_{2}$		Coll.%
Train	D	N	D	N	D	N	D	N
W	0.74	0.90	2.14	2.19	0.62	0.94	1.87	2.16
C	7.57	8.13	6.98	5.51	8.32	9.33	6.78	5.82
W+C	0.66	0.84	1.17	1.65	0.53	0.83	1.09	1.78

Table 9: Ablation: training data composition for trajectory planning.

L_{2}

error (m) and Collision Rate (%), averaged over 0.5/1.0/2.0s horizons, on the WorkIng-Highway test set. W = WorkIng-Highway; C = WorkIng-CARLA; D/N = Day/Night. Lower is better for both metrics. Bold : best per model/scene.

Figure 2: Overhead view of the sensor layout. Labels denote the positions of six Surround-View Cameras and an IMU, with a legend defining the vehicle-fixed coordinate frame $(x,y,z)$ .

Attributes	Categories
emergency, regular, public transport	Car, Van, Bus, Truck
car trailer, truck trailer, cyclist trailer	Trailer
standing, walking, sitting	Pedestrian
bendy, rigid	Bus, Truck
None	Motorcycle, Bicycle, Barrier

Table 10: Mapping of object categories to specific attribute types within the annotation scheme.

Panel of Figure 3 — Figure 3: Range distribution per category

Figure 4: Vehicle motion distributions per class: speed (top), acceleration (middle), angular velocity (bottom). Light/dark green = WorkIng-Highway Day/Night; light/dark blue = DrivIng Day/Night. Black tick = median; dotted line separates ego from NPC classes.

Region/ Standard	Speed Limit (Shape, Border, Bg)	Warning Sign (Shape, Border, Bg)	Cone/ Barrier (Color)	Temp. Lane (Marking Color)
Australia	Circular, Red, White [ 31 , 38 ]	Diamond, Black (inset), Yellow [ 31 ]	Fluor. Orange [ 38 ]	Yellow/White [ 38 ]
China	Circular, Red, White [ 39 ]	Triangle, Black, Yellow [ 39 ]	Red/Orange [ 39 ]	Yellow/Orange [ 37 ]
Nordic	Circular, Red, Yellow [ 16 , 42 , 21 ]	Triangle, Red, Yellow [ 16 , 42 , 21 ]	Red/Orange [ 44 , 11 ]	Yellow [ 44 , 16 , 21 ]
USA	Rectangle (Vert.), Black/None, White [ 15 ]	Diamond, Black, Orange [ 15 ]	Fluor. Orange [ 15 ]	White/Yellow [ 15 ]
Europe (most) ${}^{\,\ddagger}$	Circular, Red, White [ 45 ]	Triangle, Red, Yellow [ 45 , 14 ]	Red/Orange [ 10 , 11 ]	Yellow/Orange [ 14 ]
DACH	Circular, Red, White [ 45 , 5 ]	Triangle, Red, White [ 5 ]	Red/Orange [ 10 , 11 ]	Yellow [ 5 ]

Table 11: Regional comparison of work-zone traffic signs, barriers, and temporary road markings. In the two sign columns, the sub-labels (Shape, Border, Bg) give each sign’s shape, border color, and background (Bg) color; the last two columns give cone/barrier and temporary lane-marking colors. “Nordic” covers Finland, Sweden, and Iceland. In “Europe (most)” (Western and Southern EU/EEA countries), a yellow warning-sign background signals a temporary hazard, whereas “DACH” retains the standard white warning-sign background even in work zones.

Figure 5: CARLA Asset Catalog: Traffic Cone

Figure 8: CARLA Asset catalog: Speed Regulatory Signage (30–120 km/h)

Figure 9: CARLA Asset catalog: Temporary and Fixed Warning Signage

Figure 10: CARLA Asset catalog: Road Marking

Figure 11: CARLA Asset catalog: Road Debris

Figure 12: Quantitative characterization of WorkIng-CARLA environment. (a) Distribution of maneuver-based scenarios across the dataset. (b) Regional distribution of CARLA scenarios

Scene	Model	Car	Tru.	Bus	Trai.	Bar.	Mot.	Ped.	Total
Day	CenterPoint [ 47 ]	0.95	0.89	0.37	0.87	0.86	0.68	0.71	0.76
	PETR [ 28 ]	0.60	0.47	0.18	0.39	0.39	0.45	0.12	0.37
	BEVFusion [ 30 ]	0.95	0.85	0.37	0.83	0.80	0.80	0.64	0.75
Night	CenterPoint [ 47 ]	0.88	0.87	-	0.44	0.92	-	-	0.78
	PETR [ 28 ]	0.43	0.57	-	0.01	0.30	-	-	0.33
	BEVFusion [ 30 ]	0.93	0.85	-	0.42	0.90	-	-	0.78

Table 12: Class-specific 3D object detection performance. Detailed mean Average Precision (mAP) breakdown across all evaluated object categories under Day, Night, and CARLA simulation conditions. Entries marked with “-” denote classes excluded from evaluation in that scenario, either absent from the split or below the annotation-density threshold (Section 4 ). Bold values denote the best performance in each scenario.

Autonomous Driving Benchmarks

Autonomous driving benchmarks evaluate how well vehicles perceive, predict, plan, and act safely across varied roads and conditions.

See all 149 papers

WZPlanner: Safe End-to-End Path Planning for Autonomous Driving in Work Zones
Sep 16, 2026Nishad Sahu, Changzhong Qian, Guangzhou Cai +3Autonomous Driving Datasets End-to-End Autonomous Driving
Raj · Department of Electrical and Computer Engineering Carnegie Mellon University, Pittsburgh, PA, USA
WorkDrive: Roadwork Chain of Causation for Autonomous Driving
Jul 16, 2026Tianyi Jiang, Wen Zhang, Sihan Yang +2VLMs for Autonomous Driving Autonomous Driving Perception
Peking University · Beijing, China · Xiaomi EV
Dash2Sim: Closed-Loop Driving Simulation from in-the-wild Dashcam Videos
Jun 5, 2026Anurag Ghosh, Francesco Pittaluga, Khiem Vuong +4Autonomous Driving Datasets Autonomous Driving Benchmarks
1Carnegie Mellon University · 2NEC Labs America · 3MIT∗ +1

Sep 16, 2026J/K move · Enter open · S save

WZPlanner: Safe End-to-End Path Planning for Autonomous Driving in Work Zones

Nishad Sahu, Changzhong Qian, Guangzhou Cai, Shounak Sural+2

Raj · Department of Electrical and Computer Engineering Carnegie Mellon University, Pittsburgh, PA, USA

Autonomous Driving Datasets End-to-End Autonomous Driving Autonomous Driving Autonomous Driving Perception Safe Motion Planning Autonomous Driving Planning

Open paper AskPDF arXiv

Work zones alter lane geometry through temporary traffic controls and closures that may be absent from on-board maps, challenging autonomous vehicle (AV) perception and planning. Generalization is also limited by scarce public datasets with structured geometric supervision. We present WorkZonePlan, a dataset comprising 149K+ synthetic and 5K+ real-world multimodal samples with 3D annotations for lane boundaries, work zone boundaries, and driving trajectory options. It also provides 76 closed-loop CARLA scenarios replayed under three weather conditions, yielding 228 Bench2Drive-format evaluation routes. We introduce WAVE (Work-zone-focused AV data generation in Virtual and rEal Environments), a semi-automated pipeline for creating the dataset, and BoundaryFormer (BF), a transformer-based model that jointly predicts lane and work zone boundary polynomials and driving trajectories. BF uses slot attention for boundary prediction. Ablations show that a separate trajectory decoder using boundary slot features substantially improves trajectory prediction over a slot-attention-only approach. Building on this finding, BF++ offers Camera and Camera+LiDAR variants with metric ground-plane encoding, typed boundary/trajectory queries, long-range point anchors, image-space curve refinement, and conservative gated LiDAR fusion. On the 211 routes common to all four models at the evaluation freeze, BF++-Camera and BF++-Camera+LiDAR achieve Driving Scores of 63.0 and 64.4, respectively, compared with 59.3 for SimLingo and 26.1 for TransFuser++ (TF++). BF++ is 40 times smaller than SimLingo and more than 10 times smaller than TF++, while achieving higher Driving Scores. These results support jointly predicting lane boundaries, work zone boundaries, and driving trajectories as a promising direction toward safer AV operation in work zones. Code and dataset: https://github.com/Nishad-Sahu/WZPlanner.

Abstract

Autonomous Driving Benchmarks

Explore similar work

WZPlanner: Safe End-to-End Path Planning for Autonomous Driving in Work Zones

WorkDrive: Roadwork Chain of Causation for Autonomous Driving

Dash2Sim: Closed-Loop Driving Simulation from in-the-wild Dashcam Videos