Trajectory Prediction for Autonomous Driving in Road Construction Zones Using Human-in-the-Loop Driving Data

Shuo Zhang; Aman Sunesh; Semiha Ergan; Kaan Ozbay

cs.ROOct 8, 2026

Trajectory Prediction for Autonomous Driving in Road Construction Zones Using Human-in-the-Loop Driving Data

Authors: Shuo Zhang, Aman Sunesh, Semiha Ergan, Kaan Ozbay

Organizations: New York University New York City, NY, USA · New York University New York, USA

Abstract

Ego-trajectory prediction in urban roadway work zones remains an underexplored topic in autonomous-driving research, with limited dedicated data and systematic evaluation. Roadway work zones involve temporary changes in the lane layouts and connectivity of the roads, complicating interactions with surrounding vehicles and workers. These changes make future vehicle trajectories difficult to predict. To address these challenges, we collect WZDrive-Data, a multimodal dataset comprising 1,183 work-zone driving segments from 33 drivers, using a human-in-the-loop driving simulator with prerecorded worker animations and VR-controlled workers. Driver control and gaze signals are recorded from human participants, while vehicle trajectories and scene information are obtained from the simulator. Our proposed multimodal ego-trajectory predictor, WZDrive, achieves a mean minADE (K=6) of 1.09 m and minFDE (K=6) of 2.12 m. Compared with the best-performing state-of-the-art baseline for each metric in our benchmark, WZDrive reduces minADE (K=6) by 7.18% and minFDE (K=6) by 13.18%. This research addresses a gap in autonomous-driving research on urban work zones by combining human-in-the-loop data collection with multimodal learning from human driving signals and in the context of work zones.

Figures & tables

Figure 1. Overview of the DualSense platform used to collect WZDrive-Data . (a) Platform architecture connecting the human driver, VR roadway worker, simulation environment, and sensing devices. (b) User-study setup enabling real-time driver–worker interaction. (c) Data-collection procedure across six randomized scenario conditions.

Characteristic	Value
Valid drivers	33
Work-zone configurations	3
Scenario conditions	6
Distinct encounter segments	1,183
Overlapping prediction windows	26,892
Trajectory coverage	$\approx 3.7$ h

Table 1. WZDrive-Data scale and multimodality

Panel of Figure 2 — Figure 2. Distribution of prediction windows by (a) work-zone configuration and (b) scenario condition.

Figure 3. Overview of WZDrive pretraining and staged training. (a) Modality-specific encoders and fusion modules learn representations through reconstruction, contrastive, topology, route-coverage, and travel-distance objectives. (b) The six-stage training schedule, from encoder pretraining to temporal anchor calibration.

Model	Features	ADE (m) $\downarrow$	FDE (m) $\downarrow$
Stay	Ego	21.635	42.053
CV	Ego	7.237	16.896
LSTM	Ego + control + WZ	2.988	7.513
Transformer	Ego + control + WZ	2.876	7.284
GRU	Ego + control + WZ + lane	2.860	7.320

Table 2. Rule-based and time-series baseline performance (

K=1

) with selected input combinations. Lower is better.

	Without WZ		With WZ
Model	minADE 6 $\downarrow$	minFDE 6 $\downarrow$	minADE 6 $\downarrow$	minFDE 6 $\downarrow$
PGP (2021) ( Deo et al., 2022 )	1.286	2.749	1.260 ( $\downarrow$ 2.02%)	2.742 ( $\downarrow$ 0.25%)
MTR (2022) ( Shi et al., 2022 )	1.317	2.660	1.299 ( $\downarrow$ 1.37%)	2.567 ( $\downarrow$ 3.50%)
Forecast-MAE (2023) ( Cheng et al., 2023 )	1.373	2.650	1.264 ( $\downarrow$ 7.94%)	2.475 ( $\downarrow$ 6.60%)
LAFormer (2024) ( Liu et al., 2024 )	1.204	2.662	1.179 ( $\downarrow$ 2.08%)	2.550 ( $\downarrow$ 4.21%)
DeMo (2024) ( Zhang et al., 2024 )	1.514	2.436	1.281 ( $\downarrow$ 15.39%)	2.462 ( $\uparrow$ 1.07%)

Table 3. Comparison with SOTA baselines (

K=6

). Errors are in meters.

Variant	minADE 6	minFDE 6
WZDrive (full)	1.103	2.119
Without gaze	1.108	2.116
Without driver controls	1.293	2.419
Without WZ and workers	1.187	2.390

Table 4. Effects of input modalities and training stages across participant partitions. Errors are in meters; lower is better.

Figure 4. Qualitative failure cases. From left to right: pole collision, work-zone collision, unexpected stopping, cone-induced deviation, and lane-change correction. The top row shows driving views with gaze overlays; the bottom row shows observed motion, ground-truth futures, and the lowest-ADE prediction among six candidates. VR-worker markers indicate schematic positions.

Vehicle Trajectory Prediction

Vehicle trajectory prediction forecasts how vehicles will move from their histories, road maps, traffic interactions, and driving intentions.

See all 64 papers

WZPlanner: Safe End-to-End Path Planning for Autonomous Driving in Work Zones
Sep 16, 2026Nishad Sahu, Changzhong Qian, Guangzhou Cai +3Autonomous Driving Datasets End-to-End Autonomous Driving
Raj · Department of Electrical and Computer Engineering Carnegie Mellon University, Pittsburgh, PA, USA
WorkDrive: Roadwork Chain of Causation for Autonomous Driving
Jul 16, 2026Tianyi Jiang, Wen Zhang, Sihan Yang +2VLMs for Autonomous Driving Autonomous Driving Perception
Peking University · Beijing, China · Xiaomi EV
PersonaDrive: Controllable Trajectory Prediction with Multi-Dimensional Driving Personas
Aug 15, 2026Chan Lee, Kimin Yun, Yuseok Bae +2Multimodal Trajectory Prediction Autonomous Driving Datasets
Kyung Hee University, Yong-in, South Korea · ETRI, Daejeon, South Korea

Sep 16, 2026J/K move · Enter open · S save

WZPlanner: Safe End-to-End Path Planning for Autonomous Driving in Work Zones

Nishad Sahu, Changzhong Qian, Guangzhou Cai, Shounak Sural+2

Raj · Department of Electrical and Computer Engineering Carnegie Mellon University, Pittsburgh, PA, USA

Autonomous Driving Datasets End-to-End Autonomous Driving Autonomous Driving Autonomous Driving Perception Safe Motion Planning Autonomous Driving Planning

Open paper AskPDF arXiv

Work zones alter lane geometry through temporary traffic controls and closures that may be absent from on-board maps, challenging autonomous vehicle (AV) perception and planning. Generalization is also limited by scarce public datasets with structured geometric supervision. We present WorkZonePlan, a dataset comprising 149K+ synthetic and 5K+ real-world multimodal samples with 3D annotations for lane boundaries, work zone boundaries, and driving trajectory options. It also provides 76 closed-loop CARLA scenarios replayed under three weather conditions, yielding 228 Bench2Drive-format evaluation routes. We introduce WAVE (Work-zone-focused AV data generation in Virtual and rEal Environments), a semi-automated pipeline for creating the dataset, and BoundaryFormer (BF), a transformer-based model that jointly predicts lane and work zone boundary polynomials and driving trajectories. BF uses slot attention for boundary prediction. Ablations show that a separate trajectory decoder using boundary slot features substantially improves trajectory prediction over a slot-attention-only approach. Building on this finding, BF++ offers Camera and Camera+LiDAR variants with metric ground-plane encoding, typed boundary/trajectory queries, long-range point anchors, image-space curve refinement, and conservative gated LiDAR fusion. On the 211 routes common to all four models at the evaluation freeze, BF++-Camera and BF++-Camera+LiDAR achieve Driving Scores of 63.0 and 64.4, respectively, compared with 59.3 for SimLingo and 26.1 for TransFuser++ (TF++). BF++ is 40 times smaller than SimLingo and more than 10 times smaller than TF++, while achieving higher Driving Scores. These results support jointly predicting lane boundaries, work zone boundaries, and driving trajectories as a promising direction toward safer AV operation in work zones. Code and dataset: https://github.com/Nishad-Sahu/WZPlanner.

Abstract

Vehicle Trajectory Prediction

Explore similar work

WZPlanner: Safe End-to-End Path Planning for Autonomous Driving in Work Zones

WorkDrive: Roadwork Chain of Causation for Autonomous Driving

PersonaDrive: Controllable Trajectory Prediction with Multi-Dimensional Driving Personas