TriDrive: Joint Driver, Vehicle, and Road Modeling for Forecasting and Driver Monitoring

Yuhang Wang; Jingxin Yang; Chuheng Wei; Yuechen Guo; Jinghan Xu; Zhao Han; Hao Zhou

cs.ROSep 26, 2026

TriDrive: Joint Driver, Vehicle, and Road Modeling for Forecasting and Driver Monitoring

Authors: Yuhang Wang, Jingxin Yang, Chuheng Wei, Yuechen Guo, Jinghan Xu, Zhao Han, Hao Zhou

Organizations: University of South Florida · NVIDIA · Purdue University · Hunan University

Abstract

Predicting how drivers, vehicles, and road scenes interact and evolve together is central to driver monitoring. Prior work models in-cabin activity or traffic-conditioned driver motion in isolation, motivating joint driver, vehicle, and road modeling with real-time on-vehicle evaluation. We introduce TriDrive, to our knowledge the first unified framework that jointly forecasts driver kinematics, vehicle dynamics, and road demands through an automation-conditioned transition model. Modality-specific encoders (an anchored kinematic representation of the driver, causal CAN-bus dynamics, and frozen V-JEPA 2 road latents with structured road margins) are connected by directed residual connections through which driver and road context refine vehicle forecasts. We evaluate TriDrive on three downstream tasks. On the public AIDE benchmark, its kinematic encoder recipe sets a new full-set state of the art (SOTA) among published baselines (48.05 versus 71.47 All-MPJPE). On 197.2 hours of naturalistic BATON subset, directed connections and road margins raise assistance-engaged PR-AUC by 0.084 for steering onset and 0.286 for time-to-collision drops. For real-time use, we distill the road encoders and run TriDrive on a comma four with an external 8 GB GPU, where a lightweight current-state warning probe updates at 5 Hz with 177 ms p95 latency while the joint model forecasts concurrently. The probe is above an openpilot-based baseline on human-labeled manual-driving warnings (AUROC 0.725 versus 0.563), and in a paired on-road study 14 drivers rate its warnings as more appropriate (+1.79) and timely (+2.67) than those of openpilot's driver-monitoring system.

Figures & tables

Figure 1: Overview of TriDrive. Top: Driver kinematics, vehicle dynamics, and road latents from a frozen V-JEPA 2 ( Assran et al., 2025 ) encoder support automation-conditioned joint forecasting (slow horizon). Bottom: On-vehicle deployment and a paired user study comparing TriDrive with openpilot ( comma.ai, 2026b ) and chestnut ( comma.ai, 2026a ) , involving 14 participants (8.4 hours).

Figure 2: Architecture of TriDrive. Driver, vehicle, and road histories feed joint forecasting; locks mark frozen modules, coral marks the vehicle state that receives the residual connections, and the dashed path marks forecast features evaluated offline as probe inputs.

Published (Driver-WM paper)
Model	All-MPJPE $\downarrow$	[email protected] $\uparrow$	$h{=}5$ , top 5% $\downarrow$	$h{=}5$ , top 20% $\downarrow$	$h{=}5$ , top 50% $\downarrow$
ST-GCN	110.98	60.97	–	–	–
SiMLPe	106.38	63.45	–	–	–
MotionBERT	73.51	78.01	–	–	–
Driver-WM	71.47	71.66	–	–	–
Zero velocity	52.89	85.95	–	–	–

Table 1: Keypoint forecasting on AIDE (mean

\pm

seed SD): All-MPJPE (pixels), [email protected] (%), and

h{=}5

error on the 5%, 20%, and 50% highest-motion clips; bold: best per block.

Model	DBR $\uparrow$	DER $\uparrow$	TCR $\uparrow$	VCR $\uparrow$	All-MPJPE $\downarrow$
Driver-WM (published)	68.07	72.61	90.15	68.34	71.47
Ours, view-gate variant	71.13 $\pm$ .32	74.67 $\pm$ .36	90.29 $\pm$ .79	70.30 $\pm$ 1.02	49.71 $\pm$ .75

Table 2: Semantic recognition on AIDE: macro-F1 (%) for driver behavior (DBR), emotion (DER), traffic context (TCR), and vehicle condition (VCR), and All-MPJPE of the same checkpoint.

Figure 3: TriDrive on one study drive: sustained phone use (left) and a brief scratch during attentive driving (right). Curves: $P$ (transition within 3 s); bands: observed states.

A. Connections (ECAN inputs, no forecast features); $\Delta$ = directed minus indep. branches / no links / all-to-all
Driving	Event	$\Delta$ indep.	$\Delta$ no links	$\Delta$ all-to-all	Directed (default)
Manual	Steering	+.030 ∗	+.028 ∗	$-$ .007 ∗	.774 $\pm$ .002
	Gas	+.020 ∗	$-$ .0000	$-$ .011	.590 $\pm$ .014
	Brake	+.037 ∗	+.039 ∗	$-$ .007	.381 $\pm$ .003
Engaged	Steering	+.084 ∗	+.070 ∗	+.002	.463 $\pm$ .005
	Gas	$-$ .003	$-$ .002	+.002	.033 $\pm$ .003

Table 3: Joint forecasting on BATON: onset PR-AUC within 3 s (mean

\pm

seed SD); peach cells with

\ast

meet our criterion.

Figure 4: Paired on-road ratings (1–7) with 95% bootstrap intervals; grey lines join each participant’s two ratings;

\Delta

is TriDrive minus openpilot (interval, Wilcoxon

p

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Configuration	Parameters	Role
BATON, visual-only	3,502,925	Active forward-path count before forecast injection; road side vector is visual.
BATON, ECAN	3,502,925	Same active count; 40 visual side components replaced by road-margin features.
ECAN registered model	3,649,913	Includes an additional off-path pressure head.
Default registered joint model	3,658,957	Adds the 9,044-parameter forecast injector.
Compact pose forecaster	850,627	Trained separately, then frozen; retained at inference.
Default combined registry	4,509,584	Joint registry plus frozen forecaster, not 4.51M jointly trainable parameters.

Appendix

Table 4: Model variants. Registered counts include modules that may not participate in the evaluated forward path.

Input	Shape	Contents
Vehicle history	$40\times 6$	Five standardized vehicle signals and relative time, at 10 Hz.
Driver history	$40\times 18$	Eleven kinematic channels, validity, five vehicle signals, relative time.
Road token bank	$16\times 1\times 256$	Pooled clip features; separate presence flags and actual time lags.
Road side vector	$256$	Visual PCA components, or 216 visual and 40 road-margin components in ECAN.
Automation state	scalar	Manual, engaged, or unknown.
Forecast input	$10$	Six driver-state probabilities, three mode probabilities, and validity.

Appendix

Table 5: BATON input tensors, omitting the batch dimension. Targets are not network inputs.

Stage	Learning rate	Weight decay	Batch	Epochs / patience
Vehicle base	$3\times 10^{-4}$	.01	1,024	40 / 8
Driver base	$10^{-3}$	$10^{-4}$	4,096	12 / 2
Road base	$3\times 10^{-4}$	.01	1,024	40 / 8
Joint, new/base	$3\times 10^{-4}/3\times 10^{-5}$	.01	1,024	15 / 4
AIDE	$3\times 10^{-4}$	.05	32	200 / 40
BATON pose pretraining	$3\times 10^{-4}$	.05	256	6 executed

Appendix

Table 6: Training schedules used in the experiments. Epoch counts are caps except where an executed pretraining duration is shown.

Family	Event	Manual	Engaged
Demand	TTC	$+.113\!\pm\!.018$ ∗	$+.098\!\pm\!.007$ ∗
	TTC drop	$+.206\!\pm\!.008$ ∗	$+.286\!\pm\!.028$ ∗
	Lane offset	$+.016\!\pm\!.001$ ∗	$+.016\!\pm\!.001$ ∗
	Deceleration	$+.051\!\pm\!.004$ ∗	$+.054\!\pm\!.008$ ∗
	Steering rate	$+.017\!\pm\!.002$ ∗	$-.011\!\pm\!.028$
Pressure	TTC	$+.123\!\pm\!.018$ ∗	$+.106\!\pm\!.016$ ∗

Appendix

Table 7: ECAN minus visual-only environment input, before driver forecast injection. Entries are paired PR-AUC differences at 3 s, mean

\pm

seed SD.

\ast

meets our criterion.

Driving	Target	Persistence	Constant velocity	Learned residual	+ video
Manual	Lead distance	6.177	5.147	$4.735\!\pm\!.034$	$4.755\!\pm\!.031$
	Relative speed	1.154	1.154	$.947\!\pm\!.006$	$.969\!\pm\!.012$
	Lead speed	1.452	1.452	$1.303\!\pm\!.012$	$1.316\!\pm\!.011$
	Lane offset	.168	.301	$.151\!\pm\!.001$	$.152\!\pm\!.002$
Engaged	Lead distance	4.892	5.328	$4.501\!\pm\!.011$	$4.549\!\pm\!.081$
	Relative speed	.807	.807	$.813\!\pm\!.012$	$.823\!\pm\!.016$

Appendix

Table 8: Road-state MAE at three seconds. Distance and lane offset are in meters; speeds are in meters per second. Learned values are mean

\pm

SD across three seeds.

Change	Evidence and interpretation
Road-to-driver connection	Inconclusive against the two-connection model; omitted from the default.
Driver-WM-style encoder (21.2M Transformer)	The adapted trunk decreases manual hand-leave PR-AUC by .018 and steering by .009/.028 in manual/engaged driving. Not parameter-matched.
Matched driver encoders	A TCN decreases manual hand-leave by .020. A small Transformer improves head-turn ranking but worsens its hazard NLL.
Rollout (recurrent or stochastic) decoders	Some onset gains over fresh direct heads, but worse probabilistic or continuous scores and poorer results than pretrained direct heads.
Gaze-proxy chain	Head-turn improves by .023/.028, but adding its predictions gives no warning AUROC gain. Human gaze validation is incomplete.
Identity-invariance objectives	Adversarial heads and per-route normalization fail the joint identity-removal and prediction rule; driver endpoints worsen.

Appendix

Table 9: Additional comparisons and their scope. An inconclusive result does not establish equivalence or noninferiority.

Published method	All-MPJPE	HM-MPJPE	[email protected]
ST-GCN	110.98	158.10	60.97
SiMLPe	106.38	156.29	63.45
MotionBERT	73.51	141.53	78.01
Offline encoder–decoder	75.87	144.05	77.17
Static pooling	68.50	134.56	72.55
Single stream	90.87	147.44	59.63

Appendix

Table 10: Additional AIDE geometry references. HM is horizon-averaged high-motion MPJPE.

Model	All-MPJPE	HM-MPJPE	PCK	HM $h=5$
Zero velocity (published)	52.89	139.19	85.95	178.62
Driver-WM (published)	71.47	138.03	71.66	155.82
Driver-WM, our reproduction (123.7M, 3)	$130.86\!\pm\!3.11$	$176.33\!\pm\!3.29$	$54.16\!\pm\!.80$	$194.34\!\pm\!2.65$
Zero velocity (ours)	52.93	139.23	85.95	178.67
ST-GCN, anchored (3 seeds)	$49.22\!\pm\!.18$	$130.15\!\pm\!.52$	$86.86\!\pm\!.02$	$162.97\!\pm\!1.05$
MotionBERT, fine-tuned (3)	$48.71\!\pm\!.26$	$128.84\!\pm\!.47$	$86.92\!\pm\!.08$	$158.38\!\pm\!.79$

Appendix

Table 11: AIDE geometry. MPJPE is in pixels (lower is better); PCK is a percentage. Published values are unpaired references from Driver-WM. Our learned rows report mean

\pm

seed SD. HM denotes the 60 highest-motion clips;

h=5

is the last predicted sample. Ours is our AIDE pose forecaster (5.77M parameters), which applies TriDrive’s anchored kinematic-forecasting design.

Configuration	All-MPJPE	Status
Released weights, anchored output	$48.71\!\pm\!.26$	Evaluated
Released weights, absolute output	$53.18\!\pm\!.37$	Evaluated
Random initialization, anchored	$48.70\!\pm\!.19$	Evaluated
Released weights, absolute, no EMA	$49.31\!\pm\!.22$	Post hoc
Ours (pose forecaster), no pretraining	$48.63\!\pm\!.15$	Reference
Ours (pose forecaster) + BATON, matched seeds	$48.01\!\pm\!.03$	Reference

Appendix

Table 12: MotionBERT controls on AIDE, three seeds. The post-hoc no-EMA row addresses an optimization confound in the absolute-coordinate comparison.

Model	5% (30)	10% (60)	20% (121)	30% (182)	50% (304)	all (609)
Zero velocity	230.28	178.67	140.66	120.46	97.28	63.74
Ours (5)	$212.34\!\pm\!.45$	$163.37\!\pm\!.63$	$127.46\!\pm\!.77$	$109.85\!\pm\!.49$	$88.94\!\pm\!.32$	$57.64\!\pm\!.18$
Ours + BATON pretraining (5)	$210.51\!\pm\!.90$	$159.92\!\pm\!.92$	$125.64\!\pm\!.85$	$108.53\!\pm\!.57$	$87.87\!\pm\!.38$	$57.02\!\pm\!.18$
Ours, view-gate semantic variant	$211.05\!\pm\!1.93$	$161.93\!\pm\!.84$	$126.92\!\pm\!.16$	$109.64\!\pm\!.30$	$88.69\!\pm\!.19$	$57.47\!\pm\!.14$
ST-GCN, anchored	$215.26\!\pm\!2.13$	$162.97\!\pm\!1.05$	$127.22\!\pm\!.47$	$110.29\!\pm\!.43$	$89.60\!\pm\!.24$	$58.47\!\pm\!.21$
SiMLPe, anchored	$214.48\!\pm\!1.25$	$162.86\!\pm\!.93$	$128.47\!\pm\!.38$	$111.46\!\pm\!.19$	$91.21\!\pm\!.25$	$59.48\!\pm\!.14$

Appendix

Table 13: MPJPE (pixels) of the last predicted pose (

h{=}5

) on the top-

q

high-motion subsets of the AIDE test split (number of clips in parentheses).

Model	5% (30)	10% (60)	20% (121)	30% (182)	50% (304)	all (609)
Zero velocity	175.66	139.23	111.54	97.16	79.36	52.93
Ours (5)	$166.13\!\pm\!.81$	$131.57\!\pm\!.57$	$103.97\!\pm\!.59$	$91.00\!\pm\!.39$	$74.01\!\pm\!.27$	$48.60\!\pm\!.12$
Ours + BATON pretraining (5)	$163.85\!\pm\!.88$	$128.82\!\pm\!.39$	$102.48\!\pm\!.38$	$89.81\!\pm\!.26$	$73.10\!\pm\!.14$	$48.05\!\pm\!.06$
Ours, view-gate semantic variant	$164.92\!\pm\!1.49$	$130.28\!\pm\!.86$	$103.40\!\pm\!.49$	$90.68\!\pm\!.50$	$73.69\!\pm\!.30$	$48.36\!\pm\!.17$
ST-GCN, anchored	$166.10\!\pm\!.90$	$130.15\!\pm\!.52$	$103.34\!\pm\!.23$	$91.01\!\pm\!.26$	$74.38\!\pm\!.22$	$49.22\!\pm\!.18$
SiMLPe, anchored	$166.36\!\pm\!.33$	$131.22\!\pm\!.27$	$104.91\!\pm\!.31$	$92.32\!\pm\!.28$	$76.04\!\pm\!.20$	$50.26\!\pm\!.07$

Appendix

Table 14: Horizon-averaged MPJPE (pixels) on the same top-

q

high-motion subsets.

Model / checkpoint	DBR	DER	TCR	VCR	All-MPJPE
Driver-WM, published single run	68.07	72.61	90.15	68.34	71.47
View-gate, geometry-selected	67.56	73.06	91.39	69.94	48.64
View-gate, F1-selected	71.13	74.67	90.29	70.30	49.71
Three-seed logit ensemble	73.40	77.29	91.16	72.33	48.82

Appendix

Table 15: AIDE semantic macro-F1 (%). DBR: driver behavior; DER: driver emotion; TCR: traffic context; VCR: vehicle condition.

Operating point	System	Recall	FA/h	Precision
Val 10	A	.008	1.25	.493
	B	.064	1.05	.478
	C	.000	11.52	.122
Val 20	A	.233	4.03	.513
	B	.087	8.63	.288
	C	.000	12.41	.175

Appendix

Table 16: Manual-driving human evaluation, 20 weighted-partition events. Point estimates are descriptive. Recall is weighted event recall; FA/h is estimated false alarms/hour. C’s ten/hour rows use its attainable floor.

Operating point	System	Recall (95% interval)	FA/h	Precision
Val 10	A	.409 (.00,.93)	2.15	.325
	B	.000 (.00,.00)	.00	1.000
	C	.409 (.00,.93)	1.58	.793
Val 20	A	.437 (.00,.99)	5.27	.421
	B	.410 (.00,.93)	9.61	.299
	C	.000 (.00,.01)	13.76	.383

Appendix

Table 17: Engaged-driving human evaluation, 33 weighted-partition events. Large uncertainty reflects concentration of weight in a few background events. Parentheses show the reported 95% interval for recall.

Point	Pair	Recall difference [95% interval]	Effective lead, seconds [interval]
Val 20	A $-$ C	+.233 [.022,.543]	-.139 [-.452,-.019]
	B $-$ C	+.087 [.000,.422]	-.041 [-.205,.000]
	A $-$ B	+.146 [-.106,.413]	-.097 [-.331,.041]
Val 10	A $-$ B	-.056 [-.302,.000]	+.013 [-.013,.087]

Appendix

Table 18: Paired manual-driving differences. Every row is descriptive under the registered fewer-than-60-events rule. An interval containing zero is inconclusive, not evidence of equivalence.

Driving	System	Rows / positives	AUROC [95% interval]
Manual	A	139 / 32	.781 [.68,.87]
	B	139 / 32	.750 [.65,.85]
	C	134 / 27	.478 [.33,.70]
Engaged	A	142 / 48	.641 [.54,.74]
	B	142 / 48	.654 [.55,.77]
	C	136 / 43	.758 [.69,.83]

Appendix

Table 19: Secondary unweighted AUROC on the 299-clip set. Rows differ across systems (139 for A and B versus 134 for C in manual driving), so these values are unpaired and are not the paper’s headline warning result; the paired 509-anchor comparison is in Appendix D and Section 4.4 .

Driving	Alert cell	$N$	Covered	Coverage	Labeled
Manual	A	212	160	.755	14
	A+B	169	119	.704	12
	A+B+C	74	43	.581	6
	A+C	56	33	.589	10
	B	216	179	.829	14
	B+C	62	44	.710	10

Appendix

Table 20: S1 coverage used in weighting.

N

is the eligible cell count; covered denotes eligible events from drivers with positive sampling probability.

Student	Raw cosine	Centered cosine	PCA $R^{2}$	Mean PCA $R^{2}$	Side $R^{2}$
v5 ViT-S/16	.979	.886	.796	.281	.400
v6 ViT-S/16, full data	.986	.925	.866	.451	.541
v7-S, region targets	.988	.936	.888	.495	.589
v7-B, region targets	.987	.932	.880	.477	.569
Stated fidelity targets	.970	–	.900	–	.850

Appendix

Table 21: Validation fidelity of the best checkpoint of each six-epoch student run. PCA

R^{2}

is weighted by explained variance; side

R^{2}

refers to the visual side-vector target. The deployed model is v5.

Student	Token $R^{2}_{w}$	Token $R^{2}_{p}$	Side $\overline{R^{2}}$	Side $R^{2}_{p}$	Agreement
v5	.832	.827	.472	.815	1.000
v6	.856	.851	.534	.840	1.000
v7-S	.872	.868	.578	.857	1.000
v7-B	.863	.859	.552	.848	1.000

Appendix

Table 22: Token-bank fidelity over 60 routes and 299,398 anchors. Subscripts

w

and

p

denote variance-weighted and pooled estimates; the bar denotes a component mean. Availability agrees exactly because the flags depend on timing.

Route	Teacher / student events	q99 $\|\Delta\|$	max $\|\Delta\|$	Events
R1	2 / 2	.009	.030	Identical
R2	6 / 6	.036	.081	Identical
R3	6 / 6	.016	.031	Identical
R4	2 / 2	.007	.023	Identical

Appendix

Table 23: Replay warning agreement on four routes (two validation, two test) with the deployed v5 student’s road features and risk substituted for the teacher’s. Score differences are the 99th percentile and maximum of

|\Delta|

; the median is zero on every route.

Quantity	Drive 1	Drive 2	Drive 3
Duration (s)	1,347	1,450	2,453
Pose / joint rate (Hz)	9.67 / 4.61	8.50 / 2.26	9.38 / 2.35
Road-student rate (Hz)	–	0.93	1.89
Warning-readout rate (Hz)	5.00	5.00	5.00
End-to-end latency p50 / p95 (ms)	83 / 153	138 / 239	135 / 177
Ready ticks (%)	94.0	94.2	97.9

Appendix

Table 24: Achieved rates, module latencies, and input validity in three instrumented measurement drives. Drive 1 (September 20) was the experimenter’s test drive and is not part of the study; drives 2 and 3 were driven by study participants on September 22. Drive 1 omits the road path, drive 2 retains a heavy risk chain for logging, and drive 3 uses the student alone. End-to-end latency runs from the driver-frame timestamp to the warning score; module latency is median/p95 in milliseconds; “–” marks road-path quantities that drive 1 did not run.

ID	APP C	APP S	TIM C	TIM S	ANN C	ANN S	Preference
P01	4.00	2.50	7	1	6	–	S
P02	6.33	4.00	7	1	2	2	C
P03	4.00	1.67	4	2	2	1	C
P04	6.33	1.67	6	–	3	–	C
P05	6.00	5.00	7	2	4	4	C
P06	4.33	3.67	6	4	5	1	S

Appendix

Table 25: Participant-level construct scores. C is TriDrive; S is openpilot’s driver-monitoring system (stock system). Missing values are shown as dashes. Preference is C, S, or equal.

Driver Monitoring

Driver monitoring uses in-cabin cameras and other sensors to detect driver states and support safer, more responsive vehicles.

See all 30 papers

Sep 10, 2026

MC-DeTra: Motion-Consistent Joint Object Detection and Socially-Aware Trajectory Forecasting in Bird's-Eye-View Images

Unified models for object detection and trajectory forecasting aim to merge perception and prediction for autonomous driving, refining actor trajectories directly over shared bird's-eye-view (BEV) images rasterized from LiDAR and high-definition maps. Their accuracy on dynamic, moving actors, however, remains the hardest part of the task, and the strongest such model, DeTra, has no public implementation. We contribute an openly released DeTra reimplementation with documented approximations, and on top of it MC-DeTra: a family of motion-consistency mechanisms that add supervision through two annotation-derived auxiliary signals -- each actor's observed past motion and the occupancy of the surrounding traffic that forms its social context -- and one inter-output consistency constraint that aligns an actor's predicted heading with its predicted direction of motion. Every proposed loss is train-only and inference-safe: it shapes the shared BEV representation during training and is removed at test time, adding no inference latency. On the Waymo Open Dataset, evaluated under a strict, detection-conditioned forecasting protocol, MC-DeTra improves dynamic, socially-situated trajectory forecasting while preserving or improving detection accuracy; a gradient-based loss-calibration analysis exposes how the auxiliary objectives compete at the shared backbone, and our ablation identifies which signals contribute most. We release code, configurations, and evaluation tooling at https://github.com/diuzhevVlad/MC-DeTra.

Vladislav Diuzhev, Dmitry Yudin

Moscow Institute of Physics and Technology, Moscow, Russia · Artificial Intelligence Research Institute, Moscow, Russia

Multi-Agent Trajectory Prediction Autonomous Driving Consistency Regularization Vehicle Trajectory Prediction

Ask PDF

Sep 8, 2026

DriveMotion: A Large-Scale Multi-Source Benchmark for Driver Motion Sequence Modeling and Forecasting

Driver motion can provide cues to ongoing behavior, attention, and near-term driving intent. However, most existing driver-centric datasets focus on recognizing predefined driver behaviors from short video clips, while human motion forecasting benchmarks largely target motion outside the vehicle. We introduce DriveMotion, a multi-source benchmark for continuous driver motion forecasting. DriveMotion contains 393 hours of 133-keypoint motion sequences at 10 Hz from 360 drivers, integrating naturalistic driving data, curated public in-cabin videos, and the AIDE dataset into a unified representation with per-joint validity masks and synchronized driving context. Naturalistic driving contains long periods of limited body movement, making uniformly sampled evaluation dominated by persistence and less sensitive to brief but behaviorally meaningful motion. To address this, we use dynamics-anchored evaluation, placing forecasting windows around vehicle maneuvers identified offline from CAN signals without providing CAN to the model at inference. Arm motion in pre-maneuver windows is 3.4x greater than in route-matched stable-driving controls. On these anchored windows, learned models reduce forecasting error over persistence by up to 15%, while maneuver-enriched training improves forecast-derived Part-State F1 by 44% over the zero-motion reference. Training on the full multi-source corpus further reduces forecasting error on held-out web drivers by 38% compared with BATON-only training. DriveMotion provides identity-disjoint splits, fixed evaluation subsets, and reference implementations for reproducible evaluation of continuous driver motion forecasting. The dataset and benchmark are available at https://huggingface.co/datasets/HenryYHW/DriveMotion

Yuhang Wang, Chuheng Wei, Jingxin Yang +2

University of South Florida · Purdue University · Stanford University +1

Autonomous Driving Datasets Autonomous Driving Benchmarks Autonomous Driving Human Motion Prediction

Ask PDF

May 6, 2026

Driver-WM: A Driver-Centric Traffic-Conditioned Latent World Model for In-Cabin Dynamics Rollout

Safe L2/L3 driving automation requires anticipating human-in-the-loop reactions during shared-control transitions. While most driving world models forecast the external environment, in-cabin intelligence remains strictly recognition-oriented and lacks multi-step rollout capabilities for driver dynamics. We introduce Driver-WM, a driver-centric latent world model that rolls out in-cabin dynamics causally conditioned on out-cabin traffic context. This formulation unifies physical kinematics forecasting with auxiliary behavioral and emotional semantic recognition. Operating in a compact latent space constructed from frozen vision-language features, Driver-WM adopts a dual-stream architecture to separately encode external traffic and internal driver states. These streams are directionally coupled via a gated causal injection mechanism, which uses a learned vector gate to modulate external contextual perturbations while strictly enforcing temporal causality. Experiments on AIDE show robust long-horizon forecasting on reactive high-motion clips, improved driver/traffic semantic alignment, and controlled interventions that expose the external-to-internal mechanism.

Haozhuang Chi, Daosheng Qiu, Hao Su +4

Nanyang Technological University, Singapore · Hubei University, Wuhan, China · Osaka University, Osaka, Japan

World Model Learning Causal World Models World Models Autonomous Driving World Models for Autonomous Driving Latent World Models

Ask PDF

Abstract

Driver Monitoring

Explore similar work

MC-DeTra: Motion-Consistent Joint Object Detection and Socially-Aware Trajectory Forecasting in Bird's-Eye-View Images

DriveMotion: A Large-Scale Multi-Source Benchmark for Driver Motion Sequence Modeling and Forecasting

Driver-WM: A Driver-Centric Traffic-Conditioned Latent World Model for In-Cabin Dynamics Rollout