PersonaDrive: Controllable Trajectory Prediction with Multi-Dimensional Driving Personas

Chan Lee; Kimin Yun; Yuseok Bae; Seong Tae Kim; Jung Uk Kim

cs.CVAug 15, 2026

PersonaDrive: Controllable Trajectory Prediction with Multi-Dimensional Driving Personas

Authors: Chan Lee, Kimin Yun, Yuseok Bae, Seong Tae Kim, Jung Uk Kim

Organizations: Kyung Hee University, Yong-in, South Korea · ETRI, Daejeon, South Korea

Abstract

Although recent trajectory prediction and end-to-end autonomous driving methods improve robustness in urban environments, they still lack meaningful controllability. Existing benchmarks either provide no persona-conditioned annotations or support only a single urgency spectrum (i.e., emergency, normal, relaxed), which cannot distinguish personas that share the same urgency level but require different driving dynamics. To address this, we propose (i) the Persona-Conditioned Trajectory (PCT) dataset, which decomposes driving personas along two axes, Temporal Urgency and Ride Comfort, and combines three levels of each to form a grid of nine personas, each paired with natural-language descriptions and trajectories, and (ii) PersonaDrive, a framework that can learn driving personas from language and can generate persona-specific trajectories. PersonaDrive incorporates Persona-Conditioned Anchor Transform (PCAT), which hierarchically reshapes anchors along both axes, and Persona-Conditioned Multi-Modal Fusion (PCMF) for BEV-level persona fusion. Training is supervised by a Hierarchical Guide Loss enforcing axis-aligned physical orderings and an Axis-Decomposed Diversity Loss preventing diagonal mode collapse. Experimental results show that PersonaDrive consistently improves over the compared baselines across multi-dimensional scenarios. The code and PCT dataset are available at https://github.com/VisualAIKHU/PersonaDrive

Figures & tables

Figure 1: Conceptual comparison of (a) existing trajectory predictors, which output a single persona-agnostic plan, (b) one-dimensional persona methods conditioned on a single urgency axis, and (c) our PersonaDrive, which conditions on natural-language persona descriptions spanning Temporal Urgency and Ride Comfort, yielding nine distinct driving personas.

Figure 2: Four-stage generation and validation pipeline for the PCT dataset. Stage 1 generates five candidate sets of nine trajectory parameters via GPT-4o-mini and selects the best through voting. Stages 2 and 3 run in parallel: rule-based validation checks dynamic bounds, lane compliance, progress ordering, and pairwise diversity, while GPT-4o evaluates trajectory quality and verifies persona consistency. Samples failing either stage are regenerated in Stage 4, where failure context is injected into the prompt.

Figure 3: Multi-dimensional behavioral profiles of the nine personas. (a) With urgency fixed, longitudinal metrics (speed, distance) separate by urgency level while comfort lines overlap. (b) With comfort fixed, handling metrics (jerk, yaw rate) separate by comfort level while urgency lines overlap. This confirms that the two axes control orthogonal aspects of driving behavior.

	Comf. Low	Comf. Mid	Comf. High
Urg. High	89.9	89.9	92.2
Urg. Mid	98.4	82.2	82.9
Urg. Low	94.6	73.6	67.4

Table 1: Persona distinguishability of the PCT dataset. Each cell reports the accuracy (%) for identifying the correct persona from shuffled trajectory–text pairs.

Figure 4: Overall architecture of the proposed PersonaDrive framework. PCAT modulates the anchor bank along the urgency and comfort axes using scalars derived from the text query. PCMF fuses BEV, ego, persona, and transformed anchor queries to produce persona-aware trajectories. Training is supervised by a Hierarchical Guide Loss and an Axis-Decomposed Diversity Loss. $\otimes$ denotes element-wise multiplication.

Figure 5: Illustration of the PCMF module. (a) Query Prototype Pool compresses

Q_{bev}

into a conditional vector

C

. (b) Conditional Resampler modulates learnable slots with

\mathbf{C}

and resamples each source. (c) Source Gate applies persona-dependent gating, and the PCMF fuser combines the resulting slots with BEV features.

Methods	Avg. ADE $\downarrow$	Avg. FDE $\downarrow$	Avg. PDMS $\uparrow$
LTF (TPAMI’22) [ 9 ]	2.49	3.87	52.0
Transfuser (TPAMI’22) [ 9 ]	2.53	3.96	51.9
UniAD (CVPR’23) [ 18 ]	2.47	3.83	52.5
Hydra-MDP (arXiv’24) [ 30 ]	6.70	11.85	41.4
PARA-Drive (CVPR’24) [ 54 ]	4.84	9.00	51.9
Traj-LLM (T-IV’24) [ 26 ]	2.66	4.20	54.1

Table 2: Comparison of closed-loop metrics on the planning-oriented NAVSIM navtest split. We report Avg. ADE, Avg. FDE, and Avg. PDMS averaged across all nine persona categories. Bold / underlined fonts indicate the best/second-best results.

Method	Metric	UH-CL	UH-CM	UH-CH	UM-CL	UM-CM	UM-CH	UL-CL	UL-CM	UL-CH	Avg.
1D-Urg.	ADE $\downarrow$	3.30	2.96	3.07	2.76	2.35	2.35	2.73	1.93	2.56	2.67
1D-Comf.		4.79	4.85	4.66	2.43	2.35	2.34	5.70	6.26	7.11	4.50
Multi-D		2.98	2.96	2.69	2.43	2.35	2.34	2.02	1.93	1.82	2.39
1D-Urg.	FDE $\downarrow$	5.81	5.15	5.42	4.14	3.52	3.53	4.06	2.63	4.21	4.27
1D-Comf.		11.76	11.40	10.32	3.69	3.52	3.50	9.84	10.88	12.87	8.64
Multi-D		5.25	5.15	4.47	3.69	3.52	3.50	2.82	2.63	2.33	3.71

Table 3: One-dimensional vs. multi-dimensional persona conditioning on the NAVSIM navtest split. Multi-D is our full model. 1D-Urg. uses the mid-comfort (CM) prediction for all cells in the same urgency row; 1D-Comf. uses the mid-urgency (UM) prediction for all cells in the same comfort column. Bold indicates the best result per cell.

PCAT	PCMF	$\mathcal{L}_{AD}$	Avg. ADE $\downarrow$	Avg. FDE $\downarrow$	Avg. PDMS $\uparrow$
-	-	-	3.93	5.79	55.3
✓	-	-	3.52	5.43	56.0
✓	✓	-	3.23	4.85	56.9
-	✓	✓	2.55	3.87	57.2
✓	-	✓	2.61	3.97	57.5
✓	✓	✓	2.39	3.71	57.7

Table 4: Effect of our proposed components on the NAVSIM navtest split. All metrics are averaged across all nine persona categories. All variants include the text encoder. Detailed per-persona results are in the supplementary material.

Text Encoders	Avg. ADE $\downarrow$	Avg. FDE $\downarrow$	Avg. PDMS $\uparrow$
DeBERTa [ 17 ]	5.99	11.56	57.1
RoBERTa [ 37 ]	4.72	8.76	62.0
MiniLM [ 51 ]	2.39	3.71	57.7

Table 5: Comparison of different text encoders on the NAVSIM navtest split. We report Avg. ADE, Avg. FDE, and Avg. PDMS averaged across all nine persona categories. Detailed per-persona results are in the supplementary material.

Methods	Avg. ADE $\downarrow$	Avg. FDE $\downarrow$	Avg. PDMS $\uparrow$
DiffusionDrive (One-Hot) [ 33 ]	2.67	4.20	55.6
PersonaDrive (One-Hot)	2.54	4.01	56.6
PersonaDrive (Text)	2.39	3.71	57.7

Table 6: Comparison of one-hot persona encoding and text-based persona description on the NAVSIM navtest split. We report Avg. ADE, Avg. FDE, and Avg. PDMS for DiffusionDrive with one-hot labels and our framework with one-hot and text-based persona inputs, averaged across all nine persona categories. More results are in the supplementary material.

Figure 6: Qualitative results on the NAVSIM dataset. (a) Predicted trajectories under nine personas spanning the Urgency and Comfort axes. (b) Corresponding natural-language persona descriptions used as text input for each cell of the 3×3 grid. Each cell shows the ground-truth persona trajectory (waypoints highlighted in yellow) for one Urgency×Comfort persona.

Figure S.1 : Human evaluation interface for the PCT dataset. For each scene, annotators view a $3\times 3$ grid of BEV images, each displaying one persona-conditioned ground-truth trajectory, alongside the corresponding persona descriptions. Annotators rate fifteen Likert-scale items on trajectory quality (urgency ordering, comfort ordering, smoothness, safety, and overall diversity) and text-persona clarity.

Question	Mean	Std	Median
Q1-1	4.84	0.46	5.0
Q1-2	4.76	0.55	5.0
Q1-3	4.83	0.43	5.0
Q1-4	4.64	0.50	5.0
Q1-5	4.64	0.62	5.0
Q1-6	4.75	0.66	5.0

Table S.1 : Summary of human evaluation scores for the PCT dataset. Ratings are on a five point Likert scale from 1 (poor) to 5 (excellent).

Figure S.2 : Distribution comparison between OpenScene GT (dotted gray) and PCT persona trajectories. Top: urgency axis (average speed, maximum lateral acceleration). Bottom: comfort axis (average jerk, maximum yaw rate). The OpenScene GT falls between Mid and Low Urgency in longitudinal metrics, and between Mid and High Comfort in handling metrics, confirming that PCT trajectories bracket the real-world driving envelope.

Methods	Metric	UH-CL	UH-CM	UH-CH	UM-CL	UM-CM	UM-CH	UL-CL	UL-CM	UL-CH	Avg.
LTF (TPAMI’22) [ 9 ]	ADE $\downarrow$	3.01	2.99	2.78	2.62	2.47	2.47	2.10	2.03	1.92	2.49
Transfuser (TPAMI’22) [ 9 ]		3.09	3.05	2.80	2.66	2.52	2.53	2.17	2.05	1.93	2.53
UniAD (CVPR’23) [ 18 ]		2.99	2.98	2.76	2.53	2.46	2.44	2.20	2.03	1.89	2.47
Hydra-MDP (arXiv’24) [ 30 ]		4.83	4.75	4.57	5.11	5.42	5.70	8.97	9.86	11.11	6.70
PARA-Drive (CVPR’24) [ 54 ]		5.05	4.74	4.32	3.16	2.99	3.01	5.78	6.60	7.88	4.84
DiffusionDrive (CVPR’25) [ 33 ]		4.47	4.47	4.27	4.05	3.95	3.91	3.54	3.43	3.28	3.93

Table S.2 : Per-persona comparison of closed-loop metrics on the NAVSIM navtest split. We report ADE and FDE for each of the nine persona categories. Bold indicates the best result per cell.

Metric	UH-CL	UH-CM	UH-CH	UM-CL	UM-CM	UM-CH	UL-CL	UL-CM	UL-CH	Mean
NC $\uparrow$	66.9	74.6	80.6	85.3	91.2	91.8	86.3	93.1	91.0	84.5
DAC $\uparrow$	70.7	78.0	80.4	80.3	87.1	87.6	83.8	91.2	89.9	83.2
TTC $\uparrow$	49.9	56.9	62.4	75.1	83.7	84.9	76.0	84.3	85.0	73.1
Comf. $\uparrow$	45.8	55.9	69.6	79.3	80.9	82.3	84.0	76.6	63.0	70.8
EP $\uparrow$	43.8	56.1	62.7	61.0	70.1	69.5	48.9	52.4	41.9	56.3
PDMS $\uparrow$	35.2	47.0	54.4	59.6	71.1	71.7	56.7	65.1	58.3	57.7

Table S.3 : Per-persona decomposition of PDMS on the NAVSIM navtest split (higher is better). The multiplicative safety gates (NC/DAC/DDC) stay within a bounded floor even at the most against-the-grain cell (UH-CL), so the score does not collapse; the low aggregate PDMS there comes from the quality terms (Comfort, Ego-Progress) and speed-coupled TTC, reflecting an intended trade-off rather than degraded safety.

Metric	PCAT	PCMF	$\mathcal{L}_{AD}$	UH-CL	UH-CM	UH-CH	UM-CL	UM-CM	UM-CH	UL-CL	UL-CM	UL-CH	Avg.
ADE $\downarrow$	-	-	-	4.47	4.47	4.27	4.05	3.95	3.91	3.54	3.43	3.28	3.93
	✓	-	-	4.10	4.14	3.88	3.58	3.48	3.48	3.12	3.03	2.92	3.52
	✓	✓	-	3.80	3.82	3.55	3.29	3.20	3.18	2.85	2.77	2.63	3.23
	-	✓	✓	3.12	3.11	2.86	2.63	2.54	2.52	2.17	2.14	1.95	2.55
	✓	-	✓	3.22	3.21	2.96	2.69	2.59	2.55	2.19	2.12	1.99	2.61
	✓	✓	✓	2.98	2.96	2.69	2.43	2.35	2.34	2.02	1.93	1.82	2.39

Table S.4 : Effect of our proposed components on the NAVSIM navtest split. All metrics are averaged across all nine persona categories. All variants include the text encoder.

Models	Metric	UH-CL	UH-CM	UH-CH	UM-CL	UM-CM	UM-CH	UL-CL	UL-CM	UL-CH	Avg.
DeBERTa [ 17 ]	ADE $\downarrow$	9.27	9.10	8.60	6.22	5.86	6.52	2.19	3.13	3.02	5.99
RoBERTa [ 37 ]		3.98	4.49	4.12	4.74	4.08	2.92	4.53	6.20	7.43	4.72
MiniLM [ 51 ]		2.98	2.96	2.69	2.43	2.35	2.34	2.02	1.93	1.82	2.39
DeBERTa [ 17 ]	FDE $\downarrow$	20.78	19.52	17.61	10.76	9.97	11.56	3.32	5.39	5.11	11.56
RoBERTa [ 37 ]		8.35	9.54	8.24	8.47	6.95	4.97	7.85	10.91	13.58	8.76
MiniLM [ 51 ]		5.25	5.15	4.47	3.69	3.52	3.50	2.82	2.63	2.33	3.71

Table S.5 : Comparison of different text encoders on the NAVSIM navtest split, with per-persona ADE and FDE reported across all nine persona categories.

Methods	Metric	UH-CL	UH-CM	UH-CH	UM-CL	UM-CM	UM-CH	UL-CL	UL-CM	UL-CH	Avg.
DiffusionDrive (One-Hot) [ 33 ]	ADE $\downarrow$	3.32	3.29	3.03	2.71	2.65	2.59	2.24	2.16	2.03	2.67
PersonaDrive (One-Hot)		3.15	3.16	2.89	2.60	2.53	2.48	2.11	2.03	1.90	2.54
PersonaDrive (Text)		2.98	2.96	2.69	2.43	2.35	2.34	2.02	1.93	1.82	2.39
DiffusionDrive (One-Hot) [ 33 ]	FDE $\downarrow$	6.02	5.87	5.18	4.15	4.00	3.88	3.13	2.94	2.61	4.20
PersonaDrive (One-Hot)		5.66	5.60	4.87	3.98	3.86	3.76	3.01	2.83	2.52	4.01
PersonaDrive (Text)		5.25	5.15	4.47	3.69	3.52	3.50	2.82	2.63	2.33	3.71

Table S.6 : Comparison of one-hot persona encoding and text-based persona encoding on the NAVSIM navtest split. We report ADE and FDE for DiffusionDrive with one-hot labels and our framework with one-hot and text-based persona inputs, across all nine persona categories.

Methods	Metric	UH-CL	UH-CM	UH-CH	UM-CL	UM-CM	UM-CH	UL-CL	UL-CM	UL-CH	Avg.
langTraj (ICCV’25) [ 4 ]	ADE $\downarrow$	3.64	3.35	3.17	2.93	2.46	2.47	2.37	2.09	2.01	2.72
iMotionLLM (WACV’26) [ 12 ]		3.44	3.51	3.33	2.97	2.94	2.88	2.37	2.29	2.15	2.87
PersonaDrive		2.98	2.96	2.69	2.43	2.35	2.34	2.02	1.93	1.82	2.39
langTraj (ICCV’25) [ 4 ]	FDE $\downarrow$	6.70	6.50	5.99	4.83	4.07	4.04	3.45	3.07	2.95	4.66
iMotionLLM (WACV’26) [ 12 ]		6.27	6.27	5.76	4.63	4.56	4.41	3.34	3.16	2.80	4.58
PersonaDrive		5.25	5.15	4.47	3.69	3.52	3.50	2.82	2.63	2.33	3.71

Table S.7 : Comparison with language-conditioned trajectory prediction methods on the NAVSIM navtest split. All methods use the same frozen MiniLM text encoder and persona descriptions. We report per-persona ADE and FDE across all nine persona categories.

Methods	Metric	UH-CL	UH-CM	UH-CH	UM-CL	UM-CM	UM-CH	UL-CL	UL-CM	UL-CH	Avg.
1D-Urg.	ADE $\downarrow$	3.44	3.05	3.00	2.98	2.47	2.40	2.79	1.94	2.37	2.72
1D-Comf.		4.82	4.88	4.75	2.48	2.40	2.36	5.55	6.06	6.89	4.47
Multi-D		2.98	2.96	2.69	2.43	2.35	2.34	2.02	1.93	1.82	2.39
1D-Urg.	FDE $\downarrow$	6.37	5.43	5.11	4.62	3.77	3.59	4.26	2.65	3.84	4.40
1D-Comf.		11.91	11.53	10.61	3.75	3.60	3.51	9.48	10.48	12.37	8.58
Multi-D		5.25	5.15	4.47	3.69	3.52	3.50	2.82	2.63	2.33	3.71

Table S.8 : One-dimensional vs. multi-dimensional training on the NAVSIM navtest split. Multi-D is our full model. 1D-Urg. trains on the CM column only; 1D-Comf. trains on the UM row only; Multi-D trains on all nine personas jointly. We report per-persona ADE and FDE across all nine persona categories.

Metric	$\alpha_{u}$	$\alpha_{c}$	$\mathcal{L}_{\text{Guide}}$	UH-CL	UH-CM	UH-CH	UM-CL	UM-CM	UM-CH	UL-CL	UL-CM	UL-CH	Avg.
ADE $\downarrow$	✓	-	-	3.28	3.25	2.97	2.80	2.64	2.63	2.26	2.20	2.05	2.68
	✓	✓	-	3.24	3.29	3.05	2.75	2.66	2.61	2.23	2.15	2.03	2.67
	✓	-	✓	3.02	3.05	2.79	2.49	2.41	2.40	2.05	2.01	1.88	2.46
	✓	✓	✓	2.98	2.96	2.69	2.43	2.35	2.34	2.02	1.93	1.82	2.39
FDE $\downarrow$	✓	-	-	5.80	5.68	4.96	4.25	3.95	3.94	3.16	3.00	2.65	4.15
	✓	✓	-	5.69	5.66	5.00	4.10	3.96	3.84	3.09	2.90	2.58	4.09

Table S.9 : Effect of PCAT components on the NAVSIM navtest split. We report ADE and FDE for configurations using only the urgency scalar

\alpha_{u}

, adding the comfort modulation

\alpha_{c}

, adding the guide loss, and using all three together, across all nine persona categories.

Figure S.3 : Effect of guide loss on persona-conditioned anchor trajectories. Each column shows anchor sets for three representative personas (high urgency, medium, and low urgency), and rows compare (a) PCAT without guide loss and (b) PCAT with guide loss, where the guide loss enforces a more consistent progress and smoothness ordering across personas.

Methods	Metric	UH-CL	UH-CM	UH-CH	UM-CL	UM-CM	UM-CH	UL-CL	UL-CM	UL-CH	Avg.
Naive Concatenate	ADE $\downarrow$	3.79	3.79	3.69	3.46	3.68	3.38	3.08	2.94	2.77	3.40
Naive Cross-Attention		3.18	3.18	2.90	2.65	2.59	2.56	2.20	2.08	1.93	2.59
PCMF		2.98	2.96	2.69	2.43	2.35	2.34	2.02	1.93	1.82	2.39
Naive Concatenate	FDE $\downarrow$	6.44	6.33	5.94	5.17	5.49	4.94	4.30	4.05	3.60	5.14
Naive Cross-Attention		5.72	5.62	4.86	4.02	3.89	3.83	3.09	2.81	2.49	4.04
PCMF		5.25	5.15	4.47	3.69	3.52	3.50	2.82	2.63	2.33	3.71

Table S.10 : Comparison of text-scene fusion strategies on the NAVSIM navtest split. We report ADE and FDE for Naive Concatenate, Naive Cross-Attention, and the proposed PCMF, across all nine persona categories.

Metric	$\mathcal{L}_{\text{Intra}}$	$\mathcal{L}_{\text{Inter}}$	UH-CL	UH-CM	UH-CH	UM-CL	UM-CM	UM-CH	UL-CL	UL-CM	UL-CH	Avg.
ADE $\downarrow$	✓	-	3.27	3.25	3.02	2.73	2.64	2.59	2.26	2.20	2.05	2.67
	-	✓	3.27	3.26	2.99	2.72	2.60	2.56	2.22	2.11	2.00	2.64
	✓	✓	2.98	2.96	2.69	2.43	2.35	2.34	2.02	1.93	1.82	2.39
FDE $\downarrow$	✓	-	5.75	5.63	4.95	4.10	3.92	3.83	3.12	2.95	2.64	4.10
	-	✓	5.73	5.59	4.94	4.07	3.90	3.80	3.07	2.83	2.53	4.05
	✓	✓	5.25	5.15	4.47	3.69	3.52	3.50	2.82	2.63	2.33	3.71

Table S.11 : Effect of diversity objective components on the NAVSIM navtest split. We report ADE and FDE for configurations using

\mathcal{L}_{\text{Intra}}

only,

\mathcal{L}_{\text{Inter}}

only, and both losses together, across all nine persona categories.

Panel of Figure 27 — Figure S.4 : Qualitative results of persona-conditioned trajectory prediction on the NAVSIM dataset. Each row shows one scene, and the $3\times 3$ grid indicates predicted trajectories under nine personas spanning Urgency and Comfort. (a) and (b) show the predicted trajectories and the persona text inputs of PersonaDrive, respectively. Each cell shows the ground-truth persona trajectory (waypoints highlighted in yellow) for one Urgency×Comfort persona.

Figure S.5 : Representative failure cases and their recovered results. Each scene overlays the ground-truth trajectory (dashed gray), the initial failed generation (dashed red), and the recovered trajectory after failure-aware regeneration (solid green), where the specific failure reason is injected into the regeneration prompt. Over the full OpenScene generation, 31.4% of scenes were flagged for rule-based and 4.7% for quality-based (LLM-judge) regeneration; after up to three retries, 0.18% remained unrecovered and fell back to the best-scoring candidate.

Figure S.6 : Interpolation between UH-CL and UM-CL: (a) between cell embeddings, (b) between cell-defining prompts. Intermediate trajectories lie smoothly between the two cell predictions, showing that the model treats text as a continuous control signal rather than nine discrete labels.

Judge	Top-1	$\kappa$	Urgency	Comfort
Gemini Flash	82.6	0.804	90.2	90.6
Claude Opus	98.0	0.978	100.0	98.0

Table S.12 : Cross-family LLM-judge agreement with PCT cell labels on 1,000 samples (%). We report top-1 cell accuracy, Cohen’s

\kappa

, and per-axis accuracy.

Configuration	Avg. ADE $\downarrow$	Avg. FDE $\downarrow$	Avg. PDMS $\uparrow$
(a) KL temperature $\tau$
$\tau=0.10$	2.892	4.406	56.3
$\tau=0.25^{*}$	2.391	3.706	57.7
$\tau=0.50$	2.759	4.265	56.8
$\tau=1.00$	2.772	4.274	57.4
(b) Inter-axis weight $w_{\text{Inter}}$

Table S.13 : Loss-related hyperparameter sweep on the NAVSIM navtest split. Default settings are marked with ∗ . All three sit in a smooth basin.

Multimodal Trajectory Prediction

Multimodal trajectory prediction estimates several plausible future paths for people, vehicles, and other agents from motion and scene context.

See all 56 papers

Mar 9, 2025

PDB: Not All Drivers Are the Same -- A Personalized Dataset for Understanding Driving Behavior

Driving behavior varies across individuals, yet most existing driving datasets do not preserve driver identity or control for factors that may affect driving style. We introduce the Personalized Driving Behavior (PDB) dataset, a multimodal dataset collected from 9 drivers using the same vehicle and route under comparable conditions. PDB includes data from a 128-line LiDAR, external and driver-facing cameras, GNSS, IMU, CAN-bus signals, and heart-rate measurements, with 111,060 frames of LiDAR point cloud data and more than 1.33 million images. Based on these recordings, we construct PDB-Traj, a personalized trajectory prediction benchmark with 1,309 10-second trajectory segments sampled at 10 Hz. Each segment includes the ego trajectory, surrounding-agent trajectories, and driver identity. We further introduce Gated Selective Personalization (Gate-SP), a driver-aware baseline that selectively incorporates driver identity into trajectory prediction. Experiments on PDB-Traj provide reference results for personalized motion forecasting and show that driver information can improve prediction when applied selectively. PDB and PDB-Traj provide a benchmark for studying driver-specific behavior and personalized trajectory prediction.

Chuheng Wei, Ziye Qin, Siyan Li +8

University of California, Riverside · Southwest Jiaotong University · InfoTech Labs, Toyota Motor North America

Autonomous Driving Datasets Trajectory Prediction Autonomous Driving Benchmarks Multimodal Dataset

Ask PDF

Sep 29, 2026

doPlan: A Variable-Horizon Dataset for Multi-Stage Language-Conditioned Planning in Autonomous Driving

Autonomous vehicles interacting with passengers through natural language must reason beyond immediate commands. Passenger intent may span multiple stages of behavior, depend on future events, refer to surrounding agents or landmarks, and remain relevant as driving conditions evolve. Existing language-enabled driving datasets largely focus on short, localized interactions, leaving these longer-horizon forms of passenger intent comparatively underexplored. We introduce doPlan, to our knowledge the first publicly available, human-annotated real-world dataset designed to study passenger language as persistent task context. Built on nuPlan, doPlan contains 5,154 human-written passenger instructions spanning 169.1 hours of cumulative instruction-aligned context over 50.9 hours of unique driving, with annotation windows ranging from 30.0 to 508.8 s. The annotations capture immediate, deferred, event-conditioned, persistent, and multi-stage passenger intent. The dataset, annotation interface, and supporting resources are publicly available at https://github.com/Mi3-Lab/doPlan. We evaluate four language-conditioned driving models and find that sensitivity to passenger language does not reliably translate into behavior consistent with the requested direction. More broadly, among 2,161 examples with a matched future maneuver, the first associated maneuver occurs a median of 24.6 s after the evaluation point, and only 9.8% occur within the models' common 5 s prediction horizon. These findings highlight the need to connect persistent passenger intent with successive planning decisions. doPlan provides a setting for studying how unresolved goals can be retained, grounded in evolving scenes, and tracked across multiple stages, including how a planner determines when a future goal becomes relevant to the current plan.

Parthib Roy, Yash Tandon, Marcus Blennemann +4

Machine Intelligence, Interaction, and Imagination (Mi3) Laboratory, University of California, Merced, Merced, CA, USA. · Laboratory for Intelligent & Safe Automobiles (LISA), University of California, San Diego, La Jolla, CA, USA.

Autonomous Driving Datasets Long-Horizon Planning Autonomous Driving Autonomous Driving Planning

Ask PDF

Mar 26, 2026

Drive My Way: Preference Alignment of Vision-Language-Action Model for Personalized Driving

Human driving behavior is inherently personal, which is shaped by long-term habits and influenced by short-term intentions. Individuals differ in how they accelerate, brake, merge, yield, and overtake across diverse situations. However, existing end-to-end autonomous driving systems either optimize for generic objectives or rely on fixed driving modes, lacking the ability to adapt to individual preferences or interpret natural language intent. To address this gap, we propose Drive My Way (DMW), a personalized Vision-Language-Action (VLA) driving framework that aligns with users' long-term driving habits and adapts to real-time user instructions. DMW learns a user embedding from our personalized driving dataset collected across multiple real drivers and conditions the policy on this embedding during planning, while natural language instructions provide additional short-term guidance. Closed-loop evaluation on the Bench2Drive benchmark demonstrates that DMW improves style instruction adaptation, and user studies show that its generated behaviors are recognizable as each driver's own style, highlighting personalization as a key capability for human-centered autonomous driving. Our data and code are available at https://dmw-cvpr.github.io/.

Zehao Wang, Huaide Jiang, Shuaiwu Dong +3

University of California, Riverside · University of Michigan

Autonomous Driving Autonomous Driving Planning VLMs for Autonomous Driving

Ask PDF

Abstract

Multimodal Trajectory Prediction

Explore similar work

PDB: Not All Drivers Are the Same -- A Personalized Dataset for Understanding Driving Behavior

doPlan: A Variable-Horizon Dataset for Multi-Stage Language-Conditioned Planning in Autonomous Driving

Drive My Way: Preference Alignment of Vision-Language-Action Model for Personalized Driving