Drive My Way: Preference Alignment of Vision-Language-Action Model for Personalized Driving

Zehao Wang; Huaide Jiang; Shuaiwu Dong; Yuping Wang; Hang Qiu; Jiachen Li

cs.ROMar 26, 2026

Drive My Way: Preference Alignment of Vision-Language-Action Model for Personalized Driving

Authors: Zehao Wang, Huaide Jiang, Shuaiwu Dong, Yuping Wang, Hang Qiu, Jiachen Li

Organizations: University of California, Riverside · University of Michigan

Abstract

Human driving behavior is inherently personal, which is shaped by long-term habits and influenced by short-term intentions. Individuals differ in how they accelerate, brake, merge, yield, and overtake across diverse situations. However, existing end-to-end autonomous driving systems either optimize for generic objectives or rely on fixed driving modes, lacking the ability to adapt to individual preferences or interpret natural language intent. To address this gap, we propose Drive My Way (DMW), a personalized Vision-Language-Action (VLA) driving framework that aligns with users' long-term driving habits and adapts to real-time user instructions. DMW learns a user embedding from our personalized driving dataset collected across multiple real drivers and conditions the policy on this embedding during planning, while natural language instructions provide additional short-term guidance. Closed-loop evaluation on the Bench2Drive benchmark demonstrates that DMW improves style instruction adaptation, and user studies show that its generated behaviors are recognizable as each driver's own style, highlighting personalization as a key capability for human-centered autonomous driving. Our data and code are available at https://dmw-cvpr.github.io/.

Figures & tables

Figure 1: Drive My Way (DMW) achieves end-to-end personalized driving via both long-term preference alignment and short-term style instruction adaptation.

Figure 2: An overview of the Personal Driving Dataset, which consists of the driving data and structured driver profile data.

Figure 3: An overview of the DMW framework with a pretrained VLA backbone. The model takes in front-view camera images, instructions, route target points, and user profile as inputs, while the motion predictor outputs route and speed waypoints, which derive the base action (throttle, steer angle). The residual decoder outputs a discrete residual applied to the base to produce the final personalized action.

Figure 4: The contrastive learning mechanism on the long-term preference encoder and route processor.

Method	Style	DS	SR	Efficiency	Comfort	Speed	Acceleration	LC	Headway	TT
SimLingo [ 42 ]	Aggressive	78.56	65.83	247.60	18.61	7.66	5.39	0.75	25.99	25.35
	Neutral	78.15	65.85	241.44	24.67	7.37	5.22	0.75	27.81	31.41
	Conservative	78.18	65.56	238.77	26.99	7.21	5.29	0.70	29.12	33.02
StyleDrive [ 11 ]	Aggressive	75.68	60.89	256.71	16.79	7.23	5.59	0.74	24.95	27.76
	Neutral	76.26	62.13	249.07	21.35	6.98	5.43	0.66	23.62	29.12
	Conservative	77.02	61.96	242.18	23.67	6.82	5.39	0.70	27.19	29.98

Table 1: Bench2Drive closed-loop driving metrics with different style instructions. We compare SimLingo and StyleDrive under different style instructions with our policy fine-tuning with fixed rewards weights (DMW-Vanilla) and style-aware rewards weights ( DMW ).

Figure 5: The fine-tuning process and reward generation for short-term instruction alignment.

Driver	Scenario	Style	DS	Speed	Efficiency	Acceleration	Headway	AS	Ratings
D1	Emergency Brake	None $\downarrow$ Aggressive	95.10 → 84.02	8.06 → 9.58	150.94 → 167.12	6.31 → 6.39	18.94 → 17.11	1.00	8.8
D2			98.64 → 96.71	4.61 → 4.97	96.02 → 99.21	4.94 → 5.35	23.10 → 21.18	0.67	8.3
D3			97.88 → 95.93	5.39 → 6.34	109.83 → 121.06	5.63 → 6.16	21.61 → 19.22	1.00	8.0
D4			94.02 → 93.15	7.76 → 8.35	145.22 → 159.56	6.03 → 6.92	21.03 → 18.19	0.67	7.8
D1	Merging	None $\downarrow$ Conservative	90.05 → 94.38	9.32 → 8.76	270.84 → 260.10	5.15 → 4.68	96.44 → 101.16	0.67	8.4
D2			86.94 → 91.72	6.37 → 5.76	196.82 → 175.31	4.29 → 4.16	118.92 → 121.03	1.00	8.1

Table 2: Driving metrics with and without style instructions, for each driver profile with a different long-term preference. Note that for Alignment Score (AS) and Ratings, we compute their value regardless of the style since it only measures long-term alignment.

Figure 6: Driving preference under aggressive and conservative instructions. Red waypoints denote distance parametrized (every $1\text{\,}\mathrm{m}$ ) navigation path and green waypoints denote time parametrized (every $0.25\text{\,}\mathrm{s}$ ) trajectory.

	Alignment Score				Average Ratings
Methods	ID		OOD		ID		OOD
Methods	D1	D2	D3	D4	D1	D2	D3	D4
MORL-PD [ 14 ]	0.42	0.58	0.25	0.33	5.1	6.2	3.9	3.5
DMW	0.92	0.92	0.83	0.83	8.7	8.3	7.8	8.0

Table 3: Long-term preference alignment. D: Driver ID.

Driver	Encoder	DS	Speed	Effic.	Acce.	Headway	AS	Ratings
D1	w/o AAP	82.74	4.52	161.38	5.84	50.11	0.67	6.1
D2		88.02	4.21	163.07	5.88	47.56	0.58	5.9
D3		87.65	4.49	133.12	5.81	41.63	0.25	4.6
D4		80.13	4.46	172.92	5.79	38.94	0.50	5.3
D1	w/ AAP	93.38	8.54	262.68	6.46	40.38	0.92	8.7
D2		95.43	5.57	163.38	5.15	49.82	0.92	8.3

Table 4: Averaged driving metrics using long-term preference encoder with or without Adaptive Average Pooling (AAP).

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Driver	DS	Speed	Effic.	Acce.	Headway	AS	Ratings
D1	93.38	8.54	262.68	6.46	40.38	0.92	8.7
D2	95.43	5.57	163.38	5.15	49.82	0.92	8.3
D3	95.30	6.40	185.83	5.56	47.05	0.83	7.8
D4	91.96	8.77	274.19	6.17	39.70	0.83	8.0
D5	92.58	7.48	229.93	5.58	39.08	0.83	7.9
D6	97.71	6.16	190.74	6.08	42.41	0.75	7.4

Appendix

Table 5: Driving metrics across all scenario types.

Scenario	Model	Style	E1	E2	E3	E4	E5
Emergency Brake	SimLingo [ 42 ]	Conservative	7.4	6.7	6.4	7.3	6.5
		Neutral	7.0	7.3	7.5	7.0	7.1
		Aggressive	6.5	7.6	7.2	6.6	7.4
	StyleDrive [ 11 ]	Conservative	8.2	7.5	7.3	8.0	7.2
		Neutral	7.8	8.0	8.2	7.6	7.9
		Aggressive	7.2	8.4	7.9	7.1	8.2

Appendix

Table 6: User study ratings (0-10) evaluating how well trajectories match intended instructions. Five evaluators (E1-E5) rate trajectories from SimLingo [ 42 ] , StyleDrive [ 11 ] , and DMW .

Panel of Figure 7 — Figure 7: Driving preference under aggressive and conservative instructions. Red waypoints denote distance parametrized (every $1\text{\,}\mathrm{m}$ ) navigation path and green waypoints denote time parametrized (every $0.25\text{\,}\mathrm{s}$ ) trajectory.

Autonomous Driving

Autonomous driving develops vehicles that perceive road scenes, predict interactions, and plan and control safe movement with limited or no human input.

See all 536 papers

Vega: Learning to Drive with Natural Language Instructions
Mar 26, 2026Sicheng Zuo, Yuxuan Li, Wenzhao Zheng +3Autonomous Driving Autonomous Driving Planning
Tsinghua University · GigaAI
Driving Like Yourself: A Benchmark for Closed-Loop Personalized End-to-End Autonomous Driving
Feb 21, 2026Xiaoru Dong, Ruiqin Li, Xiao Han +7Autonomous Driving Benchmarks End-to-End Autonomous Driving
The University of Hong Kong · ShanghaiTech University · The Chinese University of Hong Kong
StyleVLA: Driving Style-Aware Vision Language Action Model for Autonomous Driving
Mar 10, 2026Yuan Gao, Dengyuan Hua, Mattia Piccinini +4Autonomous Driving Vision-Language-Action Models
Y. Gao, D. Hua, M. Piccinini, F. Schäfer, K. Moller, J. Betz are with the Professorship of Autonomous Vehicle Systems, TUM School of Engineering and Design, Technical University of Munich, 85748 Garching, Germany; Munich Institute of Robotics and Machine Intelligence (MIRMI) · L. Li is with the School of Mechanical and Aerospace Engineering, Nanyang Technological University

Mar 26, 2026J/K move · Enter open · S save

Vega: Learning to Drive with Natural Language Instructions

Sicheng Zuo, Yuxuan Li, Wenzhao Zheng, Zheng Zhu+2

Tsinghua University · GigaAI

Autonomous Driving Autonomous Driving Planning Vision-Language-Action Models World Models for Autonomous Driving Autonomous Driving Datasets

Open paper AskPDF arXiv

Vision-language-action models have reshaped autonomous driving to incorporate languages into the decision-making process. However, most existing pipelines only utilize the language modality for scene descriptions or reasoning and lack the flexibility to follow diverse user instructions for personalized driving. To address this, we first construct a large-scale driving dataset (InstructScene) containing around 100,000 scenes annotated with diverse driving instructions with the corresponding trajectories. We then propose a unified Vision-Language-World-Action model, Vega, for instruction-based generation and planning. We employ the autoregressive paradigm to process visual inputs (vision) and language instructions (language) and the diffusion paradigm to generate future predictions (world modeling) and trajectories (action). We perform joint attention to enable interactions between the modalities and use individual projection layers for different modalities for more capabilities. Extensive experiments demonstrate that our method not only achieves superior planning performance but also exhibits strong instruction-following abilities, paving the way for more intelligent and personalized driving systems.

Abstract

Autonomous Driving

Explore similar work

Vega: Learning to Drive with Natural Language Instructions

Driving Like Yourself: A Benchmark for Closed-Loop Personalized End-to-End Autonomous Driving

StyleVLA: Driving Style-Aware Vision Language Action Model for Autonomous Driving