Vision-Language Models (VLMs) and world models have demonstrated strong capabilities in semantic understanding and future scene evolution for autonomous driving. However, effectively integrating both capabilities for trajectory generation remains challenging. In this work, we propose DriveWLA, a unified World-Language-Action model based on Mixture-of-Transformers (MoT) that jointly models semantic understanding, future scene dynamics, and trajectory generation. DriveWLA consists of three experts: a World Expert for future prediction, a Language Expert for semantic understanding, and an Action Expert for trajectory generation. We further introduce Cross-Expert Attention, which enables controlled information flow across experts while aligning semantic and dynamic representations. To improve trajectory generation beyond imitation learning, we apply Flow-GRPO reinforcement learning post-training to refine the Action Expert. Extensive experiments show that DriveWLA achieves superior performance in both open-loop and closed-loop evaluation. DriveWLA achieves 92.3 PDMS on NAVSIM v1 and 90.1 EPDMS on NAVSIM v2, demonstrating strong performance among VLM-based and world-model-based planners. Without specific fine-tuning, DriveWLA further demonstrates strong zero-shot generalization to HUGSIM, nuScenes, and Bench2Drive. These results validate the effectiveness of our unified World-Language-Action modeling, establishing DriveWLA as a generalizable framework for autonomous driving.
Figures & tables
Figure 1: Model architecture and training strategy. DriveWLA unifies World, Language, and Action Experts through Cross-Expert Attention. The training strategy consists of trajectory imitation learning, which jointly optimizes all experts, followed by reinforcement learning refinement of the Action Expert.
Figure 2: Query-Key/Value Visibility in Cross-Expert Attention.
Method
VLM Backbone
Video Backbone
NC ↑
DAC ↑
TTC ↑
Comf. ↑
EP ↑
PDMS ↑
VLM-based planners
ReCogDrive ( Xiong et al., 2026 )
InternVL3-2B
–
97.9
97.3
94.9
100.0
87.3
90.8
SGDrive ( Li et al., 2026a )
InternVL3-2B
–
98.6
97.8
96.2
100.0
85.8
91.1
Qwen-Drive-1.0 ( Zhou et al., 2026a )
Qwen3.5-4B
–
98.6
98.2
95.9
100.0
84.8
90.7
World-Model-based planners
DriveLaW ( Xia et al., 2026a )
–
LTX-2B
99.0
97.1
96.7
100.0
81.3
89.1
Table 1: Comparison of planners based on different paradigms on NAVSIM v1 navtest . Bold values denote the best result for each metric.
Method
NC ↑
DAC ↑
DDC ↑
TLC ↑
EP ↑
TTC ↑
LK ↑
HC ↑
EC ↑
EPDMS ↑
DiffusionDrive ( Liao et al., 2025 )
98.2
95.9
99.4
99.8
87.5
97.3
96.8
98.3
87.7
84.5
DiffusionDriveV2 ( Zou et al., 2025 )
97.7
96.6
99.2
99.8
88.9
97.2
96.0
97.8
91.0
87.5
DriveWorld-VLA ( Liu et al., 2026b )
98.6
99.1
99.6
99.8
87.4
97.9
97.0
97.8
78.6
86.8
Latent-WAM ( Wang et al., 2026a )
98.1
97.3
99.6
99.8
87.7
97.3
97.6
98.1
87.3
89.3
CoWorld-VLA ( Huang et al., 2026 )
99.1
97.0
99.6
99.9
87.9
98.5
97.7
98.2
86.2
90.0
DriveFuture ( Hong et al., 2026 )
98.8
99.1
99.6
99.9
86.6
98.4
96.4
98.3
74.8
89.9
Table 2: Comparison on the NAVSIM v2 navtest split. All results use the corrected official evaluator with human-reference filtering. Bold values denote the best result for each metric.
Method
All 436 Scenarios
HD-Score
NC ↑
DAC ↑
TTC ↑
Comf. ↑
PDMS ↑
RC ↑
Easy ↑
Medium ↑
Hard ↑
Extreme ↑
Avg. ↑
VAD ( Jiang et al., 2023 )
41.20
90.30
28.00
95.30
28.30
30.10
42.00
8.50
7.70
6.30
13.90
UniAD ( Hu et al., 2023b )
65.60
93.20
51.60
66.30
49.40
43.80
64.00
37.20
21.00
6.30
31.20
LTF ( Chitta et al., 2022 ; Zhou et al., 2025 )
44.30
92.80
37.50
94.80
36.50
38.00
66.10
15.50
12.00
11.70
23.10
DrivoR ( Kirby et al., 2026 )
52.20
95.60
46.20
93.90
44.80
47.20
78.00
29.10
20.00
14.10
32.50
DriveWLA (Ours)
65.80
95.06
59.27
75.78
54.92
54.99
73.87
47.91
31.41
15.87
41.47
Table 3: Zero-shot closed-loop evaluation on HUGSIM. We report driving metrics over all 436 scenarios and HD-Score across different difficulty levels, together with the overall average.
Dataset
Method
Fine-tune
L2 (m) ↓
Collision (%) ↓
1s
2s
3s
Avg.
1s
2s
3s
Avg.
nuScenes
UniAD ( Hu et al., 2023b )
✓
0.48
0.96
1.65
1.03
0.05
0.17
0.71
0.31
OpenDriveVLA ( Zhou et al., 2026b )
✓
0.19
0.58
1.24
0.67
0.02
0.18
0.70
0.30
UniDriveVLA-L ( Li et al., 2026c )
✓
0.30
0.66
1.25
0.74
0.02
0.18
0.40
0.20
DriveVLA-W0 ( Li et al., 2026b )
✗
0.43
1.26
2.60
1.43
0.22
0.66
1.42
0.77
PWM ( Zhao et al., 2026a )
✗
2.06
3.91
6.00
3.99
0.12
0.15
0.86
0.36
Table 4: Zero-shot open-loop evaluation on nuScenes and Bench2Drive. We report L2 trajectory error (m) and collision rate (%) over 1–3 seconds and their averages. “Fine-tune” indicates target-dataset adaptation. Bold denotes the best result for each metric within each dataset.
Table 7Table 8
Figure 3: Qualitative visualization of expert complementarity and RL improvement. (a) Removing either the Language or World Expert leads to planning failures in different scenes, highlighting their complementary contributions. (b) RL refinement corrects the imitation policy and discovers a valid alternative trajectory distinct from the expert demonstration.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 4: World Expert latent features. Each row shows one driving scene. The columns show the front-view image, history latents 1 and 4, and future latents 1 and 2. All PCA-projected latent maps use the same color scale.
World models have become central to autonomous driving, where accurate scene understanding and future prediction are crucial for safe control. Recent work has explored using vision-language models (VLMs) for planning, yet existing approaches typically treat perception, prediction, and planning as separate modules. We propose UniDrive-WM, a unified VLM-based world model that jointly performs driving-scene understanding, trajectory planning, and trajectory-conditioned future image generation within a single architecture. UniDrive-WM's trajectory planner predicts a future trajectory, which conditions a VLM-based image generator to produce plausible future frames. These predictions provide additional supervisory signals that enhance scene understanding and iteratively refine trajectory generation. We further compare discrete and continuous output representations for future image prediction, analyzing their influence on downstream driving performance. Experiments on the challenging Bench2Drive benchmark show that UniDrive-WM produces high-fidelity future images and improves planning performance by 7.3% in L2 trajectory error and 10.4% in collision rate over the previous best method. These results demonstrate the advantages of tightly integrating VLM-driven reasoning, planning, and generative world modeling for autonomous driving. The project page is available at https://unidrive-wm.github.io/UniDrive-WM.
Zhexiao Xiong, Xin Ye, Burhan Yaman +5
Bosch Research North America · Bosch Center for Artificial Intelligence (BCAI) · Arizona State University +2
Vision-Language-Action (VLA) models have recently achieved notable progress in end-to-end autonomous driving by integrating perception, reasoning, and control within a unified multimodal framework. However, they often lack explicit modeling of temporal dynamics and global world consistency, which limits their foresight and safety. In contrast, world models can simulate plausible future scenes but generally struggle to reason about or evaluate the imagined future they generate. In this work, we present VLA-World, a simple yet effective VLA world model that unifies predictive imagination with reflective reasoning to improve driving foresight. VLA-World first uses an action-derived feasible trajectory to guide the generation of the next-frame image, capturing rich spatial and temporal cues that describe how the surrounding environment evolves. The model then reasons over this self-generated future imagined frame to refine the predicted trajectory, achieving higher performance and better interpretability. To support this pipeline, we curate nuScenes-GR-20K, a generative reasoning dataset derived from nuScenes, and employ a three-stage training strategy that includes pretraining, supervised fine-tuning, and reinforcement learning. Extensive experiments demonstrate that VLA-World consistently surpasses state-of-the-art VLA and world-model baselines on both planning and future-generation benchmarks. Project page: https://vlaworld.github.io
Guoqing Wang, Pin Tang, Xiangxuan Ren +3
MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University · Central Research Institute, Huawei
Vision-Language Models (VLMs) provide powerful semantic understanding and commonsense reasoning for End-to-End Autonomous Driving (E2E-AD) planning. However, trajectories directly generated by VLMs often encode only coarse driving intentions and remain insufficient for geometrically accurate, future-aware, and multi-view-grounded planning. To address these limitations, we develop the Layer-Wise World-Model-Guided Driving framework (LWDrive). LWDrive is a VLM planning framework that refines coarse trajectories through layer-wise world-model guidance. Instead of treating the VLM output as the final trajectory, LWDrive uses it as an intent-aware coarse plan, expands a diverse candidate space around it, and progressively refines the candidates through a Foresight Cascade Planner (FCP). Specifically, we introduce future-frame generation supervision to encourage the VLM to learn forward-looking scene representations, thereby injecting planning-relevant predictive dynamics into its internal hidden states. Built upon these world-model-supervised representations, FCP exploits VLM features across multiple layers and integrates historical temporal states, Action-Query representations, and current-frame multi-view Bird's-Eye-View (BEV) features to refine candidate trajectories in a coarse-to-fine manner. This design enables progressive correction of spatial positions and motion trends while grounding trajectory refinement with multi-view scene cues and preserving the high-level driving intention produced by the large model. Finally, a score head evaluates the refined candidates and selects the best trajectory as the final planning output. Experiments show that LWDrive achieves a score of 92.0 on the NAVSIM benchmark and 89.6 on NAVSIM-v2. Code and models will be made publicly available.