Vision-Language Models (VLMs) and world models have demonstrated strong capabilities in semantic understanding and future scene evolution for autonomous driving. However, effectively integrating both capabilities for trajectory generation remains challenging. In this work, we propose DriveWLA, a unified World-Language-Action model based on Mixture-of-Transformers (MoT) that jointly models semantic understanding, future scene dynamics, and trajectory generation. DriveWLA consists of three experts: a World Expert for future prediction, a Language Expert for semantic understanding, and an Action Expert for trajectory generation. We further introduce Cross-Expert Attention, which enables controlled information flow across experts while aligning semantic and dynamic representations. To improve trajectory generation beyond imitation learning, we apply Flow-GRPO reinforcement learning post-training to refine the Action Expert. Extensive experiments show that DriveWLA achieves superior performance in both open-loop and closed-loop evaluation. DriveWLA achieves 92.3 PDMS on NAVSIM v1 and 90.1 EPDMS on NAVSIM v2, demonstrating strong performance among VLM-based and world-model-based planners. Without specific fine-tuning, DriveWLA further demonstrates strong zero-shot generalization to HUGSIM, nuScenes, and Bench2Drive. These results validate the effectiveness of our unified World-Language-Action modeling, establishing DriveWLA as a generalizable framework for autonomous driving.
Figures & tables
Figure 1: Model architecture and training strategy. DriveWLA unifies World, Language, and Action Experts through Cross-Expert Attention. The training strategy consists of trajectory imitation learning, which jointly optimizes all experts, followed by reinforcement learning refinement of the Action Expert.
Figure 2: Query-Key/Value Visibility in Cross-Expert Attention.
Method
VLM Backbone
Video Backbone
NC ↑
DAC ↑
TTC ↑
Comf. ↑
EP ↑
PDMS ↑
VLM-based planners
ReCogDrive ( Xiong et al., 2026 )
InternVL3-2B
–
97.9
97.3
94.9
100.0
87.3
90.8
SGDrive ( Li et al., 2026a )
InternVL3-2B
–
98.6
97.8
96.2
100.0
85.8
91.1
Qwen-Drive-1.0 ( Zhou et al., 2026a )
Qwen3.5-4B
–
98.6
98.2
95.9
100.0
84.8
90.7
World-Model-based planners
DriveLaW ( Xia et al., 2026a )
–
LTX-2B
99.0
97.1
96.7
100.0
81.3
89.1
Table 1: Comparison of planners based on different paradigms on NAVSIM v1 navtest . Bold values denote the best result for each metric.
Method
NC ↑
DAC ↑
DDC ↑
TLC ↑
EP ↑
TTC ↑
LK ↑
HC ↑
EC ↑
EPDMS ↑
DiffusionDrive ( Liao et al., 2025 )
98.2
95.9
99.4
99.8
87.5
97.3
96.8
98.3
87.7
84.5
DiffusionDriveV2 ( Zou et al., 2025 )
97.7
96.6
99.2
99.8
88.9
97.2
96.0
97.8
91.0
87.5
DriveWorld-VLA ( Liu et al., 2026b )
98.6
99.1
99.6
99.8
87.4
97.9
97.0
97.8
78.6
86.8
Latent-WAM ( Wang et al., 2026a )
98.1
97.3
99.6
99.8
87.7
97.3
97.6
98.1
87.3
89.3
CoWorld-VLA ( Huang et al., 2026 )
99.1
97.0
99.6
99.9
87.9
98.5
97.7
98.2
86.2
90.0
DriveFuture ( Hong et al., 2026 )
98.8
99.1
99.6
99.9
86.6
98.4
96.4
98.3
74.8
89.9
Table 2: Comparison on the NAVSIM v2 navtest split. All results use the corrected official evaluator with human-reference filtering. Bold values denote the best result for each metric.
Method
All 436 Scenarios
HD-Score
NC ↑
DAC ↑
TTC ↑
Comf. ↑
PDMS ↑
RC ↑
Easy ↑
Medium ↑
Hard ↑
Extreme ↑
Avg. ↑
VAD ( Jiang et al., 2023 )
41.20
90.30
28.00
95.30
28.30
30.10
42.00
8.50
7.70
6.30
13.90
UniAD ( Hu et al., 2023b )
65.60
93.20
51.60
66.30
49.40
43.80
64.00
37.20
21.00
6.30
31.20
LTF ( Chitta et al., 2022 ; Zhou et al., 2025 )
44.30
92.80
37.50
94.80
36.50
38.00
66.10
15.50
12.00
11.70
23.10
DrivoR ( Kirby et al., 2026 )
52.20
95.60
46.20
93.90
44.80
47.20
78.00
29.10
20.00
14.10
32.50
DriveWLA (Ours)
65.80
95.06
59.27
75.78
54.92
54.99
73.87
47.91
31.41
15.87
41.47
Table 3: Zero-shot closed-loop evaluation on HUGSIM. We report driving metrics over all 436 scenarios and HD-Score across different difficulty levels, together with the overall average.
Dataset
Method
Fine-tune
L2 (m) ↓
Collision (%) ↓
1s
2s
3s
Avg.
1s
2s
3s
Avg.
nuScenes
UniAD ( Hu et al., 2023b )
✓
0.48
0.96
1.65
1.03
0.05
0.17
0.71
0.31
OpenDriveVLA ( Zhou et al., 2026b )
✓
0.19
0.58
1.24
0.67
0.02
0.18
0.70
0.30
UniDriveVLA-L ( Li et al., 2026c )
✓
0.30
0.66
1.25
0.74
0.02
0.18
0.40
0.20
DriveVLA-W0 ( Li et al., 2026b )
✗
0.43
1.26
2.60
1.43
0.22
0.66
1.42
0.77
PWM ( Zhao et al., 2026a )
✗
2.06
3.91
6.00
3.99
0.12
0.15
0.86
0.36
Table 4: Zero-shot open-loop evaluation on nuScenes and Bench2Drive. We report L2 trajectory error (m) and collision rate (%) over 1–3 seconds and their averages. “Fine-tune” indicates target-dataset adaptation. Bold denotes the best result for each metric within each dataset.
Table 7Table 8
Figure 3: Qualitative visualization of expert complementarity and RL improvement. (a) Removing either the Language or World Expert leads to planning failures in different scenes, highlighting their complementary contributions. (b) RL refinement corrects the imitation policy and discovers a valid alternative trajectory distinct from the expert demonstration.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 4: World Expert latent features. Each row shows one driving scene. The columns show the front-view image, history latents 1 and 4, and future latents 1 and 2. All PCA-projected latent maps use the same color scale.