Autonomous driving requires reasoning about how the environment evolves and planning actions accordingly. Existing world-model-based approaches typically predict future scenes first and plan afterwards, resulting in open-loop imagination that may drift from the actual decision process. In this paper, we present Uni-World VLA, a unified vision-language-action (VLA) model that tightly interleaves future frame prediction and trajectory planning. Instead of generating a full world rollout before planning, our model alternates between predicting future frames and ego actions step by step, allowing planning decisions to be continuously conditioned on the imagined future observations. This interleaved generation forms a closed-loop interaction between world modeling and control, enabling more adaptive decision-making in dynamic traffic scenarios. In addition, we incorporate monocular depth information into frames to provide stronger geometric cues for world modeling, improving long-horizon scene prediction. Experiments on the NAVSIM benchmark show that our approach achieves competitive closed-loop planning performance while producing high-fidelity future frame predictions. These results demonstrate that tightly coupling world prediction and planning is a promising direction for scalable VLA driving systems.
Figures & tables
Figure 1: Different generative paradigms of unified world models for autonomous driving. (a) Unified world models perform video generation and planning as separate tasks; (b) World-conditioned trajectory prediction, where future trajectories are predicted conditioned on the generated world states; (c) Interleaved world modeling and planning (ours). Visual tokens and action queries are generated alternately, forming a closed-loop interaction that respects the temporal causality of driving.
Figure 2: Overview of the paradigm of alternative generation in Uni-World VLA. (a) The construction of multi-model historical information; (b) The interleaved frame-action generative paradigm.
Figure 3: Schematic illustration of training and inference process. (a) Interleaved sequence for joint video generation and trajectory supervision. (b) Causal attention mask. (c) Autoregressive interleaved inference with KV-cache reuse.
Method
Input
NC ↑
DAC ↑
EP ↑
TTC ↑
Comf. ↑
PDMS ↑
Traditional End-to-End Methods
VADv2- ν8192 [ 6 ]
C
97.2
89.1
76.0
91.6
100.0
80.9
UniAD [ 17 ]
C
97.8
91.9
78.8
92.9
100.0
83.4
TransFuser [ 9 ]
C&L
97.7
92.8
79.2
92.8
100.0
84.0
ReCogDrive-IL [ 30 ]
SC
98.1
94.7
80.9
94.2
100.0
86.5
DiffusionDrive [ 33 ]
C&L
98.2
96.2
82.2
94.7
100.0
88.1
Table 1: Comparison with state-of-the-art methods on the NAVSIM test split. PDMS and its sub-metrics (NC, DAC, EP, TTC, and Comfort; see Sec. 4.1 ) evaluate closed-loop driving performance. Input modalities include multi-view cameras (C), single-view camera (SC), and multi-view cameras with LiDAR (C&L). Best results are shown in bold .
Metric & Settings
WoVoGen [ 35 ]
DriveDreamer [ 48 ]
SVD [ 2 , 7 ]
DrivingGPT [ 7 ]
GenAD [ 55 ]
Ours
FVD ↓
417.7
340.8
227.5
142.6
184.0
141.8
Max Duration/Fps
2.5 s/2 Hz
4 s/2 Hz
4 s/2 Hz
4 s/2 Hz
4 s/2 Hz
4 s/2 Hz
Dataset
nuScenes
nuScenes
NAVSIM
NAVSIM
OpenDV
NAVSIM
View
Multi
Multi
Front
Front
Front
Front
Table 2: Comparison of video generation quality across driving world models. FVD 4.1 measures the realism of generated future video sequences. We additionally report the maximum prediction horizon, frame rate, dataset, and camera view used by each method.
Figure 4: Visualization of predicted frames and BEV trajectories
Pretrain
Future Frames
Depth
NC ↑
DAC ↑
EP ↑
TTC ↑
Comf. ↑
PDMS ↑
FVD ↓
×
×
×
97.1
91.4
77.4
91.5
100.0
82.1
-
✓
×
×
98.8
95.8
82.0
95.8
100.0
88.2
-
✓
✓
×
98.8
96.5
82.9
96.4
100.0
89.2
164.2
✓
✓
✓
98.7
96.7
83.2
96.1
100.0
89.4
141.8
Table 3: Ablation study on the effects of pretraining, future-frame modeling, and depth conditioning. NC, DAC, EP, TTC, Comfort, and PDMS measure planning quality, while FVD evaluates the quality of generated future frames.
Figure 5: Comparison of predicted future frames with and without depth fusion
Scheme
NC ↑
DAC ↑
EP ↑
TTC ↑
Comf. ↑
PDMS ↑
A-cross-frequency alternation
98.8
96.4
81.2
96.0
100.0
88.3
B-high-frequency actions-frames
98.7
95.1
77.3
96.4
100.0
86.1
C-hybrid dense-then-coarse
98.7
95.8
80.6
96.0
100.0
87.8
D-sliding 1s action windows
98.6
94.5
78.4
95.2
99.7
85.7
E-2Hz aligned interleaving(Ours)
98.8
96.5
82.9
96.4
100.0
89.2
Table 4: Ablation over five alternative generation schemes (without depth fusion).
Historical Visual Info.
NC ↑
DAC ↑
EP ↑
TTC ↑
Comf. ↑
PDMS ↑
FVD ↓
2.0 s Context+Dynamic (Ours)
98.8
96.5
82.9
96.4
100.0
89.2
164.2
1.0 s Context+Dynamic
99.0
96.4
81.4
96.7
100.0
88.8
170.7
Context Only
98.6
96.8
82.3
96.2
100.0
89.1
165.5
Dynamic Only
97.4
90.8
76.2
92.3
100.0
81.7
203.6
Table 5: Effect of historical visual information on planning (without depth fusion).
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Details of contextual, dynamic and action tokens
Figure 7: Visualization of predicted future frames (a) and the corresponding ground truth (b) in representative and challenging driving scenarios.
Figure 8: BEV visualization of the corresponding six scenarios (from left to right and top to bottom). Green polyline: ground-truth trajectory; red polyline: planned future trajectory.
Vision-Language-Action (VLA) models have recently achieved notable progress in end-to-end autonomous driving by integrating perception, reasoning, and control within a unified multimodal framework. However, they often lack explicit modeling of temporal dynamics and global world consistency, which limits their foresight and safety. In contrast, world models can simulate plausible future scenes but generally struggle to reason about or evaluate the imagined future they generate. In this work, we present VLA-World, a simple yet effective VLA world model that unifies predictive imagination with reflective reasoning to improve driving foresight. VLA-World first uses an action-derived feasible trajectory to guide the generation of the next-frame image, capturing rich spatial and temporal cues that describe how the surrounding environment evolves. The model then reasons over this self-generated future imagined frame to refine the predicted trajectory, achieving higher performance and better interpretability. To support this pipeline, we curate nuScenes-GR-20K, a generative reasoning dataset derived from nuScenes, and employ a three-stage training strategy that includes pretraining, supervised fine-tuning, and reinforcement learning. Extensive experiments demonstrate that VLA-World consistently surpasses state-of-the-art VLA and world-model baselines on both planning and future-generation benchmarks. Project page: https://vlaworld.github.io
Guoqing Wang, Pin Tang, Xiangxuan Ren +3
MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University · Central Research Institute, Huawei
World models have become central to autonomous driving, where accurate scene understanding and future prediction are crucial for safe control. Recent work has explored using vision-language models (VLMs) for planning, yet existing approaches typically treat perception, prediction, and planning as separate modules. We propose UniDrive-WM, a unified VLM-based world model that jointly performs driving-scene understanding, trajectory planning, and trajectory-conditioned future image generation within a single architecture. UniDrive-WM's trajectory planner predicts a future trajectory, which conditions a VLM-based image generator to produce plausible future frames. These predictions provide additional supervisory signals that enhance scene understanding and iteratively refine trajectory generation. We further compare discrete and continuous output representations for future image prediction, analyzing their influence on downstream driving performance. Experiments on the challenging Bench2Drive benchmark show that UniDrive-WM produces high-fidelity future images and improves planning performance by 7.3% in L2 trajectory error and 10.4% in collision rate over the previous best method. These results demonstrate the advantages of tightly integrating VLM-driven reasoning, planning, and generative world modeling for autonomous driving. The project page is available at https://unidrive-wm.github.io/UniDrive-WM.
Zhexiao Xiong, Xin Ye, Burhan Yaman +5
Bosch Research North America · Bosch Center for Artificial Intelligence (BCAI) · Arizona State University +2
Vision-Language Models (VLMs) and world models have demonstrated strong capabilities in semantic understanding and future scene evolution for autonomous driving. However, effectively integrating both capabilities for trajectory generation remains challenging. In this work, we propose DriveWLA, a unified World-Language-Action model based on Mixture-of-Transformers (MoT) that jointly models semantic understanding, future scene dynamics, and trajectory generation. DriveWLA consists of three experts: a World Expert for future prediction, a Language Expert for semantic understanding, and an Action Expert for trajectory generation. We further introduce Cross-Expert Attention, which enables controlled information flow across experts while aligning semantic and dynamic representations. To improve trajectory generation beyond imitation learning, we apply Flow-GRPO reinforcement learning post-training to refine the Action Expert. Extensive experiments show that DriveWLA achieves superior performance in both open-loop and closed-loop evaluation. DriveWLA achieves 92.3 PDMS on NAVSIM v1 and 90.1 EPDMS on NAVSIM v2, demonstrating strong performance among VLM-based and world-model-based planners. Without specific fine-tuning, DriveWLA further demonstrates strong zero-shot generalization to HUGSIM, nuScenes, and Bench2Drive. These results validate the effectiveness of our unified World-Language-Action modeling, establishing DriveWLA as a generalizable framework for autonomous driving.
Shuzheng Wang, Haoxuan Li, Shengzhen Li +5
Huazhong University of Science and Technology · Yinwang Intelligent Technology Co. Ltd.