Recently, world-action models (WAM) have emerged to bridge vision-language-action (VLA) models and world models, unifying their reasoning and instruction-following capabilities and spatio-temporal world modeling. However, existing WAM approaches often focus on modeling 2D appearance or latent representations, with limited geometric grounding-an essential element for embodied systems operating in the physical world. We present DriveDreamer-Policy, a unified driving world-action model that integrates depth generation, future video generation, and motion planning within a single modular architecture. The model employs a large language model to process language instructions, multi-view images, and actions, followed by three lightweight generators that produce depth, future video, and actions. By learning a geometry-aware world representation and using it to guide both future prediction and planning within a unified framework, the proposed model produces more coherent imagined futures and more informed driving actions, while maintaining modularity and controllable latency. Experiments on the Navsim v1 and v2 benchmarks demonstrate that DriveDreamer-Policy achieves strong performance on both closed-loop planning and world generation tasks. In particular, our model reaches 89.2 PDMS on Navsim v1 and 88.7 EPDMS on Navsim v2, outperforming existing world-model-based approaches while producing higher-quality future video and depth predictions. Ablation studies further show that explicit depth learning provides complementary benefits to video imagination and improves planning robustness.
Figures & tables
Figure 1: Comparison of our DriveDreamer-Policy with existing models. Items with dashed lines are optional. Vision-based and VLA planners directly map observations (and optional inputs) to actions without explicitly predicting the future world. World models generate future observations but often rely on external action signals. Recent world–action models unify future world generation and planning, but typically operate on image/video representations. DriveDreamer-Policy extends this line by explicitly generating depth alongside video and actions, enabling geometry-grounded imagination and planning within a unified model.
Figure 2: Overview of our DriveDreamer-Policy pipeline. The large language model takes the language instruction, multi-view images and current action, along with a set of learnable queries as inputs to reason and generate world and action embeddings. The generated embeddings are then passed into our three generative expert models as cross-attention conditions to generate depth, future images, and future action.
Methods
Venue
Sensors
NC ↑
DAC ↑
TTC ↑
C ↑
EP ↑
PDMS ↑
Human
–
–
100.0
100.0
100.0
99.9
87.5
94.8
Vision-Based End-to-End Methods
TransFuser ( Chitta et al., 2022 )
TPAMI’23
3 × C + L
97.7
92.8
92.8
100.0
79.2
84.0
UniAD ( Hu et al., 2023 )
CVPR’23
6 × C
97.8
91.9
92.9
100.0
78.8
83.4
PARA-Drive ( Weng et al., 2024 )
CVPR’24
6 × C
97.9
92.4
93.0
99.8
79.3
84.0
DiffusionDrive ( Liao et al., 2025a )
CVPR’25
3 × C + L
98.2
96.2
94.7
100.0
82.2
88.1
Table 1: Comparison with state-of-the-art methods on the the navtest of Navsim v1 ( Dauner et al., 2024 ) benchmark. The best result outlines as bold. “*” indicates results with imitation learning. Methods fall into three categories: vision-based E2E models, VLA-based models, and world-model-based models.
Methods
Venue
NC ↑
DAC ↑
DDC ↑
TLC ↑
EP ↑
TTC ↑
LK ↑
HC ↑
EC ↑
EPDMS ↑
Vision-Based End-to-End Methods
TransFuser ( Chitta et al., 2022 )
TPAMI’23
96.9
89.9
97.8
99.7
87.1
95.4
92.7
98.3
87.2
76.7
DiffusionDrive ( Liao et al., 2025a )
CVPR’25
98.2
95.9
99.4
99.8
87.5
97.3
96.8
98.3
87.7
84.5
Drivesuprim ( Yao et al., 2025 )
AAAI’26
97.5
96.5
99.4
99.6
88.4
96.6
95.5
98.3
77.0
83.1
ARTEMIS ( Feng et al., 2026 )
RAL’26
98.3
95.1
98.6
99.8
81.5
97.4
96.5
98.3
89.1
83.1
Vision-Language-Action Methods
Table 2: Comparison with state-of-the-art methods on the navtest of Navsim v2 ( Cao et al., 2025 ) .
Strategy
World Representation
Planning Metrics
Depth
Video
NC ↑
DAC ↑
TTC ↑
C ↑
EP ↑
PDMS ↑
Without World Learning
✗
✗
98.0
96.3
94.4
100.0
82.5
88.0
With World Learning
✔
✗
98.1
96.7
94.9
100.0
82.8
88.5
✗
✔
98.1
97.0
95.0
100.0
83.1
88.9
✔
✔
98.4
97.1
95.1
100.0
83.5
89.2
Table 4: Ablations on World Learning for Planning. All world learning strategies effectively improve planning performance compared to training from scratch.
Strategy
World Representation
Video Metrics
Depth
LPIPS ↓
PSNR ↑
FVD ↓
Without Depth Learning
✗
0.22
19.89
65.82
With Depth Learning
✔
0.20
21.05
53.59
Table 5: Ablations on Depth Learning for Video Generation. Using depth as a prior in joint learning improves video generation accuracy.
Number of Queries
Metrics
Depth
Video
Action
Depth
Video
Action
AbsRel ↓
δ1↑
δ2↑
δ3↑
LPIPS ↓
PSNR ↑
FVD ↓
NC ↑
DAC ↑
TTC ↑
C ↑
EP ↑
PDMS ↑
32
32
4
9.7
90.2
97.9
99.4
0.20
20.67
57.97
98.2
97.0
95.0
100.0
83.2
88.9
64
64
8
8.1
92.8
98.6
99.5
0.20
21.05
53.59
98.4
97.1
95.1
100.0
83.5
89.2
Table 6: Ablations on Number of Queries. More query tokens provide higher-capacity slots for storing relevant context, thereby enhancing both generation and planning.
Figure 3: Visualization Results of our method. We show the generated depth, video, and actions, respectively. Depth is truncated to below 80 meters for better visualization. Our generation results remain spatially stable, and the planning performs well compared with human trajectories ( e.g. , aligns with human trajectories (top) and slows down more effectively than human trajectories (bottom)).
Figure 4: Visualization of world learning for planning. Columns compare Action-Only, Depth-Action, Video-Action, and Depth-Video-Action variants. Green denotes the human (expert) trajectory and red denotes the predicted trajectory. The three rows correspond to (top) avoiding potential collision by a slower trajectory, (middle) correcting an initially wrong maneuver, and (bottom) aligning more closely with the human trajectory. Depth and video provide complementary world cues that improve safety margins and trajectory consistency.
Vision-Language-Action (VLA) models have recently achieved notable progress in end-to-end autonomous driving by integrating perception, reasoning, and control within a unified multimodal framework. However, they often lack explicit modeling of temporal dynamics and global world consistency, which limits their foresight and safety. In contrast, world models can simulate plausible future scenes but generally struggle to reason about or evaluate the imagined future they generate. In this work, we present VLA-World, a simple yet effective VLA world model that unifies predictive imagination with reflective reasoning to improve driving foresight. VLA-World first uses an action-derived feasible trajectory to guide the generation of the next-frame image, capturing rich spatial and temporal cues that describe how the surrounding environment evolves. The model then reasons over this self-generated future imagined frame to refine the predicted trajectory, achieving higher performance and better interpretability. To support this pipeline, we curate nuScenes-GR-20K, a generative reasoning dataset derived from nuScenes, and employ a three-stage training strategy that includes pretraining, supervised fine-tuning, and reinforcement learning. Extensive experiments demonstrate that VLA-World consistently surpasses state-of-the-art VLA and world-model baselines on both planning and future-generation benchmarks. Project page: https://vlaworld.github.io
Guoqing Wang, Pin Tang, Xiangxuan Ren +3
MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University · Central Research Institute, Huawei
Generalization is a central challenge in autonomous driving, as real-world deployment requires robust performance under unseen scenarios, sensor domains, and environmental conditions. Recent world-model-based planning methods have shown strong capabilities in scene understanding and multi-modal future prediction, yet their generalization across datasets and sensor configurations remains limited. In addition, their loosely coupled planning paradigm often leads to poor video-trajectory consistency during visual imagination. To overcome these limitations, we propose DriveVA, a novel autonomous driving world model that jointly decodes future visual forecasts and action sequences in a shared latent generative process. DriveVA inherits rich priors on motion dynamics and physical plausibility from well-pretrained large-scale video generation models to capture continuous spatiotemporal evolution and causal interaction patterns. To this end, DriveVA employs a DiT-based decoder to jointly predict future action sequences (trajectories) and videos, enabling tighter alignment between planning and scene evolution. We also introduce a video continuation strategy to strengthen long-duration rollout consistency. DriveVA achieves an impressive PDM-based planning performance of 90.9 PDM score on the NAVSIM benchmark. Extensive experiments also demonstrate the zero-shot capability and cross-domain generalization of DriveVA, which reduces average L2 error and collision rate by 78.9% and 83.3% on nuScenes and 52.5% and 52.4% on the Bench2Drive built on CARLA v2 compared with the state-of-the-art world-model-based planner.
Mengmeng Liu, Diankun Zhang, Jiuming Liu +7
University of Twente, The Netherlands · Xiaomi EV, China · University of Cambridge, United Kingdom +1
Vision-Language Models (VLMs) and world models have demonstrated strong capabilities in semantic understanding and future scene evolution for autonomous driving. However, effectively integrating both capabilities for trajectory generation remains challenging. In this work, we propose DriveWLA, a unified World-Language-Action model based on Mixture-of-Transformers (MoT) that jointly models semantic understanding, future scene dynamics, and trajectory generation. DriveWLA consists of three experts: a World Expert for future prediction, a Language Expert for semantic understanding, and an Action Expert for trajectory generation. We further introduce Cross-Expert Attention, which enables controlled information flow across experts while aligning semantic and dynamic representations. To improve trajectory generation beyond imitation learning, we apply Flow-GRPO reinforcement learning post-training to refine the Action Expert. Extensive experiments show that DriveWLA achieves superior performance in both open-loop and closed-loop evaluation. DriveWLA achieves 92.3 PDMS on NAVSIM v1 and 90.1 EPDMS on NAVSIM v2, demonstrating strong performance among VLM-based and world-model-based planners. Without specific fine-tuning, DriveWLA further demonstrates strong zero-shot generalization to HUGSIM, nuScenes, and Bench2Drive. These results validate the effectiveness of our unified World-Language-Action modeling, establishing DriveWLA as a generalizable framework for autonomous driving.
Shuzheng Wang, Haoxuan Li, Shengzhen Li +5
Huazhong University of Science and Technology · Yinwang Intelligent Technology Co. Ltd.