Autonomous driving requires anticipating how traffic will evolve while responding quickly to new observations. Video foundation models offer the first ability, but repeated history encoding and iterative video generation make them costly to use for driving. We present AR-WAM, an efficient streaming world-action model that combines autoregressive temporal modeling with single-pass future conditioning. Because its video backbone is causal, AR-WAM updates its visual context incrementally, reusing past computation and conditioning future prediction on observed motion. We further hypothesize that future features computed from recent observations can improve generalization in dynamic traffic without iterative video generation. AR-WAM therefore obtains these features in one forward pass and uses them to condition trajectory generation, and reinforcement learning then improves the policy through direct optimization of driving rewards. With a single front camera, AR-WAM achieves 91.5 PDMS after supervised training and 94.0 after reinforcement learning on NAVSIM-v1, outperforming the compared strong end-to-end world-action models. In a dynamic setting of continuous driving, supervised AR-WAM keeps its visual history across decisions and outperforms strong world-action baselines.
Figures & tables
Figure 1: AR-WAM plans from cached visual history and a single predictive pass. (a) Predicted futures and NAVHARD EPDMS against latency. One video pass scores above the variant without future conditioning (38.6 versus 37.3) and matches ten video passes (38.8) at under a quarter of their latency, while SimWAM and UNIVERSE score lower at higher latency. (b) AR-WAM encodes only the new frame and reuses its cached history, whereas a sliding-window policy re-encodes its window at every decision. (c) Reward refinement with Dr. GRPO raises NAVTEST EPDMS to 92.2; the scored samples show 32 plans for one scene, colored by reward.
Figure 2: AR-WAM architecture. (a) Each decision runs one pass of the backbone, a causal video DiT, over the current frame and noised future slots, attending to the cached K/V of earlier frames; the ego state and driving command enter this pass but not the cache. (b) At matching backbone depths, the trajectory expert reads the shared visual K/V. (c) The predicted trajectory for a NAVTEST scene.
Future
PDMS ↑
EPDMS ↑
L2 ↓
Passes
Latency
No future
91.5
37.3
0.87
1
0.8 ×
One pass (AR-WAM)
91.5
38.6
0.87
1
1.0 ×
Clear future
91.5
38.8
0.88
10
4.2 ×
Table 1: Future modeling at inference. PDMS: NAVTEST; EPDMS: NAVHARD two-stage; L2: nuScenes zero-shot average in metres (Sec. 5.5 ); Passes: backbone world passes per decision; Latency: median per decision relative to one pass, which takes 146 ms on an H200. Bold: best result in each column.
NAVTEST, NAVSIM-v1
NAVTEST
NAVHARD
Method
Ref.
Cam.
NC
DAC
TTC
Comf.
EP
PDMS
EPDMS
EPDMS
Reference planners without a world model
Human driver [ 7 ]
–
–
100.0
100.0
100.0
99.9
87.5
94.8
–
–
DiffusionDrive [ 20 ]
CVPR’25
3+L
98.2
96.2
94.7
100.0
82.2
88.1
–
–
ReCogDrive † [ 17 ]
ICLR’26
1
97.9
97.3
94.9
100.0
87.3
90.8
–
–
DrivoR ‡ [ 15 ]
CVPR’26
4
98.9
98.3
96.2
100.0
89.1
93.1
–
48.3
Table 2: Comparison on NAVTEST and NAVHARD. Baseline values are publication-reported. Cam.: cameras (L: LiDAR; M: multiview). † : reward post-training. ‡ : proposal scoring. Bold: best result in each column, not counting the human driver and ‡ rows.
Method
Hist.
Latency (ms)
NC
DAC
DDC
TLC
EP
TTC
LK
HC
EC
EPDMS
SimWAM [ 50 ]
1
499.1
98.0
99.1
99.4
99.5
88.0
98.2
98.1
98.7
87.2
90.4
UNIVERSE [ 23 ]
5
256.9
98.5
99.0
99.5
99.8
87.8
98.7
97.9
98.6
64.5
88.3
DriveVA [ 24 ]
5
258.8
98.5
99.0
99.5
99.8
87.7
98.6
98.2
98.5
57.4
87.4
DriveWAM [ 30 ]
1
895.7
98.0
99.1
99.4
99.4
88.3
98.1
98.3
98.7
91.8
90.9
AR-WAM, current frame
1
106.9
98.2
99.1
99.5
99.7
88.0
98.3
98.5
98.7
84.8
90.7
AR-WAM
5
146.3
98.7
99.1
99.5
99.6
88.0
98.8
98.8
98.7
88.0
91.5
Table 3: Dynamic setting: the 47,220 non-waiting frames of 138 NAVTEST logs, run in order at 2 Hz with a persistent history cache. Columns are one-stage NAVSIM-v2 sub-scores. Hist.: camera frames used per decision. Latency: median per decision at batch size one on an H200. Current frame: AR-WAM retrained without visual history. Bold: best result in each column.
All samples
Non-waiting
Waiting
L2
Col.
L2
Col.
L2
Col.
Method
1 s
2 s
3 s
Avg.
Avg.
Avg.
Avg.
Avg.
Avg.
Constant velocity
0.33
1.13
2.34
1.27
0.47
1.46
0.54
0.15
0.06
SimWAM [ 50 ]
0.29
0.83
1.77
0.97
0.05
0.94
0.05
1.12
0.05
UNIVERSE [ 23 ]
0.30
0.75
1.44
0.83
0.07
0.91
0.08
0.37
0.01
DriveVA [ 24 ]
0.34
0.79
1.48
0.87
0.10
0.97
0.08
0.30
0.20
Table 4: NAVSIM-to-nuScenes transfer. Lower is better. L2 is in metres; Col. is the horizon-averaged collision percentage. The Adaptive-WAM row is publication-reported; the others are our evaluations on 4,519 samples (Appendix B ). Of the 4,437 samples with all horizons, 635 are waiting: the logged vehicle moves less than 1 m within 3 s. Bold: best result in each column.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
NAVTEST, NAVSIM-v1
NAVTEST
NAVHARD
Method
Ref.
Cam.
NC
DAC
TTC
Comf.
EP
PDMS
EPDMS
EPDMS
Reference planners without a world model
Human driver [ 7 ]
–
–
100.0
100.0
100.0
99.9
87.5
94.8
–
–
DiffusionDrive [ 20 ]
CVPR’25
3+L
98.2
96.2
94.7
100.0
82.2
88.1
–
–
ReCogDrive † [ 17 ]
ICLR’26
1
97.9
97.3
94.9
100.0
87.3
90.8
–
–
DrivoR ‡ [ 15 ]
CVPR’26
4
98.9
98.3
96.2
100.0
89.1
93.1
–
48.3
Appendix
Table 5: All compared methods on NAVTEST and NAVHARD, with the columns and notes of Table 2 . ∘ : concurrent work. † : reward or preference post-training; the DriveReferee row is its distilled single-pass policy. ‡ : proposal scoring. Bold: best result in each column, not counting the human driver and proposal-scoring rows.
Hist.
Latency (ms)
Mem.
Method
(fr.)
Future
Med.
P90
(GB)
PDMS
DiffusionDrive †
1
none
40.7
52.5
0.3
88.1
SimWAM
1
dropped
499.1
933.9
13.8
90.3
Adaptive-WAM ‡
1
early exit
170
–
–
90.8
UNIVERSE
5
dropped
256.9
327.2
24.4
91.0
UNIVERSE
5
3 steps
341.8
350.9
24.4
–
Appendix
Table 6: Per-decision latency, peak GPU memory and NAVTEST PDMS at batch size one on an H200. Hist.: camera frames used per decision, which AR-WAM reads from its cache. Future: future-video computation at inference. Baseline PDMS values are publication-reported. Bold: best result in each column. † Three cameras and LiDAR. ‡ Publication-reported mean on an A100.
Pretrained foundation models have become an important basis for end-to-end autonomous driving. In contrast to vision-language models pretrained primarily on static image-text pairs, video generative models capture temporal dynamics and motion priors that are naturally suited for driving. We present DriveWAM, a driving world-action model that adapts a pretrained video diffusion transformer into an autoregressive video-action policy. DriveWAM organizes video and action streams into a unified temporal token sequence and trains them under a joint flow-matching objective, preserving the pretrained video-generation architecture while adapting its large-scale video priors to action generation. To incorporate high-level scene understanding, we introduce scene-evolving driving guidance, where a frozen VLM produces chunk-specific semantic intent to guide video-action generation. To keep long-horizon rollout bounded, we further introduce selective KV memory, which maintains bounded modality-aware video and action memory pools through relevance-redundancy cache selection at inference time. Experiments on NAVSIM and the PhysicalAI-Autonomous-Vehicles benchmark show that DriveWAM achieves strong planning performance, and a data-scaling study from 4k to 100k driving clips further confirms the scaling potential of world-action modeling for end-to-end autonomous driving.
Chen Shi, Jinrui Xu, Shaoshuai Shi +3
The Chinese University of Hong Kong, Shenzhen · Voyager Research, Didi Chuxing
In autonomous driving, World-Action Models (WAMs) have improved end-to-end planning by transferring video dynamics priors to action prediction, but many still couple planning with future-video generation at inference, incurring substantial computational overhead. We present SimWAM, a simple yet effective WAM that leverages future-video prediction solely as a training-time supervision signal. It co-trains a pretrained video expert and a lightweight action expert with joint flow matching. An isolated attention mask keeps action prediction independent of future frames, allowing trajectory prediction without future-frame generation at inference. This design supports multiple pretrained video backbones and independent action-expert scaling within a shared attention interface, while preserving the joint learning objective. Moreover, we apply reinforcement learning to optimize a compositional driving reward beyond trajectory imitation. Experiments show that SimWAM achieves 91.9 PDMS on NAVSIM with a favorable trade-off between accuracy and latency among world-model-based planners, while transferring zero-shot to nuScenes. It also achieves competitive planning accuracy on WOD-E2E and PhysicalAI-Autonomous-Vehicles. These results position SimWAM as a plain yet solid baseline for efficient autonomous driving. The code and model weights are available at https://github.com/H-EmbodVis/SimWAM/.
Zongchuang Zhao, Xin Zhou, Tianyang Xu +6
Huazhong University of Science & Technology · Dongfeng Research & Development Institute
Direct-policy World Action Models (WAM) have demonstrated the benefits of decoupling action generation from explicit future visual prediction for efficient inference in autonomous driving and embodied control. However, they still require tens of denoising steps to achieve competitive action quality, incurring substantial latency and hindering real-time deployment. Simply reducing the number of denoising steps will collapse the nonlinear multi-step denoising trajectory into coarse updates, substantially degrading action quality. To reduce the denoising steps while enabling real-time action generation, we present an effective world action model with one-step action generation, namely OSWAM, for autonomous driving and embodied control. We argue that it is the absence of structured action priors in random noise initialization that forces direct-policy WAMs to recover coherent action sequences through iterative denoising. Hence, we construct imagination tokens to initialize the diffusion process, providing physical priors and action-conditioned representations. We further introduce privileged information distillation with future conditioning to enhance future-aware action generation. As a result, OSWAM can efficiently and effectively generate high quality actions in just one diffusion step. Our experiments demonstrate that OSWAM achieves comparable or even better results in both autonomous driving and embodied control than direct-policy WAM and Joint-WAM that require tens of steps. It obtains approximately 3× speedup over direct-policy WAM baselines and supports real-time compiled inference with latency as low as 40ms on an NVIDIA RTX 5090 GPU. The project page is available at https://gwenzhang.github.io/OSWAM/