Organizations: Nanyang Technological University, Singapore · Tongji University, Shanghai, China · Agency for Science, Technology and Research (A*STAR), Singapore
Occupancy world models provide a structured, metric representation of future driving scenes, making them a promising basis for end-to-end autonomous driving. However, existing occupancy-aware planners typically couple occupancy prediction and action generation through either hierarchical coupling or parallel multi-task learning. Hierarchical methods use occupancy either as a fully predicted future preceding action generation or as an action-conditioned rollout for trajectory evaluation, whereas parallel multi-task methods predict occupancy alongside planning through a separate branch or auxiliary objective. These interfaces retain either an explicit generation order or a role asymmetry, limiting reciprocal interaction between world prediction and action generation during trajectory formation. We propose OccWAM, an occupancy-native world-action model that jointly generates dense future occupancy and continuous ego trajectories via flow matching conditioned on scene representations. Building on this joint formulation, we introduce scene-to-ego world-action flow matching, a two-stage formulation. The first stage predicts scene-centric occupancy together with an absolute ego trajectory, providing a stable representation of future scene evolution. The same predicted world is then re-indexed into the future ego frames induced by the initial trajectory, where ego motion is represented through adjacent pose increments and a second joint flow refines ego-centric occupancy and motion for action-aligned planning. Unlike video-based world-action models, OccWAM preserves explicit semantic geometry for direct planning reasoning without pixel synthesis. Extensive experiments on Bench2Drive and nuScenes demonstrate state-of-the-art performance in closed-loop driving, open-loop planning, and future occupancy prediction.
Figures & tables
Figure 1: Comparison of occupancy–action interfaces. Hierarchical methods impose an action-first or world-first order, while parallel methods retain separate prediction pathways. OccWAM instead jointly generates occupancy and action through coupled conditional flows. Multi M. Traj : multimodal trajectory candidates.
Figure 2: Overall architecture of OccWAM. A frozen Understanding Expert provides semantic driving context, while the World and Action Experts interact through layer-wise joint attention to parameterize a coupled occupancy–action vector field. Stage I jointly generates scene-centric occupancy and an absolute ego trajectory from Gaussian sources. The predicted trajectory then defines future ego frames for warping the occupancy and is converted into ego-motion increments; together, they form the source of Stage II. Using the same three-expert backbone, Stage II applies a second joint flow in ego-centric coordinates to produce the final occupancy and trajectory.
Method
Venue
Expert
WM
DS ↑
SR (%) ↑
UniAD-Base [ 1 ]
CVPR’23
Think2Drive
×
45.81
16.36
DiffusionDrive [ 3 ]
CVPR’25
PDM-Lite
×
77.68
57.72
ORION [ 28 ]
ICCV’25
Think2Drive
×
77.74
54.62
AutoVLA [ 29 ]
NeurIPS’25
PDM-Lite
×
78.84
57.73
TransFuser++ [ 30 ]
ICCV’23
PDM-Lite
×
84.21
67.27
SimLingo [ 31 ]
CVPR’25
PDM-Lite
×
85.07
67.27
Table 1: Closed-loop driving performance on Bench2Drive. Expert denotes the expert policy used for collecting training data, while WM denotes an explicit learned future-world model. Best and second-best results are shown in bold and underlined, respectively.
L2 (m) ↓
Collision (%) ↓
Method
Ego status
1s
2s
3s
Avg.
1s
2s
3s
Avg.
UniAD [ 1 ]
Vector
0.44
0.67
0.96
0.69
0.04
0.08
0.23
0.12
VAD [ 2 ]
Vector
0.17
0.34
0.60
0.37
0.07
0.10
0.24
0.14
DriveTransformer-Large [ 36 ]
Vector
0.16
0.30
0.55
0.33
0.01
0.06
0.15
0.07
AutoVLA [ 29 ]
Text
0.21
0.38
0.60
0.40
0.13
0.18
0.28
0.20
OmniDrive [ 37 ]
Vector
0.14
0.29
0.55
0.33
0.00
0.13
0.78
0.30
Table 2: Open-loop planning and future-occupancy evaluation on nuScenes.
Control output
DS ↑
SR (%) ↑
Stage-I trajectory
88.21
73.63
Stage-II trajectory
90.45
77.27
Table 3: Ablation of the second joint flow and World-to-Action information flow.
Configuration
L2 ↓
Coll. ↓
mIoU ↑
IoU ↑
Without Action-to-World
0.32
0.10
23.11
32.59
Full OccWAM
0.30
0.07
24.23
33.88
Table 4: Complementary ablations on nuScenes. Planning metrics (L2 and collision) use camera observations, whereas occupancy metrics (mIoU and IoU) use four historical 3D occupancy frames. Comparisons within each metric group follow the same task-specific input and evaluation protocol.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1: Qualitative comparison of future 4D occupancy predictions and ground truth across future time steps. Each frame shows the predicted 3D semantic occupancy of OccWAM and its corresponding ground truth.
Setting
Bench2Drive
nuScenes
Planning input
Front-view RGB image and one LiDAR BEV representation
Camera observations
Occupancy-prediction input
The same front-view RGB and LiDAR BEV observations; no historical or current ground-truth occupancy is used
Four historical 3D occupancy frames
Predicted occupancy
Temporal BEV occupancy over (t,x,y) , without an explicit vertical dimension
Future volumetric occupancy over (t,x,y,z)
Temporal configuration
6 future steps at 2 Hz
4 historical and 6 future occupancy frames for occupancy prediction
Spatial discretization
64m×64m ego-centered BEV; 1m per cell
[−40,40]×[−40,40]×[−1,5.4]m ; 0.4m voxels
Semantic output
64×64 grid with 11 categorical classes, including free space
200×200×16 grid with 17 semantic classes plus free space
Appendix
Table S1: Benchmark-specific model and training configurations. The planning and occupancy-prediction rows distinguish the task-specific nuScenes input protocols.
World models for autonomous driving should anticipate the consequences that matter for driving decisions rather than reconstruct every detail of future scenes. However, predictive representations alone are not guaranteed to distinguish candidate actions that lead to different planning outcomes.We propose DA-WAM, a decision-aligned world action model that couples a Joint-Embedding Predictive Architecture (JEPA) with candidate-level trajectory scoring. During joint training, the JEPA objective grounds future representations in observed scene dynamics, while the planning objectives encourage these representations to capture distinctions relevant to safety and utility. An action-conditioned predictor generates a distinct future latent for each candidate trajectory, and a multi-head trajectory scorer uses the corresponding latent to estimate planning factors and overall utility. In addition, safety-critical post-training with hard negatives further improves discrimination between geometrically similar trajectories with different safety outcomes. DA-WAM achieves state-of-the-art performance among learned camera-based planners on NAVSIM-v1, NAVSIM-v2, and Bench2Drive. Ablation studies demonstrate the effectiveness of each component in driving decision-making.
Ruiguo Zhong, Xiaolong Chen, Benshan Ma +5
The Hong Kong University of Science and Technology (Guangzhou) · Leapmotor · The Hong Kong University of Science and Technology
World models in end-to-end autonomous driving predict future scene evolution to provide foresight for trajectory planning. Existing methods mainly study how to predict the future and how to use it, but less often ask which future representation is actually most useful for planning. To this end, we propose PlanWAM, a Planning-Shaped World Action Model. The key idea is to let the planning task shape the future-state representation, so that it retains the information most useful for planning. A latent world model then predicts this planning-shaped future latent representation from historical observations and uses it for planning, enabling foresighted planning. Specifically, we first use a Temporal Register Pyramid to compress multi-frame historical information in a recency-aware manner, learning a compact history representation oriented toward future reasoning and planning. We then introduce a privileged future posterior branch that observes ground-truth future frames, and shape its future latent representation with trajectory-planning objectives to obtain a planning-shaped future latent representation. Hindsight-to-Foresight Distillation trains a prior branch that depends only on history to predict this future latent representation. The predicted future latent representation serves as planning context and guides trajectory generation and selection. PlanWAM achieves 93.8 PDMS / 90.9 EPDMS on NAVSIM-v1/v2 navtest and reaches 38.7 HD-Score on closed-loop HUGSIM in a zero-shot setting, demonstrating leading planning performance across both open-loop and closed-loop evaluations. Extensive experiments further demonstrate that planning-shaped future representations provide an effective and deployable form of foresight for world-action models.
Jinchang Xu, Hongda Yu, Fengwei Dong +9
Southeast University · NIO · Nanyang Technological University +1
Future 3D semantic occupancy forecasting and motion planning are central to autonomous driving, as they require models to reason about how surrounding scenes evolve and how the ego vehicle should act. Existing occupancy world models commonly discretize scenes into latent embeddings, volumetric features, or quantized tokens, and forecast future states through fixed-step autoregressive generation. This limits temporal flexibility, obscures scene evolution, accumulates errors over long horizons, and poorly matches the continuous-time dynamics of real driving scenes. We propose GEM, a Gaussian Evolution Model for non-autoregressive occupancy world modeling, where driving scenes are represented as explicit continuous 4D Gaussian primitives with learned dynamics. Instead of rolling out future occupancy states step by step, GEM directly queries the Gaussian world representation at arbitrary timestamps and splats the corresponding conditional 3D Gaussians into semantic occupancy volumes. This enables efficient forecasting over the full horizon while retaining a compact and interpretable scene representation. By decoupling spatial geometry, temporal support, and primitive motion, GEM makes the predicted world easier to inspect, as each primitive's evolution can be followed continuously over time. The same representation also supports motion planning by predicting future ego trajectories from the learned Gaussian world. Extensive experiments show that GEM achieves state-of-the-art future semantic occupancy forecasting and strong motion planning performance, while providing flexible temporal querying.