Organizations: School of Intelligent Systems Engineering, Shenzhen Campus of Sun Yat-sen University, Shenzhen 518107, China. · Peng Cheng Laboratory, Shenzhen 518108, China. · School of Electronic and Computer Engineering, Shenzhen Graduate School, Peking University, 518066, China. · School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, 518055, China. · School of Computer Science, Peking University, Beijing 100871, China.
Occupancy World Models (OWMs) aim to predict future 3D occupancy scenes from historical observations and future ego motions, providing a useful world simulation tool for autonomous driving. Existing methods usually rely on large networks to implicitly align multi-frame historical states and predict future occupancy in a full-state manner, which is costly and redundant because most scene regions remain unchanged over short time intervals. In this paper, we propose Aligned Delta-Triplane Transformer (ADTT), a compact 4D OWM that explicitly aligns historical triplanes with ego-motion compensation and predicts only future scene changes. Based on the aligned triplane prior, a query-conditioned Transformer predicts residual changes, where ego-motion queries guide controllable future motion and learnable external queries capture scene changes beyond ego motion. The predicted changes are added to the aligned prior and decoded into future occupancy scenes. Experiments on Occ3D-nus show that ADTT achieves state-of-the-art forecasting performance with fewer parameters and higher FPS. Our code is publicly available online: https://github.com/kyoran/ADTT.
Figures & tables
Figure 1: (a-b) Comparison between the full occupancy and occupancy changes ( Δ ) of a scene. In the change map, red indicates newly appeared voxels, while blue denotes disappeared voxels. (c) Distribution of latent-space feature values for full occupancy versus occupancy changes.
Figure 2: ADTT follows a simple yet effective procedure: it first represents occupancy scenes with pretrained triplanes, then explicitly aligns historical triplanes through historical ego-motion compensation, and finally employs a Transformer to predict triplane changes ( Δ ) based on the aligned triplane features, future ego-motion query, and external learnable query.
Figure 3: ADTT performs motion compensation and tokenization in the triplane latent space. The Transformer encoder then encodes historical context from triplane tokens, while the decoder predicts next-frame triplane changes using deterministic ego-motion queries and learnable external queries. These predicted changes are added to the latest triplane and finally decoded into future occupancy states.
Figure 4: Pretrained triplane for OWM.
Figure 5: Motion compensation in the triplane latent space.
Models
Input
mIoU (%) ↑
IoU (%) ↑
FPS ↑
Params ↓
1s
2s
3s
Avg.
1s
2s
3s
Avg.
OccWorld [ 47 ]
Occ
25.78
15.14
10.51
17.14
34.63
25.07
20.18
26.63
18
72M
OccSora [ 34 ]
Occ
32.77
22.04
14.40
23.07
41.39
33.68
29.97
35.01
20
174M
OccLLaMA [ 37 ]
Occ&Text
25.05
19.49
15.26
19.93
34.56
28.53
24.41
29.17
-
> 7B
OccLLM [ 43 ]
Occ&Text
24.02
21.65
17.29
20.99
36.65
32.14
28.77
32.52
-
> 7B
DOME [ 8 ]
Occ
35.11
25.89
20.29
27.10
43.99
35.36
29.74
36.36
6.54
397M
Table 1: Comparison on Occ3D-nus 4D occupancy forecasting at 1s, 2s, and 3s. The best and second-best results are shown in bold and underlined , respectively.
Table 7
Figure 6: Occupancy forecasting comparison on the Occ3D-nus val set (zoom in for details).
Table 9Figure 10
Figure 8: Controllable future occupancy prediction under different ego-motion inputs.
Occupancy world models provide a structured, metric representation of future driving scenes, making them a promising basis for end-to-end autonomous driving. However, existing occupancy-aware planners typically couple occupancy prediction and action generation through either hierarchical coupling or parallel multi-task learning. Hierarchical methods use occupancy either as a fully predicted future preceding action generation or as an action-conditioned rollout for trajectory evaluation, whereas parallel multi-task methods predict occupancy alongside planning through a separate branch or auxiliary objective. These interfaces retain either an explicit generation order or a role asymmetry, limiting reciprocal interaction between world prediction and action generation during trajectory formation. We propose OccWAM, an occupancy-native world-action model that jointly generates dense future occupancy and continuous ego trajectories via flow matching conditioned on scene representations. Building on this joint formulation, we introduce scene-to-ego world-action flow matching, a two-stage formulation. The first stage predicts scene-centric occupancy together with an absolute ego trajectory, providing a stable representation of future scene evolution. The same predicted world is then re-indexed into the future ego frames induced by the initial trajectory, where ego motion is represented through adjacent pose increments and a second joint flow refines ego-centric occupancy and motion for action-aligned planning. Unlike video-based world-action models, OccWAM preserves explicit semantic geometry for direct planning reasoning without pixel synthesis. Extensive experiments on Bench2Drive and nuScenes demonstrate state-of-the-art performance in closed-loop driving, open-loop planning, and future occupancy prediction.
Qihang Huang, Wenhui Huang, Collister Chua +6
Nanyang Technological University, Singapore · Tongji University, Shanghai, China · Agency for Science, Technology and Research (A*STAR), Singapore
This letter proposes CascadeOcc, a novel occupancy world model that prioritizes intrinsic structural hierarchy over extrinsic auxiliary modalities for autonomous driving. Occupancy world models -- forecasting the future driving environment and planning the driving trajectory -- effectively bridge perception and planning, but current approaches often heavily rely on external modalities or large language models, failing to fully exploit the inherent structural potential of occupancy representations themselves. To enhance representational capacity for complex 3D scenes, we integrate a cascaded Vector Quantized (VQ) mechanism into an autoregressive framework. Following a coarse-to-fine principle, CascadeOcc progressively refines fine-grained details from global structures through a multi-scale architecture. Additionally, we incorporate a TimeMixer to capture multi-scale temporal dependencies, establishing a dual-hierarchy mechanism in both space and time. Experimental results on 4D occupancy forecasting and motion planning benchmarks demonstrate that CascadeOcc achieves superior performance among vision-centric approaches, validating that optimizing inherent representations is a powerful alternative to relying on external foundation models.
Kyumin Hwang, Wonhyeok Choi, Jaeyeul Kim +3
Daegu Gyeongbuk Institute of Science and Technology (DGIST), Daegu 42988, South Korea
Future 3D semantic occupancy forecasting and motion planning are central to autonomous driving, as they require models to reason about how surrounding scenes evolve and how the ego vehicle should act. Existing occupancy world models commonly discretize scenes into latent embeddings, volumetric features, or quantized tokens, and forecast future states through fixed-step autoregressive generation. This limits temporal flexibility, obscures scene evolution, accumulates errors over long horizons, and poorly matches the continuous-time dynamics of real driving scenes. We propose GEM, a Gaussian Evolution Model for non-autoregressive occupancy world modeling, where driving scenes are represented as explicit continuous 4D Gaussian primitives with learned dynamics. Instead of rolling out future occupancy states step by step, GEM directly queries the Gaussian world representation at arbitrary timestamps and splats the corresponding conditional 3D Gaussians into semantic occupancy volumes. This enables efficient forecasting over the full horizon while retaining a compact and interpretable scene representation. By decoupling spatial geometry, temporal support, and primitive motion, GEM makes the predicted world easier to inspect, as each primitive's evolution can be followed continuously over time. The same representation also supports motion planning by predicting future ego trajectories from the learned Gaussian world. Extensive experiments show that GEM achieves state-of-the-art future semantic occupancy forecasting and strong motion planning performance, while providing flexible temporal querying.