Aligned Delta-Triplane Transformers as Occupancy World Models

Haoran Xu; Peixi Peng; Shijun Guo; Guang Tan; Yiqian Chang; Yisen Zhao; Shuaixian Wang; Yonghong Tian

cs.ROMar 10, 2025

Aligned Delta-Triplane Transformers as Occupancy World Models

Authors: Haoran Xu, Peixi Peng, Shijun Guo, Guang Tan, Yiqian Chang, Yisen Zhao, Shuaixian Wang, Yonghong Tian

Organizations: School of Intelligent Systems Engineering, Shenzhen Campus of Sun Yat-sen University, Shenzhen 518107, China. · Peng Cheng Laboratory, Shenzhen 518108, China. · School of Electronic and Computer Engineering, Shenzhen Graduate School, Peking University, 518066, China. · School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, 518055, China. ·

Abstract

Occupancy World Models (OWMs) aim to predict future 3D occupancy scenes from historical observations and future ego motions, providing a useful world simulation tool for autonomous driving. Existing methods usually rely on large networks to implicitly align multi-frame historical states and predict future occupancy in a full-state manner, which is costly and redundant because most scene regions remain unchanged over short time intervals. In this paper, we propose Aligned Delta-Triplane Transformer (ADTT), a compact 4D OWM that explicitly aligns historical triplanes with ego-motion compensation and predicts only future scene changes. Based on the aligned triplane prior, a query-conditioned Transformer predicts residual changes, where ego-motion queries guide controllable future motion and learnable external queries capture scene changes beyond ego motion. The predicted changes are added to the aligned prior and decoded into future occupancy scenes. Experiments on Occ3D-nus show that ADTT achieves state-of-the-art forecasting performance with fewer parameters and higher FPS. Our code is publicly available online: https://github.com/kyoran/ADTT.

Figures & tables

Figure 1: (a-b) Comparison between the full occupancy and occupancy changes ( $\Delta$ ) of a scene. In the change map, red indicates newly appeared voxels, while blue denotes disappeared voxels. (c) Distribution of latent-space feature values for full occupancy versus occupancy changes.

Figure 2: ADTT follows a simple yet effective procedure: it first represents occupancy scenes with pretrained triplanes, then explicitly aligns historical triplanes through historical ego-motion compensation, and finally employs a Transformer to predict triplane changes ( $\Delta$ ) based on the aligned triplane features, future ego-motion query, and external learnable query.

Figure 3: ADTT performs motion compensation and tokenization in the triplane latent space. The Transformer encoder then encodes historical context from triplane tokens, while the decoder predicts next-frame triplane changes using deterministic ego-motion queries and learnable external queries. These predicted changes are added to the latest triplane and finally decoded into future occupancy states.

Figure 5: Motion compensation in the triplane latent space.

Models	Input	mIoU (%) $\uparrow$				IoU (%) $\uparrow$				FPS $\uparrow$	Params $\downarrow$
Models	Input	1s	2s	3s	Avg.	1s	2s	3s	Avg.	FPS $\uparrow$	Params $\downarrow$
OccWorld [ 47 ]	Occ	25.78	15.14	10.51	17.14	34.63	25.07	20.18	26.63	18	72M
OccSora [ 34 ]	Occ	32.77	22.04	14.40	23.07	41.39	33.68	29.97	35.01	20	174M
OccLLaMA [ 37 ]	Occ&Text	25.05	19.49	15.26	19.93	34.56	28.53	24.41	29.17	-	> 7B
OccLLM [ 43 ]	Occ&Text	24.02	21.65	17.29	20.99	36.65	32.14	28.77	32.52	-	> 7B
DOME [ 8 ]	Occ	35.11	25.89	20.29	27.10	43.99	35.36	29.74	36.36	6.54	397M

Table 1: Comparison on Occ3D-nus 4D occupancy forecasting at 1s, 2s, and 3s. The best and second-best results are shown in bold and underlined , respectively.

Table 7

Figure 6: Occupancy forecasting comparison on the Occ3D-nus val set (zoom in for details).

Table 9

Figure 8: Controllable future occupancy prediction under different ego-motion inputs.

3D Scene Prediction

See all 19 papers

OccWAM: Occupancy World-Action Models with Joint Scene-to-Ego Flow Matching
Oct 13, 2026Qihang Huang, Wenhui Huang, Collister Chua +6World Action Models World Models for Autonomous Driving
Nanyang Technological University, Singapore · Tongji University, Shanghai, China · Agency for Science, Technology and Research (A*STAR), Singapore
CascadeOcc: Rethinking 3D Occupancy World Models with Cascaded VQ Representations
Jun 26, 2026Kyumin Hwang, Wonhyeok Choi, Jaeyeul Kim +3World Model Learning 3D Occupancy Prediction
Daegu Gyeongbuk Institute of Science and Technology (DGIST), Daegu 42988, South Korea
GEM: Gaussian Evolution Model for Occupancy Forecasting and Motion Planning
May 17, 2026Cheng Chen, Hao Huang, Saurabh Bagchi3D Gaussian Splatting World Model Learning
Purdue University · New York University Abu Dhabi

Oct 13, 2026J/K move · Enter open · S save

OccWAM: Occupancy World-Action Models with Joint Scene-to-Ego Flow Matching

Qihang Huang, Wenhui Huang, Collister Chua, Shiyu Fang+5

Nanyang Technological University, Singapore · Tongji University, Shanghai, China · Agency for Science, Technology and Research (A*STAR), Singapore

World Action Models World Models for Autonomous Driving Autonomous Driving Planning 3D Occupancy Prediction Flow Matching

Open paper AskPDF arXiv

Occupancy world models provide a structured, metric representation of future driving scenes, making them a promising basis for end-to-end autonomous driving. However, existing occupancy-aware planners typically couple occupancy prediction and action generation through either hierarchical coupling or parallel multi-task learning. Hierarchical methods use occupancy either as a fully predicted future preceding action generation or as an action-conditioned rollout for trajectory evaluation, whereas parallel multi-task methods predict occupancy alongside planning through a separate branch or auxiliary objective. These interfaces retain either an explicit generation order or a role asymmetry, limiting reciprocal interaction between world prediction and action generation during trajectory formation. We propose OccWAM, an occupancy-native world-action model that jointly generates dense future occupancy and continuous ego trajectories via flow matching conditioned on scene representations. Building on this joint formulation, we introduce scene-to-ego world-action flow matching, a two-stage formulation. The first stage predicts scene-centric occupancy together with an absolute ego trajectory, providing a stable representation of future scene evolution. The same predicted world is then re-indexed into the future ego frames induced by the initial trajectory, where ego motion is represented through adjacent pose increments and a second joint flow refines ego-centric occupancy and motion for action-aligned planning. Unlike video-based world-action models, OccWAM preserves explicit semantic geometry for direct planning reasoning without pixel synthesis. Extensive experiments on Bench2Drive and nuScenes demonstrate state-of-the-art performance in closed-loop driving, open-loop planning, and future occupancy prediction.

Abstract

3D Scene Prediction

Explore similar work

OccWAM: Occupancy World-Action Models with Joint Scene-to-Ego Flow Matching

CascadeOcc: Rethinking 3D Occupancy World Models with Cascaded VQ Representations

GEM: Gaussian Evolution Model for Occupancy Forecasting and Motion Planning