4D occupancy forecasting models the spatio-temporal evolution of 3D scenes and is crucial for autonomous driving, especially for corner-case simulation. Existing methods often rely on discrete tokenization followed by autoregressive prediction, yet struggle with geometric distortion in static structures and inconsistent temporal coherence over the forecasting horizon. In this work, we propose a Geometry-Aware Spatio-Temporal context modeling method (GAST) for 4D occupancy forecasting, built upon progressive explicit-implicit generation and dual-path spatio-temporal modeling. Specifically, the generation module produces per-frame occupancy with high geometric fidelity and semantic plausibility through pose-driven warping, motion-aware feature modulation, and attention-based feature refinement. Subsequently, the spatio-temporal module enhances spatial consistency through global context aggregation while capturing scene evolution through temporal dynamics extraction. This unified design enables joint optimization of historical reconstruction and future forecasting in an end-to-end manner. Extensive experiments on Occ3D-nuScenes demonstrate the superiority of our method, outperforming the state-of-the-art by 7.67% in mIoU and 6.44% in IoU with a 2.84x speedup, while maintaining strong performance in long-term forecasting.
Figures & tables
Figure 1: Comparison with the existing paradigm [ 45 , 16 ] . It adopts a two-stage pipeline that encodes occupancy into discrete tokens and autoregressively generates future tokens. In contrast, our end-to-end framework directly models spatio-temporal representations in continuous BEV space via per-frame generation and cross-frame modeling.
Figure 2: General architecture of GAST. We first encode historical occupancy into BEV features and predict future ego-poses. The generation module outputs per-frame future features via geometric warping, feature modulation, and deformable cross-attention. The spatio-temporal module enhances global consistency through spatial context aggregation while modeling scene evolution via temporal extraction. Finally, an occupancy decoder jointly reconstructs past observations and forecasts future occupancy.
Method
Input
Ego traj.
mIoU (%) ↑
IoU (%) ↑
1s
2s
3s
Avg.
1s
2s
3s
Avg.
OccWorld-D [ 45 ]
Camera
Pred.
11.55
8.10
6.22
8.62
18.90
16.26
14.43
16.53
OccWorld-T [ 45 ]
Camera
Pred.
4.68
3.36
2.63
3.56
9.32
8.23
7.47
8.34
OccWorld-S [ 45 ]
Camera
Pred.
0.28
0.26
0.24
0.26
5.05
5.01
4.95
5.00
OccWorld-F [ 45 ]
Camera
Pred.
8.03
6.91
3.54
6.16
23.62
18.13
15.22
18.99
RenderWorld [ 38 ]
Camera
Pred.
2.83
2.55
2.37
2.58
14.61
13.61
12.98
13.73
Table 1: 4D occupancy forecasting performance on Occ3D-nuScenes validation set. The settings vary by input modality and whether the ego trajectory is predicted (Pred.) or provided as ground truth (GT). “Avg." denotes the average performance across 1s, 2s, and 3s horizons. The bold numbers indicate the best results.
Method
mIoU (%) ↑
1s
2s
3s
4s
5s
6s
7s
8s
Avg.
DOME [ 7 ]
30.10
21.35
17.36
14.86
12.61
11.03
10.00
9.34
15.83
COME [ 23 ]
33.78
24.57
21.35
18.25
15.84
13.85
12.99
11.96
19.07
GAST (Ours)
36.23
30.61
27.13
24.36
21.98
19.81
17.83
15.96
24.12
Method
IoU (%) ↑
1s
2s
3s
4s
5s
6s
7s
8s
Avg.
Table 2: Long-term 4D occupancy forecasting performance on Occ3D-nuScenes validation set. Ground-truth 3D occupancy and trajectories are used as inputs. “Avg.” denotes the average performance over the 1-8 second prediction horizon. The bold numbers indicate the best results.
Table 5
Figure 3: Qualitative results of GAST on Occ3D-nuScenes.
Camera-only 4D occupancy forecasting enables autonomous vehicles to predict future 3D semantic scenes solely from historical multi-view images, which is critical for driving safety. Even though current methods have achieved good performance, the strong spatial-temporal modeling between the input multi-view frames is still underexplored, which limits the performance of those methods in future 4D forecasting. To address this gap, we introduce a novel framework, InterOCF, for 4D occupancy forecasting that jointly models temporal dynamics in both 3D voxel-based representations and multi-view segmentation sequences, while explicitly incorporating feature interaction between the 2D and 3D branches. Our framework incorporates three core components: 1) A 3D Spatio-Temporal (3DST) module that learns volumetric dynamics from historical voxel states to predict future voxel states; 2) A 2D Spatio-Temporal (2DST) module employing an auxiliary multi-view temporal segmentation forecasting task to enhance temporal semantic dynamics; 3) A Spatio-Temporal Interaction Modeling (STIM) module that enables feature interaction between 2D and 3D representations. Experiments on the nuScenes, Lyft-Level5, and nuScenes-Occupancy datasets show that InterOCF consistently outperforms existing baseline approaches.
Qi Zhang, Xinquan Yu, Kaiyi Zhang +1
College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China.
Roadside 4D occupancy forecasting predicts future semantic occupancy in a fixed infrastructure coordinate system. Roadside scenes are globally structured yet locally evolving, requiring comprehensive scene-level context while precisely modeling sparse localized motion. However, existing forecasting paradigms struggle to tackle both requirements. To address this limitation, we propose InfraWorld, a world model formulating roadside occupancy forecasting as selective state evolution. InfraWorld maintains a dense BEV state for the complete scene and persistent motion queries for moving regions, assigning complementary states to global context and local dynamics. Specifically, we propose Dynamic Query Initiation (DQI) to initialize queries in regions supported by motion in the observed history. During rollout, the dense state is updated through Long--Short Temporal Fusion (LST), which combines short-term adjacent-frame continuity with long-term motion trends to preserve stable context while capturing scene evolution. Motion-Aware Query Injection (MAQI) then refines the queries and injects confidence-weighted query information into corresponding BEV tokens, enabling targeted dynamic updates without perturbing global context. Together, InfraWorld concentrates predictive capacity on evolving traffic regions. Extensive experiments on the InfraOcc and Occ3D--nus datasets demonstrate that InfraWorld achieves state-of-the-art performance for roadside 4D occupancy forecasting while retaining its effectiveness under ego-centric settings. Code will be released.
Songkai Wang, Xiaokai Bai, Lei Yang +1
College of Information Science and Electronic Engineering, Zhejiang University. · School of Mechanical and Aerospace Engineering, Nanyang Technological University.
Future 3D semantic occupancy forecasting and motion planning are central to autonomous driving, as they require models to reason about how surrounding scenes evolve and how the ego vehicle should act. Existing occupancy world models commonly discretize scenes into latent embeddings, volumetric features, or quantized tokens, and forecast future states through fixed-step autoregressive generation. This limits temporal flexibility, obscures scene evolution, accumulates errors over long horizons, and poorly matches the continuous-time dynamics of real driving scenes. We propose GEM, a Gaussian Evolution Model for non-autoregressive occupancy world modeling, where driving scenes are represented as explicit continuous 4D Gaussian primitives with learned dynamics. Instead of rolling out future occupancy states step by step, GEM directly queries the Gaussian world representation at arbitrary timestamps and splats the corresponding conditional 3D Gaussians into semantic occupancy volumes. This enables efficient forecasting over the full horizon while retaining a compact and interpretable scene representation. By decoupling spatial geometry, temporal support, and primitive motion, GEM makes the predicted world easier to inspect, as each primitive's evolution can be followed continuously over time. The same representation also supports motion planning by predicting future ego trajectories from the learned Gaussian world. Extensive experiments show that GEM achieves state-of-the-art future semantic occupancy forecasting and strong motion planning performance, while providing flexible temporal querying.