InfraWorld: Motion-Aware World Modeling for Roadside 4D Occupancy Forecasting
Authors: Songkai Wang, Xiaokai Bai, Lei Yang, Hui-liang Shen
Organizations: College of Information Science and Electronic Engineering, Zhejiang University. · School of Mechanical and Aerospace Engineering, Nanyang Technological University.
Roadside 4D occupancy forecasting predicts future semantic occupancy in a fixed infrastructure coordinate system. Roadside scenes are globally structured yet locally evolving, requiring comprehensive scene-level context while precisely modeling sparse localized motion. However, existing forecasting paradigms struggle to tackle both requirements. To address this limitation, we propose InfraWorld, a world model formulating roadside occupancy forecasting as selective state evolution. InfraWorld maintains a dense BEV state for the complete scene and persistent motion queries for moving regions, assigning complementary states to global context and local dynamics. Specifically, we propose Dynamic Query Initiation (DQI) to initialize queries in regions supported by motion in the observed history. During rollout, the dense state is updated through Long--Short Temporal Fusion (LST), which combines short-term adjacent-frame continuity with long-term motion trends to preserve stable context while capturing scene evolution. Motion-Aware Query Injection (MAQI) then refines the queries and injects confidence-weighted query information into corresponding BEV tokens, enabling targeted dynamic updates without perturbing global context. Together, InfraWorld concentrates predictive capacity on evolving traffic regions. Extensive experiments on the InfraOcc and Occ3D--nus datasets demonstrate that InfraWorld achieves state-of-the-art performance for roadside 4D occupancy forecasting while retaining its effectiveness under ego-centric settings. Code will be released.
Figures & tables
Figure 1: (a) Voxel distribution in InfraOcc: static scene (SS), moving participants (MP), and stationary participants (SP). (b) InfraWorld maintains the stable scene layout with a dense BEV state, while queries reason about localized dynamics and inject their updates into the scene context.
Figure 2: Sparse and localized dynamics in roadside occupancy. Across five consecutive frames, the static scene layout remains largely unchanged, while only a few moving traffic participants exhibit temporal changes.
Figure 3: Dynamic Query Initiation (DQI). Top: DQI process for selecting and localizing motion-supported queries. Bottom: query evolution during rollout, where gray regions denote GT moving objects and highlighted trajectories show how queries capture localized motion evolution.
Figure 4: Architecture overview of InfraWorld. A frozen tokenizer encodes the observed occupancy into dense BEV latents, while DQI initializes persistent queries in predicted moving regions. At each autoregressive step, LST fuses spatial and temporal context to update the dense state and refine the queries, after which MAQI injects reliable query features into local BEV regions. The updated dense latent and queries are then propagated to the next forecasting step.
Figure 5: Motion-Aware Query Injection (MAQI). MAQI injects confidence-weighted query features into neighboring BEV tokens through distance-biased cross-attention.
Method
Input
mIoU dyn (%) ↑
mIoU sta (%) ↑
1 s
2 s
3 s
Avg.
1 s
2 s
3 s
Avg.
OccWorld-O (ECCV 2024)
3D-Occ
25.64
21.24
19.76
22.21
97.52
97.42
97.37
97.44
I 2 -World-O (ICCV 2025)
3D-Occ
33.34
29.67
28.66
30.56
97.30
97.27
97.07
97.21
InfraWorld-O (Ours)
3D-Occ
46.03
38.08
34.63
39.58
97.79
97.72
97.68
97.73
OccWorld-STC (ECCV 2024)
Camera
16.20
14.29
13.69
14.73
78.29
78.27
78.24
78.27
I 2 -World-STC (ICCV 2025)
Camera
19.47
16.95
15.73
17.38
78.71
78.70
78.69
78.70
Table 1: 4D occupancy forecasting performance on InfraOcc validation set.
Method
Input
mIoU (%) ↑
1 s
2 s
3 s
Avg.
OccWorld-O
Occ
25.78
15.14
10.51
17.14
OccLLaMA-O
Occ & Text
25.05
19.49
15.26
19.93
DFIT-OccWorld
Occ & Camera
31.68
21.29
15.19
22.71
DOME
Occ
35.11
25.89
20.29
27.10
UniScene
Occ
35.37
29.59
25.08
31.76
Table 2: Forecasting performance on Occ3D–nus.
Figure 6: Qualitative comparison between InfraWorld and I 2 -World on InfraOcc. Yellow boxes highlight dynamic regions where InfraWorld more closely matches the ground truth.
LST
MAQI
DQI
mIoU dyn (%) ↑
1 s
2 s
3 s
Avg.
–
–
–
33.68
32.54
26.58
30.93
✓
–
–
35.20
32.95
30.46
32.87
–
✓
–
36.81
34.33
31.91
34.35
–
✓
✓
42.76
34.87
32.62
36.75
✓
✓
✓
46.03
38.08
34.63
39.58
Table 3: Component-wise ablation of InfraWorld.
Figure 7: Qualitative comparison of occupancy forecasts with and without dynamic queries. Yellow boxes highlight moving regions where dynamic queries improve agreement with the GT.
Fusion method
Writeback method
mIoU dyn (%) ↑
1 s
2 s
3 s
Avg.
w/o short-term fusion
MAQI
44.06
36.67
34.03
38.25
w/o long-term fusion
MAQI
45.02
36.18
33.09
38.10
LST
MAQI
46.03
38.08
34.63
39.58
LST
Bilinear Splat
42.83
36.18
34.34
37.78
LST
MAQI w/o conf.
45.11
36.32
33.11
38.18
Table 4: Joint ablation of LST and MAQI on InfraOcc.
4D occupancy forecasting models the spatio-temporal evolution of 3D scenes and is crucial for autonomous driving, especially for corner-case simulation. Existing methods often rely on discrete tokenization followed by autoregressive prediction, yet struggle with geometric distortion in static structures and inconsistent temporal coherence over the forecasting horizon. In this work, we propose a Geometry-Aware Spatio-Temporal context modeling method (GAST) for 4D occupancy forecasting, built upon progressive explicit-implicit generation and dual-path spatio-temporal modeling. Specifically, the generation module produces per-frame occupancy with high geometric fidelity and semantic plausibility through pose-driven warping, motion-aware feature modulation, and attention-based feature refinement. Subsequently, the spatio-temporal module enhances spatial consistency through global context aggregation while capturing scene evolution through temporal dynamics extraction. This unified design enables joint optimization of historical reconstruction and future forecasting in an end-to-end manner. Extensive experiments on Occ3D-nuScenes demonstrate the superiority of our method, outperforming the state-of-the-art by 7.67% in mIoU and 6.44% in IoU with a 2.84x speedup, while maintaining strong performance in long-term forecasting.
Sitao Chen, Zhuangwei Zhuang, Hui Luo +2
South China University of Technology, Guangzhou, China · Institute of Optics and Electronics, CAS, Chengdu, China
Fixed roadside cameras repeatedly observe a stable scene overlaid by sparse moving traffic. Temporal memory can recover weak observations, but reusing moving evidence at stale locations can corrupt occupancy predictions. Motion compensation addresses displacement, while reliance on the resulting history remains a separate learning problem. We introduce RoadOcc, which learns soft routing among fixed-coordinate history (Persist), velocity-addressed history (Transport), and current evidence (Refresh). Motion state and class-consistent historical support supervise these source choices. Dynamic-aware cross-attention (DCA) updates candidate locations, multi-scale voxel velocity estimation (VVE) constructs transport addresses from multi-scale current--history correspondence, and velocity-guided dynamic sparse fusion (VDSF) combines routed evidence under fixed sparse-token budgets. On InfraOcc, RoadOcc reaches 65.29 mIoU and 32.37 dynamic mIoU, gains of 4.44 and 4.71 over STCOcc. Controlled address experiments show that VVE raises dynamic mIoU by 0.87 over fixed-coordinate reading. Across three seeds, supervised P/T/R adds 1.40 dynamic points over motion-corrected retrieval, while removing Refresh costs 0.32 points. Results from two transfer models, Occ3D-nuScenes, and longer intervals provide additional support. Code will be released.
Xiaokai Bai, Lei Yang, Songkai Wang +3
College of Information Science and Electronic Engineering, Zhejiang University · School of Mechanical and Aerospace Engineering, Nanyang Technological University · School of Automotive Studies, Tongji University
Camera-only 4D occupancy forecasting enables autonomous vehicles to predict future 3D semantic scenes solely from historical multi-view images, which is critical for driving safety. Even though current methods have achieved good performance, the strong spatial-temporal modeling between the input multi-view frames is still underexplored, which limits the performance of those methods in future 4D forecasting. To address this gap, we introduce a novel framework, InterOCF, for 4D occupancy forecasting that jointly models temporal dynamics in both 3D voxel-based representations and multi-view segmentation sequences, while explicitly incorporating feature interaction between the 2D and 3D branches. Our framework incorporates three core components: 1) A 3D Spatio-Temporal (3DST) module that learns volumetric dynamics from historical voxel states to predict future voxel states; 2) A 2D Spatio-Temporal (2DST) module employing an auxiliary multi-view temporal segmentation forecasting task to enhance temporal semantic dynamics; 3) A Spatio-Temporal Interaction Modeling (STIM) module that enables feature interaction between 2D and 3D representations. Experiments on the nuScenes, Lyft-Level5, and nuScenes-Occupancy datasets show that InterOCF consistently outperforms existing baseline approaches.
Qi Zhang, Xinquan Yu, Kaiyi Zhang +1
College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China.