InfraWorld: Motion-Aware World Modeling for Roadside 4D Occupancy Forecasting

Songkai Wang; Xiaokai Bai; Lei Yang; Hui-liang Shen

cs.CVOct 10, 2026

InfraWorld: Motion-Aware World Modeling for Roadside 4D Occupancy Forecasting

Authors: Songkai Wang, Xiaokai Bai, Lei Yang, Hui-liang Shen

Organizations: College of Information Science and Electronic Engineering, Zhejiang University. · School of Mechanical and Aerospace Engineering, Nanyang Technological University.

Abstract

Roadside 4D occupancy forecasting predicts future semantic occupancy in a fixed infrastructure coordinate system. Roadside scenes are globally structured yet locally evolving, requiring comprehensive scene-level context while precisely modeling sparse localized motion. However, existing forecasting paradigms struggle to tackle both requirements. To address this limitation, we propose InfraWorld, a world model formulating roadside occupancy forecasting as selective state evolution. InfraWorld maintains a dense BEV state for the complete scene and persistent motion queries for moving regions, assigning complementary states to global context and local dynamics. Specifically, we propose Dynamic Query Initiation (DQI) to initialize queries in regions supported by motion in the observed history. During rollout, the dense state is updated through Long--Short Temporal Fusion (LST), which combines short-term adjacent-frame continuity with long-term motion trends to preserve stable context while capturing scene evolution. Motion-Aware Query Injection (MAQI) then refines the queries and injects confidence-weighted query information into corresponding BEV tokens, enabling targeted dynamic updates without perturbing global context. Together, InfraWorld concentrates predictive capacity on evolving traffic regions. Extensive experiments on the InfraOcc and Occ3D--nus datasets demonstrate that InfraWorld achieves state-of-the-art performance for roadside 4D occupancy forecasting while retaining its effectiveness under ego-centric settings. Code will be released.

Figures & tables

Figure 1: (a) Voxel distribution in InfraOcc: static scene (SS), moving participants (MP), and stationary participants (SP). (b) InfraWorld maintains the stable scene layout with a dense BEV state, while queries reason about localized dynamics and inject their updates into the scene context.

Figure 2: Sparse and localized dynamics in roadside occupancy. Across five consecutive frames, the static scene layout remains largely unchanged, while only a few moving traffic participants exhibit temporal changes.

Figure 3: Dynamic Query Initiation (DQI). Top: DQI process for selecting and localizing motion-supported queries. Bottom: query evolution during rollout, where gray regions denote GT moving objects and highlighted trajectories show how queries capture localized motion evolution.

Figure 4: Architecture overview of InfraWorld. A frozen tokenizer encodes the observed occupancy into dense BEV latents, while DQI initializes persistent queries in predicted moving regions. At each autoregressive step, LST fuses spatial and temporal context to update the dense state and refine the queries, after which MAQI injects reliable query features into local BEV regions. The updated dense latent and queries are then propagated to the next forecasting step.

Figure 5: Motion-Aware Query Injection (MAQI). MAQI injects confidence-weighted query features into neighboring BEV tokens through distance-biased cross-attention.

Method	Input	mIoU dyn (%) $\uparrow$				mIoU sta (%) $\uparrow$
		1 s	2 s	3 s	Avg.	1 s	2 s	3 s	Avg.
OccWorld-O (ECCV 2024)	3D-Occ	25.64	21.24	19.76	22.21	97.52	97.42	97.37	97.44
I 2 -World-O (ICCV 2025)	3D-Occ	33.34	29.67	28.66	30.56	97.30	97.27	97.07	97.21
InfraWorld-O (Ours)	3D-Occ	46.03	38.08	34.63	39.58	97.79	97.72	97.68	97.73
OccWorld-STC (ECCV 2024)	Camera	16.20	14.29	13.69	14.73	78.29	78.27	78.24	78.27
I 2 -World-STC (ICCV 2025)	Camera	19.47	16.95	15.73	17.38	78.71	78.70	78.69	78.70

Table 1: 4D occupancy forecasting performance on InfraOcc validation set.

Method	Input	mIoU (%) $\uparrow$
Method	Input	1 s	2 s	3 s	Avg.
OccWorld-O	Occ	25.78	15.14	10.51	17.14
OccLLaMA-O	Occ & Text	25.05	19.49	15.26	19.93
DFIT-OccWorld	Occ & Camera	31.68	21.29	15.19	22.71
DOME	Occ	35.11	25.89	20.29	27.10
UniScene	Occ	35.37	29.59	25.08	31.76

Table 2: Forecasting performance on Occ3D–nus.

Figure 6: Qualitative comparison between InfraWorld and I 2 -World on InfraOcc. Yellow boxes highlight dynamic regions where InfraWorld more closely matches the ground truth.

LST	MAQI	DQI	mIoU dyn (%) $\uparrow$
			1 s	2 s	3 s	Avg.
–	–	–	33.68	32.54	26.58	30.93
$\checkmark$	–	–	35.20	32.95	30.46	32.87
–	$\checkmark$	–	36.81	34.33	31.91	34.35
–	$\checkmark$	$\checkmark$	42.76	34.87	32.62	36.75
$\checkmark$	$\checkmark$	$\checkmark$	46.03	38.08	34.63	39.58

Table 3: Component-wise ablation of InfraWorld.

Figure 7: Qualitative comparison of occupancy forecasts with and without dynamic queries. Yellow boxes highlight moving regions where dynamic queries improve agreement with the GT.

Fusion method	Writeback method	mIoU dyn (%) $\uparrow$
		1 s	2 s	3 s	Avg.
w/o short-term fusion	MAQI	44.06	36.67	34.03	38.25
w/o long-term fusion	MAQI	45.02	36.18	33.09	38.10
LST	MAQI	46.03	38.08	34.63	39.58
LST	Bilinear Splat	42.83	36.18	34.34	37.78
LST	MAQI w/o conf.	45.11	36.32	33.11	38.18

Table 4: Joint ablation of LST and MAQI on InfraOcc.

Method	Params. (M)	FLOPs (G)	Lat. (ms)	mIoU dyn $\uparrow$
OccWorld	71.1	1291	162	22.21
I 2 -World	22.5	91	85	30.56
I 2 -World-L	28.9	169	91	32.90
InfraWorld	29.1	166	119	39.58

Table 5: Efficiency–accuracy comparison. Lat: bs=1 inference latency.

Spatiotemporal Forecasting

Spatiotemporal forecasting predicts how quantities change across locations and time, from traffic and weather to disease and environmental systems.

See all 423 papers

Geometry-Aware Spatio-Temporal Context Modeling for 4D Occupancy Forecasting
Aug 15, 2026Sitao Chen, Zhuangwei Zhuang, Hui Luo +23D Occupancy Prediction 4D Scene Understanding
South China University of Technology, Guangzhou, China · Institute of Optics and Electronics, CAS, Chengdu, China
RoadOcc Learns When to Persist, Transport, or Refresh Memory for Roadside Occupancy Prediction
Sep 23, 2026Xiaokai Bai, Lei Yang, Songkai Wang +3Autonomous Driving Perception 3D Occupancy Prediction
College of Information Science and Electronic Engineering, Zhejiang University · School of Mechanical and Aerospace Engineering, Nanyang Technological University · School of Automotive Studies, Tongji University
InterOCF: Spatio-Temporal 2D-3D Interaction for Camera-Only 4D Occupancy Forecasting
Jul 27, 2026Qi Zhang, Xinquan Yu, Kaiyi Zhang +13D Occupancy Prediction Spatiotemporal Forecasting
College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China.

Aug 15, 2026J/K move · Enter open · S save

Geometry-Aware Spatio-Temporal Context Modeling for 4D Occupancy Forecasting

Sitao Chen, Zhuangwei Zhuang, Hui Luo, Qingyao Wu+1

South China University of Technology, Guangzhou, China · Institute of Optics and Electronics, CAS, Chengdu, China

3D Occupancy Prediction 4D Scene Understanding 3D Scene Understanding Autonomous Driving

Open paper AskPDF arXiv

4D occupancy forecasting models the spatio-temporal evolution of 3D scenes and is crucial for autonomous driving, especially for corner-case simulation. Existing methods often rely on discrete tokenization followed by autoregressive prediction, yet struggle with geometric distortion in static structures and inconsistent temporal coherence over the forecasting horizon. In this work, we propose a Geometry-Aware Spatio-Temporal context modeling method (GAST) for 4D occupancy forecasting, built upon progressive explicit-implicit generation and dual-path spatio-temporal modeling. Specifically, the generation module produces per-frame occupancy with high geometric fidelity and semantic plausibility through pose-driven warping, motion-aware feature modulation, and attention-based feature refinement. Subsequently, the spatio-temporal module enhances spatial consistency through global context aggregation while capturing scene evolution through temporal dynamics extraction. This unified design enables joint optimization of historical reconstruction and future forecasting in an end-to-end manner. Extensive experiments on Occ3D-nuScenes demonstrate the superiority of our method, outperforming the state-of-the-art by 7.67% in mIoU and 6.44% in IoU with a 2.84x speedup, while maintaining strong performance in long-term forecasting.

Abstract

Spatiotemporal Forecasting

Explore similar work

Geometry-Aware Spatio-Temporal Context Modeling for 4D Occupancy Forecasting

RoadOcc Learns When to Persist, Transport, or Refresh Memory for Roadside Occupancy Prediction

InterOCF: Spatio-Temporal 2D-3D Interaction for Camera-Only 4D Occupancy Forecasting