Occupancy world models struggle to forecast moving objects due to motion uncertainty and shape distortion. A single trajectory can average plausible futures into an implausible path, and repeated re-voxelization can erode or inflate object shape even under correct motion. We introduce ShoWorld to address these challenges through separate motion and shape representations. The motion forecaster predicts probabilistic trajectories conditioned on scene layout, nearby agents and the future ego trajectory. Each hypothesis is rendered into occupancy and fused according to its probability to retain alternative futures. The shape component predicts sub-voxel points with occupancy probabilities and offsets from local geometry. These points move rigidly with the object, and voxel occupancy follows the probability that at least one occupied point falls within it. This representation reduces shape distortion under motion. ShoWorld achieves state-of-the-art results on Occ3D-nuScenes. With ground-truth occupancy and ego trajectory, it improves mIoU and IoU by 6.56 and 9.16 points overall and by 6.06 and 5.53 points on dynamic classes.
Figures & tables
Figure 1: Static and dynamic mIoU of occupancy world models on Occ3D-nuScenes [ 29 ] . Moving objects remain harder to forecast than static regions, and the gap widens with the prediction horizon.
Figure 2: Motion uncertainty and shape preservation. (a) Voxel-wise supervision can pull a single trajectory between plausible futures. Our fusion retains each hypothesis with its probability. (b) Re-voxelization of rotated voxel centers leaves holes. Sub-voxel points improve shape coverage with offsets shown as intended module outputs.
Figure 3: Overview of ShoWorld. The scene is decomposed into a static background and rigid objects grouped from the current occupancy. Each object predicts K trajectories with probabilities from the layout, nearby agents, and the ego vehicle. Its sub-voxel points, inferred once from local geometry, are rendered under every hypothesis, fused by the probabilities, and composited over the background.
Figure 4: Gain of each component on the dynamic mIoU. Each bar replaces one component at inference, using only the most likely hypothesis or voxel centers instead of sub-voxel points.
Figure 5: Two forecasts from the validation set. For each scene, we show the hypotheses of one vehicle on the current frame and the dynamic objects forecast at 3 s by the fused hypotheses and by the most likely one. The most likely hypothesis of the vehicle ends over 4 m from the annotated position, while another one with a probability above 0.15 ends within 2 m.
Figure 6: Shape of moving objects under ground-truth motion. Objects moved by their annotated motion are rendered by each method and compared with ground truth.
Method
Input
Ego traj.
mIoU (%) ↑
IoU (%) ↑
Venue
1s
2s
3s
Avg.
1s
2s
3s
Avg.
OccWorld-D [ 46 ]
Camera
Pred.
11.55
8.10
6.22
8.62
18.90
16.26
14.43
16.53
ECCV 2024
OccWorld-T [ 46 ]
Camera
Pred.
4.68
3.36
2.63
3.56
9.32
8.23
7.47
8.34
ECCV 2024
OccWorld-S [ 46 ]
Camera
Pred.
0.28
0.26
0.24
0.26
5.05
5.01
4.95
5.00
ECCV 2024
OccWorld-F [ 46 ]
Camera
Pred.
8.03
6.91
3.54
6.16
23.62
18.13
15.22
18.99
ECCV 2024
RenderWorld [ 40 ]
Camera
Pred.
2.83
2.55
2.37
2.58
14.61
13.61
12.98
13.73
ICRA 2025
Table 1: 4D occupancy forecasting on Occ3D-nuScenes. Methods are grouped by input modality and by whether the ego trajectory is predicted (Pred.) or given as ground truth (GT). Avg. denotes the average over the 1 s, 2 s, and 3 s horizons. Results are taken from [ 9 ] .
Method
mIoU (%) ↑
IoU (%) ↑
1s
2s
3s
4s
5s
6s
7s
8s
Avg.
1s
2s
3s
4s
5s
6s
7s
8s
Avg.
DOME [ 11 ]
30.10
21.35
17.36
14.86
12.61
11.03
10.00
9.34
15.83
39.04
31.20
27.14
24.73
22.32
20.28
19.05
17.97
25.21
COME [ 27 ]
33.78
24.57
21.35
18.25
15.84
13.85
12.99
11.96
19.07
44.20
36.25
32.86
30.03
26.93
24.70
23.30
21.44
29.96
GAST [ 9 ]
36.23
30.61
27.13
24.36
21.98
19.81
17.83
15.96
24.12
49.09
45.11
42.11
39.52
37.19
34.99
32.87
30.87
39.06
ShoWorld
63.82
52.68
45.60
39.43
35.55
32.26
28.62
26.07
40.50
70.30
64.61
60.41
53.81
50.08
46.73
41.82
38.86
53.33
Table 2: Long-term forecasting results on the Occ3D-nuScenes dataset over 8 s with GT 3D occupancy and ego trajectory as inputs.
Figure 7: Qualitative comparison with COME [ 27 ] for long-term forecasting on Occ3D-nuScenes, shown for two scenes in bird’s-eye view. Black outlines mark the GT positions of dynamic objects, static classes are drawn in gray, and the triangle marks the ego vehicle.
Figure 8: Long-term forecasting of dynamic classes on Occ3D-nuScenes. GAST is not included as its 8 s weight is not released.
Method
Rate
mIoU (%) ↑
IoU (%) ↑
Copy&Paste [ 46 ]
2 Hz
17.17
28.21
I 2 -World [ 20 ]
2 Hz
36.38
52.36
GAST [ 9 ]
2 Hz
46.70
60.09
ShoWorld
2 Hz
57.84
68.74
Copy&Paste [ 46 ]
10 Hz
28.34
40.09
I 2 -World [ 20 ]
10 Hz
43.73
60.97
Table 3: Zero-shot forecasting on Occ3D-Waymo.
Method
Params ↓
FLOPs ↓
Latency ↓
mIoU ↑
IoU ↑
(M)
(G)
(ms)
(%)
(%)
DOME [ 11 ]
444.07
2928.66
1899.15
27.10
36.36
COME [ 27 ]
692.97
4461.46
4377.67
34.23
44.13
I 2 -World [ 20 ]
22.67
494.58
227.09
39.73
49.80
GAST [ 9 ]
12.09
416.27
80.03
47.40
56.24
ShoWorld
7.93
123.22
84.91
53.96
65.40
Table 4: Complexity analysis on the Occ3D-nuScenes dataset.
Method
Context
Multi-Hypothesis
Sub-Voxel Shape
mIoU (%)
Dynamic
Encoding
2
4
6
nearest
any-hit
mIoU (%)
(a)
45.13
37.65
(b)
✓
46.84
38.27
(c)
✓
✓
48.06
41.19
(d)
✓
✓
49.35
42.62
(e)
✓
✓
50.33
44.61
Table 5: Ablation studies of our method on Occ3D-nuScenes.
Current 4D occupancy world models use heavy cross-attention or deep recurrence to disentangle ego-motion from scene dynamics. This requires significant computational capacity. However, this effort is largely redundant since the ego-vehicle's intended future motion is provided by the downstream motion planner. We present FreeOcc, a lightweight continuous-time world model that introduces explicit SE(2) geometric decoupling into a conditional flow-matching framework. By mathematically factoring out self-motion, we create a purely dynamic feature space. This space supports attention-free temporal fusion and allows physics-constrained ODEs to operate strictly on residual agent kinematics. To maximize deployment efficiency, FreeOcc introduces an uncertainty-aware blending mechanism. This routes well-observed voxels to a single-pass decoder, reserving the ODE solver exclusively for ambiguous, high-uncertainty regions. Requiring only 28.3M parameters-roughly 40% of comparable models-our optimal ensemble mode (FreeOcc-E) runs at 27 FPS. On nuScenes, FreeOcc achieves state-of-the-art forecasting performance with a 0.47 m Near Field Chamfer Distance (NFCD) and 32.1% mIoU. Furthermore, the model transfers zero-shot to SemanticKITTI. It also proves highly robust, retaining 92.8% of its ideal mIoU under significant sensor packet loss and timestamp jitter.
Zeping Zhang, Zhuoya Zhao, Samy Metari +1
University of Ottawa · National Research Council Canada
Roadside 4D occupancy forecasting predicts future semantic occupancy in a fixed infrastructure coordinate system. Roadside scenes are globally structured yet locally evolving, requiring comprehensive scene-level context while precisely modeling sparse localized motion. However, existing forecasting paradigms struggle to tackle both requirements. To address this limitation, we propose InfraWorld, a world model formulating roadside occupancy forecasting as selective state evolution. InfraWorld maintains a dense BEV state for the complete scene and persistent motion queries for moving regions, assigning complementary states to global context and local dynamics. Specifically, we propose Dynamic Query Initiation (DQI) to initialize queries in regions supported by motion in the observed history. During rollout, the dense state is updated through Long--Short Temporal Fusion (LST), which combines short-term adjacent-frame continuity with long-term motion trends to preserve stable context while capturing scene evolution. Motion-Aware Query Injection (MAQI) then refines the queries and injects confidence-weighted query information into corresponding BEV tokens, enabling targeted dynamic updates without perturbing global context. Together, InfraWorld concentrates predictive capacity on evolving traffic regions. Extensive experiments on the InfraOcc and Occ3D--nus datasets demonstrate that InfraWorld achieves state-of-the-art performance for roadside 4D occupancy forecasting while retaining its effectiveness under ego-centric settings. Code will be released.
Songkai Wang, Xiaokai Bai, Lei Yang +1
College of Information Science and Electronic Engineering, Zhejiang University. · School of Mechanical and Aerospace Engineering, Nanyang Technological University.
This letter proposes CascadeOcc, a novel occupancy world model that prioritizes intrinsic structural hierarchy over extrinsic auxiliary modalities for autonomous driving. Occupancy world models -- forecasting the future driving environment and planning the driving trajectory -- effectively bridge perception and planning, but current approaches often heavily rely on external modalities or large language models, failing to fully exploit the inherent structural potential of occupancy representations themselves. To enhance representational capacity for complex 3D scenes, we integrate a cascaded Vector Quantized (VQ) mechanism into an autoregressive framework. Following a coarse-to-fine principle, CascadeOcc progressively refines fine-grained details from global structures through a multi-scale architecture. Additionally, we incorporate a TimeMixer to capture multi-scale temporal dependencies, establishing a dual-hierarchy mechanism in both space and time. Experimental results on 4D occupancy forecasting and motion planning benchmarks demonstrate that CascadeOcc achieves superior performance among vision-centric approaches, validating that optimizing inherent representations is a powerful alternative to relying on external foundation models.
Kyumin Hwang, Wonhyeok Choi, Jaeyeul Kim +3
Daegu Gyeongbuk Institute of Science and Technology (DGIST), Daegu 42988, South Korea