3D semantic occupancy prediction provides dense geometric and semantic scene representations for autonomous driving, where both high accuracy and low response latency are crucial for safe downstream forecasting and planning. Existing methods usually exhibit different accuracy-latency characteristics. Performance-oriented models provide stronger geometric and semantic predictions, while deployment-friendly models provide faster responses. This motivates EvoOcc, a fast-slow evolutionary occupancy prediction framework that jointly exploits their complementary strengths to improve the accuracy-latency balance without changing the upstream architectures. EvoOcc models evolutionary occupancy prediction as a continuous time controlled dynamical system, enabling flexible adaptation to variable slow system delays and irregular fast system arrivals. The framework further decomposes the evolution process into dual state evolution where ego state evolution compensates for the deterministic coordinate shifts and scene state evolution models the genuine geometric and semantic scene changes. Extensive experiments on three different fast-slow system pairs and irregular prediction arrivals show that EvoOcc consistently improves over fast system baselines while maintaining much lower latency than the slow system, indicating a favorable accuracy-latency balance and stable behavior under practical inference situations.
Figures & tables
Figure 1: Overview of fast-slow evolutionary occupancy prediction. (a) Deployment-friendly models provide timely occupancy predictions but sacrifice geometric and semantic accuracy. (b) Performance-oriented models are more accurate, yet their delayed outputs may become stale for real-time perception. (c) EvoOcc exploits high-quality information from the slow system without sacrificing the responsiveness of the fast system by switching between two operational modes.
Figure 2: Main framework of EvoOcc. Along the timeline, EvoOcc switches between two operational modes depending on which logits arrive at the current timestamp. When slow system logits arrive, EvoOcc uses them to calibrate the state anchor. When only the fast system logits arrive, EvoOcc evolves the previous state by ego state evolution and scene state evolution, then decodes it as a residual correction to the current fast logits.
Fast / Slow pair
Method
mIoU
mIoU D
RayIoU 1m
RayIoU 2m
RayIoU 4m
RayIoU
Fast System
33.4
31.3
32.9
40.1
44.8
39.3
ALOcc-2D-mini [ 23 ] /
Slow System
38.3
35.6
37.7
44.7
48.8
43.7
ALOcc-3D
Warped delayed slow
26.5
20.0
25.1
30.5
34.4
30.0
EvoOcc (ours)
39.3 ↑ 5.9
33.7 ↑ 2.4
34.7 ↑ 1.8
41.3 ↑ 1.2
45.6 ↑ 0.8
40.5 ↑ 1.2
Fast System
33.2
28.7
31.7
39.2
44.3
38.4
OPUS-T [ 12 ] /
Slow System
38.6
34.0
37.9
44.9
49.1
44.0
Table 1: Results for EvoOcc in offline evolution. Fast system predictions arrive every 0.5 s, and each current timestamp is paired with its own slow system prediction from 2.0 s earlier. The result of slow system evaluated at the current timestamp is reported only as a reference. Warped delayed slow applies only ego-motion warping to align the delayed slow predictions to the current timestamp. Values after ↑ denote absolute gains over the fast system.
Figure 3: Visualizations of 3D semantic occupancy predictions on the OPUS-T / OPUS-V2-L pair. Orange and blue circles highlight representative dynamic-object and static-object regions.
Fast / Slow pair
Method
mIoU
mIoU D
RayIoU
ALOcc-2D-mini / ALOcc-3D
Fast System
33.4
31.3
39.3
Slow System
38.3
35.6
43.7
EvoOcc (ours)
40.3 ↑ 6.9
34.5 ↑ 3.2
41.5 ↑ 2.2
OPUS-T / OPUS-V2-L
Fast System
33.2
28.7
38.4
Slow System
38.6
34.0
44.0
EvoOcc (ours)
37.5 ↑ 4.3
32.2 ↑ 3.5
40.3 ↑ 1.9
Table 2: Results for EvoOcc in streaming evolution. Fast system predictions arrive every 0.5 s. A slow prediction sensed at tm becomes available at tn=tm+0.5 s, recalibrates the state anchor, and is evolved to the current timestamp tn . Values after ↑ denote absolute gains over the fast system.
Accuracy ↑
Cost ↓
Method
Interval (s)
mIoU
mIoU D
RayIoU
TFLOPs/scene
Fast System
–
33.4
31.3
39.3
0 7.1
Slow System
–
38.3
35.6
43.7
15.4
EvoOcc (ours)
0 0.5
40.3
34.5
41.5
23.6
0 2.0
39.7
34.0
40.9
13.2
0 5.0
38.9
33.5
40.4
11.0
Table 3: Results for EvoOcc under different slow system invocation intervals in streaming evolution, on the ALOcc-2D-mini / ALOcc-3D pair. Interval denotes the time between consecutive slow system invocations. The standalone fast and slow systems are listed for reference.
Accuracy ↑
E2E cost ↓
EvoOcc overhead ↓
Method
mIoU
mIoU D
RayIoU
Lat. (ms)
TFLOPs
Mem. (GB)
Lat. (ms)
TFLOPs
Fast System
33.4
31.3
39.3
63.3
0.177
0.82
–
–
Slow System
38.3
35.6
43.7
216.6
0.384
8.40
–
–
EvoOcc (ours)
38.9
33.5
40.4
101.8
0.273
8.80
23.2
0.061
Table 4: System-level results for EvoOcc in streaming evolution on the ALOcc-2D-mini / ALOcc-3D pair with a 5.0 s slow system invocation interval. E2E cost covers the fast system, the slow system, and all EvoOcc operations; latency and FLOPs are averaged per frame, and Mem. denotes peak memory. EvoOcc overhead is the incremental cost of EvoOcc itself.
Fast / Slow pair
Fast rate
mIoU
mIoU D
RayIoU
ALOcc-2D-mini [ 23 ] / ALOcc-3D
Fixed 3 Hz
38.7
33.3
40.5
Alt. 3/6 Hz
38.2
33.3
40.5
Rand. 2/3/6 Hz
38.6 ± 0.21
33.3 ± 0.04
40.5 ± 0.02
OPUS-T [ 12 ] / OPUS-V2-L
Fixed 3 Hz
36.5
31.4
39.2
Alt. 3/6 Hz
35.9
31.2
39.1
Rand. 2/3/6 Hz
36.4 ± 0.31
31.4 ± 0.12
39.2 ± 0.08
Table 5: Results for EvoOcc under irregular fast system arrival rates. We evaluate EvoOcc without retraining under fixed 3 Hz, alternating 3/6 Hz, and random {2,3,6} Hz intervals. Random results are reported as mean ± std over 10 seeds.
Accuracy ↑
Module cost ↓
Method
mIoU
mIoU D
RayIoU
GFLOPs
Latency (ms)
Fast System
33.4
31.3
39.3
–
–
Slow System
38.3
35.6
43.7
–
–
w/o control increment
39.9
34.2
41.1
60.2
22.9
w/o Lterm
40.6
34.4
40.4
60.8
23.2
Learned fusion (convolution)
39.8
34.0
41.1
62.9
22.1
Table 6: Ablation study in streaming evolution on the ALOcc-2D-mini / ALOcc-3D pair. w/o control increment replaces ΔUtn with the elapsed time Δtn ; learned fusion replaces scene state evolution by directly fusing fast and warped delayed slow logits. Module costs exclude the fast and slow systems.
Metric
w/o Lterm
EvoOcc
Extra occ. (%)
4.30
3.01
Extra length (m)
0.72
0.65
Extra thickness (m)
0.15
0.10
Table 7: Surface-geometry diagnostics ( ↓ ) for the ray-termination loss on the ALOcc-2D-mini / ALOcc-3D pair.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Method
mIoU
others
barrier
bicycle
bus
car
cons. veh.
motorcycle
pedestrian
tfc. cone
trailer
truck
drive. surf.
other flat
sidewalk
terrain
manmade
vegetation
ALOcc-2D-mini / ALOcc-3D
Fast System
33.38
13.18
46.24
26.06
40.45
46.89
22.58
26.61
26.64
29.73
25.93
35.53
59.82
41.29
42.10
36.08
24.70
23.68
Slow System
38.29
14.27
49.09
31.48
42.11
50.26
27.74
31.63
29.82
36.30
32.98
38.73
64.80
44.37
46.93
42.45
35.57
32.32
EvoOcc (ours)
39.31
14.27
49.43
26.70
43.36
50.94
23.54
27.80
28.59
32.64
30.37
38.29
76.26
45.52
53.06
52.84
39.30
35.36
OPUS-T / OPUS-V2-L
Fast System
33.19
10.70
39.83
21.26
39.75
45.25
23.40
21.83
17.81
19.24
27.49
33.20
71.61
37.11
45.14
43.59
33.80
33.18
Appendix
Table 8: Per-class results on the Occ3D-nuScenes [ 2 ] dataset in the offline evolution scenario. The slow system is evaluated directly at the current timestamp and reported only as a reference.
Vision-based 3D semantic occupancy prediction is essential for autonomous driving, yet dense voxel representations waste computation on largely empty space, while BEV and TPV projections compromise fine-grained 3D structure. Fully sparse representations offer an attractive alternative, but existing methods, including SparseOcc, entangle scene completion with semantic prediction by indiscriminately propagating high-dimensional features into empty regions and applying voxel-wise classification. This creates excessive activations, computational overhead, and geometric ambiguity. We present SparseOcc++, a geometry-aware sparse framework that explicitly decouples scene completion from semantic segmentation. SparseOcc++ reformulates completion as signed-distance regression on sparse anchor voxels through a scene completion field (SCF). To model complex outdoor geometry robustly, it combines orthogonal decomposition with discretized distance learning. A geometry-guided propagation module then converts the SCF into a complete volumetric scene and restricts semantic segmentation to geometrically verified regions. Experiments establish new state of the art: SparseOcc++ improves IoU by 2.3 points and is 3.9x faster than SparseOcc on nuScenes, while achieving a 5.9x speedup over OccFormer on SemanticKITTI.
Pin Tang, Zhongdao Wang, Guoqing Wang +2
MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, Shanghai 200240, China · Huawei Noah’s Ark Lab, China
Future 3D semantic occupancy forecasting and motion planning are central to autonomous driving, as they require models to reason about how surrounding scenes evolve and how the ego vehicle should act. Existing occupancy world models commonly discretize scenes into latent embeddings, volumetric features, or quantized tokens, and forecast future states through fixed-step autoregressive generation. This limits temporal flexibility, obscures scene evolution, accumulates errors over long horizons, and poorly matches the continuous-time dynamics of real driving scenes. We propose GEM, a Gaussian Evolution Model for non-autoregressive occupancy world modeling, where driving scenes are represented as explicit continuous 4D Gaussian primitives with learned dynamics. Instead of rolling out future occupancy states step by step, GEM directly queries the Gaussian world representation at arbitrary timestamps and splats the corresponding conditional 3D Gaussians into semantic occupancy volumes. This enables efficient forecasting over the full horizon while retaining a compact and interpretable scene representation. By decoupling spatial geometry, temporal support, and primitive motion, GEM makes the predicted world easier to inspect, as each primitive's evolution can be followed continuously over time. The same representation also supports motion planning by predicting future ego trajectories from the learned Gaussian world. Extensive experiments show that GEM achieves state-of-the-art future semantic occupancy forecasting and strong motion planning performance, while providing flexible temporal querying.
The point-based occupancy prediction paradigm has achieved an attractive trade-off between accuracy and efficiency by modeling 3D space sparsely. However, its predictions inherently mismatch the dense voxel-based occupancy required by self-driving systems, necessitating hand-crafted heuristics during training and inference that limit final performance. To overcome these limitations, we propose OPUS-V2, a novel framework built upon the pioneering OPUS (occupancy prediction using a sparse set) point-based approach. OPUS-V2 incorporates a lightweight point-voxel transformation (PVT) module behind the decoder to adaptively map sparse predictions into the dense voxel space, eliminating the need for suboptimal operations and improving model accuracy. Furthermore, our architecture decouples feature and occupancy generation processes, allowing OPUS-V2 to adapt to arbitrary occupancy resolutions. OPUS-V2 achieves a state-of-the-art rayIoU of 44.0 on the Occ3D dataset. On the more challenging OpenOccupancy dataset, it attains a competitive 16.4 mIoU while running in real time at 20.6 FPS.