LO-World: Unified BEV Latent Modeling for Joint 4D Occupancy and LiDAR Forecasting
Authors: Zhimin Liao, Ping Wei, Kai Mao, Shuaijia Chen
Organizations: National Key Laboratory of Human-Machine Hybrid Augmented Intelligence Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University
LiDAR point clouds and 3D semantic occupancy represent two distinct modalities capturing fine-grained metric geometry and dense volumetric semantics, respectively, holding natural potential for mutual complementarity. However, existing world models typically forecast these modalities independently or cascade them heuristically, missing the opportunity for reciprocal cross-modal regularization. To address this, we propose LO-World, a unified 4D world model that forecasts both modalities within a shared BEV latent space. This single-stream formulation enables dense semantics and metric geometry to mutually reinforce each other, while eliminating multi-pipeline redundancy. Specifically, LO-World couples an efficient hybrid range-voxel autoencoder with a flow-matching generator that caches historical context to reduce iterative sampling latency. Extensive experiments demonstrate that LO-World outperforms prior state-of-the-art baselines, improving occupancy forecasting IoU by 10.9% and reducing LiDAR Chamfer Distance by 30.1%. Meanwhile, it sustains a real-time throughput exceeding 25~FPS on a single RTX 4090. Our code is available at https://github.com/lzzzzzm/LO-World.
Figures & tables
Figure 1: (a) Unified forecasting. LO-World jointly encodes occupancy and LiDAR into a shared BEV latent, forecasting both modalities through a single generative process. (b) Performance comparison. Under comparable control conditions, LO-World achieves a 10.9% relative improvement in 4D IoU and a 30.1% reduction in Chamfer Distance over prior state-of-the-art baselines.
Figure 2: Overall architecture of LO-World. Our framework consists of two components: (1) LO-VAE, which jointly tokenizes occupancy and LiDAR into a unified BEV latent space; and (2) LO-Former, a flow-matching-based generation framework that efficiently generates future unified BEV latents from Gaussian noise.
Method
Input/Condition
mIoU (%) ↑
IoU (%) ↑
FPS ↑
1s
2s
3s
Avg.
1s
2s
3s
Avg.
OccWorld-O [ 42 ]
O /Traj.
25.78
15.14
10.51
17.14
34.63
25.07
20.18
26.63
18.00
UniScene [ 13 ]
O/Traj.&Box&Map
35.37
29.59
25.08
30.01
38.34
32.70
29.09
33.38
-
I2 -World [ 19 ]
O/Traj.
47.62
38.58
32.98
39.73
54.29
49.43
45.69
49.80
37.04
GenieDrive [ 37 ]
O/Traj.
50.47
41.47
35.83
42.59
56.87
51.46
47.08
51.80
41.38
LO-World-T ∗ (Ours)
O/Traj.
52.92
44.54
38.37
45.28
57.64
53.10
49.67
53.47
48.25
Table 1: 4D occupancy forecasting performance on the Occ3D-nus validation set. “C”, “O”, and “L” denote camera, occupancy, and LiDAR modalities, respectively. ∗ denotes the single-modality variant of our method.
Method
Input/Condition
Forecasting Quality( m2 ) ↓
Generation Quality
FPS-E2E ↑
CD1s
CD2s
CD3s
Avg.
FPD ↓
B-MMD ↓
B-JSD ↓
HERMES [ 44 ]
C/Traj.
0.78
0.95
1.17
0.97
105.96
21.04
4.01
0.51
OpenDWM-DiT [ 5 ]
L/Traj.&Box&Map
1.96
2.78
3.21
2.65
381.91
5.73
19.90
0.10
LaGen [ 43 ]
L/Traj.&Box&Map
0.61
0.81
1.08
0.83
-
-
3.97
-
UniScene [ 13 ]
O/Traj.&Box&Map
-
-
-
-
976.47
13.61
31.55
0.19
LO-World-T ∗ (Ours)
L/Traj.
0.46
0.81
1.18
0.82
8.84
2.68
4.54
28.16
Table 2: 4D LiDAR forecasting performance on the nuScenes validation set. “C”, “O”, and “L” denote camera, occupancy, and LiDAR, respectively. B-MMD and B-JSD are computed from BEV point-count histograms. Values are multiplied by 104 and 102 , respectively. ∗ denotes the single-modality variant of our method.
Method
Occ. mIoU ( % ) ↑
LiDAR ( m2 ) ↓
4s
5s
6s
CD4.0s
CD5.5s
I2 -World [ 19 ]
28.59
25.32
22.55
-
-
GenieDrive [ 37 ]
31.16
27.17
23.66
-
-
LaGen [ 43 ]
-
-
-
1.32
1.73
LO-World-T
34.01
30.70
27.48
1.49
2.12
LO-World-TBM
48.16
45.74
43.52
1.10
1.52
Table 3: Long-horizon evaluation on nuScenes val set.
Method
Occupancy ( % ) ↑
LiDAR ( m2 ) ↓
Avg.mIoU
Avg.IoU
CD1s
CD3s
OpenDWM † [ 5 ]
-
-
2.27
3.35
I2 -World [ 19 ]
36.38
52.36
-
-
LO-World-T
41.31
58.30
0.91
1.77
LO-World-T †
46.00
61.08
0.51
1.28
Table 4: Performance comparison on the Waymo dataset. † denotes models fine-tuned on Waymo, while unmarked models are evaluated in a zero-shot setting without target domain fine-tuning.
Method
Frames
Condition
FVD ↓
mIoU ↑
mAP ↑
MagicDrive [ 7 ]
60
BEV + Box
217.94
18.27
11.86
UniScene [ 13 ]
32
2 Hz GT Occ.
610.15
18.59
6.24
GenieDrive [ 37 ]
81
2 Hz GT Occ.
92.78
31.03
18.89
Ours
81
12 Hz Pred. Occ.
60.41
34.83
27.15
Oracle
81
12 Hz GT Occ.
40.78
36.47
30.24
Table 5: Downstream video generation on the nuScenes validation set. “Oracle” denotes the ground-truth-conditioned reference using 12 Hz occupancy.
Method
Avg. IoU ( % ) ↑
Avg. CD ( m2 ) ↓
FPD ↓
JC ↑
FPS-E2E ↑
(a)
54.08
0.84
9.01
0.52
17.76
(b)
54.71
0.80
8.94
0.55
10.24
(c)
57.46
0.70
5.16
0.77
26.26
w/o Hb
50.01
0.92
75.23
0.65
26.30
w/o Hb ⋄
55.87
0.75
32.59
0.67
6.72
w/o Sem
–
0.97
56.12
0.60
26.27
Table 6: Ablation study on the unified multimodal design. “Hb” indicates the hybrid range-voxel representation for LiDAR, “Sem” specifies whether semantic labels are incorporated into the occupancy volume, and ⋄ denotes the high-resolution voxelization setting.
Method
Avg. mIoU ↑
Avg. CD ↓
FPS-E2E ↑
Architecture Ablation
Full-Model
46.01
0.70
26.26
w/o Context
37.19
1.58
34.34
w/o Context ‡
43.60
0.74
18.43
Layers Ablation (Context, Flow)
(4, 4)
42.81
0.90
31.71
Table 7: Ablation studies on LO-Former. ‡ indicates the depth-matched variant with 12 flow blocks and no context blocks.
Figure 3: Qualitative forecasting on nuScenes. (a) LiDAR and occupancy forecasts at 2, 4, and 6 s versus OpenDWM and I2 -World , respectively. (b) Paired forecasts under straight, right-turn, and left-turn ego-vehicle trajectories.
Setting
Value
Ego-frame bounds (m)
[−40,40)×[−40,40)×[−1,5.4)
Occupancy voxel size
0.4×0.4×0.4 m
Occupancy grid (Nx,Ny,Nz)
200×200×16
Occupancy labels
18 (free-space ID 17)
Range-image size (Hr,Wr)
32×1024
Vertical projection bounds
−30∘,+10∘
Table A1: Spatial and temporal settings for the six-frame nuScenes task. Spatial bounds follow (x,y,z) , and the ego exclusion region uses sensor-frame (x,y) coordinates. Observed frames include the current frame.
Setting
Value
LO-VAE
Occupancy embedding width
16
Stem width per modality
64
Encoder stage widths
64, 128, 256
Spatial downsampling factor
4
Latent size
50×50×16
Table A2: Architecture and sampling settings for LO-VAE and LO-Former. Latent sizes list spatial dimensions followed by channel width. Encoder widths follow stage order.
Setting
LO-VAE
LO-Former
2-frame
6-frame
12-frame
Learning rate
10−4
5×10−4
10−4
10−5
Total batch size
16
8
8
8
Training epochs
40
72
20
8
AdamW β2
0.95
0.999
0.999
0.999
Gradient clipping norm
1
15
15
15
Table A3: Training settings for LO-VAE and the successive LO-World-T forecasting stages on nuScenes. The 12-frame stage fine-tunes the six-frame model. Batch sizes are totals across eight GPUs. Warmup ratios are relative to the listed learning rates (LR).
Figure A1 : Examples with higher and lower JC. Each row pairs occupancy and LiDAR views from one example. For each modality, we show ground truth (GT), prediction, and their superimposition. The associated JC score is shown at the right.
Time (s)
LO-World-T
LO-World-TBM
mIoU ↑ (%)
IoU ↑ (%)
CD ↓ ( m2 )
mIoU ↑ (%)
IoU ↑ (%)
CD ↓ ( m2 )
0.5
60.02
64.58
0.3164
63.32
64.95
0.3066
1.0
53.56
61.72
0.4206
59.65
62.49
0.3896
1.5
48.89
59.31
0.5440
57.01
60.46
0.4829
2.0
45.15
57.14
0.6833
54.86
58.60
0.5817
2.5
42.00
55.02
0.7906
52.93
56.77
0.6526
Table A4: Twelve-frame forecasting on nuScenes at 0.5 s intervals. The divider after 3 s separates the two six-frame chunks.
Steps
Occupancy-only
Joint
LiDAR-only
mIoU ↑ (%)
IoU ↑ (%)
mIoU ↑ (%)
IoU ↑ (%)
CD ↓ ( m2 )
CD ↓ ( m2 )
1
45.85
53.60
47.20
57.52
0.83
1.01
2
45.72
53.52
47.19
57.50
0.83
0.98
3
45.47
53.49
46.77
57.48
0.75
0.89
4
45.28
53.47
46.01
57.46
0.70
0.82
8
42.51
51.46
43.80
55.33
0.68
0.81
Table A5: Sampling-step ablation for unimodal and joint LO-World-T.
Module
Parameters (M)
Time (ms)
LO-VAE
16.7
125.8
LO-Former
43.2
102.7
Total
59.9
228.5
Table A6 : Parameter count and execution time of each module.
Figure A2 : Occupancy, LiDAR, and RGB views for three scenes.
Figure A3 : Two failure cases under reverse ego-motion trajectories. The cases are shown side by side, with occupancy and LiDAR predictions paired in adjacent columns. Successive frames run from top to bottom.