Camera-based 3D semantic occupancy models can be trained without 3D labels by rendering their predictions into the images and supervising them with 2D pseudo labels. In sparse Gaussian representations trained this way, Gaussian positions are learned only through pixel-wise rendering losses, which we hypothesize give small foreground objects little weight. In trained GaussianFlowOcc models, the Gaussian density on foreground voxels is on average only 1.62 times that on background voxels. We propose a training-only foreground attraction loss that back-projects foreground pseudo-label pixels with pseudo depth and pulls nearby Gaussians toward these 3D points, favoring Gaussians that already predict the target class; the inference network is unchanged. On Occ3D-nuScenes, the loss raises this concentration to 2.40 over three seeds. Over three seeds, the mean foreground IoU increases from 6.29 to 6.96 and is higher for each seed, whereas the change in mean mIoU (13.53 to 13.82) is within the seed-to-seed variation of the baseline. Single-run ablations further indicate that pseudo-label targets outperform targets placed at ground-truth voxels and that acting on the Gaussian positions alone is not sufficient, and a single-seed test shows that the model degrades less when fewer Gaussians are kept at inference.
Figures & tables
Fig. 1: Overview. The baseline learns Gaussian positions only through 2D rendering losses on pseudo semantic labels and pseudo depth. Our foreground attraction loss (dashed, training only) back-projects foreground pseudo-label pixels to 3D and pulls nearby Gaussians toward them, favoring Gaussians that already predict the target class. The inference network is unchanged. Image thumbnails are schematic illustrations.
Method
mIoU
FG mIoU
Occ. IoU
GaussianFlowOcc
13.53 ± 0.62
6.29 ± 0.50
37.72 ± 1.25
Ours
13.82 ± 0.10
6.96 ± 0.27
37.52 ± 0.74
TABLE I: Main comparison on Occ3D-nuScenes (separate-mode pseudo labels, trained on 100 scenes, evaluated on all 150 validation scenes, camera-visible voxels). Mean ± sample standard deviation over three seeds. mIoU: mean IoU over 15 semantic classes; FG mIoU: mean IoU over the 10 foreground classes; Occ. IoU: occupancy IoU.
Method
barr. ↑
bicy. ↑
bus ↑
car ↑
c.veh. ↑
moto. ↑
ped. ↑
cone ↑
trail. ↑
truck ↑
drive. ↑
sidew. ↑
terr. ↑
manm. ↑
veg. ↑
mIoU ↑
GaussianFlowOcc
5.44
2.55
9.05
14.11
1.42
6.37
7.04
6.68
0.42
9.79
57.64
28.81
31.83
11.94
9.88
13.53
Ours
6.24
3.67
12.99
14.25
0.84
6.88
7.77
7.59
0.31
9.05
58.26
26.92
31.25
11.92
9.45
13.82
TABLE II: Per-class IoU on Occ3D-nuScenes validation (camera-visible voxels), averaged over three seeds with separate-mode pseudo labels; same setting as Table I . Many per-class differences are smaller than the seed-to-seed variation. The first ten columns are foreground classes.
Method
FG/frame
FG dens.
BG dens.
Ratio
Baseline
132.6 ± 11.9
1.91 ± 0.17
1.18 ± 0.05
1.62 ± 0.21
Ours
228.8 ± 19.9
3.30 ± 0.29
1.38 ± 0.12
2.40 ± 0.11
TABLE III: Gaussian placement on all 150 validation scenes (separate-mode pseudo labels; mean ± sample standard deviation over three seeds). A Gaussian is assigned to the ground-truth voxel containing its center; Gaussians in free voxels are ignored. FG/frame: Gaussians in foreground voxels per frame. FG/BG dens.: Gaussians per m 3 of ground-truth foreground/background voxels. Ratio: FG/BG density (concentration).
Fig. 2: Gaussian placement on a validation frame (scene-0016, frame 8). Left: ground-truth occupancy (camera-visible voxels). Middle: GaussianFlowOcc. Right: ours. Each Gaussian is drawn at its center and scale and colored by its predicted class; background classes are shown in gray. Both models are trained with separate-mode pseudo labels and seed 0.
Variant
Pos.
Cls.
Detach
mIoU ↑
IoU ↑
No attraction
–
–
–
13.40
38.38
Pseudo (ours)
pseudo
pseudo
no
13.80
39.54
Pseudo + detach
pseudo
pseudo
yes
12.14
36.77
GT
GT
GT
no
12.91
37.80
GT + detach
GT
GT
yes
13.61
39.46
Mix
GT
pseudo
no
12.21
36.38
TABLE IV: Ablation of the attraction target and gradient path (single run per row, same seed, combine-mode pseudo labels; 150 validation scenes). Pos./Cls.: source of the target positions and of the target classes; Detach: stop-gradient through the class term of the assignment. IoU: occupancy IoU. Best in bold.
Fig. 3: Accuracy when only the B Gaussians with the highest opacity are kept at inference (no retraining). All 150 validation scenes; separate-mode models with seed 0.