Class-Distribution Guided Active Learning for 3D Occupancy Prediction in Autonomous Driving
Authors: Wonjune Kim, In-Jae Lee, Sihwan Hwang, Sanmin Kim, Dongsuk Kum
Organizations: Robotics Program, KAIST, Daejeon 34141, South Korea · Interdisciplinary Program in Artificial Intelligence, Seoul National University, Seoul 08826, South Korea · Graduate School of Mobility, KAIST, Daejeon 34051, South Korea · Department of Automobile and IT Convergence, Kookmin University, Seoul 02707, South Korea
3D occupancy prediction provides dense spatial understanding critical for safe autonomous driving. However, this task suffers from a severe class imbalance due to its volumetric representation, where safety-critical objects (bicycles, traffic cones, pedestrians) occupy minimal voxels compared to dominant backgrounds. Additionally, voxel-level annotation is costly, yet dedicating effort to dominant classes is inefficient. To address these challenges, we propose a class-distribution guided active learning framework for selecting training samples to annotate in autonomous driving datasets. Our approach combines three complementary criteria to select the training samples. Inter-sample diversity prioritizes samples whose predicted class distributions differ from those of the labeled set, intra-set diversity prevents redundant sampling within each acquisition cycle, and frequency-weighted uncertainty emphasizes rare classes by reweighting voxel-level entropy with inverse per-sample class proportions. We ensure evaluation validity by using a geographically disjoint train/validation split of Occ3D-nuScenes, which reduces train-validation overlap and mitigates potential map memorization. With only 42.4% labeled data, our framework reaches 26.62 mIoU, comparable to full supervision and outperforming active learning baselines at the same budget. We further validate generality on SemanticKITTI using a different architecture, demonstrating consistent effectiveness across datasets.
Figures & tables
Fig. 1: Overview of the proposed class-distribution guided acquisition strategy based on predictive uncertainty and class-distribution diversity. Unlabeled samples receive an uncertainty score and a diversity score computed from similarity to the current selection and to the labeled set. The method selects samples that are uncertain and dissimilar to both sets and rejects redundant or certain samples. Sample 1 is selected because uncertainty is high and similarity to both sets is low. Sample 2 is rejected because it is similar to the selected Sample 1, which lowers intra-set diversity. Sample 3 is rejected because it is similar to the labeled reference Sample 4 and its uncertainty is low. Sample 4 is the labeled anchor used for similarity assessment. The bars show predicted class distributions.
Fig. 2: Overview of our active learning method for 3D occupancy prediction. An initial labeled set trains the model, which then performs inference on the unlabeled pool. In each iteration, candidate samples are scored using three complementary metrics: frequency-weighted uncertainty, inter-sample diversity relative to the labeled set, and intra-set diversity within the current selected samples. Scores are normalized and combined into a unified acquisition score. Top-ranked samples are selected for annotation and added to the labeled subset. The process repeats for T cycles, each time updating the labeled set and retraining the model.
Cycle ( N samples, %)
AL Method
mIoU
■ others
■ barrier
■ bicycle
■ bus
■ car
■ const. veh.
■ motorcycle
■ pedestrian
■ traffic cone
■ trailer
■ truck
■ drive. surf.
■ other flat
■ sidewalk
■ terrain
■ manmade
■ vegetation
Cycle 1 (2,000, 8.5%)
Random
18.33
0.25
30.48
0.00
23.79
37.11
0.15
0.00
6.53
0.00
11.61
23.93
68.64
8.85
30.47
31.99
21.20
16.52
Cycle 2 (4,000, 17.0%)
Random
23.35
1.31
36.37
0.92
33.18
40.03
10.43
3.95
14.52
10.12
26.03
28.42
70.54
11.82
33.03
34.60
22.31
19.33
Entropy
22.70
2.01
35.08
0.33
32.24
39.04
7.94
7.98
13.79
7.89
23.61
27.57
70.13
10.87
32.48
34.25
22.23
18.49
Coreset [ 13 ]
22.69
2.02
35.77
0.04
31.26
39.70
10.77
1.40
12.96
8.90
23.11
28.25
70.37
11.45
33.28
34.68
22.55
19.20
Ours
24.17
2.75
35.72
0.81
34.04
40.15
16.84
5.00
14.51
12.78
25.94
30.21
70.32
12.43
33.85
34.22
22.40
18.99
Cycle 3 (6,000, 25.4%)
Random
24.40
3.38
35.88
0.89
32.87
40.21
13.64
8.84
14.45
12.04
27.01
28.83
71.02
12.50
34.26
35.43
23.20
20.30
TABLE I: Performance Comparison of Active Learning Methods on the Occ3D-nuScenes validation set [ 4 ] , constructed using a geographically disjoint split (train/val non-overlapping) following the protocol of Lilja et al. [ 17 ] . ‘Const. Veh.’ denotes Construction Vehicles, while ‘Drive. Surf.’ represents Drivable Surfaces.
AL Method
Cycle 1 (7.8%)
Cycle 2 (15.6%)
Cycle 3 (23.4%)
Cycle 4 (31.2%)
Cycle 5 (39.0%)
Fully Sup.
Random
9.66
10.13
10.50
10.69
11.17
13.46
Entropy
10.32
10.81
11.24
11.35
Coreset [ 13 ]
10.74
11.25
11.74
11.77
Ours
10.79
11.82
12.23
12.51
TABLE II: Cycle-wise Semantic Scene Completion (SSC) mIoU on the SemanticKITTI [ 8 ] validation set using OccFormer [ 6 ] . Cumulative labeled ratios at each cycle are shown in parentheses.
Fig. 3: Qualitative comparison of active learning strategies for 3D occupancy prediction. Each example shows six surround-view images and the ground truth occupancy grid, followed by model outputs when samples are chosen by random, entropy, Coreset [ 13 ] , and our active learning methods. Red dashed boxes mark regions where our approach more accurately predicts occupancy than the baselines. This improvement comes from our sample selection strategy, which emphasizes rare-class uncertainty and diversity in sample-level class distributions.
AL Method
Night ( mIoU / #samples )
Rainy ( mIoU / #samples )
C2
C3
C4
C5
C2
C3
C4
C5
Random
13.54
14.31
14.54
14.23
24.96
26.33
27.55
26.88
229
214
215
237
355
406
402
339
Entropy
14.74
14.48
14.26
14.37
24.44
25.96
26.64
27.13
795
525
149
275
464
337
374
407
Coreset [ 13 ]
13.10
13.76
13.91
14.07
24.65
26.38
26.66
27.41
TABLE III: Cycle-wise mIoU on the Occ3D-nuScenes geographically disjoint validation split, evaluated on the Night and Rainy subsets. C2–C5 correspond to Cycle 2–5. Values show mIoU (top) and number of selected samples ( #samples , bottom). Best is in bold .
Fig. 4: Frequency-weighted uncertainty visualization. Two representative training samples are shown. Top: A high-uncertainty sample with diverse classes, including rare objects. Bottom: A low-uncertainty sample dominated by common classes. Frequency weighting highlights rare-class regions and suppresses dominant backgrounds. Dashed boxes indicate zoomed-in regions.
Ufw
Dinter
Dintra
mIoU (%)
Cycle 2
Cycle 3
Cycle 4
Cycle 5
23.35
24.40
25.84
25.67
✓
23.55
24.93
26.00
26.15
✓
23.54
24.65
25.99
25.96
✓
23.42
24.70
25.96
26.08
✓
✓
23.48
24.82
25.98
26.10
TABLE IV: Ablation study of frequency-weighted uncertainty ( Ufw ), inter-sample diversity ( Dinter ), and intra-set diversity ( Dintra ) across active learning cycles on the Occ3D-nuScenes geographically disjoint split validation set.
Init. Size
mIoU NL
Cycle 1
Cycle 2
Cycle 3
Cycle 4
Cycle 5
2,000
18.33 2k
24.17 4k
25.57 6k
26.38 8k
26.62 10k
4,000
20.12 4k
23.77 6k
25.23 8k
26.19 10k
26.63 12k
6,000
22.94 6k
25.22 8k
25.63 10k
26.18 12k
26.60 14k
8,000
24.36 8k
25.81 10k
26.12 12k
26.40 14k
26.66 16k
Fully sup.
26.83 23.5k
TABLE V: Cycle-wise mIoU (%) for different initial labeled set sizes on the Occ3D-nuScenes geographically disjoint split validation set (FlashOCC). Gray superscripts denote cumulative labeled samples ( {\color[rgb]{0.5,0.5,0.5}N_{L}} ), where {\color[rgb]{0.5,0.5,0.5}k} denotes thousands.
3D semantic occupancy prediction is central to autonomous driving, yet current methods are vulnerable to long-tailed class bias and out-of-distribution (OOD) inputs, often overconfidently assigning anomalies to rare classes. We present ProOOD, a lightweight, plug-and-play method that couples prototype-guided refinement with training-free OOD scoring. ProOOD comprises (i) prototype-guided semantic imputation that fills occluded regions with class-consistent features, (ii) prototype-guided tail mining that strengthens rare-class representations to curb OOD absorption, and (iii) EchoOOD, which fuses local logit coherence with local and global prototype matching to produce reliable voxel-level OOD scores. Extensive experiments on five datasets demonstrate that ProOOD achieves state-of-the-art performance on both in-distribution 3D occupancy prediction and OOD detection. On SemanticKITTI, it surpasses baselines by +3.57% mIoU overall and +24.80% tail-class mIoU; on VAA-KITTI, it improves AuPRCr by +19.34 points, with consistent gains across benchmarks. These improvements yield more calibrated occupancy estimates and more reliable OOD detection in safety-critical urban driving. The source code is publicly available at https://github.com/7uHeng/ProOOD.
Yuheng Zhang, Mengfei Duan, Kunyu Peng +5
Hunan University · Karlsruhe Institute of Technology · INSAIT, Sofia University “St. Kliment Ohridski”
3D semantic occupancy prediction is crucial for autonomous driving perception, offering comprehensive geometric scene understanding and semantic recognition. However, existing methods struggle with geometric misalignment in view transformation due to the lack of pixel-level accurate depth estimation, and severe spatial class imbalance where semantic categories exhibit strong spatial anisotropy. To address these challenges, we propose Dr. Occ, a depth- and region-guided occupancy prediction framework. Specifically, we introduce a depth-guided 2D-to-3D View Transformer (D2-VFormer) that effectively leverages high-quality dense depth cues from MoGe-2 to construct reliable geometric priors, thereby enabling precise geometric alignment of voxel features. Moreover, inspired by the Mixture-of-Experts (MoE) framework, we propose a region-guided Expert Transformer (R/R2-EFormer) that adaptively allocates region-specific experts to focus on different spatial regions, effectively addressing spatial semantic variations. Thus, the two components make complementary contributions: depth guidance ensures geometric alignment, while region experts enhance semantic learning. Experiments on the Occ3D--nuScenes benchmark demonstrate that Dr. Occ improves the strong baseline BEVDet4D by 7.43% mIoU and 3.09% IoU under the full vision-only setting.
Xubo Zhu, Haoyang Zhang, Fei He +4
School of Electronic Information, Wuhan University · Horizon Robotics · Wuhan University Shenzhen Research Institute
Panoptic occupancy prediction aims to jointly infer voxel-wise semantics and instance identities within a unified 3D scene representation. Nevertheless, progress in this field remains constrained by the absence of high-quality 3D mesh resources, instance-level annotations, and physically consistent occupancy datasets. Existing benchmarks typically provide incomplete and low-resolution geometry without instance-level annotations, limiting the development of models capable of achieving precise geometric reconstruction, reliable occlusion reasoning, and holistic 3D understanding. To address these challenges, this paper presents an instance-centric benchmark for the 3D panoptic occupancy prediction task. Specifically, we introduce ADMesh, the first unified 3D mesh library tailored for autonomous driving, which integrates over 15K high-quality 3D models with diverse textures and rich semantic annotations. Building upon ADMesh, we further construct CarlaOcc, a large-scale, physically consistent panoptic occupancy dataset generated using the CARLA simulator. This dataset contains over 100K frames with fine-grained, instance-level occupancy ground truth at voxel resolutions as fine as 0.05 m. Furthermore, standardized evaluation metrics are introduced to quantify the quality of existing occupancy datasets. Finally, a systematic benchmark of representative models is established on the proposed dataset, which provides a unified platform for fair comparison and reproducible research in the field of 3D panoptic perception. Code and dataset are available at https://mias.group/CarlaOcc.