AmodalBox: Automated 3D Bounding Box Annotation under Partial Visibility
Organizations: Seoul National University · University of Pittsburgh · POSTECH
Abstract
Recent progress in vision foundation models (VFMs) has made automated open-vocabulary 3D bounding box annotation increasingly feasible by jointly leveraging LiDAR point clouds and images. However, partial observations remain a major challenge in sparse driving scenes, where visible LiDAR points often cover only a small fraction of the object's surface and provide insufficient cues for estimating its full spatial extent. As a result, existing methods often produce mislocalized boxes or leave objects unlabeled. To address this limitation, we propose \Ours{}, an automated annotation framework that leverages image-generated 3D assets as amodal geometric priors for partially observed objects. By establishing correspondences between input images and rendered views of 3D assets, \Ours{} robustly aligns canonical amodal geometry with partially observed world-space LiDAR point clouds. To improve temporal consistency and reduce computational cost, \Ours{} generates a single asset per instance, estimates its metric scale once at a single frame and propagates it across frames for dynamic instances, and aggregates multi-frame observations for static instances. 3D object detectors trained on annotations from \Ours{} improve performance over strong baselines by 14.1 mAP on the nuScenes dataset and by 12.45 AP at IoU 0.5 for the vehicle class on the Waymo Open Dataset (WOD). Our project page is available at: https://oliver0922.github.io/AmodalBox/
Figures & tables
| Method | mAP | NDS | Car | Ped. | Truck | M.C. | Bicycle | Cone | Bus | Barr. | C.V. | Trail. |
| UP-VL [ 36 ] | 11.3 | 17.2 | 19.4 | 30.8 | 0 7.9 | 16.8 | 14.1 | 15.6 | 0 3.1 | 0 0.5 | 0 4.3 | 0 0.4 |
| Find n’ Prop. [ 12 ] | 16.7 | 22.4 | 24.3 | 22.8 | 0 8.6 | 35.8 | 34.7 | 21.3 | 11.1 | 0 4.4 | 0 4.1 | 0 0.1 |
| OpenSight [ 54 ] | 22.9 | 23.3 | 25.3 | 52.5 | 11.6 | 30.4 | 26.1 | 25.6 | 0 5.1 | 42.2 | 0 8.7 | 0 0.8 |
| AnnofreeOD [ 42 ] | 22.4 | 26.4 | 50.2 | 51.2 | 17.1 | 27.1 | 13.4 | 17.1 | 26.1 | 0 9.6 | 0 2.2 | 0 0.0 |
| OV-SCAN [ 7 ] | 31.1 | 32.8 | 61.6 | 60.1 | 30.3 | 39.8 | 31.0 | 39.6 | 22.0 | 18.8 | 0 6.8 | 0 0.6 |
| OV-SCAN † [ 7 ] | 28.7 | 31.4 | 62.2 | 59.1 | 29.8 | 29.5 | 26.3 | 34.6 | 19.9 | 17.3 | 0 7.1 | 0 0.7 |
| Method | Vehicle | Ped. | Cyclist |
| OpenBox [ 22 ] | 40.9 | 62.7 | 0 5.2 |
| AmodalBox | 59.5 | 46.7 | 25.1 |
| AmodalBox ‡ | 60.1 | 62.0 | 26.6 |
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
| Config | Voxel R-CNN [ 9 ] | TransFusion [ 3 ] |
| Optimizer | AdamW | AdamOneCycle |
| Base learning rate | ||
| Weight decay | ||
| Momentum | ||
| Momentum range | ||
| Learning rate decay |
| Hyperparameter | Value | Description |
| 0.05 | Lower quantile for robust source extent | |
| 0.95 | Upper quantile for robust source extent | |
| 1.0 | Strength of the class-size scale prior | |
| Inlier threshold | 0.3 m | RANSAC inlier distance threshold |
| Minimum inliers | 10 | Minimum inliers for accepting a hypothesis |
| Number of iterations | 1000 | Maximum LO-RANSAC iterations |
| Stage | Unit | nuScenes [ 5 ] | WOD [ 43 ] |
| 2D instance & motion cues ( Sec. 3.1 ) | frame | 4 h | 18 h |
| Asset generation & rendering | track | 11 h | 38 h |
| Matching, alignment & box fitting | track-frame | 35 h | 100 h |
| Total | split | 50 h | 156 h |
| Cross-view association & NMS | mAP | NDS | Box/frame |
| Off | 29.2 | 29.6 | 18.7 |
| On (AmodalBox) | 31.4 | 31.1 | 13.8 |
| Method | mAP | |||||||||||
| 0-10m | 10-20m | 20-30m | 30-40m | |||||||||
| Low | Mid | High | Low | Mid | High | Low | Mid | High | Low | Mid | High | |
| OV-SCAN [ 7 ] | 2.18 | 4.75 | 54.15 | 3.05 | 11.67 | 43.04 | 2.88 | 18.05 | 27.46 | 2.63 | 15.41 | 10.45 |
| AmodalBox | 7.37 | 11.27 | 64.92 | 7.27 | 22.77 | 60.48 | 6.55 | 32.20 | 46.77 | 4.17 | 25.87 | 23.81 |
| Method | NDS | |||||||||||
| 0-10m | 10-20m | 20-30m | 30-40m | |||||||||
3D Object Detection
3D object detection identifies objects and estimates their locations, sizes, and orientations from images, point clouds, radar, or fused sensors.
See all 195 papers