Occlusion for Free: Detecting Occlusion from the Disagreement Between Two Monocular Distance Cues
Organizations: University of Lagos · ML Collective · Federal University of Technology, Minna
Abstract
Knowing that a pedestrian is occluded is useful to a robot even when the detector cannot recover the hidden pixels, but occlusion estimators are normally trained on occlusion annotations that deployment settings rarely have. We show that occlusion can be read off two distance estimates a camera-only pipeline already computes. Apparent height is largely unchanged by occlusion, because the detected box still spans the whole person; monocular metric depth is pulled toward the occluder, because the nearest surface inside the box is what a depth map reports. Only their ordering matters, so no focal length, stature, or depth scale is needed, provided the depth model's error is close to a pure gain, a condition that can be checked without labels. The score detects heavy occlusion on KITTI with AUC 0.909 (0.888 over all annotated pedestrians, including those the detector misses), requires no training and no occlusion labels, and is not a proxy for range: within fixed distance bands it holds AUC 0.894-0.925 while true distance alone sits near chance. On synthetic CARLA scenes with measured visibility it reaches 0.819, against 0.644 for distance alone. We arrived here after measuring three standard architectural responses to occlusion, depth fusion, occlusion-simulating augmentation, and attention, none of which improved detection outside its training distribution in the configurations we tested, and after finding that a detector trained on German roads loses half its accuracy on self-annotated street scenes in Lagos, Nigeria. Signals that need no local annotation are worth more in settings that benchmarks do not represent.
Figures & tables
| Distance band | vis/occ (peds) | AUC of score [95% CI] | AUC of distance |
|---|---|---|---|
| 5–10 m | 409/26 (137/10) | 0.917 [0.843, 0.971] | 0.594 |
| 10–20 m | 466/140 (156/55) | 0.894 [0.840, 0.939] | 0.457 |
| 20 m | 318/100 (112/41) | 0.925 [0.874, 0.964] | 0.469 |
| all | 1237/269 (420/107) | 0.909 [0.876, 0.938] | 0.624 |
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
| Target | Band | vis/occ (peds) | AUC of score [95% CI] | AUC of distance |
|---|---|---|---|---|
| Any (1–3) | 5–10 m | 409/124 (137/44) | 0.773 [0.676, 0.863] | 0.549 |
| Any (1–3) | 10–20 m | 466/297 (156/109) | 0.807 [0.750, 0.860] | 0.471 |
| Any (1–3) | 20 m | 318/267 (112/102) | 0.874 [0.828, 0.919] | 0.513 |
| Any (1–3) | all | 1237/704 (420/262) | 0.826 [0.792, 0.857] | 0.600 |
| Partly (1) | 5–10 m | 409/98 (137/34) | 0.735 [0.615, 0.840] | 0.538 |
| Partly (1) | 10–20 m | 466/157 (156/54) | 0.728 [0.648, 0.809] | 0.485 |
| Read-out | Calib. | AUC |
|---|---|---|
| Ranking, | none | 0.909 |
| Signed, | 0.908 | |
| Absolute difference | 0.872 | |
| Absolute, gain 25% high | 0.470 |
| Depth source | any | heavy |
|---|---|---|
| MiDaS, as-is | 0.572 | 0.596 |
| MiDaS, inverted | 0.674 | 0.798 |
| DAv2, metric | 0.826 | 0.909 |
| Variant | Heavy AUC [95% CI] |
|---|---|
| Paper score, in predicted box | 0.909 [0.876, 0.938] |
| annotated, predicted box | 0.899 [0.866, 0.928] |
| predicted, annotated box | 0.902 [0.868, 0.931] |
| Annotated boxes only, matched pedestrians | 0.904 [0.870, 0.931] |
| Annotated boxes, all 709 pedestrians | 0.888 [0.852, 0.921] |
| 1st / 5th / 10th / 25th percentile instead of min | 0.911 / 0.919 / 0.921 / 0.910 |
| Intervention | Setting | mAP50 | Hard mAP50 |
|---|---|---|---|
| Occlusion augmentation (GridMask) | KITTI, YOLOv8x | ||
| Occlusion augmentation (H&S) | CityPersons, YOLOv8x | ||
| Depth fusion (MiDaS) | KITTI, YOLOv8x | ||
| Depth fusion (MiDaS) † | KITTI, YOLOv8n | ||
| Depth fusion (DAv2) † | KITTI, YOLOv8n | ||
| Attention (POPAM) † | KITTI, YOLOv8n |
| Trained on | Configuration | In-dist. mAP50 | Lagos mAP50 |
|---|---|---|---|
| KITTI | Baseline | 0.8809 | 0.4482 0.0094 |
| KITTI | GridMask | 0.8903 | 0.4150 0.0530 |
| CityPersons | Baseline | 0.7154 | 0.7356 0.0351 |
| CityPersons | Hide-and-Seek | 0.7166 | 0.7075 0.0198 |
| Distance band | AUC of score [95% CI] | AUC of distance | |
|---|---|---|---|
| 0–5 m | 154 | 0.969 [0.939, 0.989] | 0.506 |
| 5–10 m | 668 | 0.895 [0.868, 0.919] | 0.585 |
| 10–15 m | 687 | 0.839 [0.804, 0.871] | 0.562 |
| 15–25 m | 1277 | 0.797 [0.771, 0.821] | 0.509 |
| 25–60 m | 507 | 0.739 [0.693, 0.783] | 0.481 |
| all | 3293 | 0.819 [0.801, 0.834] | 0.644 |
| Depth input | Detection | Depth | Score | Total | FPS |
|---|---|---|---|---|---|
| 518 1722 (default) | 77 | 6233 | 0.3 | 6336 | 0.16 |
| 294 980 | 66 | 1229 | 0.3 | 1314 | 0.76 |
Autonomous Driving Perception
Autonomous driving perception enables vehicles to detect and interpret road users, drivable space, and hazards using onboard and shared sensors.
See all 411 papers