OASIC: Occlusion-Agnostic and Severity-Informed Classification

Kay Gijzen; Gertjan J. Burghouts; Daniël M. Pelt

cs.CVApr 5, 2026

OASIC: Occlusion-Agnostic and Severity-Informed Classification

Authors: Kay Gijzen, Gertjan J. Burghouts, Daniël M. Pelt

Organizations: Leiden University · TNO

Abstract

Severe occlusions of objects pose a major challenge for computer vision. We show that two root causes are (1) the loss of visible information and (2) the distracting patterns caused by the occluders. Our approach addresses both causes at the same time. First, the distracting patterns are removed at test-time, via masking of the occluding patterns. This masking is independent of the type of occlusion, by handling the occlusion through the lens of visual anomalies w.r.t. the object of interest. Second, to deal with less visual details, we follow standard practice by masking random parts of the object during training, for various degrees of occlusions. We discover that (a) it is possible to estimate the degree of the occlusion (i.e. severity) at test-time, and (b) that a model optimized for a specific degree of occlusion also performs best on a similar degree during test-time. Combining these two insights brings us to a severity-informed classification model called OASIC: Occlusion Agnostic Severity Informed Classification. We estimate the severity of occlusion for a test image, mask the occluder, and select the model that is optimized for the degree of occlusion. This strategy performs better than any single model optimized for any smaller or broader range of occlusion severities. Experiments show that combining gray masking with adaptive model selection improves $\text{AUC}_\text{occ}$ by +18.5 over standard training on occluded images and +23.7 over finetuning on unoccluded images.

Figures & tables

Figure 1: Occlusion handling by OASIC. At test time, an occlusion map is inferred by scoring against the memory bank $\mathcal{M}$ . The segmented mask is turned into gray to suppress distraction, while the estimated occlusion severity informs the selection of the most suitable classification model $f^{\ast}$ from the pool $\mathcal{F}$ , to better handle reduced visual information.

Panel of Figure 2 — Figure 2: Comparison of an occluded image and its masked versions at different thresholds $\tau$ . From left to right: the original occluded image, and masks applied with thresholds 0.3, 0.5, and 0.7.

Figure 3: Larger degrees of occlusions (severity) deteriorate the performance of finegrained classification, trained without occlusions. Textured occlusions (vegetation, rubble) are more problematic than dull occlusion (smoke) or gray occlusion.

Figure 4: Textured occluders draw away the attention from the object. The first column shows the original (unoccluded) images, and the second column displays their corresponding saliency maps. The occlusion mask applied to each row is shown in column 3 and remains the same across all occlusion types. Columns 4–6 present the occluded images: gray, vegetation and rubble. Each is overlaid with its respective attention map.

Method	Vegetation		Smoke		Rubble
Method	mAUROC	mAP	mAUROC	mAP	mAUROC	mAP
	20% Occluded
AnomalyDINO	95.64 $\pm$ 1.51	79.35 $\pm$ 7.46	94.46 $\pm$ 2.33	76.59 $\pm$ 8.09	94.40 $\pm$ 2.34	74.56 $\pm$ 8.54
OVSeg	80.38 $\pm$ 9.09	63.25 $\pm$ 15.26	49.75 $\pm$ 5.55	24.16 $\pm$ 8.38	51.24 $\pm$ 6.30	25.54 $\pm$ 9.05
	40% Occluded
AnomalyDINO	93.02 $\pm$ 3.33	88.02 $\pm$ 5.76	91.76 $\pm$ 3.51	85.52 $\pm$ 6.24	90.31 $\pm$ 4.47	82.21 $\pm$ 7.63

Table 1: Across various types of occlusions, the occlusion-agnostic AnomalyDINO performs better than OVSeg when prompted with the occlusion type. Mean AUROC (mAUROC) and mean Average Precision (mAP) is reported across occlusion severities (in rows) for occlusion types (vegetation, smoke, rubble; in columns).

Figure 5: OASIC is much more robust to severe occlusions, improving 5x compared to standard training. Both the gray-masking of occlusion (to suppress distraction) and the severity-informed model selection (to handle limited visibility) contribute to OASIC’s performance.

Explore similar work

Occlusion-Aware Panoptic Segmentation with Joint Position Embedding and Occlusion-Level Attention
Jul 20, 2026Wenbo Wei, Jun Wang, Shan Raza +1Panoptic Segmentation Occlusion Reasoning
Department of Computer Science, University of Warwick, Coventry, UK
OccludeBench: Can VLMs Find What's Missing?
Oct 9, 2026Jasin Cekinmez, Addison J. Wu, Ryo Mitsuhashi +3Occlusion Reasoning VLM Reasoning
Princeton University
Mitigating Objectness Bias and Region-to-Text Misalignment for Open-Vocabulary Panoptic Segmentation
Mar 22, 2026Nikolay Kormushev, Josip Šarić, Matej KristanPanoptic Segmentation Vision-Language Alignment
University of Ljubljana Faculty of Comp. and Inf. Science · ETH Zurich Dept. of Computer Science · University of Zagreb Faculty of Elec. Eng. and Computing

Jul 20, 2026J/K move · Enter open · S save

Occlusion-Aware Panoptic Segmentation with Joint Position Embedding and Occlusion-Level Attention

Wenbo Wei, Jun Wang, Shan Raza, Abhir Bhalerao

Department of Computer Science, University of Warwick, Coventry, UK

Panoptic Segmentation Occlusion Reasoning

Open paper AskPDF arXiv

Panoptic segmentation in complex scenes remains challenging because of occlusions, yet modern approaches often neglect occlusion modelling. In this paper, we propose Position Embedding Modulation with Occlusion Level Attention (PEMOLA), a novel occlusion-aware module that can be seamlessly integrated into transformer-based panoptic segmentation. To obtain occlusion cues, we train an occlusion classifier on the COCO-OLAC dataset. The classifier derives the occlusion-level attention, which serves as spatial guidance, while the occlusion labels are encoded into a learnable embedding to produce channel-wise weights. Through joint modulation, PEMOLA elegantly introduces the occlusion priors into the position embedding, thereby improving the occlusion modelling. We further annotate the Cityscapes dataset with occlusion levels, termed Cityscapes Occlusion Labels for All Computer Vision Tasks (Cityscapes-OLAC), following the same labelling protocol as COCO-OLAC, to evaluate the cross-dataset generalisation ability of PEMOLA. Extensive experiments on COCO-OLAC and Cityscapes-OLAC demonstrate that PEMOLA consistently improves panoptic segmentation quality while introducing minimal computational overhead. These results highlight the importance of occlusion modelling, where incorporating occlusion-level attention helps deliver robust panoptic segmentation under occlusion. Code and dataset are available at https://github.com/wenbo-wei/PEMOLA.

Abstract

Explore similar work

Occlusion-Aware Panoptic Segmentation with Joint Position Embedding and Occlusion-Level Attention

OccludeBench: Can VLMs Find What's Missing?

Mitigating Objectness Bias and Region-to-Text Misalignment for Open-Vocabulary Panoptic Segmentation