Knowing the Unknown: Interpretable Open-World Object Detection via Concept Decomposition Model

Xueqiang Lv; Shizhou Zhang; Yinghui Xing; Di Xu; Peng Wang; Yanning Zhang

cs.CVFeb 24, 2026

Knowing the Unknown: Interpretable Open-World Object Detection via Concept Decomposition Model

Authors: Xueqiang Lv, Shizhou Zhang, Yinghui Xing, Di Xu, Peng Wang, Yanning Zhang

Organizations: School of Computer Science, Northwestern Polytechnical Unviersity, Xi’an, China · Huawei Technologies Ltd.

Abstract

Open-world object detection (OWOD) requires incrementally detecting known categories while reliably identifying unknown objects. Existing methods primarily focus on improving unknown recall, yet overlook interpretability, often leading to known-unknown confusion and reduced prediction reliability. This paper aims to make the entire OWOD framework interpretable, enabling the detector to truly "knowing the unknown". To this end, we propose a concept-driven InterPretable OWOD framework(IPOW) by introducing a Concept Decomposition Model (CDM) for OWOD, which explicitly decomposes the coupled RoI features in Faster R-CNN into discriminative, shared, and background concepts. Discriminative concepts identify the most discriminative features to enlarge the distances between known categories, while shared and background concepts, due to their strong generalization ability, can be readily transferred to detect unknown categories. Leveraging the interpretable framework, we identify that known-unknown confusion arises when unknown objects fall into the discriminative space of known classes. To address this, we propose Concept-Guided Rectification (CGR) to further resolve such confusion. Extensive experiments show that IPOW significantly improves unknown recall while mitigating confusion, and provides concept-level interpretability for both known and unknown predictions.

Figures & tables

Figure 1: Left: The proposed IPOW reformulates OWOD as a problem of concept decomposition modeling, consisting of discriminative concepts responsible for known-class recognition, along with shared and background concepts that support unknown object detection. Right: Within the IPOW framework, known–unknown confusion can be viewed as unknown objects falling into the discriminative space learned for known classes.

Figure 2: Overview of the IPOW framework. GMM-RPN is utilized to mitigate the proposal generation bias towards known-categories. A concept head is adopted before decomposition. Each RoI feature is decomposed into discriminative, shared, and background concepts for known object recognition, transferable unknown discovery, and contextual modeling.

Figure 3: Score distributions on M-OWODB Task 1 for known, confusion (unknown falsely predicted as known), and unknown samples. Discriminative concept activations (left) and the geometric mean of class-specific shared concept activations (right).

Figure 4: Ablation study on the number of shared concepts on M-OWODB.

K

and

M

denote the numbers of LLM-derived shared concepts and residual shared concepts, respectively.

Figure 5: Qualitative results contrasting IPOW with RandBox, and CROWD, demonstrating that IPOW offers interpretable concept-level reasoning, reduces known–unknown confusion, and enables effective generalization to unknown objects.

Figure 6: Qualitative results comparing IPOW with OrthogonalDet on M-OWODB, S-OWODB, and the DIOR dataset, showing that IPOW provides interpretable concept-level reasoning and more effectively reduces known–unknown confusion across diverse scenarios.

10 + 10 setting	aero	cycle	bird	boat	bottle	bus	car	cat	chair	cow	table	dog	horse	bike	person	plant	sheep	sofa	train	tv	mAP
ILOD ( Shmelkov et al., 2017 )	69.9	70.4	69.4	54.3	48	68.7	78.9	68.4	45.5	58.1	59.7	72.7	73.5	73.2	66.3	29.5	63.4	61.6	69.3	62.2	63.2
Faster ILOD ( Peng et al., 2020 )	72.8	75.7	71.2	60.5	61.7	70.4	83.3	76.6	53.1	72.3	36.7	70.9	66.8	67.6	66.1	24.7	63.1	48.1	57.1	43.6	62.1
ORE ( Joseph et al., 2021a )	63.5	70.9	58.9	42.9	34.1	76.2	80.7	76.3	34.1	66.1	56.1	70.4	80.2	72.3	81.8	42.7	71.6	68.1	77.0	67.7	64.5
Meta-ILOD ( Joseph et al., 2021b )	76.0	74.6	67.5	55.9	57.6	75.1	85.4	77.0	43.7	70.8	60.1	66.4	76.0	72.6	74.6	39.7	64.0	60.2	68.5	60.7	66.3
ROSETTA ( Yang et al., 2022 )	74.2	76.2	64.9	54.4	57.4	76.1	84.4	68.8	52.4	67.0	62.9	63.3	79.8	72.8	78.1	40.1	62.3	61.2	72.4	66.8	66.8
OW-DETR ( Gupta et al., 2022 )	61.8	69.1	67.8	45.8	47.3	78.3	78.4	78.6	36.2	71.5	57.5	75.3	76.2	77.4	79.5	40.1	66.8	66.3	75.6	64.1	65.7

Table 5: Comparison with state-of-the-art methods in single-step incremental object detection settings on PASCAL VOC 2007. Best results are shown in bold , while newly introduced classes in each task are shaded in gray. Our method consistently outperforms existing approaches across all settings.

Explainable Object Detection

See all 8 papers

Detecting Unknown Objects via Energy-based Separation for Open World Object Detection
Mar 31, 2026Jun-Woo Heo, Keonhee Park, Gyeong-Moon ParkContinual Object Detection Object Detection
Korea University, South Korea · Seoul National University, South Korea
Towards Sparsely Annotated Open-World Object Detection
Aug 13, 2026HeeJu Han, AJeong Kim, Jinsun ParkOpen-Vocabulary Object Detection Object Detection
Pusan National University, Republic of Korea
Multimodal Semantic-Probabilistic Objectness for Open World Object Detection
Jul 27, 2026Weijun Tian, Rui LiuObject Detection Multimodal Object Detection
School of Computer Science and Engineering, Beihang University

Mar 31, 2026J/K move · Enter open · S save

Detecting Unknown Objects via Energy-based Separation for Open World Object Detection

Jun-Woo Heo, Keonhee Park, Gyeong-Moon Park

Korea University, South Korea · Seoul National University, South Korea

Continual Object Detection Object Detection Open-Vocabulary Object Detection OOD Detection

Open paper AskPDF arXiv

In this work, we tackle the problem of Open World Object Detection (OWOD). This challenging scenario requires the detector to incrementally learn to classify known objects without forgetting while identifying unknown objects without supervision. Previous OWOD methods have enhanced the unknown discovery process and employed memory replay to mitigate catastrophic forgetting. However, since existing methods heavily rely on the detector's known class predictions for detecting unknown objects, they struggle to effectively learn and recognize unknown object representations. Moreover, while memory replay mitigates forgetting of old classes, it often sacrifices the knowledge of newly learned classes. To resolve these limitations, we propose DEUS (Detecting Unknowns via energy-based Separation), a novel framework that addresses the challenges of Open World Object Detection. DEUS consists of Equiangular Tight Frame (ETF)-Subspace Unknown Separation (EUS) and an Energy-based Known Distinction (EKD) loss. EUS leverages ETF-based geometric properties to create orthogonal subspaces, enabling cleaner separation between known and unknown object representations. Unlike prior energy-based approaches that consider only the known space, EUS utilizes energies from both spaces to better capture distinct patterns of unknown objects. Furthermore, EKD loss enforces the separation between previous and current classifiers, thus minimizing knowledge interference between previous and newly learned classes during memory replay. We thoroughly validate DEUS on OWOD benchmarks, demonstrating outstanding performance improvements in unknown detection while maintaining competitive known class performance.

Abstract

Explainable Object Detection

Explore similar work

Detecting Unknown Objects via Energy-based Separation for Open World Object Detection

Towards Sparsely Annotated Open-World Object Detection

Multimodal Semantic-Probabilistic Objectness for Open World Object Detection