Open World Object Detection (OWOD) built on multimodal foundation models often suffers from semantic ambiguity caused by unidirectional text-to-vision matching, while rigid outlier penalties may over-suppress unknown objects near known-class decision boundaries. We propose CODE (Cross-Modal Calibration and Dynamic Suppression), a unified inference-time framework with three complementary components. Cross-Modal Joint Confidence Calibration injects global visual prototypes to calibrate text-driven known-class predictions. Uncertainty-Guided Universal Objectness Enhancement measures classification hesitation from local visual responses to strengthen potential unknown objects. Dynamic Outlier Suppression via Confidence Margin replaces rigid suppression with a margin-aware adjustment that preserves ambiguous out-of-distribution instances. Experiments on the Real-World Detection benchmark demonstrate that, with the OWL-ViT L/14 backbone, CODE achieves 21.7 U-mAP and 40.8 K-mAP in Task 1, surpassing the previous state of the art by 2.6 and 2.3 points, respectively.
Figures & tables
Figure 1. Paradigm comparison between attribute-based OWOD methods and CODE. Existing methods derive text-driven probabilities followed by fixed outlier suppression, whereas CODE introduces known-class visual references for cross-modal calibration and dynamic suppression. A paradigm comparison between existing attribute-driven open-world object detection methods and CODE. The existing method independently encodes the test image and attribute text, computes a text-driven probability, and applies fixed outlier suppression. CODE additionally encodes known-category visual data, computes visual similarities for cross-modal probability calibration, and applies dynamic suppression before producing the final prediction.
Figure 2. Inference architecture of CODE. CMJCC calibrates known logits with cached visual prototypes, UGUOE enhances unknown logits from visual uncertainty, and DOSCM converts confidence margins into dynamic OOD scores. The inference architecture of CODE. An input image is encoded into candidate visual embeddings, while optimized textual attributes are encoded and mapped to initial known and unknown logits. The CMJCC branch compares candidate embeddings with pre-computed known-class visual prototypes to calibrate known logits. The UGUOE branch combines visual-response uncertainty with universal objectness to enhance unknown logits. The DOSCM branch uses the confidence margin of calibrated known predictions to compute a dynamic out-of-distribution score and produce the final known and unknown probabilities.
Dataset ( → )
Aquatic
Aerial
Game
Medical
Surgery
Overall
Task ID ( → )
Task 1
Task 2
Task 1
Task 2
Task 1
Task 2
Task 1
Task 2
Task 1
Task 2
Task 1
Task 2
U
K
PK
CK
U
K
PK
CK
U
K
PK
CK
U
K
PK
CK
U
K
PK
CK
U
K
PK
CK
B/16 Backbone:
BASE-ZS+GT †
29.8
45.0
45.0
36.7
1.3
5.7
5.7
1.4
15.0
0.4
0.4
0.1
0.5
0.0
0.0
0.1
5.6
1.5
1.4
0.3
10.4
10.5
10.5
7.7
BASE-ZS
6.2
45.0
45.0
36.7
0.9
5.7
5.7
1.4
15.7
0.4
0.4
0.1
0.2
0.0
0.0
0.1
1.4
1.5
1.4
0.3
4.9
10.5
10.5
7.7
BASE-ZS+IN
26.5
45.1
45.1
36.7
1.9
5.7
5.7
1.4
2.4
0.3
0.3
0.0
0.6
0.0
0.0
0.1
1.7
1.4
1.0
0.3
6.6
10.5
10.4
7.7
Table 1. OWOD results on the five RWD datasets. We report U- and K-mAP in Task 1 and PK- and CK-mAP in Task 2. Results of previous methods are taken from their original papers, while CODE uses our final unified configuration. The Δ rows report CODE minus PASS; positive values indicate improvements. † GT baselines use ground-truth class names to detect unknown objects and serve as an open-vocabulary upper reference.
Figure 3. Qualitative comparison on five RWD datasets. Rows show ground truth, PASS, and CODE; blue and yellow boxes denote known and unknown objects, respectively. Columns show Aquatic and Surgery (Task 1), and Aerial, Medical, and Game (Task 2). A grid of 15 images arranged in 3 rows and 5 columns comparing object detection results. The columns represent five datasets: Aquatic, Surgery, Aerial, Medical, and Game. The top row shows the original images with ground truth bounding boxes. The middle row displays predictions from the PASS baseline, which visibly misses some objects or misclassifies them. The bottom row displays results from the proposed CODE framework, demonstrating bounding boxes that align much more accurately with the ground truth. Blue boxes indicate known objects and yellow boxes indicate unknown objects. Overall, the visual comparison shows that the CODE framework is more robust at localizing and correctly identifying both known and unknown targets across various complex environments compared to the baseline.
Configuration
U-mAP
K-mAP
Baseline
16.2
43.0
+ CMJCC
17.0
45.0
+ CMJCC + UGUOE
22.4
42.3
+ CMJCC + UGUOE + DOSCM
25.7
43.9
Table 2. Incremental ablation on Surgery Task 1 using OWL-ViT L/14.
Configuration
Aquatic
Surgery
U
K
U
K
CODE Full
24.7
62.7
25.7
43.9
w/o CMJCC
20.6
54.3
23.7
41.5
w/o UGUOE
9.1
63.0
2.3
44.2
w/o DOSCM
8.3
62.0
22.4
42.3
Table 3. Removal-based ablation on Aquatic and Surgery Task 1 using OWL-ViT L/14.
Dataset
10-shot
50-shot
75-shot
100-shot
Aquatic
18.0/35.3
21.0/62.5
20.5/63.0
24.7/62.7
Surgery
13.5/39.8
26.1/42.2
27.4/43.5
25.7/43.9
Table 4. Sample-size sensitivity of visual prototypes using OWL-ViT L/14. Values are U-/K-mAP on Task 1.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Total Images
Classes (K+U)
Attributes
% in Tokenizer
Aquatic
637
7 (4+3)
385
100.0
Aerial
10,000
20 (10+10)
1,229
55.0
Game
1,575
59 (30+29)
390
35.6
Medical
182
12 (6+6)
390
15.4
Surgery
1,829
13 (6+7)
808
7.69
Appendix
Table S1. Comprehensive summary of the RWD benchmark datasets. We report the total images, class distribution for Task 1, attribute pool size, and tokenizer vocabulary coverage (%).
Dataset ( → )
Aquatic
Surgery
Method Variant
PK
CK
PK
CK
CODE Full (Ours)
65.3
59.6
46.3
35.6
w/ Fixed τmin Only
61.7
55.6
42.3
27.6
w/o Threshold (Direct Simvis )
59.9
49.1
42.1
25.4
Appendix
Table S2. Ablation study of the CMJCC module’s dynamic gating mechanism. We report PK-mAP (PK) and CK-mAP (CK) on the Aquatic and Surgery datasets under Task 2 evaluation using the L/14 backbone.
Dataset ( → )
Aquatic
Surgery
Method Variant
U
K
U
K
CODE Full (Ours)
24.7
62.7
25.7
43.9
w/o Objunk (No Enhancement)
9.1
63.0
2.3
44.2
w/o Wunc (No Uncertainty Weight)
20.1
62.9
23.9
43.5
Appendix
Table S3. Ablation study of the UGUOE module’s internal components. We report U-mAP (U) and K-mAP (K) on the Aquatic and Surgery datasets under Task 1 evaluation.
Parameter
Value
Aquatic
Surgery
PK
CK
PK
CK
Gating Margin ( m )
0
50.2
41.3
40.0
23.4
0.1
64.0
58.0
41.8
27.6
0.2
65.3
59.6
46.3
35.6
0.3
65.2
59.6
46.4
36.3
0.4
62.4
58.4
43.3
32.4
Appendix
Table S4. Parameter sensitivity analysis of the visual gating margin m and modulation coefficient α . Default parameter values are highlighted in bold in the Value column, while the best metric values within each parameter sweep are highlighted in bold. PK and CK indicate Previously Known and Currently Known mAP, respectively.
Parameter
Value
Aquatic
Surgery
U
K
U
K
Local Size ( Ku )
1
20.4
63.0
16.4
43.4
3
20.1
63.0
25.7
43.9
All
17.4
63.0
18.7
44.0
Half
18.8
62.9
21.5
44.0
Sensitivity ( γ )
10
20.2
62.8
24.0
43.9
Appendix
Table S5. Parameter sensitivity analysis of the UGUOE module. We evaluate the local neighborhood size Ku , sensitivity factor γ , and activation threshold τunk . Default parameter values are highlighted in bold in the Value column, while the best metric values within each parameter sweep are highlighted in bold. U and K denote U-mAP and K-mAP under Task 1, respectively.
Figure S1. Logit distribution comparisons on the Aquatic (top), Medical (middle), and Surgery (bottom) datasets. For each dataset, the left plot illustrates the known logit distribution, while the right plot displays the unknown logit distribution. CODE significantly shifts the probability mass rightward into the high-confidence region ( >0 ).
Figure S2. Scatter visualizations of DOSCM on the Aquatic (left), Game (middle), and Surgery (right) datasets. The x-axis represents the Top-1 known Softmax probability, and the y-axis is the final OOD score ( pOOD ). The purple line denotes the rigid baseline suppression bottleneck, while the green curve illustrates our dynamic suppression boundary.
Figure S3. Qualitative detection results on the Aerial and Surgery datasets for Task 1. Top row: Ground Truth annotations. Middle row: Predictions by the SOTA baseline PASS. Bottom row: Predictions by our proposed CODE framework. Blue boxes denote known categories, while yellow boxes denote unknown categories.
Figure S4. Qualitative detection results on Task 2 across the Aerial, Game, Medical, and Surgery datasets. Top row: Ground Truth annotations, where yellow boxes indicate Currently Known (CK) classes. Middle row: Predictions by the baseline PASS. Bottom row: Predictions by our CODE framework. CODE recalls significantly more known targets (blue boxes) through visual-to-visual calibration.
Figure S5. Qualitative detection results focusing on class confusion and error cases on the Aquatic and Aerial datasets for Task 1. Top row: Ground Truth annotations. Middle row: Predictions by the SOTA baseline PASS. Bottom row: Predictions by our proposed CODE framework. Blue boxes denote known categories, while yellow boxes denote unknown categories.
Open-world object detection (OWOD) requires a detector to recognize known categories, discover unnamed objects from unseen categories, and incrementally learn newly annotated classes. PROB improves unknown discovery by modeling class-agnostic probabilistic objectness in the decoder-query space. However, visual objectness alone cannot determine whether an object-like query corresponds to a hard known instance, an unseen-category object, or background clutter, resulting in an ambiguous known-unknown decision boundary. We propose MSPO, a lightweight semantic calibration framework that augments PROB with task-aware known-category language priors while preserving its detector architecture and incremental learning protocol. For each currently known category, MSPO constructs an extended text description covering category attributes, visual appearance, typical scenes, and functional usage, and encodes it using a frozen CLIP text encoder. Decoder query features are projected into the same semantic space to estimate their support from the current known-category semantics. This semantic evidence is fused with PROB's visual objectness to calibrate known and unknown predictions without turning OWOD into open-vocabulary classification. Importantly, MSPO never uses future-category names, and all unseen categories remain unnamed during evaluation. Experiments on M-OWODB and S-OWODB show that MSPO improves the strong PROB baseline on the main aggregate metrics while retaining competitive unknown recall. It also improves early unknown-confusion metrics and raises PASCAL VOC final mAP by up to 2.7 points. These results demonstrate that known-category language semantics provide an effective calibration signal for probabilistic objectness under the standard OWOD setting.
Weijun Tian, Rui Liu
School of Computer Science and Engineering, Beihang University
Open-vocabulary object detection (OVOD) aims to detect both seen and unseen categories, yet existing methods often struggle to generalize to novel objects due to limited integration of global and local contextual cues. We propose DetRefiner, a simple yet effective plug-and-play framework that learns to fuse global and local features to refine open-vocabulary detection. DetRefiner processes global image features and patch-level image features from foundational models (e.g., DINOv3) through a lightweight Transformer encoder. The encoder produces a class vector capturing image-level attributes and patch vectors representing local region attributes, from which attribute reliability is inferred to recalibrate the base model's confidence. Notably, DetRefiner is trained independently of the base OVOD model, requiring neither access to its internal features nor retraining. At inference, it operates solely on the base detector's predictions, producing auxiliary calibration scores that are merged with the base detector's scores to yield the final refined confidence. Despite this simplicity, DetRefiner consistently enhances multiple OVOD models across COCO, LVIS, ODinW13, and Pascal VOC, achieving gains of up to +10.1 AP on novel categories. These results highlight that learning to fuse global and local representations offers a powerful and general mechanism for advancing open-world object detection. Our codes and models are available at https://github.com/hitachi-rd-cv/detrefiner.
Soichiro Okazaki, Tatsuya Sasaki, Hiroki Ohashi
Hitachi, Ltd. Research and Development Group, Japan
Open-world object detection (OWOD) requires incrementally detecting known categories while reliably identifying unknown objects. Existing methods primarily focus on improving unknown recall, yet overlook interpretability, often leading to known-unknown confusion and reduced prediction reliability. This paper aims to make the entire OWOD framework interpretable, enabling the detector to truly "knowing the unknown". To this end, we propose a concept-driven InterPretable OWOD framework(IPOW) by introducing a Concept Decomposition Model (CDM) for OWOD, which explicitly decomposes the coupled RoI features in Faster R-CNN into discriminative, shared, and background concepts. Discriminative concepts identify the most discriminative features to enlarge the distances between known categories, while shared and background concepts, due to their strong generalization ability, can be readily transferred to detect unknown categories. Leveraging the interpretable framework, we identify that known-unknown confusion arises when unknown objects fall into the discriminative space of known classes. To address this, we propose Concept-Guided Rectification (CGR) to further resolve such confusion. Extensive experiments show that IPOW significantly improves unknown recall while mitigating confusion, and provides concept-level interpretability for both known and unknown predictions.
Xueqiang Lv, Shizhou Zhang, Yinghui Xing +3
School of Computer Science, Northwestern Polytechnical Unviersity, Xi’an, China · Huawei Technologies Ltd.