Organizations: School of Computer Science and Artificial Intelligence, Zhengzhou University, Zhengzhou,China · Sinosteel Zhengzhou Research Institute of Steel Wire Products Co., Ltd., China
Open-vocabulary object detection aims to detect both seen and unseen categories by leveraging the semantic knowledge of vision-language models. However, existing methods often struggle with small objects due to insufficient preservation of high-resolution details, ineffective information aggregation across feature levels, and limited region-level discriminability. To address these challenges, we propose FocusOVD, an open-vocabulary object detector that enhances small-object representations without modifying the frozen VLM backbone. FocusOVD introduces three complementary components: a Reweighted Cross-scale Feature Fusion module, which learns normalized level-wise weights to adaptively balance shallow appearance details and deep semantic cues during feature pyramid aggregation; a Residual Pinwheel-shaped Gated Convolution module, which strengthens local geometric responses in high-resolution features; and a Residual Convolutional Gated Linear Unit module, which suppresses background-dominated activations and enhances RoI discriminability before prediction. Extensive experiments on COCO-OVD and LVIS-OVD demonstrate the effectiveness of FocusOVD. On COCO-OVD, FocusOVD achieves a novel-category AP of 36.3%, outperforming CORA, UniDetector, and SAMP by 1.3%, 1.1%, and 1.5%, respectively; on LVIS-OVD, it achieves an overall AP of 33.7%, surpassing the strongest competing method by 0.6%. These results demonstrate the strong generalization ability of FocusOVD for open-vocabulary small-object detection.
Figures & tables
Figure 1: Overview of FocusOVD.
Figure 2: Structure of the re-parameterizable convolution used in RCFF.
Model
AP novel
AP base
AP all
AP Snovel
AP Mnovel
AP Lnovel
ViLD [ 4 ]
27.6
58.7
51.3
14.5
29.1
39.3
RegionCLIP [ 5 ]
31.4
57.1
50.4
18.2
33.5
44.4
OV-DETR [ 17 ]
29.4
61.0
52.7
17.8
32.6
41.0
Detic [ 18 ]
27.8
47.1
45.0
15.7
30.0
39.9
CoDet [ 10 ]
30.2
52.4
46.7
17.5
32.8
41.1
BARON [ 20 ]
34.0
60.4
53.5
20.3
36.5
46.3
Table 1: Open-vocabulary detection results on COCO-OVD, including size-specific AP for novel categories.
Figure 3: Novel-category AP on COCO-OVD for UniDetector and FocusOVD across different object scales.
Figure 4: Qualitative comparison of CORA, RegionCLIP, UniDetector, and FocusOVD on COCO validation images.
Model
APr
APc
APf
AP
ViLD [ 4 ]
16.1
23.9
32.5
24.9
RegionCLIP [ 5 ]
21.3
30.8
36.0
30.7
OV-DETR [ 17 ]
17.4
25.0
32.5
26.6
Detic [ 18 ]
17.8
26.3
31.6
26.8
CoDet [ 10 ]
21.8
30.2
34.9
30.6
MEDet [ 11 ]
22.2
32.7
36.9
33.1
Table 2: Open-vocabulary detection results on LVIS-OVD.
Model
COCO
Objects365 v2
AP
AP50
AP75
AP
AP50
AP75
ViLD [ 4 ]
36.6
55.6
39.8
11.8
18.2
12.6
F-VLM [ 7 ]
32.5
53.1
34.6
11.9
19.2
12.6
DetPro [ 19 ]
34.9
53.8
34.7
12.1
18.8
12.9
BARON [ 20 ]
36.2
55.7
38.1
13.6
21.0
14.5
UniDetector [ 9 ]
36.4
54.9
38.5
13.3
21.4
14.7
Table 3: Cross-dataset zero-shot transfer results on COCO and Objects365 v2.
Open-vocabulary object detection (OVOD) aims to detect both seen and unseen categories, yet existing methods often struggle to generalize to novel objects due to limited integration of global and local contextual cues. We propose DetRefiner, a simple yet effective plug-and-play framework that learns to fuse global and local features to refine open-vocabulary detection. DetRefiner processes global image features and patch-level image features from foundational models (e.g., DINOv3) through a lightweight Transformer encoder. The encoder produces a class vector capturing image-level attributes and patch vectors representing local region attributes, from which attribute reliability is inferred to recalibrate the base model's confidence. Notably, DetRefiner is trained independently of the base OVOD model, requiring neither access to its internal features nor retraining. At inference, it operates solely on the base detector's predictions, producing auxiliary calibration scores that are merged with the base detector's scores to yield the final refined confidence. Despite this simplicity, DetRefiner consistently enhances multiple OVOD models across COCO, LVIS, ODinW13, and Pascal VOC, achieving gains of up to +10.1 AP on novel categories. These results highlight that learning to fuse global and local representations offers a powerful and general mechanism for advancing open-world object detection. Our codes and models are available at https://github.com/hitachi-rd-cv/detrefiner.
Soichiro Okazaki, Tatsuya Sasaki, Hiroki Ohashi
Hitachi, Ltd. Research and Development Group, Japan
Open-vocabulary detectors are proposed to locate and recognize objects in novel classes. However, variations in vision-aware language vocabulary data used for open-vocabulary learning can lead to unfair and unreliable evaluations. Recent evaluation methods have attempted to address this issue by incorporating object properties or adding locations and characteristics to the captions. Nevertheless, since these properties and locations depend on the specific details of the images instead of classes, detectors can not make accurate predictions without precise descriptions provided through human annotation. This paper introduces 3F-OVD, a novel task that extends supervised fine-grained object detection to the open-vocabulary setting. Our task is intuitive and challenging, requiring a deep understanding of Fine-grained captions and careful attention to Fine-grained details in images in order to accurately detect Fine-grained objects. Additionally, due to the scarcity of qualified fine-grained object detection datasets, we have created a new dataset, NEU-171K, tailored for both supervised and open-vocabulary settings. We benchmark state-of-the-art object detectors on our dataset for both settings. Furthermore, we propose a simple yet effective post-processing technique. Our data, annotations and codes are available at https://github.com/tengerye/3FOVD.
Ying Liu, Yijing Hua, Haojiang Chai +2
department of software engineering, Northeastern University, China. · His work was done in Articul8 AI.
Open-vocabulary object detection (OVOD) enables models to recognize objects beyond predefined categories, but existing approaches remain limited in practical deployment. On the one hand, multimodal designs often incur substantial computational overhead due to their reliance on text encoders at inference time. On the other hand, tightly coupled training objectives introduce a trade-off between closed-set detection accuracy and open-world generalization. Thus, we propose Decoupled Cognition DETR (DeCo-DETR), a vision-centric framework that addresses these challenges through a unified decoupling paradigm. Instead of depending on online text encoding, DeCo-DETR constructs a hierarchical semantic prototype space from region-level descriptions generated by pre-trained LVLMs and aligned via CLIP, enabling efficient and reusable semantic representation. Building upon this representation, the framework further disentangles semantic reasoning from localization through a decoupled training strategy, which separates alignment and detection into parallel optimization streams. Extensive experiments on standard OVOD benchmarks demonstrate that DeCo-DETR achieves competitive zero-shot detection performance while significantly improving inference efficiency. These results highlight the effectiveness of decoupling semantic cognition from detection, offering a practical direction for scalable OVOD systems.