Open-vocabulary object detection (OVOD) aims to detect known and unknown objects in the open world by leveraging text prompts. Benefiting from the emergence of large-scale vision--language pre-trained models, OVOD has demonstrated strong zero-shot generalization capabilities. However, when dealing with camouflaged objects, the detector often fails to distinguish and localize objects because the visual features of the objects and the background are highly similar. To bridge this gap, we construct a benchmark named OVCOD-D by augmenting carefully selected camouflaged object images with fine-grained textual descriptions. Due to the limited scale of available camouflaged object datasets, we adopt detectors pre-trained on large-scale object detection datasets as our baseline methods, as they possess stronger zero-shot generalization ability. In the specificity-aware sub-descriptions generated by multimodal large models, there still exist confusing and overly decorative modifiers. To mitigate such interference, we design a sub-description principal component contrastive fusion strategy that reduces noisy textual components. Furthermore, to address the challenge that the visual features of camouflaged objects are highly similar to those of their surrounding environment, we propose a specificity-guided regional weak alignment and dynamic focusing method, which aims to strengthen the detector's ability to discriminate camouflaged objects from background. Under the open-set evaluation setting, the proposed method achieves an AP of 56.4 on the OVCOD-D benchmark.
Figures & tables
Figure 1: We perform zero-shot detection with the YOLO-World-M [ 2 ] on both the LVIS [ 10 ] dataset and our OVCOD-D dataset. By comparing the AP of the overlapping categories across the two datasets, we observe a substantial performance decline on OVCOD-D, indicating that open-vocabulary detectors face significant challenges when dealing with camouflaged objects.
Figure 2: We conducted a statistical analysis of per-class instance counts in the OVCOD-D dataset. In the bar chart, blue bars denote base classes and red bars denote novel classes, revealing a pronounced long-tailed distribution of class frequencies. Additionally, using the fine-grained textual descriptions associated with each category, we selected 25 categories characterized by higher lexical richness to construct a multidimensional quality-analysis heatmap. The horizontal axis reports, in sequence, lexical diversity, average tokens, unique words, average unique-word ratio, and the standard deviation of sentence length.
Figure 3: Overall architecture of the proposed specificity-driven open-vocabulary camouflaged object detector. Fine-grained textual sub-descriptions are encoded by a text encoder and decorrelated via SVD, then refined through an adapter and integrated with visual object embeddings in the contrastive fusion module. The image is processed by a lightweight YOLO‑style backbone and a PAN to extract multi‑scale features, which are transformed into object embeddings and fed in parallel into the SF‑GLU and the box head.
Figure 4: Construction pipeline of OVCOD-D dataset. We extend COD10K-D, NC4K-D, and clean CAMO-D with YOLO-style detection labels and an additional red imported fire ant nest subset, then reorganize them into 40 base and 47 novel classes. Qwen3-VL-Plus generates fine-grained image descriptions from which we derive a semantic prompt library for open-vocabulary camouflaged object detection.
Method
Backbone
Params
Pre-train
AP
AP 50
AP 75
AP m
AP l
GLIP-T [ 20 ]
Swin-T
232M
O365,GoldG
22.6
31.9
28.2
-
-
GLIPv2-T [ 44 ]
Swin-T
232M
O365,GoldG
25.6
36.4
29.7
-
-
Grounding DINO-T [ 25 ]
Swin-T
172M
O365,GoldG
34.8
43.9
37.7
-
-
YOLO-World-S [ 2 ]
YOLOv8-S
77M
O365,GoldG
38.5
58.0
41.4
18.7
41.1
YOLO-World-M [ 2 ]
YOLOv8-M
92M
O365,GoldG
43.3
62.3
46.5
24.9
46.1
YOLO-World-L [ 2 ]
YOLOv8-L
110M
O365,GoldG
45.7
63.2
48.9
22.9
48.4
Table 1: Comparison with classical open-vocabulary object detectors on the OVCOD-D dataset is conducted under an open-set setting, where all models are fine-tuned on the base classes and evaluated on the union of base and novel classes.
Method
Pre-train
AP
AP 50
AP 75
FPS
Training from scratch.
YOLOv8-S [ 34 ]
×
18.6
26.5
19.0
-
YOLOv8-M [ 34 ]
×
21.7
31.8
22.6
-
YOLOv8-L [ 34 ]
×
23.7
32.0
23.4
-
Fine-tuned w/ DuVL-PAN and sub-descriptions
YOLO-World-S [ 2 ]
O+G
39.3
57.3
41.3
35
Table 2: Comparisons under the closed-set setting are conducted to evaluate the performance of classical object detectors and open-vocabulary object detectors on the OVCOD-D dataset. The FPS is measured on a single NVIDIA RTX 4090 using the raw trained checkpoints without TensorRT, re-parameterization, or other lightweight optimizations.
Fusion Method
AP
AP 50
AP 75
Orthogonal+Contrastive
46.5
70.1
48.9
Contrastive
47.5
70.6
49.6
Orthogonal
44.0
66.4
45.6
Table 3: Ablation study on sub-description fusion strategies for open-vocabulary camouflaged object detection on OVCOD-D dataset. The coverage auxiliary loss weight is uniformly set to 5, and SF-GLU is disabled for all variants to ensure fair comparison.
Dynamic Focusing Module
AP
AP 50
AP 75
w/o GLU
47.5
70.6
49.6
GLU
46.7
69.7
49.8
SwiGLU
47.1
70.3
49.2
SF-GLU
48.1
70.7
50.3
Table 4: Ablation study on SF-GLU module for open-vocabulary camouflaged object detection on the OVCOD-D dataset. All variants adopt the contrastive fusion framework with coverage auxiliary loss set to 5.
Weight for auxiliary loss
AP
AP 50
AP 75
0
47.3
71.0
49.5
2
47.2
70.5
48.7
5
47.5
70.6
49.6
7
46.3
69.3
49.0
10
46.6
69.7
49.0
Table 5: Ablation study on different weights of coverage-based auxiliary loss for open-vocabulary camouflaged object detection on the OVCOD-D dataset. All variants adopt the same contrastive fusion framework without using the SF-GLU module.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Metric
Accuracy
Relevance
Hallucination-free
Qualified category
Rate (%)
98.15
99.77
92.61
100.00
Appendix
Table A1 : Human evaluation of generated textual descriptions.
Method
AP
AP50
AP75
SINet-V2 [ 5 ]
40.2
69.3
39.4
FSPNet [ 16 ]
47.9
76.2
49.4
CamoFormer [ 42 ]
55.6
80.2
59.0
HDPNet [ 11 ]
56.3
81.5
59.6
SDDF-L (Ours)
56.4
76.4
60.7
Appendix
Table A2 : Comparison with SOTA COD methods on OVCOD-D.
Method
APmild
APmod
APsevere
SDDF-S
64.2
49.8
39.0
Appendix
Table A3 : Multi-level camouflability performance of SDDF-S on OVCOD-D.
SVD Configuration
AP
AP 50
AP 75
w/o SVD
44.8
67.4
46.8
w/ SVD (5)
46.5
69.8
48.3
w/ SVD (10)
46.8
69.5
49.2
w/ SVD (15)
46.1
69.2
48.4
Appendix
Table A4 : Ablation study on different maximum number of components obtainable from SVD on the OVCOD-D dataset. All model variants adopt the fusion approach defined in Equation A1 . The coverage auxiliary loss and SF-GLU are disabled.
Conv Layers
AP
AP50
AP75
1
48.1
70.7
50.3
2
47.1
69.6
49.8
4
46.0
68.7
48.6
Appendix
Table A5 : Ablation study on the number of convolutional layers.
Figure A1 : Quantitative comparison is conducted via visualization of the detection bounding boxes.
Figure A2 : Quantitative comparisons are further conducted by visualizing heatmap representations of the response differences in feature maps with and without the proposed SF-GLU. Specifically, these feature maps are extracted from the PAN outputs.
Camouflaged Object Detection (COD) aims to segment objects that are highly integrated with the background in terms of color, texture, and structure, making it a highly challenging task in computer vision. Although existing methods introduce multi-scale fusion and attention mechanisms to alleviate the above issues, they generally lack the guidance of textual semantic priors, which limits the model's ability to focus on camouflaged regions in complex scenes. To address this issue, this paper proposes a Language-Guided Structure-Aware Network (LGSAN). Specifically, based on the visual backbone PVT-v2, we introduce CLIP to generate masks from text prompts and RGB images, thereby guiding the multi-scale features extracted by PVT-v2 to focus on potential target regions. On this foundation, we further design a Fourier Edge Enhancement Module (FEEM), which integrates multi-scale features with high-frequency information in the frequency domain to extract edge enhancement features. Furthermore, we propose a Structure-Aware Attention Module (SAAM) to effectively enhance the model's perception of object structures and boundaries. Finally, we introduce a Coarse-Guided Local Refinement Module (CGLRM) to enhance fine-grained reconstruction and boundary integrity of camouflaged object regions. Extensive experiments demonstrate that our method consistently achieves highly competitive performance across multiple COD datasets, validating its effectiveness and robustness.
Min Zhang
School of Artificial Intelligence, Chongqing University of Technology, Chongqing, 401120, China
Camouflaged object detection (COD) aims to segment objects that are visually concealed in their surroundings and has attracted increasing attention in recent years. However, most existing COD methods are developed under a closed-world assumption, where each input image is assumed to contain a camouflaged object. This assumption ignores realistic scenarios with pure backgrounds or non-camouflaged objects, causing existing models to produce severe false positives when deployed in open-world environments. To address this limitation, we propose OPC16K, a large-scale benchmark for realistic COD. OPC16K contains 16,245 images from 14 sources and is carefully organized into camouflaged-object images, pure background images, and non-camouflaged-object images, enabling comprehensive evaluation of both segmentation quality and negative-sample rejection. Based on this benchmark, we further propose OPCNet, a presence-aware camouflage network that reformulates COD from a pure segmentation task into a joint problem of object localization and camouflage existence reasoning. Specifically, OPCNet introduces hierarchical existence reasoning to distinguish CO, BG, and NOCOD scenarios, similarity-aware camouflage relation modeling to capture foreground-background camouflage cues, and existence-aware feature refinement to regulate segmentation features with existence predictions. Extensive experiments on OPC16K demonstrate that OPCNet achieves superior performance under the proposed realistic COD evaluation protocol, significantly reducing false positives on negative samples while maintaining accurate camouflaged-object segmentation. Code and dataset will be released at https://github.com/2231122/OPCOD.
Huafeng Chen, Yueming Lyu, Chenyang Si +3
School of Intelligence Science and Technology, Nanjing University · Department of Computing, Imperial College London
Video camouflaged object detection (VCOD) is challenging due to dynamic environments. Existing methods face two main issues: (1) SAM-based methods struggle to separate camouflaged object edges due to model freezing, and (2) MLLM-based methods suffer from poor object separability as large language models merge foreground and background. To address these issues, we propose a novel VCOD method based on SAM and MLLM, called Phantom-Insight. To enhance the separability of object edge details, we represent video sequences with temporal and spatial clues and perform feature fusion via LLM to increase information density. Next, multiple cues are generated through the dynamic foreground visual token scoring module and the prompt network to adaptively guide and fine-tune the SAM model, enabling it to adapt to subtle textures. To enhance the separability of objects and background, we propose a decoupled foreground-background learning strategy. By generating foreground and background cues separately and performing decoupled training, the visual token can effectively integrate foreground and background information independently, enabling SAM to more accurately segment camouflaged objects in the video. Experiments on the MoCA-Mask dataset show that Phantom-Insight achieves state-of-the-art performance across various metrics. Additionally, its ability to detect unseen camouflaged objects on the CAD2016 dataset highlights its strong generalization ability.
Hua Zhang, Changjiang Luo
Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences