Discover, Segment, and Select: A Progressive Mechanism for Zero-shot Camouflaged Object Segmentation
Authors: Yilong Yang, Jianxin Tian, Shengchuan Zhang, Liujuan Cao
Organizations: Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China.
Current zero-shot Camouflaged Object Segmentation methods typically employ a two-stage pipeline (discover-then-segment): using MLLMs to obtain visual prompts, followed by SAM segmentation. However, relying solely on MLLMs for camouflaged object discovery often leads to inaccurate localization, false positives, and missed detections. To address these issues, we propose the Discover-Segment-Select (DSS) mechanism, a progressive framework designed to refine segmentation step by step. The proposed method contains a Feature-coherent Object Discovery (FOD) module that leverages visual features to generate diverse object proposals, a segmentation module that refines these proposals through SAM segmentation, and a Semantic-driven Mask Selection (SMS) module that employs MLLMs to evaluate and select the optimal segmentation mask from multiple candidates. Without requiring any training or supervision, DSS achieves state-of-the-art performance on multiple COS benchmarks, especially in multiple-instance scenes.
Figures & tables
Figure 1: Comparison between the proposed DSS framework and prior zero-shot COS methods.
Figure 2: The overall pipeline of the Discovery-Segment-Selection Framework. The framework operates in three stages: (a) Feature-guided Object Discovery (FOD): the Part Composition (PC) module refines low-resolution clustering masks, followed by a Similarity based Box Generation (SBG) module to produce prompt boxes; (b) The Segment Anything Model generates segmentation mask based on the input prompts. (c) Semantic-driven Mask Selection (SMS): an MLLM evaluates all candidate masks and selects the final segmentation.
Figure 3: Visual tracking of the iterative refinement process. Left to right: the original image, Leiden clustering map, and binary maps from initial clustering Y(0) to the final refined map Y(5) . Here we select one cluster from the clustering results for demonstration. The number of iterations may vary for different clusters until convergence.
Figure 4: Visual comparison between bboxes generated from different strategies. In each 2 × 2 grid, Top left : bboxes from the ground truth mask. Top right : bboxes from leiden clustered mask. Bottom left : bboxes from PC refined mask. Bottom right : bboxes from self-similarity map.
HP
Value
Remark
r
0.5
Leiden clustering resolution.
ϵ
1.0
The energy change threshold.
τ
0.95
Correlation threshold for de-duplication.
K
5
Top-K masks for MLLM mask selection.
Table 1: Hyperparameter setting.
Figure 5: We present a visual comparison of our method against existing alternatives under challenging conditions. The red and black bounding boxes correspond to prompts generated by our approach and QWen, respectively. To aid interpretation, the third column visualizes the similarity maps from which our bounding boxes are derived.
Methods
Venue
Sup
CAMO(250)
NC4K(4121)
COD10K(2016)
CHAMELEON(87)
Fβw↑
M↓
Sα↑
Eϕ↑
Fβw↑
M↓
Sα↑
Eϕ↑
Fβw↑
M↓
Sα↑
Eϕ↑
Fβw↑
M↓
Sα↑
Eϕ↑
HitNet [ 16 ]
AAAI23
F
.809
.055
.849
.906
.834
.037
.875
.926
.806
.023
.871
.935
.897
.019
.921
.967
SENet [ 12 ]
TIP24
F
.847
.039
.888
.932
.843
.032
.889
.933
.780
.024
.865
.925
.878
.019
.918
.957
ZoomNeXt [ 30 ]
TPAMI24
F
.857
.041
.889
.945
.863
.028
.903
.951
.827
.018
.898
.956
.885
.018
.924
.975
CGCOD [ 45 ]
MM25
F
.864
.036
.896
.947
.869
.026
.904
.949
.824
.018
.890
.948
.902
.017
.931
.972
CAMF [ 19 ]
TPAMI25
F
.887
.041
.901
.933
.897
.033
.905
.933
.845
.025
.890
.923
.895
.028
.917
.944
Table 2: Quantitative comparison with state-of-the-art methods on four COD datasets “ ↑/↓ ” : the higher/lower the better. “-”: not available. “F”, “U” and “ZS” denote “Fully-supervised”, “Un-supervised” and “Zero-shot” method, repsectively. The best three results in zero-shot setting are highlighted in red , blue , and green , respectively. The best results in “F”, “U” settings are in bold.
Figure 6: (a) Each pie chart shows the proportion of images containing different numbers of camouflaged objects. “3+” denotes images contain (≥3) camouflaged objects. (b) and (c) Performance comparison of different camouflaged instance counts on NC4K and COD10K. The y-axis is score=3Sα+Eϕ+Fβw−M , indicateing the overall performance.
ProMaC
RDVP-MSD
DSS
Time (s)
130.49
18.05
41.96 (7.74, 3.63, 30.59)
GPU (GB)
32.85
31.51
17.90
Table 3: Comparison of computational efficiency, reported as average inference time per image (s). For our DSS framework, the time is decomposed into its three stages: FOD, mask generation, and SMS. Results are reported on the CHAMELEON dataset.
ID
Leiden
PC
VLOS
COD10K
CHAMELEON
CAMO
1
✓
×
×
.778
.797
.689
2
×
✓
×
.758
.814
.761
3
×
×
✓
.850
.791
.708
4
✓
✓
×
.820
.863
.790
5
✓
×
✓
.886
.872
.801
6
×
✓
✓
.881
.880
.810
Table 4: Ablation comparison of different inference settings. “Leiden” and “PC” denotes that the initial Leiden clustering map or the refined clustering map is used in the FOD module for extracting bboxes; “VLOS” denotes that the mask produced by QWen+SAM2 is included for mask selection.
Methods
NC4K 3+
COD10K 3+
Fβw↑
M↓
Sα↑
Eϕ↑
Fβw↑
M↓
Sα↑
Eϕ↑
Leiden+PC
.795
.058
.812
.897
.723
.034
.794
.881
Sim. Map
.818
.054
.839
.917
.745
.026
.821
.911
Table 5: Ablation on SBG. The segmentation performance based on bboxes generated from: (1) Clustered and refined masks. (Leiden+PC). (2). similarity maps (Sim. Map). “ 3+ ” denotes the subset of images containing three or more camouflaged objects.
Method
CAMO
NC4K
COD10K
CHAMELEON
VLOS
.708
.845
.850
.791
DSS
.737
.869
.871
.852
Ideal Seg.
.823
.895
.892
.888
Table 6: Ablation on the SMS module.
Method
CAMO
CHAMELEON
Time (s)
Heuristic Rules
68.80
71.05
1.14
Single-pass Selection
OOM
OOM
OOM
Top- K Single-pass
40.24
36.84
14.25
Top- K pairwise (random)
54.00
59.21
31.14
Top- K pairwise (descending)
50.80
43.42
30.81
Top- K pairwise (ascending)
74.40
73.68
30.59
Table 7: Comparison of different Mask Selection strategies. CAMO and CHAMELEON columnes measure selection accuracy (%), while Time indicates average inference cost (seconds per image). “OOM” denotes out-of-memory errors.
Figure 7: Prompt template for camouflaged object localization using MLLM.
Figure 8: Prompt template for mask selection using MLLM.
Video camouflaged object detection (VCOD) is challenging due to dynamic environments. Existing methods face two main issues: (1) SAM-based methods struggle to separate camouflaged object edges due to model freezing, and (2) MLLM-based methods suffer from poor object separability as large language models merge foreground and background. To address these issues, we propose a novel VCOD method based on SAM and MLLM, called Phantom-Insight. To enhance the separability of object edge details, we represent video sequences with temporal and spatial clues and perform feature fusion via LLM to increase information density. Next, multiple cues are generated through the dynamic foreground visual token scoring module and the prompt network to adaptively guide and fine-tune the SAM model, enabling it to adapt to subtle textures. To enhance the separability of objects and background, we propose a decoupled foreground-background learning strategy. By generating foreground and background cues separately and performing decoupled training, the visual token can effectively integrate foreground and background information independently, enabling SAM to more accurately segment camouflaged objects in the video. Experiments on the MoCA-Mask dataset show that Phantom-Insight achieves state-of-the-art performance across various metrics. Additionally, its ability to detect unseen camouflaged objects on the CAD2016 dataset highlights its strong generalization ability.
Hua Zhang, Changjiang Luo
Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences
Accurately segmenting objects without any manual annotations remains one of the core challenges in computer vision. In this work, we introduce Selfment, a fully self-supervised framework that segments foreground objects directly from raw images without human labels, pretrained segmentation models, or any post-processing. Selfment first constructs patch-level affinity graphs from self-supervised features and applies NCut to obtain an initial coarse foreground--background separation. We then introduce Iterative Patch Optimization (IPO), a feature-space refinement procedure that progressively enforces spatial coherence and semantic consistency through iterative patch clustering. The refined masks are subsequently used as supervisory signals to train a lightweight segmentation head with contrastive and region-consistency objectives, allowing the model to learn stable and transferable object representations. Despite its simplicity and complete absence of manual supervision, Selfment sets new state-of-the-art (SoTA) results across multiple benchmarks. It achieves substantial improvements on Fmax over previous unsupervised saliency detection methods on ECSSD (+4.0%), HKUIS (+4.6%), and PASCAL-S (+5.7%). Moreover, without any additional fine-tuning, Selfment demonstrates remarkable zero-shot generalization to camouflaged object detection tasks (e.g., 0.910Sm on CHAMELEON and 0.792Fβω on CAMO), outperforming all existing unsupervised approaches and even rivaling the SoTA fully supervised methods.
Camouflaged Object Detection (COD) aims to segment objects that are highly integrated with the background in terms of color, texture, and structure, making it a highly challenging task in computer vision. Although existing methods introduce multi-scale fusion and attention mechanisms to alleviate the above issues, they generally lack the guidance of textual semantic priors, which limits the model's ability to focus on camouflaged regions in complex scenes. To address this issue, this paper proposes a Language-Guided Structure-Aware Network (LGSAN). Specifically, based on the visual backbone PVT-v2, we introduce CLIP to generate masks from text prompts and RGB images, thereby guiding the multi-scale features extracted by PVT-v2 to focus on potential target regions. On this foundation, we further design a Fourier Edge Enhancement Module (FEEM), which integrates multi-scale features with high-frequency information in the frequency domain to extract edge enhancement features. Furthermore, we propose a Structure-Aware Attention Module (SAAM) to effectively enhance the model's perception of object structures and boundaries. Finally, we introduce a Coarse-Guided Local Refinement Module (CGLRM) to enhance fine-grained reconstruction and boundary integrity of camouflaged object regions. Extensive experiments demonstrate that our method consistently achieves highly competitive performance across multiple COD datasets, validating its effectiveness and robustness.
Min Zhang
School of Artificial Intelligence, Chongqing University of Technology, Chongqing, 401120, China