Few-shot semantic segmentation (FSS) of strip steel surface defects (S3D) has posed significant challenges distinct from natural scenes. Unlike natural images, S3D task exhibits unique characteristics including low local contrast, uneven illumination, and complex fine-grained texture patterns. Although recent methods based on Segment Anything Model (SAM) have shown promise in FSS on natural images by leveraging SAM's powerful pre-trained representations, these unique industrial characteristics of S3D images lead to performance drop when directly applying SAM to industrial defect scenarios. In this paper, we propose a novel Perceptual Parallel Prompt (P3) framework that empowers SAM, creating the P3-SAM model to address these challenges through two core strategies. First, we develop a Perceptual-Optimized Encoding (POE) strategy that enhances local contrast and preserves critical texture details for S3D segmentation. Second, we introduce the Parallel Prompt Generator (PPG) strategy that simultaneously generates both semantic and spatial prompts, enabling comprehensive guidance for SAM's decoder across varying images. Extensive experiments on three few-shot S3D benchmarks demonstrate that P3-SAM achieves state-of-the-art performance, with particularly notable improvements of 12.00% in mIoU on Surface Defects-4i dataset.
Figures & tables
Fig. 1: Visualization of comparative results of Ours with MAPTNet and VRP-SAM. From top to bottom, each row illustrates the ground truth masks of support images in green, the ground truth masks of query images in yellow, and the predictions generated by various methods in red.
Fig. 2: Overview of the proposed P 3 -SAM framework. Our method consists of two core strategies: (1) Perceptual-Optimized Encoding (POE) strategy that unifies defect representation through perceptual optimization and augments shallow features with prototype and mask guided learning, and (2) Parallel Prompt Generator (PPG) strategy that generates both semantic and spatial prompts in parallel to provide comprehensive guidance for SAM’s decoder.
Dataset
Method
Backbone
mIoU(1-shot)
FB-IoU
mIoU(5-shot)
FB-IoU
Fold-0
Fold-1
Fold-2
MEAN
Fold-0
Fold-1
Fold-2
MEAN
FSSD-12
TGRNet
ResNet50
61.80
62.00
48.30
57.70
73.60
62.40
59.70
47.80
58.50
75.10
CPANet
66.00
64.00
54.60
61.50
76.10
66.50
64.90
56.30
62.60
76.30
SCCAN
53.70
60.70
50.90
55.10
70.70
56.90
61.10
53.40
57.10
72.20
DCP
58.30
51.70
43.10
51.00
70.60
55.90
47.00
55.70
52.90
70.90
MAPTNet
69.70
71.30
58.60
66.50
80.00
69.00
72.30
62.60
68.00
80.60
TABLE I: Performance Comparison of mIoU and FB-IoU on FSSD-12, Surface Defects-4i and ESDIs-SOD datasets.
Fig. 3: Qualitative effect of parallel prompts on S 3 D segmentation. From left to right: query image Iq , feature Fbase captured from SAM image encoder, semantic prompt embedding Qsem , spatial prompt embedding Fspa , and final segmentation mask Mpre .
POE
PPG
1-shot
5-shot
mIoU
FB-IoU
mIoU
FB-IoU
71.82
82.31
71.72
82.27
✓
73.92
84.07
74.01
84.14
✓
73.69
83.58
73.68
83.58
✓
✓
75.12
85.09
75.45
85.31
TABLE II: Ablation Study of Different Strategies on FSSD-12.
Few-Shot Semantic Segmentation (FSS) focuses on segmenting novel object categories from only a handful of annotated examples. Most existing approaches rely on extensive episodic training to learn transferable representations, which is both computationally demanding and sensitive to distribution shifts. In this work, we revisit FSS from the perspective of modern vision foundation models and explore the potential of Segment Anything Model 3 (SAM3) as a training-free solution. By repurposing its Promptable Concept Segmentation (PCS) capability, we adopt a simple spatial concatenation strategy that places support and query images into a shared canvas, allowing a fully frozen SAM3 to perform segmentation without any fine-tuning or architectural changes. Experiments on PASCAL-5i and COCO-20i show that this minimal design already achieves state-of-the-art performance, outperforming many heavily engineered methods. Beyond empirical gains, we uncover that negative prompts can be counterproductive in few-shot settings, where they often weaken target representations and lead to prediction collapse despite their intended role in suppressing distractors. These findings suggest that strong cross-image reasoning can emerge from simple spatial formulations, while also highlighting limitations in how current foundation models handle conflicting prompt signals. Code at: https://github.com/WongKinYiu/FSS-SAM3
Yi-Jen Tsai, Yen-Yu Lin, Chien-Yao Wang
National Yang Ming Chiao Tung University · Institute of Information Science, Academia Sinica, Taiwan
Steel surface defect segmentation is critical for industrial quality inspection, yet existing methods struggle with elongated, anisotropic defects such as cracks and scratches due to the isotropic receptive fields of standard convolutions and rigid sampling grids that cannot adapt to irregular defect boundaries. To address these limitations, we propose Strip-based Predictor for Deformable Convolutional Networks (SPDCN) with two key innovations. The Fuzzy-enhanced Multi-scale Context Module (FMCM) employs group-wise multi-branch convolutions with an intuitionistic fuzzy channel attention mechanism to adaptively capture multi-scale contextual information across varying defect sizes. The Adaptive Direction-Aware Deformable Convolution (ADADC) replaces the conventional offset predictor with decoupled horizontal and vertical strip convolutions, enabling the deformable sampling grid to anisotropically align with the principal orientation of elongated defects. Extensive experiments on public steel surface defect benchmarks demonstrate that SPDCN consistently outperforms state-of-the-art methods, achieving 89.60% mIoU on NEU-Seg with only 3.54M parameters. The source code is publicly available at https://github.com/DWlzm .
Zhongming Liu, Bingbing Jiang, Guangxin Wan +1
School of Artificial Intelligence, Jiangxi Normal University, Nanchang, 330022, China · Jiangxi Provincial Key Laboratory of Intelligent Information Processing and Affective Computing, School of Artificial Intelligence, Jiangxi Normal University, Nanchang, 330022, China
Accurate defect segmentation is critical for industrial inspection, yet dense pixel-level annotations are rarely available. A common workaround is to convert inexpensive bounding boxes into pseudo-masks using foundation segmentation models such as the Segment Anything Model (SAM). However, these pseudo-labels are systematically noisy on industrial surfaces, often hallucinating background structure while missing sparse defects. To address this limitation, a noise-robust box-to-pixel distillation framework, Boxes2Pixels, is proposed that treats SAM as a noisy teacher rather than a source of ground-truth supervision. Bounding boxes are converted into pseudo-masks offline by SAM, and a compact student is trained with (i) a hierarchical decoder over frozen DINOv2 features for semantic stability, (ii) an auxiliary binary localization head to decouple sparse foreground discovery from class prediction, and (iii) a one-sided online self-correction mechanism that relaxes background supervision when the student is confident, targeting teacher false negatives. On a manually annotated wind turbine inspection benchmark, the proposed Boxes2Pixels improves anomaly mIoU by +6.97 and binary IoU by +9.71 over the strongest baseline trained under identical weak supervision. Moreover, online self-correction increases the binary recall by +18.56, while the model employs 80% fewer trainable parameters. Code is available at https://github.com/CLendering/Boxes2Pixels.
Camile Lendering, Erkut Akdag, Egor Bondarev
AIMS Group, Department of Electrical Engineering, Eindhoven University of Technology