PSMP-CLIP: Patch-Prompt SAM and Multi-Semantic Prompting for CLIP-Based Zero-Shot Anomaly Detection

Xuezhi Xiang; Guanghao Wu; Heqi Xiang; Jiayao Liu; Xiaoheng Li; Yiming Chen; Shanjun Zhang

cs.CVSep 15, 2026

PSMP-CLIP: Patch-Prompt SAM and Multi-Semantic Prompting for CLIP-Based Zero-Shot Anomaly Detection

Authors: Xuezhi Xiang, Guanghao Wu, Heqi Xiang, Jiayao Liu, Xiaoheng Li, Yiming Chen, Shanjun Zhang

Organizations: Information and Communication Engineering, Harbin Engineering University, Harbin, China · Key Laboratory of Advanced Marine Communication and Information Technology, Harbin, China · Department of Computer Science, University of Toronto, Toronto, ON M5S 2E4, Canada · The Department of Computer Science, Kanagawa University, Kanagawa, 221-8686, Japan

Abstract

Zero-shot anomaly detection aims to localize anomalies without target-domain samples. Existing CLIP-based methods suffer from coarse anomaly maps and limited semantic prompts. We propose PSMP-CLIP, integrating patch-prompt SAM2 segmentation (PPSS) and multi-semantic guided prompt regularization (MSGPR). PPSS samples prompts directly from intermediate patch features, avoiding threshold drift and guiding SAM2 to produce precise masks. MSGPR uses multiple learnable prompts constrained by semantic anchors to preserve generalization. Experiments on 14 datasets show highly competitive performance, achieving the best pixel-level AUROC on MVTec AD, BTAD, DTD-Synthetic, CVC-ClinicDB, TN3K, Endo, and Kvasir.

Explore similar work

Proximity-CLIP: Text-Guided Semantic Proximity Learning for Zero-Shot Anomaly Detection
Sep 7, 2026Manwen Yang, Leqian Ding, Yu Guo +1Zero-Shot Anomaly Detection Proximity Operations
EntroAD: Structural Entropy-Guided Prompt Adaptation for Zero-Shot Anomaly Detection
May 27, 2026Xinyu Zhao, Qingyun Sun, Jiayi Luo +1Zero-Shot Anomaly Detection Anomalous Frame Detection
Defect-aware Hybrid Prompt Optimization via Progressive Tuning for Zero-Shot Multi-type Anomaly Detection and Segmentation
Dec 10, 2025Nadeem Nazer, Hongkuan Zhou, Lavdim Halilaj +2Zero-Shot Anomaly Detection Anomaly

Sep 7, 2026 · cs.CVJ/K move · Enter open · S save

Proximity-CLIP: Text-Guided Semantic Proximity Learning for Zero-Shot Anomaly Detection

Manwen Yang, Leqian Ding, Yu Guo, Fei Wang

Vision-language models offer a promising approach for zero-shot anomaly detection (ZSAD). However, due to object-centric bias, normal and anomalous text prototypes exhibit a high semantic overlap. While enforcing strict orthogonality between them improves discriminability, mapping highly contiguous visual inputs onto drastically orthogonal prototypes introduces a geometric dilemma, disrupting the pre-trained structural continuity. To address this problem, we propose Proximity-CLIP, a framework that visually calibrates the semantic margin to guide visual adaptation. First, we introduce a visually-calibrated semantic proximity learning mechanism that uses a bounded dynamic regularization to learn an appropriate semantic margin, ensuring discriminative separation while preserving structural alignment. Second, we design an Anomaly Query Module (AQM) driven by these text priors. Using the calibrated anomalous prototype as a semantic query, the AQM actively retrieves localized defect cues from contextual visual patches, mitigating the dilution of subtle anomalies during global pooling. Extensive experiments demonstrate that Proximity-CLIP outperforms current state-of-the-art methods across multiple ZSAD benchmarks with minimal architectural modifications.

Zero-Shot Anomaly Detection Proximity Operations

Open paper Ask about it PDF

Abstract

Explore similar work

Proximity-CLIP: Text-Guided Semantic Proximity Learning for Zero-Shot Anomaly Detection

EntroAD: Structural Entropy-Guided Prompt Adaptation for Zero-Shot Anomaly Detection

Defect-aware Hybrid Prompt Optimization via Progressive Tuning for Zero-Shot Multi-type Anomaly Detection and Segmentation