Zero-shot anomaly detection (ZSAD) aims to detect anomalies in previously unseen classes using auxiliary images, with applications such as industrial visual inspection and medical imaging. Recent vision-language-model-based approaches improve ZSAD by aligning textual and visual representations, while distributional prompting further models prompt variability for unseen categories. However, existing methods do not explicitly combine normality-referenced distributional modeling with hierarchy-specific prompt specialization, limiting their ability to model anomalous semantics relative to normality while addressing both image-level detection and fine-grained localization. We propose Hierarchical Knowledge-Guided Distributional Prompt Learning (HiDist), a normality-referenced hierarchical distributional prompt framework for ZSAD. HiDist models each prompt token with a diagonal Gaussian distribution and generates stochastic prompt variations through Monte Carlo sampling. Rather than learning normal and anomalous distributions independently, anomalous prompt distributions are parameterized as deviations from learned normal references, explicitly encoding their dependency. The resulting distributions are specialized into global and local representations for image-level detection and pixel-level localization, respectively, while an adapted global visual representation guides hierarchical alignment. Experiments on seven industrial and medical datasets demonstrate strong cross-dataset performance across anomaly detection and localization tasks. Extensive ablation and representation analyses further support the effectiveness of the proposed distributional modeling and hierarchy-specific representation learning.
Figures & tables
Figure 1: Comparison of prompt-learning paradigms for zero-shot anomaly detection (ZSAD) and the proposed HiDist. (a) Evolution of prompt design from hand-crafted prompt ensembles and context optimization to flow-based distributional prompting. (b) HiDist structures normality-aware distributional prompts within global–local visual hierarchies and aligns them through knowledge-guided contrastive learning.
Figure 2: Overview of the proposed HiDist framework. Given an auxiliary training image, the vision encoder extracts global and multi-scale local visual representations, which are further adapted through the corresponding visual adapters. In parallel, token-wise distributional prompts are sampled from normal reference distributions and their corresponding anomalous distributions at the global and local levels, and are encoded into semantic prototypes. Knowledge-guided contrastive learning aligns the hierarchical text prototypes with the adapted global visual representation vg , which serves as a shared knowledge anchor.
Figure 3: Illustration of the comprehensive learning strategy proposed in HiDist. (a) Knowledge-guided contrastive learning. (b) Distributional loss. (c) Orthogonality constraint. (d) Kullback–Leibler (KL) divergence regularization.
Figure 4: Comparison of text encoder tuning strategies. (a) Full tuning. (b) Prefix tuning ( Li and Liang, 2021 ) . (c) Low-Rank Adaptation ( Hu et al., 2022 ) . (d) LayerNorm and head tuning.
Method
MVTec-AD
VisA
DTD-Synthetic
AUROC
AP
F1-max
AUROC
AP
F1-max
AUROC
AP
F1-max
OpenCLIP
63.7
84.1
85.0
59.4
68.8
74.7
55.6
81.3
85.7
SigLIP 2
60.5
81.9
84.6
59.7
68.7
74.3
83.1
93.3
90.1
WinCLIP
90.4
95.6
92.7
75.6
78.8
78.2
95.1
97.7
94.1
APRIL-GAN
90.8
95.4
91.7
83.0
84.6
81.6
85.5
94.0
89.0
AnomalyCLIP
91.5
96.2
92.7
81.4
84.9
80.7
93.7
97.3
93.9
Table 1: Comparison of image-level ZSAD results with state-of-the-art methods on industrial datasets. Bolded values indicate the best performance, and underlined values indicate the second-best.
Method
MVTec-AD
VisA
DTD-Synthetic
AUROC
PRO
F1-max
AUROC
PRO
F1-max
AUROC
PRO
F1-max
OpenCLIP
66.3
27.0
9.8
69.3
35.2
3.2
74.9
37.9
6.0
SigLIP 2
66.7
27.2
10.0
69.0
34.9
3.2
75.7
39.2
6.3
WinCLIP
82.3
61.9
24.8
73.2
51.1
9.0
79.5
51.5
16.1
APRIL-GAN
87.6
44.0
43.3
94.2
86.6
32.3
96.5
41.5
65.4
AnomalyCLIP
91.0
81.9
39.0
95.3
85.1
27.3
97.8
91.1
61.5
Table 2: Comparison of pixel-level ZSAD results with state-of-the-art methods on industrial datasets. Bolded values indicate the best performance, and underlined values indicate the second-best.
Method
CVC-ColonDB
CVC-ClinicDB
Kvasir
Endo
AUROC
PRO
F1-max
AUROC
PRO
F1-max
AUROC
PRO
F1-max
AUROC
PRO
F1-max
OpenCLIP
63.6
30.1
18.7
67.0
31.4
24.1
67.1
26.6
34.3
70.3
36.6
35.2
SigLIP 2
64.1
29.5
18.9
67.2
32.4
24.3
67.3
26.1
34.6
70.5
36.4
35.4
WinCLIP
64.8
28.4
21.0
70.3
32.5
27.2
69.7
24.5
35.9
68.2
28.3
32.9
APRIL-GAN
78.4
28.0
32.5
85.4
41.0
44.2
80.5
17.3
48.4
84.1
32.3
51.0
AnomalyCLIP
88.5
79.3
52.4
93.0
81.6
64.2
93.2
59.9
70.7
94.1
86.9
72.6
Table 3: Comparison of ZSAD results with state-of-the-art methods on medical datasets. Bolded values indicate the best performance, and underlined values indicate the second-best.
Figure 5: Qualitative comparison of anomaly localization results across various ZSAD methods. The left panel shows results on industrial datasets, where the first four columns correspond to MVTec-AD ( Bergmann et al., 2019 ) , followed by two columns each for VisA ( Zou et al., 2022 ) and DTD-Synthetic ( Aota et al., 2023 ) . The right panel presents results on medical datasets in the order of CVC-ClinicDB ( Bernal et al., 2015 ) , CVC-ColonDB ( Tajbakhsh et al., 2015 ) , Kvasir ( Jha et al., 2019 ) , and Endo ( Hicks et al., 2021 ) .
Domain
Level
Normality-aware
Visual hierarchy
AUROC
AP/PRO
F1-max
Industrial
Image
✗
✗
90.2
94.3
90.7
✓
✗
92.9
95.3
91.4
✗
✓
93.4
95.5
92.1
✓
✓
93.9
95.9
92.1
Pixel
✗
✗
95.3
87.1
49.8
✓
✗
95.5
87.7
50.9
Table 4: Ablation of normality-aware prompt modeling and visual hierarchy across industrial and medical domains. Results are shown for image-level (classification) and pixel-level (segmentation) tasks, under different combinations of the two proposed components. Bolded values indicate the best performance in each setting.
Figure 6: Visual comparison of anomaly localization performance under different configurations of normality-aware prompting and visual hierarchy. Each row shows examples from MVTec-AD ( Bergmann et al., 2019 ) and CVC-ClinicDB ( Bernal et al., 2015 ) . Columns represent four experimental conditions: (C1) baseline (neither method), (C2) normality-aware prompting only, (C3) visual hierarchy only, and (C4) both methods applied.
Domain
Level
Distribution-aware
AUROC
AP/PRO
F1-max
Industrial
Image
✗
91.9
93.9
91.6
✓
93.9
95.9
92.1
Pixel
✗
95.5
89.2
51.6
✓
95.7
90.3
51.8
Medical
Pixel
✗
95.5
87.1
78.2
✓
96.2
88.9
79.5
Table 5: Ablation of distribution-based prompt on industrial and medical domains. Bolded values indicate the best performance.
Domain
Level
w/o setting
AUROC
AP/PRO
F1-max
Industrial
Image
w/o visual hierarchy loss
93.1
95.2
91.6
w/o distributional loss
92.6
94.8
91.8
w/o orthogonal loss
91.9
95.0
91.6
w/o KL divergence loss
93.3
95.5
92.0
Ours
93.9
95.9
92.1
Pixel
w/o visual hierarchy loss
95.1
86.6
51.1
Table 6: Ablation of comprehensive learning strategy on industrial and medical domains. Bolded values indicate the best performance.
Figure 7: Training convergence curves for each loss component. The plots show: (a) classification loss, (b) segmentation loss, (c) visual hierarchy loss, (d) distributional loss, (e) orthogonality loss, and (f) KL divergence loss.
Domain
Level
Global adapter
Local adapters
AUROC
AP/PRO
F1-max
Industrial
Image
✓
✗
90.5
93.2
91.4
✗
✓
93.2
95.6
91.7
✓
✓
93.9
95.9
92.1
Pixel
✓
✗
91.1
79.2
24.2
✗
✓
94.3
80.3
48.6
✓
✓
95.7
90.3
51.8
Table 7: Ablation results for global and local visual adapters on industrial and medical domains. Bolded values indicate the best performance.
Domain
Level
Number of patch features
AUROC
AP/PRO
F1-max
Industrial
Image
1
90.9
93.0
91.6
2
93.6
95.7
91.8
4
93.9
95.9
92.1
Pixel
1
93.9
79.3
43.2
2
95.0
83.8
48.2
4
95.7
90.3
51.8
Table 8: Ablation of multi-scale patch features on industrial and medical domains. Bolded values indicate the best performance.
Figure 8: Anomaly localization results using multi-scale patch features extracted from the 5th, 12th, 19th, and 26th layers of the vision encoder. Each column shows the anomaly map corresponding to one of the selected layers for an input from MVTec-AD ( Bergmann et al., 2019 ) , highlighting the effect of hierarchical features.
Domain
Level
Fine-tuning method
AUROC
AP/PRO
F1-max
Industrial
Image
Frozen
93.6
95.7
92.0
Full tuning
92.7
94.6
91.8
Prefix-tuning
67.8
79.4
85.9
LoRA
92.0
94.7
91.5
LayerNorm and head tuning
93.9
95.9
92.1
Pixel
Frozen
95.3
87.2
50.6
Table 9: Ablation of text encoder fine-tuning strategies on industrial and medical domains. Bolded values indicate the best performance.
Figure 9: Visualization of learned prompt distribution parameters. The top row shows the mean vectors ( μ ) and the bottom row shows the corresponding log-variance values ( logσ2 ) for each prompt token.
Figure 10: Quantitative analysis of distributional differences across global/local and normal/anomaly prompts using KL divergence, Wasserstein distance, and JS divergence.
Figure 11: t -SNE visualizations of learned representation spaces.
Figure 12: Comparison of image- and pixel-level performance under varying prompt lengths and K -sampling on MVTec-AD ( Bergmann et al., 2019 ) .
Zero-shot anomaly detection (ZSAD) aims to detect and localize anomalies in unseen categories without access to target-specific training data. Although recent CLIP-based methods have demonstrated promising generalization through vision-language alignment, they remain limited in capturing diverse anomaly semantics and subtle local variations. To address these limitations, we propose VFAD, a unified framework that combines variational semantic prompting with frequency-adaptive representation learning. Specifically, we introduce a Variational Semantic Prompt Extractor (VSPE), which adaptively aggregates anomaly-relevant local semantics from dense patch tokens and regularizes them through a variational information bottleneck, thereby incorporating fine-grained visual cues and enabling more precise cross-modal alignment. Furthermore, we develop a Frequency-Adaptive Representation Aggregation (FARA) module that leverages wavelet-based frequency decomposition and frequency-specific expert aggregation to enhance anomaly-discriminative visual representations. By jointly strengthening semantic guidance and visual representation learning, VFAD improves both anomaly discrimination and fine-grained localization. Extensive experiments on 13 industrial and medical benchmarks demonstrate that VFAD consistently outperforms existing state-of-the-art ZSAD methods across diverse anomaly scenarios. The code will be publicly available upon publication.
Peng Chen, Kaige Li, Wei Wang +5
School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University, Shenzhen, China
Zero-shot anomaly detection (ZSAD) requires detecting and localizing anomalies without access to target-class anomaly samples. Mainstream methods rely on vision-language models (VLMs) such as CLIP: they build hand-crafted or learned prompt sets for normal and abnormal semantics, then compute image-text similarities for open-set discrimination. While effective, this paradigm depends on a text encoder and cross-modal alignment, which can lead to training instability and parameter redundancy. This work revisits the necessity of the text branch in ZSAD and presents VisualAD, a purely visual framework built on Vision Transformers. We introduce two learnable tokens within a frozen backbone to directly encode normality and abnormality. Through multi-layer self-attention, these tokens interact with patch tokens, gradually acquiring high-level notions of normality and anomaly while guiding patches to highlight anomaly-related cues. Additionally, we incorporate a Spatial-Aware Cross-Attention (SCA) module and a lightweight Self-Alignment Function (SAF): SCA injects fine-grained spatial information into the tokens, and SAF recalibrates patch features before anomaly scoring. VisualAD achieves state-of-the-art performance on 13 zero-shot anomaly detection benchmarks spanning industrial and medical domains, and adapts seamlessly to pretrained vision backbones such as the CLIP image encoder and DINOv2. Code: https://github.com/7HHHHH/VisualAD
Yanning Hou, Peiyuan Li, Zirui Liu +4
College of Intelligence Science and Technology, National University of Defense Technology, Changsha, China · School of Artificial Intelligence, Anhui University, Hefei, China · State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology, Anhui University, Hefei, China
Vision-language models have recently shown strong generalization in zero-shot anomaly detection (ZSAD), enabling the detection of unseen anomalies without task-specific supervision. However, existing approaches typically rely on fixed textual prompts, which struggle to capture complex semantics, and focus solely on spatial-domain features, limiting their ability to detect subtle anomalies. To address these challenges, we propose a wavelet-enhanced mixture-of-experts prompt learning method for ZSAD. Specifically, a variational autoencoder is employed to model global semantic representations and integrate them into prompts to enhance adaptability to diverse anomaly patterns. Wavelet decomposition extracts multi-frequency image features that dynamically refine textual embeddings through cross-modal interactions. Furthermore, a semantic-aware mixture-of-experts module is introduced to aggregate contextual information. Extensive experiments on 14 industrial and medical datasets demonstrate the effectiveness of the proposed method.
Peng Chen, Chao Huang
School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University, Shenzhen