Organizations: College of Intelligence Science and Technology, National University of Defense Technology, Changsha, China · School of Artificial Intelligence, Anhui University, Hefei, China · State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology, Anhui University, Hefei, China
Zero-shot anomaly detection (ZSAD) requires detecting and localizing anomalies without access to target-class anomaly samples. Mainstream methods rely on vision-language models (VLMs) such as CLIP: they build hand-crafted or learned prompt sets for normal and abnormal semantics, then compute image-text similarities for open-set discrimination. While effective, this paradigm depends on a text encoder and cross-modal alignment, which can lead to training instability and parameter redundancy. This work revisits the necessity of the text branch in ZSAD and presents VisualAD, a purely visual framework built on Vision Transformers. We introduce two learnable tokens within a frozen backbone to directly encode normality and abnormality. Through multi-layer self-attention, these tokens interact with patch tokens, gradually acquiring high-level notions of normality and anomaly while guiding patches to highlight anomaly-related cues. Additionally, we incorporate a Spatial-Aware Cross-Attention (SCA) module and a lightweight Self-Alignment Function (SAF): SCA injects fine-grained spatial information into the tokens, and SAF recalibrates patch features before anomaly scoring. VisualAD achieves state-of-the-art performance on 13 zero-shot anomaly detection benchmarks spanning industrial and medical domains, and adapts seamlessly to pretrained vision backbones such as the CLIP image encoder and DINOv2. Code: https://github.com/7HHHHH/VisualAD
Figures & tables
Figure 1: Exploratory study motivating this work. (a) AnomalyCLIP derives normal/abnormal prototypes via trainable text prompts and a text encoder. (b) A purely visual variant removes the text branch and directly learns two visual prototypes, achieving comparable or slightly better results on VisA and MVTec with 99% fewer parameters. Radar and line plots show that our variant maintains similar accuracy but with much smoother evaluation curves, whereas AnomalyCLIP oscillates noticeably.
Figure 2: Overview of VisualAD. Two learnable global tokens (anomaly and normal) are inserted into a frozen ViT. For intermediate layers ℓ , SCA uses a few anchor queries with positional encoding and token-guided gating to aggregate localized spatial evidence, yielding enhanced tokens t~a(ℓ),t~n(ℓ) . In parallel, each layer’s patch features are recalibrated by a SAF, denoted as Fℓ . The cosine-similarity difference between enhanced tokens and recalibrated patches forms layer-wise anomaly maps, which are upsampled and summed to obtain the final anomaly map; the image-level score is the mean of the top- k responses.
Domain
Metric
Dataset
WinCLIP [ 23 ]
APRIL-GAN [ 10 ]
CLIP-AD [ 11 ]
AnomalyCLIP [ 58 ]
AdaCLIP [ 9 ]
VisualAD(CLIP [ 41 ] )
VisualAD(DINOv2 [ 37 ] )
ViT-L/14@336px
ViT-L/14@336px
ViT-L/14@336px
ViT-L/14@336px
ViT-L/14@336px
ViT-L/14@336px
ViT-L/14
Industrial
Image-level (AUROC, F1 -max, AP)
MVTec-AD [ 6 ]
(90.4, 92.7 , 95.6)
(86.1, 90.4, 93.6)
(74.1, 86.3, 88.1)
(91.6, 92.7 , 96.2)
( 92.0 , 92.7 , 96.4 )
( 92.2 , 93.2 , 96.7 )
(90.1, 92.4, 94.8)
VisA [ 60 ]
(75.6, 78.2, 78.8)
(77.4, 78.6, 80.9)
(66.2, 74.3, 71.4)
(81.0, 80.3, 84.4)
(79.7, 79.63, 83.2)
( 84.7 , 82.5 , 87.6 )
( 83.1 , 81.4 , 86.8 )
BTAD [ 35 ]
(68.2, 67.8, 70.9)
(73.7, 68.7, 69.9)
(66.7, 65.9, 67.3)
(88.7, 86.0, 90.6)
( 90.0 , 87.2 , 91.5 )
( 94.9 , 93.9 , 97.0 )
(88.2, 84.7, 89.7)
KSDD2 [ 47 ]
(93.5, 86.4, 94.2)
(90.4, 82.9, 92.0)
(81.7, 75.0, 85.5)
(91.9, 84.5, 93.4)
(94.9, 90.3, 96.2)
( 98.0 , 93.9 , 98.3 )
( 97.7 , 93.1 , 98.1 )
DAGM [ 49 ]
(91.8, 75.8, 79.5)
(94.4, 80.3, 83.9)
(62.1, 37.1, 32.3)
(98.0, 90.6, 92.4)
( 98.3 , 91.5 , 94.2 )
( 99.5 , 95.0 , 97.8 )
(93.2, 83.9, 86.1)
Table 1: Comparison of zero-shot anomaly detection methods on industrial and medical benchmarks. For each dataset, we report image-level metrics (AUROC, F1 -max, AP) and pixel-level metrics (AUROC, F1 -max, AP, PRO). VisualAD(CLIP) and VisualAD(DINOv2) denote our method instantiated with CLIP and DINOv2 backbones, respectively. The best and second-best results for each metric and dataset are highlighted in red and blue .
Figure 3: Qualitative comparison of anomaly segmentation results for different ZSAD methods. The first five columns show images from industrial datasets, while the last three columns correspond to medical datasets.
Ablation
SCA
SAF
Focal
Dice
Ctr
Image-level
Pixel-level
Module
✗
✓
✓
✓
✓
(82.3, 85.4)
(95.3, 27.4)
✓
✗
✓
✓
✓
(50.5, 58.9)
(87.9, 3.5)
✗
✗
✓
✓
✓
(48.0, 56.1)
(38.3, 0.8)
Loss
✓
✓
✗
✓
✓
(83.5, 86.8)
(95.3, 26.6)
✓
✓
✓
✗
✓
(83.4, 86.5)
(95.3, 27.3)
✓
✓
✓
✓
✗
(84.2, 87.0)
(95.4, 27.6)
Table 2: Ablation study on different modules and loss components using ViT-L/14@336px. Both Image-level and Pixel-level metrics are reported as (AUROC, AP).
Figure 4: Radar plot comparing different pretrained ViT backbones under the ZSAD setting. We evaluate CLIP and DINO-based variants from small to large model scales. Larger backbones and higher input resolutions generally lead to consistent improvements in both pixel-level and sample-level metrics, with CLIP’s ViT-L/14@336px giving the strongest sample-level accuracy and DINO’s ViT-g/14 delivering the highest pixel-level precision.
Table 7
Positional modeling
Pixel-level
Image-level
None (no PE)
(93.0, 29.0, 22.5)
(82.4, 78.1, 84.5)
Learnable 1D Absolute
( 95.8 , 34.6 , 28.4 )
( 84.7 , 82.5 , 87.6 )
Fixed 1D Sinusoidal
( 95.2 , 33.8 , 27.6 )
( 84.1 , 81.9 , 86.8 )
Learnable 2D Absolute
(95.0, 33.2, 27.0)
(83.8, 81.6, 86.0)
Fixed 2D Sinusoidal
(94.6, 32.4, 26.1)
(83.4, 80.8, 85.4)
Learnable 2D Relative Bias
(94.4, 31.9, 25.7)
(83.0, 80.2, 84.9)
Table 5: Effect of positional modeling in the SCA module on VisA. Metrics are reported as (AUROC, F1 -max, AP). Best and second-best results are highlighted in bold and underline .
Figure 5: PCA visualization of feature distributions under three configurations. From left to right: vanilla CLIP features, CLIP with normal/anomaly tokens, and CLIP with both tokens and the MLP-based transformation. The anomaly cluster (dark) becomes progressively more compact and moves farther from the normal cluster, while the variance concentrates along a single dominant axis, indicating increasingly discriminative representations for anomaly separation.
Figure 6: Layer-wise anchor attention maps on the carpet category. Anchors within the same layer attend to similar regions, whereas attention patterns differ across layers: mid-level layers focus on fine-grained defect boundaries, while deeper layers emphasize broader normal regions. This complementarity between layers provides both local anomaly cues and holistic normal context for accurate localization.
Zero-shot anomaly detection aims to identify defects in unseen categories without target-specific training. Existing methods usually apply the same feature transformation to all samples, treating normal and anomalous data uniformly despite their fundamentally asymmetric distributions, compact normals versus diverse anomalies. We instead exploit this natural asymmetry by proposing AVA-DINO, an anomaly-aware vision-language adaptation framework with dual specialized branches for normal and anomalous patterns that adapt frozen DINOv3 visual features. During training on auxiliary data, the two branches are learned jointly with a text-guided routing mechanism and explicit routing regularization that encourages branch specialization. At test time, only the input image and fixed, predefined language descriptions are used to dynamically combine the two branches, enabling an asymmetric activation. This design prevents degenerate uniform routing and allows context-specific feature transformations. Experiments across nine industrial and medical benchmarks demonstrate state-of-the-art performance, achieving 93.5% image-AUROC on MVTec-AD and strong cross-domain generalization to medical imaging without domain-specific fine-tuning. https://github.com/aqeeelmirza/AVA-DINO
Muhammad Aqeel, Maham Nazir, Uzair Khan +2
Dept. of Engineering for Innovation Medicine, University of Verona, Italy · School of Computer Science and Engineering, Beihang University, China · Dept. of Engineering for Innovation +1
Zero-shot anomaly detection (ZSAD) aims to detect and localize anomalies in unseen categories without access to target-specific training data. Although recent CLIP-based methods have demonstrated promising generalization through vision-language alignment, they remain limited in capturing diverse anomaly semantics and subtle local variations. To address these limitations, we propose VFAD, a unified framework that combines variational semantic prompting with frequency-adaptive representation learning. Specifically, we introduce a Variational Semantic Prompt Extractor (VSPE), which adaptively aggregates anomaly-relevant local semantics from dense patch tokens and regularizes them through a variational information bottleneck, thereby incorporating fine-grained visual cues and enabling more precise cross-modal alignment. Furthermore, we develop a Frequency-Adaptive Representation Aggregation (FARA) module that leverages wavelet-based frequency decomposition and frequency-specific expert aggregation to enhance anomaly-discriminative visual representations. By jointly strengthening semantic guidance and visual representation learning, VFAD improves both anomaly discrimination and fine-grained localization. Extensive experiments on 13 industrial and medical benchmarks demonstrate that VFAD consistently outperforms existing state-of-the-art ZSAD methods across diverse anomaly scenarios. The code will be publicly available upon publication.
Peng Chen, Kaige Li, Wei Wang +5
School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University, Shenzhen, China
Vision foundation models have enabled strong training-free anomaly detection (AD). However, most existing approaches rely primarily on independent local patch features, leaving the global contextual information encoded by Vision Transformers (ViTs) underexploited. In this work, we identify the dual characteristics of the ViT [CLS] token: its embedding provides anomaly-invariant global semantic representation, while its attention maps implicitly highlight spatially abnormal regions. Building on this observation, we propose a fully automated AD framework leveraging global context to remove manual tunings. Our framework introduces (1) an automatic augmentation selection strategy driven by [CLS]-level semantic consistency, and (2) an attention-guided feature reweighting mechanism that dynamically adjusts patch contributions according to [CLS] attention saliency. By integrating these components over multi-level features, our method achieves stable anomaly scoring and precise localization without training or parameter tuning. Under the one-shot setting, it achieves Image-AUC scores of 97.7%, 93.2%, and 84.5% on MVTec-AD, VisA, and Real-IAD. Using a single fixed configuration across categories, backbones, and datasets, the method establishes a new state-of-the-art for plug-and-play, training-free anomaly detection while maintaining strong robustness and practical scalability.
Jyun-Ze Tang, Po-Han Huang, Ming-Ching Chang +2
Inventec Corporation, Taipei, Taiwan · University at Albany, State University of New York, NY, USA