SPG: Sparse-Projected Guides with Sparse Autoencoders for Zero-Shot Anomaly Detection

Tomoyasu Nanaumi; Yukino Tsuzuki; Junichi Okubo; Junichiro Fujii; Takayoshi Yamashita

cs.CVApr 3, 2026

SPG: Sparse-Projected Guides with Sparse Autoencoders for Zero-Shot Anomaly Detection

Authors: Tomoyasu Nanaumi, Yukino Tsuzuki, Junichi Okubo, Junichiro Fujii, Takayoshi Yamashita

Organizations: Yachiyo Engineering Co., Ltd., Japan · Chubu University, Japan

Abstract

We study zero-shot anomaly detection and segmentation using frozen foundation model features, where all learnable parameters are trained only on a labeled auxiliary dataset and deployed to unseen target categories without any target-domain adaptation. Existing prompt-based approaches use handcrafted or learned prompt embeddings as reference vectors for normal/anomalous states. We propose Sparse-Projected Guides (SPG), a prompt-free framework that learns sparse guide coefficients in the Sparse Autoencoder (SAE) latent space, which generate normal/anomaly guide vectors via the SAE dictionary. SPG employs a two stage learning strategy on the labeled auxiliary dataset: (i) train an SAE on patch-token features, and (ii) optimize only guide coefficients using auxiliary pixel-level masks while freezing the backbone and SAE. On MVTec AD and VisA under cross-dataset zero-shot settings, SPG achieves competitive image-level detection and strong pixel-level segmentation; with DINOv3, SPG attains the highest pixellevel AUROC among the compared methods. We also report SPG instantiated with OpenCLIP (ViT-L/14@336px) to align the backbone with CLIP-based baselines. Moreover, the learned guide coefficients trace decisions back to a small set of dictionary atoms, revealing category-general and category-specific factors.

Figures & tables

Figure 1: Overview of SPG (Sparse-Projected Guides). Stage 1 trains a Sparse Autoencoder (SAE) on patch-token features extracted by a frozen visual encoder to learn a dictionary in which normal/anomaly guide vectors can be parameterized as sparse coefficients. Stage 2 freezes both the encoder and SAE, and learns only non-negative, sparsity-regularized guide coefficients that generate normal/anomaly guide vectors via the SAE dictionary. At inference, cosine similarities between patch tokens and the two guides are converted into an anomaly probability map via a two-class softmax, which is upsampled to the image resolution. Image-level anomaly scores are computed by aggregating the anomaly map (default: max pooling; see Eq. 12 for the log-sum-exp generalization with temperature $\tau$ ).

Image-level (Detection)
Method	VisA $\rightarrow$ MVTec		MVTec $\rightarrow$ VisA
	AUROC	AP	AUROC	AP
WinCLIP	91.8	96.5	78.1	81.2
AnomalyCLIP	91.5	96.2	82.1	85.4
AdaCLIP	89.2	–	85.8	–
VCP-CLIP	92.1	96.9	83.8	87.6

Table 1: Cross-dataset zero-shot performance on MVTec AD and VisA (%). Models are trained on only the other dataset as the unseen target, following the cross-dataset zero-shot protocol with no target-domain adaptation. We report image-level AUROC / AP and pixel-level AUROC / AUPRO (higher is better). Baseline results are taken from the original papers, and ”–” denotes metrics not reported. Note that we report SPG with OpenCLIP to match the backbone used by prior CLIP-based methods, and with DINOv3 ViT-L to demonstrate that SPG is not restricted to VLM backbones.

Figure 2: Sensitivity to SAE hyperparameters. The Heatmaps summarize cross-dataset performance when sweeping the SAE dictionary width $C$ and the TopK sparsity level $k$ . The top row reports VisA $\rightarrow$ MVTec AD transfer and the bottom row reports MVTec $\rightarrow$ VisA transfer. Each column corresponds to image-level AUROC / AP and pixel-level AUROC / AUPRO, respectively, highlighting that the optimal $(C,k)$ can depend on both the transfer direction and the evaluation metric.

Figure 3: Ablation of image-level anomaly-score aggregation via temperature-controlled log-sum-exp pooling. We vary the temperature $\tau$ in log-sum-exp pooling applied to the upsampled anomaly map to produce an image-level anomaly score. Curves report image-level AUROC and AP for both transfer directions (left: VisA $\rightarrow$ MVTec; right: MVTec $\rightarrow$ VisA). The figure compares the $\tau\rightarrow 0$ (max-like) regime against larger $\tau$ (mean-like) aggregation, showing that more max-like pooling better preserves localized high-confidence anomaly responses for detection.

Figure 4: Effect of the visual backbone on SPG under cross-dataset transfer. We instantiate SPG with different frozen encoders and rerun Stage 1 (SAE training on auxiliary patch tokens) and Stage 2 (guide-coefficient learning) for each backbone under the same protocol. Results are reported for two transfer directions (left: VisA

\rightarrow

MVTec; right: MVTec

\rightarrow

VisA). Each plot reports image-level AUROC/AP and pixel-level AUROC/AUPRO, isolating how backbone choice influences both detection and segmentation quality within the same guide-based scoring rule.

Figure 5: Qualitative interpretation of SAE dictionary atoms emphasized by the learned anomaly guide. We select representative atoms from the anomaly guide’s active set by choosing those with the largest learned anomaly coefficients. Each atom is visualized by retrieving top-activating patches from the auxiliary dataset and by overlaying an upsampled activation heatmap on the corresponding images for spatial context. This qualitative analysis illustrates that SPG’s anomaly criterion can be traced to a sparse subset of SAE atoms, some of which appear broadly activated across categories while others are more category-biased.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure S1 : Stage-2 guide learning with explicit TopK sparsification. We replace the ReLU+ $\ell_{1}$ parameterization in the main paper with $k_{g}$ -sparse guide coefficients obtained by TopK projection, and plot performance against the total guide sparsity $\|\tilde{\bm{z}}_{N}\|_{0}+\|\tilde{\bm{z}}_{A}\|_{0}$ . Orange markers denote the effective sparsity obtained under the ReLU+ $\ell_{1}$ formulation in the main paper. In both transfer directions, performance improves as the sparsity budget increases, indicating that overly small TopK budgets over-constrain the guides.

Category	WinCLIP	AnomalyCLIP	AdaCLIP	VCP-CLIP	AA-CLIP	AdaptCLIP	SPG (OpenCLIP)	SPG (DINOv3)
Bottle	99.2	89.3	94.4	–	–	90.7	78.7	99.3
Cable	86.5	69.8	90.6	–	–	83.6	57.3	86.4
Capsule	72.9	89.9	91.5	–	–	95.2	90.0	89.9
Carpet	100.0	100.0	82.1	–	–	100.0	92.3	97.8
Grid	98.8	97.0	90.0	–	–	98.9	76.2	99.4
Hazelnut	93.9	97.2	80.2	–	–	94.8	84.7	93.5

Appendix

Table S1 : Category-wise image-level AUROC (%) under the cross-dataset zero-shot setting (VisA

\rightarrow

MVTec AD). Baseline results are taken from the original papers, and ”–” denotes metrics not reported.

Category	WinCLIP	AnomalyCLIP	AdaCLIP	VCP-CLIP	AA-CLIP	AdaptCLIP	SPG (OpenCLIP)	SPG (DINOv3)
Bottle	99.8	97.0	–	–	–	97.1	92.6	99.8
Cable	91.2	81.4	–	–	–	89.9	67.8	92.4
Capsule	91.5	97.9	–	–	–	99.0	97.8	98.0
Carpet	100.0	100.0	–	–	–	99.9	97.2	99.2
Grid	99.6	99.1	–	–	–	99.5	88.9	99.8
Hazelnut	96.9	98.6	–	–	–	97.0	90.5	97.1

Appendix

Table S2 : Category-wise image-level AP (%) under the cross-dataset zero-shot setting (VisA

\rightarrow

MVTec AD). Baseline results are taken from the original papers, and ”–” denotes metrics not reported.

Category	WinCLIP	AnomalyCLIP	AdaCLIP	VCP-CLIP	AA-CLIP	AdaptCLIP	SPG (OpenCLIP)	SPG (DINOv3)
Bottle	89.5	90.4	90.4	94.1	–	92.4	89.5	96.6
Cable	77.0	78.9	79.8	78.6	–	76.6	74.2	80.4
Capsule	86.9	95.8	82.3	96.6	–	95.8	93.7	95.6
Carpet	95.4	98.8	97.3	99.6	–	99.2	98.6	99.5
Grid	82.2	97.3	96.9	97.9	–	97.4	96.9	99.3
Hazelnut	94.3	97.1	97.8	97.7	–	97.6	94.6	97.7

Appendix

Table S3 : Category-wise pixel-level AUROC (%) under the cross-dataset zero-shot setting (VisA

\rightarrow

MVTec AD). Baseline results are taken from the original papers, and ”–” denotes metrics not reported.

Category	WinCLIP	AnomalyCLIP	AdaCLIP	VCP-CLIP	AA-CLIP	AdaptCLIP	SPG (OpenCLIP)	SPG (DINOv3)
Bottle	76.4	80.9	–	88.5	–	–	72.4	91.4
Cable	42.9	64.4	–	67.7	–	–	39.4	73.7
Capsule	62.1	87.2	–	94.0	–	–	70.5	86.0
Carpet	84.1	90.1	–	98.4	–	–	85.7	98.2
Grid	57.0	75.6	–	92.4	–	–	75.6	94.7
Hazelnut	81.6	92.4	–	84.9	–	–	78.1	86.4

Appendix

Table S4 : Category-wise pixel-level AUPRO (%) under the cross-dataset zero-shot setting (VisA

\rightarrow

MVTec AD). Baseline results are taken from the original papers, and ”–” denotes metrics not reported.

Category	WinCLIP	AnomalyCLIP	AdaCLIP	VCP-CLIP	AA-CLIP	AdaptCLIP	SPG (OpenCLIP)	SPG (DINOv3)
Candle	95.4	79.3	96.0	–	–	87.4	84.5	74.7
Capsules	85.0	81.5	85.1	–	–	93.9	92.4	95.8
Cashew	92.1	76.3	91.8	–	–	86.1	84.3	70.2
Chewinggum	96.5	97.4	96.4	–	–	96.7	96.2	97.7
Fryum	80.3	93.0	93.0	–	–	91.5	91.9	94.9
Macaroni1	76.2	87.2	91.6	–	–	83.3	77.3	72.7

Appendix

Table S5 : Category-wise image-level AUROC (%) under the cross-dataset zero-shot setting (MVTec AD

\rightarrow

VisA). Baseline results are taken from the original papers, and ”–” denotes metrics not reported.

Category	WinCLIP	AnomalyCLIP	AdaCLIP	VCP-CLIP	AA-CLIP	AdaptCLIP	SPG (OpenCLIP)	SPG (DINOv3)
Candle	95.8	81.1	–	–	–	90.5	88.0	81.2
Capsules	90.9	88.7	–	–	–	96.8	96.3	98.1
Cashew	96.4	89.4	–	–	–	94.0	93.0	84.0
Chewinggum	98.6	98.9	–	–	–	98.7	98.1	99.1
Fryum	90.1	96.8	–	–	–	95.9	96.4	97.7
Macaroni1	75.8	86.0	–	–	–	85.1	82.5	77.8

Appendix

Table S6 : Category-wise image-level AP (%) under the cross-dataset zero-shot setting (MVTec AD

\rightarrow

VisA). Baseline results are taken from the original papers, and ”–” denotes metrics not reported.

Category	WinCLIP	AnomalyCLIP	AdaCLIP	VCP-CLIP	AA-CLIP	AdaptCLIP	SPG (OpenCLIP)	SPG (DINOv3)
Candle	88.9	98.8	98.9	99.2	–	98.7	98.8	98.9
Capsules	81.6	95.0	98.6	98.7	–	94.2	95.6	99.4
Cashew	84.7	93.8	95.9	93.1	–	93.4	91.7	96.2
Chewinggum	93.3	99.3	99.6	99.5	–	99.5	99.5	99.5
Fryum	88.5	94.6	94.4	94.6	–	95.3	92.0	94.7
Macaroni1	70.9	98.3	99.5	99.6	–	98.1	98.4	98.4

Appendix

Table S7 : Category-wise pixel-level AUROC (%) under the cross-dataset zero-shot setting (MVTec AD

\rightarrow

VisA). Baseline results are taken from the original papers, and ”–” denotes metrics not reported.

Category	WinCLIP	AnomalyCLIP	AdaCLIP	VCP-CLIP	AA-CLIP	AdaptCLIP	SPG (OpenCLIP)	SPG (DINOv3)
Candle	83.5	96.2	–	96.2	–	–	96.5	89.9
Capsules	35.3	78.5	–	91.2	–	–	81.8	98.6
Cashew	76.4	91.6	–	95.6	–	–	93.8	88.7
Chewinggum	70.4	91.2	–	92.2	–	–	94.8	92.8
Fryum	77.4	86.8	–	92.2	–	–	90.6	92.9
Macaroni1	34.3	89.8	–	97.5	–	–	90.2	91.7

Appendix

Table S8 : Category-wise pixel-level AUPRO (%) under the cross-dataset zero-shot setting (MVTec AD

\rightarrow

VisA). Baseline results are taken from the original papers, and ”–” denotes metrics not reported.

Zero-Shot Anomaly Detection

Zero-shot anomaly detection finds unusual patterns in unseen domains without task-specific labeled examples.

See all 85 papers

Sep 15, 2026

PSMP-CLIP: Patch-Prompt SAM and Multi-Semantic Prompting for CLIP-Based Zero-Shot Anomaly Detection

Zero-shot anomaly detection aims to localize anomalies without target-domain samples. Existing CLIP-based methods suffer from coarse anomaly maps and limited semantic prompts. We propose PSMP-CLIP, integrating patch-prompt SAM2 segmentation (PPSS) and multi-semantic guided prompt regularization (MSGPR). PPSS samples prompts directly from intermediate patch features, avoiding threshold drift and guiding SAM2 to produce precise masks. MSGPR uses multiple learnable prompts constrained by semantic anchors to preserve generalization. Experiments on 14 datasets show highly competitive performance, achieving the best pixel-level AUROC on MVTec AD, BTAD, DTD-Synthetic, CVC-ClinicDB, TN3K, Endo, and Kvasir.

Xuezhi Xiang, Guanghao Wu, Heqi Xiang +4

Information and Communication Engineering, Harbin Engineering University, Harbin, China · Key Laboratory of Advanced Marine Communication and Information Technology, Harbin, China · Department of Computer Science, University of Toronto, Toronto, ON M5S 2E4, Canada +1

Promptable Image Segmentation Anomaly Localization Zero-Shot Anomaly Detection Visual Prompt Learning

Ask PDF

Mar 9, 2026

VisualAD: Language-Free Zero-Shot Anomaly Detection via Vision Transformer

Zero-shot anomaly detection (ZSAD) requires detecting and localizing anomalies without access to target-class anomaly samples. Mainstream methods rely on vision-language models (VLMs) such as CLIP: they build hand-crafted or learned prompt sets for normal and abnormal semantics, then compute image-text similarities for open-set discrimination. While effective, this paradigm depends on a text encoder and cross-modal alignment, which can lead to training instability and parameter redundancy. This work revisits the necessity of the text branch in ZSAD and presents VisualAD, a purely visual framework built on Vision Transformers. We introduce two learnable tokens within a frozen backbone to directly encode normality and abnormality. Through multi-layer self-attention, these tokens interact with patch tokens, gradually acquiring high-level notions of normality and anomaly while guiding patches to highlight anomaly-related cues. Additionally, we incorporate a Spatial-Aware Cross-Attention (SCA) module and a lightweight Self-Alignment Function (SAF): SCA injects fine-grained spatial information into the tokens, and SAF recalibrates patch features before anomaly scoring. VisualAD achieves state-of-the-art performance on 13 zero-shot anomaly detection benchmarks spanning industrial and medical domains, and adapts seamlessly to pretrained vision backbones such as the CLIP image encoder and DINOv2. Code: https://github.com/7HHHHH/VisualAD

Yanning Hou, Peiyuan Li, Zirui Liu +4

College of Intelligence Science and Technology, National University of Defense Technology, Changsha, China · School of Artificial Intelligence, Anhui University, Hefei, China · State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology, Anhui University, Hefei, China

Zero-Shot Anomaly Detection Vision Foundation Model Adaptation Medical Image Anomaly Detection

Ask PDF

Jul 31, 2026

VFAD: Variational Semantic Prompting Meets Frequency-Adaptive Representation Learning for Zero-Shot Anomaly Detection

Zero-shot anomaly detection (ZSAD) aims to detect and localize anomalies in unseen categories without access to target-specific training data. Although recent CLIP-based methods have demonstrated promising generalization through vision-language alignment, they remain limited in capturing diverse anomaly semantics and subtle local variations. To address these limitations, we propose VFAD, a unified framework that combines variational semantic prompting with frequency-adaptive representation learning. Specifically, we introduce a Variational Semantic Prompt Extractor (VSPE), which adaptively aggregates anomaly-relevant local semantics from dense patch tokens and regularizes them through a variational information bottleneck, thereby incorporating fine-grained visual cues and enabling more precise cross-modal alignment. Furthermore, we develop a Frequency-Adaptive Representation Aggregation (FARA) module that leverages wavelet-based frequency decomposition and frequency-specific expert aggregation to enhance anomaly-discriminative visual representations. By jointly strengthening semantic guidance and visual representation learning, VFAD improves both anomaly discrimination and fine-grained localization. Extensive experiments on 13 industrial and medical benchmarks demonstrate that VFAD consistently outperforms existing state-of-the-art ZSAD methods across diverse anomaly scenarios. The code will be publicly available upon publication.

Peng Chen, Kaige Li, Wei Wang +5

School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University, Shenzhen, China

VLM Adaptation Frequency-Domain Feature Learning Medical Image Anomaly Detection Industrial Anomaly Detection Anomaly Localization Zero-Shot Anomaly Detection

Ask PDF

Abstract

Zero-Shot Anomaly Detection

Explore similar work

PSMP-CLIP: Patch-Prompt SAM and Multi-Semantic Prompting for CLIP-Based Zero-Shot Anomaly Detection

VisualAD: Language-Free Zero-Shot Anomaly Detection via Vision Transformer

VFAD: Variational Semantic Prompting Meets Frequency-Adaptive Representation Learning for Zero-Shot Anomaly Detection