SPG: Sparse-Projected Guides with Sparse Autoencoders for Zero-Shot Anomaly Detection

Tomoyasu Nanaumi; Yukino Tsuzuki; Junichi Okubo; Junichiro Fujii; Takayoshi Yamashita

cs.CVApr 3, 2026

SPG: Sparse-Projected Guides with Sparse Autoencoders for Zero-Shot Anomaly Detection

Authors: Tomoyasu Nanaumi, Yukino Tsuzuki, Junichi Okubo, Junichiro Fujii, Takayoshi Yamashita

Organizations: Yachiyo Engineering Co., Ltd., Japan · Chubu University, Japan

Abstract

We study zero-shot anomaly detection and segmentation using frozen foundation model features, where all learnable parameters are trained only on a labeled auxiliary dataset and deployed to unseen target categories without any target-domain adaptation. Existing prompt-based approaches use handcrafted or learned prompt embeddings as reference vectors for normal/anomalous states. We propose Sparse-Projected Guides (SPG), a prompt-free framework that learns sparse guide coefficients in the Sparse Autoencoder (SAE) latent space, which generate normal/anomaly guide vectors via the SAE dictionary. SPG employs a two stage learning strategy on the labeled auxiliary dataset: (i) train an SAE on patch-token features, and (ii) optimize only guide coefficients using auxiliary pixel-level masks while freezing the backbone and SAE. On MVTec AD and VisA under cross-dataset zero-shot settings, SPG achieves competitive image-level detection and strong pixel-level segmentation; with DINOv3, SPG attains the highest pixellevel AUROC among the compared methods. We also report SPG instantiated with OpenCLIP (ViT-L/14@336px) to align the backbone with CLIP-based baselines. Moreover, the learned guide coefficients trace decisions back to a small set of dictionary atoms, revealing category-general and category-specific factors.

Figures & tables

Figure 1: Overview of SPG (Sparse-Projected Guides). Stage 1 trains a Sparse Autoencoder (SAE) on patch-token features extracted by a frozen visual encoder to learn a dictionary in which normal/anomaly guide vectors can be parameterized as sparse coefficients. Stage 2 freezes both the encoder and SAE, and learns only non-negative, sparsity-regularized guide coefficients that generate normal/anomaly guide vectors via the SAE dictionary. At inference, cosine similarities between patch tokens and the two guides are converted into an anomaly probability map via a two-class softmax, which is upsampled to the image resolution. Image-level anomaly scores are computed by aggregating the anomaly map (default: max pooling; see Eq. 12 for the log-sum-exp generalization with temperature $\tau$ ).

Image-level (Detection)
Method	VisA $\rightarrow$ MVTec		MVTec $\rightarrow$ VisA
	AUROC	AP	AUROC	AP
WinCLIP	91.8	96.5	78.1	81.2
AnomalyCLIP	91.5	96.2	82.1	85.4
AdaCLIP	89.2	–	85.8	–
VCP-CLIP	92.1	96.9	83.8	87.6

Table 1: Cross-dataset zero-shot performance on MVTec AD and VisA (%). Models are trained on only the other dataset as the unseen target, following the cross-dataset zero-shot protocol with no target-domain adaptation. We report image-level AUROC / AP and pixel-level AUROC / AUPRO (higher is better). Baseline results are taken from the original papers, and ”–” denotes metrics not reported. Note that we report SPG with OpenCLIP to match the backbone used by prior CLIP-based methods, and with DINOv3 ViT-L to demonstrate that SPG is not restricted to VLM backbones.

Figure 2: Sensitivity to SAE hyperparameters. The Heatmaps summarize cross-dataset performance when sweeping the SAE dictionary width $C$ and the TopK sparsity level $k$ . The top row reports VisA $\rightarrow$ MVTec AD transfer and the bottom row reports MVTec $\rightarrow$ VisA transfer. Each column corresponds to image-level AUROC / AP and pixel-level AUROC / AUPRO, respectively, highlighting that the optimal $(C,k)$ can depend on both the transfer direction and the evaluation metric.

Figure 3: Ablation of image-level anomaly-score aggregation via temperature-controlled log-sum-exp pooling. We vary the temperature $\tau$ in log-sum-exp pooling applied to the upsampled anomaly map to produce an image-level anomaly score. Curves report image-level AUROC and AP for both transfer directions (left: VisA $\rightarrow$ MVTec; right: MVTec $\rightarrow$ VisA). The figure compares the $\tau\rightarrow 0$ (max-like) regime against larger $\tau$ (mean-like) aggregation, showing that more max-like pooling better preserves localized high-confidence anomaly responses for detection.

Figure 4: Effect of the visual backbone on SPG under cross-dataset transfer. We instantiate SPG with different frozen encoders and rerun Stage 1 (SAE training on auxiliary patch tokens) and Stage 2 (guide-coefficient learning) for each backbone under the same protocol. Results are reported for two transfer directions (left: VisA

\rightarrow

MVTec; right: MVTec

\rightarrow

VisA). Each plot reports image-level AUROC/AP and pixel-level AUROC/AUPRO, isolating how backbone choice influences both detection and segmentation quality within the same guide-based scoring rule.

Figure 5: Qualitative interpretation of SAE dictionary atoms emphasized by the learned anomaly guide. We select representative atoms from the anomaly guide’s active set by choosing those with the largest learned anomaly coefficients. Each atom is visualized by retrieving top-activating patches from the auxiliary dataset and by overlaying an upsampled activation heatmap on the corresponding images for spatial context. This qualitative analysis illustrates that SPG’s anomaly criterion can be traced to a sparse subset of SAE atoms, some of which appear broadly activated across categories while others are more category-biased.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure S1 : Stage-2 guide learning with explicit TopK sparsification. We replace the ReLU+ $\ell_{1}$ parameterization in the main paper with $k_{g}$ -sparse guide coefficients obtained by TopK projection, and plot performance against the total guide sparsity $\|\tilde{\bm{z}}_{N}\|_{0}+\|\tilde{\bm{z}}_{A}\|_{0}$ . Orange markers denote the effective sparsity obtained under the ReLU+ $\ell_{1}$ formulation in the main paper. In both transfer directions, performance improves as the sparsity budget increases, indicating that overly small TopK budgets over-constrain the guides.

Category	WinCLIP	AnomalyCLIP	AdaCLIP	VCP-CLIP	AA-CLIP	AdaptCLIP	SPG (OpenCLIP)	SPG (DINOv3)
Bottle	99.2	89.3	94.4	–	–	90.7	78.7	99.3
Cable	86.5	69.8	90.6	–	–	83.6	57.3	86.4
Capsule	72.9	89.9	91.5	–	–	95.2	90.0	89.9
Carpet	100.0	100.0	82.1	–	–	100.0	92.3	97.8
Grid	98.8	97.0	90.0	–	–	98.9	76.2	99.4
Hazelnut	93.9	97.2	80.2	–	–	94.8	84.7	93.5

Appendix

Table S1 : Category-wise image-level AUROC (%) under the cross-dataset zero-shot setting (VisA

\rightarrow

MVTec AD). Baseline results are taken from the original papers, and ”–” denotes metrics not reported.

Category	WinCLIP	AnomalyCLIP	AdaCLIP	VCP-CLIP	AA-CLIP	AdaptCLIP	SPG (OpenCLIP)	SPG (DINOv3)
Bottle	99.8	97.0	–	–	–	97.1	92.6	99.8
Cable	91.2	81.4	–	–	–	89.9	67.8	92.4
Capsule	91.5	97.9	–	–	–	99.0	97.8	98.0
Carpet	100.0	100.0	–	–	–	99.9	97.2	99.2
Grid	99.6	99.1	–	–	–	99.5	88.9	99.8
Hazelnut	96.9	98.6	–	–	–	97.0	90.5	97.1

Appendix

Table S2 : Category-wise image-level AP (%) under the cross-dataset zero-shot setting (VisA

\rightarrow

MVTec AD). Baseline results are taken from the original papers, and ”–” denotes metrics not reported.

Category	WinCLIP	AnomalyCLIP	AdaCLIP	VCP-CLIP	AA-CLIP	AdaptCLIP	SPG (OpenCLIP)	SPG (DINOv3)
Bottle	89.5	90.4	90.4	94.1	–	92.4	89.5	96.6
Cable	77.0	78.9	79.8	78.6	–	76.6	74.2	80.4
Capsule	86.9	95.8	82.3	96.6	–	95.8	93.7	95.6
Carpet	95.4	98.8	97.3	99.6	–	99.2	98.6	99.5
Grid	82.2	97.3	96.9	97.9	–	97.4	96.9	99.3
Hazelnut	94.3	97.1	97.8	97.7	–	97.6	94.6	97.7

Appendix

Table S3 : Category-wise pixel-level AUROC (%) under the cross-dataset zero-shot setting (VisA

\rightarrow

MVTec AD). Baseline results are taken from the original papers, and ”–” denotes metrics not reported.

Category	WinCLIP	AnomalyCLIP	AdaCLIP	VCP-CLIP	AA-CLIP	AdaptCLIP	SPG (OpenCLIP)	SPG (DINOv3)
Bottle	76.4	80.9	–	88.5	–	–	72.4	91.4
Cable	42.9	64.4	–	67.7	–	–	39.4	73.7
Capsule	62.1	87.2	–	94.0	–	–	70.5	86.0
Carpet	84.1	90.1	–	98.4	–	–	85.7	98.2
Grid	57.0	75.6	–	92.4	–	–	75.6	94.7
Hazelnut	81.6	92.4	–	84.9	–	–	78.1	86.4

Appendix

Table S4 : Category-wise pixel-level AUPRO (%) under the cross-dataset zero-shot setting (VisA

\rightarrow

MVTec AD). Baseline results are taken from the original papers, and ”–” denotes metrics not reported.

Category	WinCLIP	AnomalyCLIP	AdaCLIP	VCP-CLIP	AA-CLIP	AdaptCLIP	SPG (OpenCLIP)	SPG (DINOv3)
Candle	95.4	79.3	96.0	–	–	87.4	84.5	74.7
Capsules	85.0	81.5	85.1	–	–	93.9	92.4	95.8
Cashew	92.1	76.3	91.8	–	–	86.1	84.3	70.2
Chewinggum	96.5	97.4	96.4	–	–	96.7	96.2	97.7
Fryum	80.3	93.0	93.0	–	–	91.5	91.9	94.9
Macaroni1	76.2	87.2	91.6	–	–	83.3	77.3	72.7

Appendix

Table S5 : Category-wise image-level AUROC (%) under the cross-dataset zero-shot setting (MVTec AD

\rightarrow

VisA). Baseline results are taken from the original papers, and ”–” denotes metrics not reported.

Category	WinCLIP	AnomalyCLIP	AdaCLIP	VCP-CLIP	AA-CLIP	AdaptCLIP	SPG (OpenCLIP)	SPG (DINOv3)
Candle	95.8	81.1	–	–	–	90.5	88.0	81.2
Capsules	90.9	88.7	–	–	–	96.8	96.3	98.1
Cashew	96.4	89.4	–	–	–	94.0	93.0	84.0
Chewinggum	98.6	98.9	–	–	–	98.7	98.1	99.1
Fryum	90.1	96.8	–	–	–	95.9	96.4	97.7
Macaroni1	75.8	86.0	–	–	–	85.1	82.5	77.8

Appendix

Table S6 : Category-wise image-level AP (%) under the cross-dataset zero-shot setting (MVTec AD

\rightarrow

VisA). Baseline results are taken from the original papers, and ”–” denotes metrics not reported.

Category	WinCLIP	AnomalyCLIP	AdaCLIP	VCP-CLIP	AA-CLIP	AdaptCLIP	SPG (OpenCLIP)	SPG (DINOv3)
Candle	88.9	98.8	98.9	99.2	–	98.7	98.8	98.9
Capsules	81.6	95.0	98.6	98.7	–	94.2	95.6	99.4
Cashew	84.7	93.8	95.9	93.1	–	93.4	91.7	96.2
Chewinggum	93.3	99.3	99.6	99.5	–	99.5	99.5	99.5
Fryum	88.5	94.6	94.4	94.6	–	95.3	92.0	94.7
Macaroni1	70.9	98.3	99.5	99.6	–	98.1	98.4	98.4

Appendix

Table S7 : Category-wise pixel-level AUROC (%) under the cross-dataset zero-shot setting (MVTec AD

\rightarrow

VisA). Baseline results are taken from the original papers, and ”–” denotes metrics not reported.

Category	WinCLIP	AnomalyCLIP	AdaCLIP	VCP-CLIP	AA-CLIP	AdaptCLIP	SPG (OpenCLIP)	SPG (DINOv3)
Candle	83.5	96.2	–	96.2	–	–	96.5	89.9
Capsules	35.3	78.5	–	91.2	–	–	81.8	98.6
Cashew	76.4	91.6	–	95.6	–	–	93.8	88.7
Chewinggum	70.4	91.2	–	92.2	–	–	94.8	92.8
Fryum	77.4	86.8	–	92.2	–	–	90.6	92.9
Macaroni1	34.3	89.8	–	97.5	–	–	90.2	91.7

Appendix

Table S8 : Category-wise pixel-level AUPRO (%) under the cross-dataset zero-shot setting (MVTec AD

\rightarrow

VisA). Baseline results are taken from the original papers, and ”–” denotes metrics not reported.

Zero-Shot Anomaly Detection

Zero-shot anomaly detection finds unusual patterns in unseen domains without task-specific labeled examples.

See all 85 papers

PSMP-CLIP: Patch-Prompt SAM and Multi-Semantic Prompting for CLIP-Based Zero-Shot Anomaly Detection
Sep 15, 2026Xuezhi Xiang, Guanghao Wu, Heqi Xiang +4Promptable Image Segmentation Anomaly Localization
Information and Communication Engineering, Harbin Engineering University, Harbin, China · Key Laboratory of Advanced Marine Communication and Information Technology, Harbin, China · Department of Computer Science, University of Toronto, Toronto, ON M5S 2E4, Canada +1
VisualAD: Language-Free Zero-Shot Anomaly Detection via Vision Transformer
Mar 9, 2026Yanning Hou, Peiyuan Li, Zirui Liu +4Zero-Shot Anomaly Detection Vision Foundation Model Adaptation
College of Intelligence Science and Technology, National University of Defense Technology, Changsha, China · School of Artificial Intelligence, Anhui University, Hefei, China · State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology, Anhui University, Hefei, China
VFAD: Variational Semantic Prompting Meets Frequency-Adaptive Representation Learning for Zero-Shot Anomaly Detection
Jul 31, 2026Peng Chen, Kaige Li, Wei Wang +5VLM Adaptation Frequency-Domain Feature Learning
School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University, Shenzhen, China

Sep 15, 2026J/K move · Enter open · S save

PSMP-CLIP: Patch-Prompt SAM and Multi-Semantic Prompting for CLIP-Based Zero-Shot Anomaly Detection

Xuezhi Xiang, Guanghao Wu, Heqi Xiang, Jiayao Liu+3

Information and Communication Engineering, Harbin Engineering University, Harbin, China · Key Laboratory of Advanced Marine Communication and Information Technology, Harbin, China · Department of Computer Science, University of Toronto, Toronto, ON M5S 2E4, Canada+1

Promptable Image Segmentation Anomaly Localization Zero-Shot Anomaly Detection Visual Prompt Learning

Open paper AskPDF arXiv

Zero-shot anomaly detection aims to localize anomalies without target-domain samples. Existing CLIP-based methods suffer from coarse anomaly maps and limited semantic prompts. We propose PSMP-CLIP, integrating patch-prompt SAM2 segmentation (PPSS) and multi-semantic guided prompt regularization (MSGPR). PPSS samples prompts directly from intermediate patch features, avoiding threshold drift and guiding SAM2 to produce precise masks. MSGPR uses multiple learnable prompts constrained by semantic anchors to preserve generalization. Experiments on 14 datasets show highly competitive performance, achieving the best pixel-level AUROC on MVTec AD, BTAD, DTD-Synthetic, CVC-ClinicDB, TN3K, Endo, and Kvasir.

Abstract

Zero-Shot Anomaly Detection

Explore similar work

PSMP-CLIP: Patch-Prompt SAM and Multi-Semantic Prompting for CLIP-Based Zero-Shot Anomaly Detection

VisualAD: Language-Free Zero-Shot Anomaly Detection via Vision Transformer

VFAD: Variational Semantic Prompting Meets Frequency-Adaptive Representation Learning for Zero-Shot Anomaly Detection