cs.CVOct 15, 2025

Adversarially Robust Few-Shot Anomaly Detection with Vision Foundation Models

Authors: Akib Mohammed Khan, Bartosz Krawczyk

Organizations: Rochester Institute of Technology Rochester, USA

Abstract

Vision foundation models such as DINOv2 enable strong few-shot anomaly detection (FSAD) through simple non-parametric k-nearest-neighbor (k-NN) scoring over frozen patch features. Existing robust anomaly detection methods assume large normal-class training sets and adversarial training of the feature extractor. The few-shot regime, where the detector consists of a frozen backbone and a non-parametric scorer, has remained unaddressed despite its widespread deployment. We develop a white-box attack framework adapted to k-NN memory-bank detectors and introduce an adversarially robust FSAD method which is training-free with respect to the backbone. We introduce the DistanceProbe (DP), a lightweight MLP trained by regression to predict per-patch k-NN distance directly from frozen features, providing the differentiable proxy required for gradient-based attacks against non-parametric scorers. We further propose a two-level combined defense that operates using only the normal reference support set images. At the input level, PatchShift (PS) applies random pixel shifts to exploit the misalignment of adversarial perturbations with the encoder's patch grid, aggregating scores across multiple shifted views via element-wise median pooling. At the feature level we employ the FeaturePurifier (FP), a lightweight residual MLP trained on clean and adversarially augmented support features to project perturbed representations back toward the clean feature manifold. Through comprehensive experiments under various adversarial settings, we show that our method exhibits robust detection and localization with performance gains of ~27% image-level AUROC and ~51% pixel-level PRO over the undefended attacked baseline across MVTec-AD, VisA, and MPDD at k=4 shots, while preserving clean accuracy within 3%, matching full-shot adversarially-trained baselines on pixel-level AUROC, and holding under adaptive attack.

Figures & tables

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Feb 26, 2026

SubspaceAD: Training-Free Few-Shot Anomaly Detection via Subspace Modeling

Detecting visual anomalies in industrial inspection often requires training with only a few normal images per category. Recent few-shot methods achieve strong results employing foundation-model features, but typically rely on memory banks, auxiliary datasets, or multi-modal tuning of vision-language models. We therefore question whether such complexity is necessary given the feature representations of vision foundation models. To answer this question, we introduce SubspaceAD, a training-free method, that operates in two simple stages. First, patch-level features are extracted from a small set of normal images by a frozen DINOv2 backbone. Second, a Principal Component Analysis (PCA) model is fit to these features to estimate the low-dimensional subspace of normal variations. At inference, anomalies are detected via the reconstruction residual with respect to this subspace, producing interpretable and statistically grounded anomaly scores. Despite its simplicity, SubspaceAD achieves state-of-the-art performance across one-shot and few-shot settings without training, prompt tuning, or memory banks. In the one-shot anomaly detection setting, SubspaceAD achieves image-level and pixel-level AUROC of 97.1% and 97.5% on the MVTec-AD dataset, and 93.2% and 98.2% on the VisA dataset, respectively, surpassing prior state-of-the-art results. Code and demo are available at https://github.com/CLendering/SubspaceAD.
Jul 27, 2026

DuoAD: Leveraging [CLS] Dual Characteristics for Training-Free Few-Shot Anomaly Detection

Vision foundation models have enabled strong training-free anomaly detection (AD). However, most existing approaches rely primarily on independent local patch features, leaving the global contextual information encoded by Vision Transformers (ViTs) underexploited. In this work, we identify the dual characteristics of the ViT [CLS] token: its embedding provides anomaly-invariant global semantic representation, while its attention maps implicitly highlight spatially abnormal regions. Building on this observation, we propose a fully automated AD framework leveraging global context to remove manual tunings. Our framework introduces (1) an automatic augmentation selection strategy driven by [CLS]-level semantic consistency, and (2) an attention-guided feature reweighting mechanism that dynamically adjusts patch contributions according to [CLS] attention saliency. By integrating these components over multi-level features, our method achieves stable anomaly scoring and precise localization without training or parameter tuning. Under the one-shot setting, it achieves Image-AUC scores of 97.7%, 93.2%, and 84.5% on MVTec-AD, VisA, and Real-IAD. Using a single fixed configuration across categories, backbones, and datasets, the method establishes a new state-of-the-art for plug-and-play, training-free anomaly detection while maintaining strong robustness and practical scalability.
Aug 27, 2026

ShiftSplit-AD: Separating Domain Shift from Defects in Foundation-Feature Visual Anomaly Detection

Visual anomaly detectors based on frozen foundation-model features commonly score distances from test patches to a memory of normal features. Benign acquisition changes can also enlarge these distances, confounding domain variation with defects. We investigate whether structured decomposition of nearest-normal DINOv2 residuals can suppress shift-induced evidence while retaining unseen defects. ShiftSplit-AD decomposes the patch residual matrix into low-rank and row-sparse components and scores the sparse component, with an optional low-rank/sparse fusion. The experiments expose a central trade-off rather than a universal separation: genuine defects can contain correlated, low-dimensional structure, so filtering broad residual activity may also remove defect information. On AeBAD-S, using settings fixed after Bottle development, sparse-only scoring improves image AUROC from 0.6780 to 0.7294 and AUPRC from 0.8052 to 0.8465. Paired bootstrap 95% intervals for the improvements are [0.0238, 0.0808] and [0.0170, 0.0650], respectively. However, sparse-only scoring reduces mean clean AUROC from 0.9890 to 0.9133 on four held-out MVTec categories and degrades Bottle localization. These findings show that residual decomposition can help when domain shift strongly contaminates anomaly evidence, but preserving defect structure remains the limiting problem.