Detecting Defects that Matter: An Application-Driven Benchmark for Anomaly Detection in Manufacturing and Retail Logistics (VAND 4.0 Challenge)

Lars Heckler-Kram; Dorian Henning; Ashwin Vaidya; Jan-Hendrik Neudeck; Ulla Scheler; Anton Milan; Samet Akcay; Paula Ramos

cs.CVOct 3, 2026

Detecting Defects that Matter: An Application-Driven Benchmark for Anomaly Detection in Manufacturing and Retail Logistics (VAND 4.0 Challenge)

Authors: Lars Heckler-Kram, Dorian Henning, Ashwin Vaidya, Jan-Hendrik Neudeck, Ulla Scheler, Anton Milan, Samet Akcay, Paula Ramos,

Organizations: MVTec Software GmbH · Technical University of Munich · Amazon · Intel ·

Abstract

Existing Anomaly Detection benchmarks are saturated and often unrealistic. As part of the VAND 4.0 Challenge, we introduce a hidden-test, application-driven benchmark across two deployment-critical domains: industrial manufacturing and retail logistics. In the Industrial Track (MVTec AD 2), the results reveal that unsupervised anomaly segmentation remains challenging: the best regular-setting method achieves only ~57% pixel-level $SegF_1$ , indicating substantial room for improvement. Zero-shot approaches trail by ~15 $SegF_1$ points, confirming that task-specific training on normal data remains essential for precise defect localization. Robustness to distribution shifts remains a key open challenge and DINOv3-backbones clearly dominate this track. In the Retail Track (Kaputt 2), the results reveal that (1) supervised defect detection is approaching saturation for common defect types; (2) the best off-the-shelf VLM approach trails specialized models by ~28 AP, confirming that currently VLMs cannot replace fine-tuned detectors, (3) reference images did not prove helpful for top-performing approaches. Performance collapses on rare defects (spillage ~53 AP, missing units ~27 AP), where the supervised ceiling is bounded by data availability. To drive future progress in this domain, we provide a new low-prevalence retail AD dataset (Kaputt-Rare). Across both tracks, computational efficiency is assessed as a first-class metric combining performance, throughput, memory, and power consumption. We introduce a novel metric for measuring efficiency and reveal that that top-performing methods rely on heavy architectures while efficiency is largely neglected. Overall, we conclude that the community needs (a) more efficiency-aware method development, and (b) true anomaly detection approaches for rare defects and shifting conditions. https://sites.google.com/view/vand4-cvpr2026/challenge

Figures & tables

$\mathcal{R}_{\text{final}}$	Method	$SegF_{1,\text{priv}}$	$SegF_{1,\text{priv\_mix}}$	Report
1	SuperADD	57.42	54.35	★
2	ISVL++	58.31	53.75	★
3	Forgis_baseline	49.02	45.80	★
4	MSAD	48.54	45.50	★
5	DL-SubspaceAD	46.88	45.36	★
6	vandseg_4	46.48	40.61	★

Table 1: VAND 4.0 Challenge industrial track ( MVTec AD 2 ) leaderboard under the regular setting.

$\mathcal{R}_{\text{final}}$	Method	$SegF_{1,\text{priv}}$	$SegF_{1,\text{priv\_mix}}$	Report
1	BALM	42.64	41.45	★
2	ProtoFuse	30.50	30.44	★
3	SPR (lin)	23.96	24.05	★
4	sim2	24.55	22.84	★
5	EthMuSAM	23.57	22.15	★

Table 2: VAND 4.0 Challenge industrial track leaderboard ( MVTec AD 2 ) under the zero-shot setting.

$\mathcal{R}_{\text{final}}$	Team	$AP_{any}$	Report
1	dinovision	90.84	★
2	sandeepsaluru	89.30	★
3	xie233	84.32	★

Table 3: VAND 4.0 Challenge retail track ( Kaputt 2 ) leaderboard under the regular setting.

$\mathcal{R}_{\text{final}}$	Team	$AP_{any}$	Report
1	rahulvippala	61.88	★
2	dinovision	61.83	★
3	ant-hy	54.87	★
4	amirparsa-sal	50.49	★
5	sandeepsaluru	46.75	★

Table 4: VAND 4.0 Challenge retail track ( Kaputt 2 ) leaderboard under the off-the-shelf (OTS) setting.

Figure 1: Per-defect-type $AP_{any}$ of the top three submissions on Kaputt 2 . Common structural defects are largely solved, while rare categories (spillage, missing unit) remain challenging.

$\mathcal{R}_{\text{eff}}$	Team/Method	$\mathcal{E}\uparrow$	$\tau\uparrow$	$\mu\downarrow$	$\pi\downarrow$	$q\uparrow$
1	xie233	0.3473	0.1093	0.7160	0.4442	0.8432
1	SPR (lin)	0.1937	0.0993	0.8964	0.4300	0.2401
2	Forgis_baseline	0.1599	0.1185	0.9791	0.4441	0.4741
3	GBE_AD	0.1415	0.0221	0.9088	0.4144	0.3398
4	BALM	0.1414	0.0250	0.9376	0.3899	0.4205
5	CAT	0.1350	0.0631	0.9772	0.4477	0.4188

Table 5: Efficiency rank

\mathcal{R}_{\text{eff}}

according to efficiency score

\mathcal{E}

for the retail track (upper part;

q:=\textbf{\mbox{

AP_{any}

}}

) and industrial track (lower part;

q:=\mbox{

SegF_{1,\text{avg}}

}

\mathcal{E}

is the geometric mean of normalized throughput

\tau

, normalized memory usage

\mu

, normalized power consumption

\pi

and model performance

q

( Eq. 6 ). Only methods runnable on the target hardware are shown. Best (second best) result per track and column is marked bold (underlined).

Method	$AP_{any}$	$AP_{major}$	AUROC
ViT-S [ 30 ]	58.38	50.09	72.99
CLIP [ 32 ]	36.20	17.15	56.05
WinCLIP-zero [ 23 ]	33.87	19.11	52.30
WinCLIP-few [ 23 ]	34.05	19.29	52.41
PatchCore50 [ 34 ]	35.86	17.80	54.69

Table 6: Baseline results on Kaputt-Rare. ViT-S is fine-tuned on the Kaputt-Rare training set (normal-heavy,

{\sim}1\%

defect rate); all other methods are training-free (zero-/few-shot). ViT-S operates on the full tray image, whereas CLIP , WinCLIP , and PatchCore50 operate on item crops. Evaluated on the test split (10067 samples; 2089 minor, 1117 major defects). All values in %.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Value	Unit
$t_{\max}$ , target throughput	$30$	fps
$m_{\max}$ , memory budget	$62470.692864$	MB
$p_{\max}$ , power budget	$56.21$	W

Appendix

Table 7: Reference operating-point parameters.

Figure 2: Efficiency score sensitivity to each component in isolation. (a) F1 score: fourth-root growth, where even modest F1 improvements noticeably raise the score. (b) Throughput: diminishing-returns normalization soft-caps the curve well below $1$ . (c) Memory usage: because the complement $1-\mu$ is used, the score decreases as memory usage grows. (d) Power consumption: because the complement $1-\pi$ is used, the score decreases as power consumption grows.

Figure 3: Composite efficiency score across the full operating range. The approximately linear response confirms that no single component dominates the metric.

Figure 4: Efficiency score surface over F1 and throughput at moderate resource usage, with memory $=50\%$ and power $=50\%$ . The concave surface illustrates the balanced trade-off between detection quality and inference speed.

Anomaly Detection

Anomaly detection finds rare or unexpected patterns in data, from defective products and system faults to fraud and security threats.

See all 646 papers

May 14, 2026

SuperADD: Training-free Class-agnostic Anomaly Segmentation -- CVPR 2026 VAND 4.0 Workshop Challenge Industrial Track

Visual anomaly detection (AD) for industrial inspection is a highly relevant task in modern production environments. The problem becomes particularly challenging when training and deployment data differ due to changes in acquisition conditions during production. In the VAND 4.0 Industrial Track, models must remain robust under distribution shifts such as varying illumination and their performance is assessed on the MVTec AD 2 dataset. To address this setting, we propose a training-free and class-agnostic anomaly detection pipeline based on the work of SuperAD. Our approach improves generalization through several modifications designed to enhance robustness under distribution shifts. These adaptations include using a DINOv3 backbone, overlapping patch-wise processing, intensity-based augmentations, improved memory-bank subsampling for better coverage of the data distribution, and iterative morphological closing for cleaner and more spatially consistent anomaly maps. Unlike methods that rely on class-specific architectures or per-class hyperparameter tuning, our method uses a single architecture and one shared hyperparameter configuration across all object classes. This makes the approach well suited for industrial deployment, where product variants and appearance changes must be handled with minimal adaptation effort. We achieve segmentation F1 scores of

62.61\%

57.42\%

, and

54.35\%

on test public, private, and private mixed of MVTec AD 2 respectively, thereby outperforming SuperAD and other state-of-the-art methods. Code is available at https://github.com/LukasRoom/SuperADD.

Lukas Roming, Felix Lehnerer, Jonas V. Funk +4

Fraunhofer IOSB, Fraunhoferstr. 1, 76131 Karlsruhe, Germany · Independent Research

OOD Generalization Industrial Anomaly Detection Anomaly Localization

Ask PDF

Aug 29, 2026

NFAD: Nuisance-Filtered Anomaly Detection Under Distribution Shift

Recent advances in anomaly detection (AD) for industrial inspection have pushed performance on standard benchmarks toward saturation. However, strong benchmark performance does not necessarily translate to real-world deployment, as these benchmarks are primarily collected under controlled acquisition conditions. Changes in illumination, background, viewpoint, and other environmental factors can shift normal samples away from the learned normal distribution and cause false anomaly responses. We address AD under such distribution shifts by explicitly modeling nuisance variation from changing imaging conditions in feature space. Without anomaly labels or target-domain data, our Nuisance-Filtered Anomaly Detection (NFAD) framework estimates a nuisance subspace from matched feature displacements induced by content-preserving perturbations and suppresses its contribution to anomaly residuals at inference. The same subspace supports two complementary branches: full projection for image-level detection and selective suppression for pixel-level localization, preserving evidence of localized defects. On AeBAD-S, a benchmark specifically designed for AD under acquisition shifts, NFAD achieves 91.0% image-level AUROC, establishing a new state of the art. Notably, this robustness does not come at the expense of conventional AD performance: NFAD remains competitive on standard benchmarks that do not explicitly evaluate distribution shift, including VisA, Real-IAD, and MVTec AD. These results show that explicitly suppressing such nuisance variation improves AD under distribution shift while preserving strong performance in standard settings.

Dat Cao, Son Nghiem, Phan Nguyen +2

Korea Advanced Institute of Science and Technology (KAIST), South Korea · Sony Computer Science Laboratories (Sony CSL), Japan · The University of Tokyo, Japan +1

Anomaly Detection Industrial Anomaly Detection Anomaly Localization Invariant Representation Learning

Ask PDF

Oct 10, 2026

InCLAD: A Continual Learning Benchmark for Industrial Visual Anomaly Detection

Industrial anomaly detection benchmarks have advanced rapidly, yet they remain largely static and therefore underrepresent deployment settings in which inspection systems must be extended from one industrial component to another over time. In our setting, continual industrial anomaly detection is framed as a task-incremental problem: each component defines a task, while the learning objective remains semi-supervised. The goal extends beyond conventional anomaly detection, since models should preserve their performance on all components as they are sequentially exposed to new components. This framing is especially relevant because many existing industrial pipelines either assume fully supervised defect labels or study only isolated datasets, leaving open the question of how semi-supervised anomaly detectors behave across a sequence of heterogeneous components. We address this gap by organizing the benchmark as a sequence of related component-level tasks and by defining multiple scenarios, including random, easy-to-hard, and hard-to-easy. In addition to single-dataset continual scenarios, we propose challenging scenarios spanning across multiple datasets. We further evaluate continual performance from two complementary perspectives: image-level anomaly detection and pixel-level anomaly localization. By recasting industrial anomaly detection as a continual benchmark rather than a static train/test problem, our work establishes a concrete path toward studying knowledge retention, adaptation, and generalization in evolving industrial environments.

Łukasz Marcjan, Dorota Wilk-Kołodziejczyk, Roberto Corizzo

AGH University of Krakow, Krakow, Poland · American University, Washington, DC, USA

Task-Incremental Learning Continual Anomaly Detection Semi-Supervised Anomaly Detection Industrial Anomaly Detection Anomaly Detection Anomaly Localization

Ask PDF

Abstract

Anomaly Detection

Explore similar work

SuperADD: Training-free Class-agnostic Anomaly Segmentation -- CVPR 2026 VAND 4.0 Workshop Challenge Industrial Track

NFAD: Nuisance-Filtered Anomaly Detection Under Distribution Shift

InCLAD: A Continual Learning Benchmark for Industrial Visual Anomaly Detection