Detecting Defects that Matter: An Application-Driven Benchmark for Anomaly Detection in Manufacturing and Retail Logistics (VAND 4.0 Challenge)

Lars Heckler-Kram; Dorian Henning; Ashwin Vaidya; Jan-Hendrik Neudeck; Ulla Scheler; Anton Milan; Samet Akcay; Paula Ramos

cs.CVOct 3, 2026

Detecting Defects that Matter: An Application-Driven Benchmark for Anomaly Detection in Manufacturing and Retail Logistics (VAND 4.0 Challenge)

Authors: Lars Heckler-Kram, Dorian Henning, Ashwin Vaidya, Jan-Hendrik Neudeck, Ulla Scheler, Anton Milan, Samet Akcay, Paula Ramos,

Organizations: MVTec Software GmbH · Technical University of Munich · Amazon · Intel ·

Abstract

Existing Anomaly Detection benchmarks are saturated and often unrealistic. As part of the VAND 4.0 Challenge, we introduce a hidden-test, application-driven benchmark across two deployment-critical domains: industrial manufacturing and retail logistics. In the Industrial Track (MVTec AD 2), the results reveal that unsupervised anomaly segmentation remains challenging: the best regular-setting method achieves only ~57% pixel-level $SegF_1$ , indicating substantial room for improvement. Zero-shot approaches trail by ~15 $SegF_1$ points, confirming that task-specific training on normal data remains essential for precise defect localization. Robustness to distribution shifts remains a key open challenge and DINOv3-backbones clearly dominate this track. In the Retail Track (Kaputt 2), the results reveal that (1) supervised defect detection is approaching saturation for common defect types; (2) the best off-the-shelf VLM approach trails specialized models by ~28 AP, confirming that currently VLMs cannot replace fine-tuned detectors, (3) reference images did not prove helpful for top-performing approaches. Performance collapses on rare defects (spillage ~53 AP, missing units ~27 AP), where the supervised ceiling is bounded by data availability. To drive future progress in this domain, we provide a new low-prevalence retail AD dataset (Kaputt-Rare). Across both tracks, computational efficiency is assessed as a first-class metric combining performance, throughput, memory, and power consumption. We introduce a novel metric for measuring efficiency and reveal that that top-performing methods rely on heavy architectures while efficiency is largely neglected. Overall, we conclude that the community needs (a) more efficiency-aware method development, and (b) true anomaly detection approaches for rare defects and shifting conditions. https://sites.google.com/view/vand4-cvpr2026/challenge

Figures & tables

$\mathcal{R}_{\text{final}}$	Method	$SegF_{1,\text{priv}}$	$SegF_{1,\text{priv\_mix}}$	Report
1	SuperADD	57.42	54.35	★
2	ISVL++	58.31	53.75	★
3	Forgis_baseline	49.02	45.80	★
4	MSAD	48.54	45.50	★
5	DL-SubspaceAD	46.88	45.36	★
6	vandseg_4	46.48	40.61	★

Table 1: VAND 4.0 Challenge industrial track ( MVTec AD 2 ) leaderboard under the regular setting.

$\mathcal{R}_{\text{final}}$	Method	$SegF_{1,\text{priv}}$	$SegF_{1,\text{priv\_mix}}$	Report
1	BALM	42.64	41.45	★
2	ProtoFuse	30.50	30.44	★
3	SPR (lin)	23.96	24.05	★
4	sim2	24.55	22.84	★
5	EthMuSAM	23.57	22.15	★

Table 2: VAND 4.0 Challenge industrial track leaderboard ( MVTec AD 2 ) under the zero-shot setting.

$\mathcal{R}_{\text{final}}$	Team	$AP_{any}$	Report
1	dinovision	90.84	★
2	sandeepsaluru	89.30	★
3	xie233	84.32	★

Table 3: VAND 4.0 Challenge retail track ( Kaputt 2 ) leaderboard under the regular setting.

$\mathcal{R}_{\text{final}}$	Team	$AP_{any}$	Report
1	rahulvippala	61.88	★
2	dinovision	61.83	★
3	ant-hy	54.87	★
4	amirparsa-sal	50.49	★
5	sandeepsaluru	46.75	★

Table 4: VAND 4.0 Challenge retail track ( Kaputt 2 ) leaderboard under the off-the-shelf (OTS) setting.

Figure 1: Per-defect-type $AP_{any}$ of the top three submissions on Kaputt 2 . Common structural defects are largely solved, while rare categories (spillage, missing unit) remain challenging.

$\mathcal{R}_{\text{eff}}$	Team/Method	$\mathcal{E}\uparrow$	$\tau\uparrow$	$\mu\downarrow$	$\pi\downarrow$	$q\uparrow$
1	xie233	0.3473	0.1093	0.7160	0.4442	0.8432
1	SPR (lin)	0.1937	0.0993	0.8964	0.4300	0.2401
2	Forgis_baseline	0.1599	0.1185	0.9791	0.4441	0.4741
3	GBE_AD	0.1415	0.0221	0.9088	0.4144	0.3398
4	BALM	0.1414	0.0250	0.9376	0.3899	0.4205
5	CAT	0.1350	0.0631	0.9772	0.4477	0.4188

Table 5: Efficiency rank

\mathcal{R}_{\text{eff}}

according to efficiency score

\mathcal{E}

for the retail track (upper part;

q:=\textbf{\mbox{

AP_{any}

}}

) and industrial track (lower part;

q:=\mbox{

SegF_{1,\text{avg}}

}

\mathcal{E}

is the geometric mean of normalized throughput

\tau

, normalized memory usage

\mu

, normalized power consumption

\pi

and model performance

q

( Eq. 6 ). Only methods runnable on the target hardware are shown. Best (second best) result per track and column is marked bold (underlined).

Method	$AP_{any}$	$AP_{major}$	AUROC
ViT-S [ 30 ]	58.38	50.09	72.99
CLIP [ 32 ]	36.20	17.15	56.05
WinCLIP-zero [ 23 ]	33.87	19.11	52.30
WinCLIP-few [ 23 ]	34.05	19.29	52.41
PatchCore50 [ 34 ]	35.86	17.80	54.69

Table 6: Baseline results on Kaputt-Rare. ViT-S is fine-tuned on the Kaputt-Rare training set (normal-heavy,

{\sim}1\%

defect rate); all other methods are training-free (zero-/few-shot). ViT-S operates on the full tray image, whereas CLIP , WinCLIP , and PatchCore50 operate on item crops. Evaluated on the test split (10067 samples; 2089 minor, 1117 major defects). All values in %.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Value	Unit
$t_{\max}$ , target throughput	$30$	fps
$m_{\max}$ , memory budget	$62470.692864$	MB
$p_{\max}$ , power budget	$56.21$	W

Appendix

Table 7: Reference operating-point parameters.

Figure 2: Efficiency score sensitivity to each component in isolation. (a) F1 score: fourth-root growth, where even modest F1 improvements noticeably raise the score. (b) Throughput: diminishing-returns normalization soft-caps the curve well below $1$ . (c) Memory usage: because the complement $1-\mu$ is used, the score decreases as memory usage grows. (d) Power consumption: because the complement $1-\pi$ is used, the score decreases as power consumption grows.

Figure 3: Composite efficiency score across the full operating range. The approximately linear response confirms that no single component dominates the metric.

Figure 4: Efficiency score surface over F1 and throughput at moderate resource usage, with memory $=50\%$ and power $=50\%$ . The concave surface illustrates the balanced trade-off between detection quality and inference speed.

Anomaly Detection

Anomaly detection finds rare or unexpected patterns in data, from defective products and system faults to fraud and security threats.

See all 654 papers

SuperADD: Training-free Class-agnostic Anomaly Segmentation -- CVPR 2026 VAND 4.0 Workshop Challenge Industrial Track
May 14, 2026Lukas Roming, Felix Lehnerer, Jonas V. Funk +4OOD Generalization Industrial Anomaly Detection
Fraunhofer IOSB, Fraunhoferstr. 1, 76131 Karlsruhe, Germany · Independent Research
NFAD: Nuisance-Filtered Anomaly Detection Under Distribution Shift
Aug 29, 2026Dat Cao, Son Nghiem, Phan Nguyen +2Anomaly Detection Industrial Anomaly Detection
Korea Advanced Institute of Science and Technology (KAIST), South Korea · Sony Computer Science Laboratories (Sony CSL), Japan · The University of Tokyo, Japan +1
InCLAD: A Continual Learning Benchmark for Industrial Visual Anomaly Detection
Oct 10, 2026Łukasz Marcjan, Dorota Wilk-Kołodziejczyk, Roberto CorizzoTask-Incremental Learning Continual Anomaly Detection
AGH University of Krakow, Krakow, Poland · American University, Washington, DC, USA

May 14, 2026J/K move · Enter open · S save

SuperADD: Training-free Class-agnostic Anomaly Segmentation -- CVPR 2026 VAND 4.0 Workshop Challenge Industrial Track

Lukas Roming, Felix Lehnerer, Jonas V. Funk, Andreas Michel+3

Fraunhofer IOSB, Fraunhoferstr. 1, 76131 Karlsruhe, Germany · Independent Research

OOD Generalization Industrial Anomaly Detection Anomaly Localization

Open paper AskPDF arXiv

Visual anomaly detection (AD) for industrial inspection is a highly relevant task in modern production environments. The problem becomes particularly challenging when training and deployment data differ due to changes in acquisition conditions during production. In the VAND 4.0 Industrial Track, models must remain robust under distribution shifts such as varying illumination and their performance is assessed on the MVTec AD 2 dataset. To address this setting, we propose a training-free and class-agnostic anomaly detection pipeline based on the work of SuperAD. Our approach improves generalization through several modifications designed to enhance robustness under distribution shifts. These adaptations include using a DINOv3 backbone, overlapping patch-wise processing, intensity-based augmentations, improved memory-bank subsampling for better coverage of the data distribution, and iterative morphological closing for cleaner and more spatially consistent anomaly maps. Unlike methods that rely on class-specific architectures or per-class hyperparameter tuning, our method uses a single architecture and one shared hyperparameter configuration across all object classes. This makes the approach well suited for industrial deployment, where product variants and appearance changes must be handled with minimal adaptation effort. We achieve segmentation F1 scores of

62.61\%

57.42\%

, and

54.35\%

on test public, private, and private mixed of MVTec AD 2 respectively, thereby outperforming SuperAD and other state-of-the-art methods. Code is available at https://github.com/LukasRoom/SuperADD.

Abstract

Anomaly Detection

Explore similar work

SuperADD: Training-free Class-agnostic Anomaly Segmentation -- CVPR 2026 VAND 4.0 Workshop Challenge Industrial Track

NFAD: Nuisance-Filtered Anomaly Detection Under Distribution Shift

InCLAD: A Continual Learning Benchmark for Industrial Visual Anomaly Detection