What Memory Composition Does Not Tell Us About Anomaly Detection

Joongwon Chae; Runming Wang; Peiwu Qin

cs.CVAug 24, 2026

What Memory Composition Does Not Tell Us About Anomaly Detection

Authors: Joongwon Chae, Runming Wang, Peiwu Qin

Organizations: Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · RatelSoft · Guangdong Provincial Laboratory of Traditional Chinese Medicine, Hengqin, Guangdong, China

Abstract

Memory-based anomaly detectors store nominal training patches and score test patches against this memory. A patch selected for coverage therefore becomes a nor- mal reference without a separate check that geometric rarity makes it safe to trust. We probe this coupling with sparse training contamination. Under fixed representa- tions and memory budgets, we compare random, medoid, local, and global coverage selectors. We then use CLEANCON, an out-of-bag cross-image support gate that changes candidate-image eligibility while fixing the representation, absolute mem- ory size, builder, and inference rule. Global coverage strongly over-represents sparse contamination. CLEANCON reduces final-memory contamination to approx- imately zero and increases category-macro P-AP in all 12 matched comparisons. Yet along a retention sweep, the lowest-contamination memory does not attain the highest P-AP; performance continues to improve while contamination rises. Mem- ory contamination therefore does not order the resulting memories by P-AP

Figures & tables

Figure 1: One selection, two roles. (a) Selected patches serve global coverage and test-time normal evidence. (b) Candidate-image eligibility changes while the absolute memory size and detector remain fixed. (c) The lowest-contamination memory is not necessarily the best-performing one.

Figure 2: Global coverage preferentially retains sparse contamination. Left: amplification over nested memory budgets from

0.1\%

10\%

. Curves show means over three contamination folds and bands show fold standard deviation. Right: fold-0 selector controls at a

1\%

memory budget; random controls use 20 repetitions.

Setting	Vanilla $p_{\mathrm{mem}}$	CleanCon $p_{\mathrm{mem}}$	Vanilla P-AP	CleanCon P-AP	$\Delta$ P-AP
MVTec–WRN50	$6.481\!\pm\!0.370\%$	$0.000\%$	$57.704\!\pm\!2.033$	$61.141\!\pm\!0.433$	$+3.438\!\pm\!1.625$
MVTec–DINOv2	$9.462\!\pm\!0.611\%$	$0.000\%$	$69.342\!\pm\!0.995$	$72.573\!\pm\!0.373$	$+3.231\!\pm\!0.645$
VisA–WRN50	$3.678\!\pm\!0.500\%$	$0.003\!\pm\!0.003\%$	$39.345\!\pm\!0.586$	$39.725\!\pm\!0.309$	$+0.380\!\pm\!0.429$
VisA–DINOv2	$4.748\!\pm\!0.814\%$	$0.002\!\pm\!0.000\%$	$47.271\!\pm\!0.901$	$51.590\!\pm\!1.821$	$+4.319\!\pm\!1.139$

Table 1: Three-fold matched unfiltered-to- CleanCon comparison.

Setting	R50	R60	R70	R80	R90
MVTec–WRN50	$61.141/0\%$	$61.717/0\%$	$61.720/0.0067\%$	$62.070/0.0089\%$	$62.084/0.0234\%$
MVTec–DINOv2	$72.573/0\%$	$72.743/0\%$	$73.027/0.0034\%$	$73.094/0.0100\%$	$73.144/0.0371\%$
VisA–WRN50	$39.725/0.0033\%$	$40.345/0.0112\%$	$40.453/0.0210\%$	$40.600/0.0462\%$	$40.475/0.1460\%$
VisA–DINOv2	$51.590/0.0018\%$	$53.509/0.0066\%$	$53.762/0.0146\%$	$53.883/0.0270\%$	$53.155/0.0947\%$

Table 2: Retention sweep: P-AP / pooled memory contamination. Values are three-fold means.

Figure 3: The cleanest memory does not attain the highest P-AP. The left axis shows the change in category-macro P-AP relative to R50 for the four dataset–representation settings and their mean. The right axis shows the setting mean of pooled

p_{\mathrm{mem}}

. R80 exceeds R50 in all 12 comparisons, and contamination is also higher in 11 of them.

Dataset	Method	Mixed-train contamination	I-AUROC	P-AUROC	P-AP	AUPRO
MVTec	InReaCh	$\approx 4.762\%$	92.32	97.01	–	–
	SoftPatch	$\approx 4.762\%$	98.40	98.17	–	–
	FUN-AD	$\approx 4.762\%$	98.45	98.43	–	–
	CleanCon –WRN50	$\approx 5\%$	97.539	–	61.016	93.539
	CleanCon –DINOv2	$\approx 5\%$	99.147	–	72.255	95.770
VisA	InReaCh	$\approx 4.762\%$	83.66	97.67	–	–

Table 3: Performance comparison on contaminated MVTec AD and VisA under No-Overlap.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	$0.1\%$	$0.5\%$	$1\%$	$5\%$	$10\%$
MVTec–WRN50	$47.47\pm 2.82$	$23.88\pm 1.81$	$16.04\pm 0.77$	$5.92\pm 0.21$	$3.99\pm 0.14$
VisA–WRN50	$97.15\pm 5.90$	$44.39\pm 2.82$	$29.97\pm 1.25$	$11.22\pm 0.31$	$6.94\pm 0.18$
MVTec–DINOv2	$43.17\pm 2.97$	$30.01\pm 2.31$	$23.19\pm 1.67$	$9.59\pm 0.60$	$6.04\pm 0.32$
VisA–DINOv2	$87.73\pm 5.89$	$57.99\pm 4.02$	$40.61\pm 2.65$	$13.41\pm 0.55$	$7.70\pm 0.26$

Appendix

Table A1: Micro-pooled amplification over nested memory budgets. Values are mean

\pm

sample standard deviation over contamination folds 0–2.

Setting	$n$	$q$	$K$	$m_{K}$	$A_{K}$	Selected $Q$
MVTec–WRN50	2,996,448	12,586	29,958	2,065	16.41	16.41%
VisA–WRN50	7,146,944	7,210	71,464	2,247	31.17	31.17%
MVTec–DINOv2	11,985,792	51,036	119,832	12,160	23.83	23.83%
VisA–DINOv2	28,587,776	25,356	285,856	11,048	43.57	43.57%

Appendix

Table A2: Fold-0 Global FF counts at a

1\%

per-memory budget. DINOv2 pools categories and four depths for one seed-0 memory per depth.

Selector	MV–WRN	VisA–WRN	MV–DINO	VisA–DINO
Patch random	$0.99\pm 0.09$	$0.99\pm 0.12$	$0.99\pm 0.04$	$1.00\pm 0.07$
Image-balanced random	$0.99\pm 0.09$	$1.01\pm 0.13$	$0.99\pm 0.04$	$1.02\pm 0.07$
SimHash	1.67	1.73	0.92	0.98
Density top- $K$	0.00	0.00	0.11	0.01
Sampled $k$ -means medoid	1.34	1.44	1.08	1.05
Voronoi landmark mass	0.41	0.33	0.30	0.29

Appendix

Table A3: Amplification by selector. Random baselines report mean

\pm

sample standard deviation over 20 repetitions; the remaining selectors use seed 0.

Setting	$\|U_{K}\|/q$	$\|P_{K}\|$	$m_{K}$	$\|P_{K}\|/m_{K}$
MVTec–WRN50	32.02%	95	2,065	4.60%
VisA–WRN50	24.98%	142	2,247	6.32%
MVTec–DINOv2	20.36%	70	12,160	0.58%
VisA–DINOv2	26.92%	92	11,048	0.83%

Appendix

Table A4: Fold-0 achieved-radius packing certificate for the same seed-0 Global FF memories as Table A2 .

Figure A1: Distribution of matched P-AP changes. Points denote contamination folds, ticks denote three-fold means, and labels report category–fold wins.

Setting	I-AUROC	P-AUROC	P-AP	AUPRO
MVTec–WRN50	$+1.239$ (3/3)	$+0.126$ (3/3)	$+0.929$ (3/3)	$+0.425$ (3/3)
MVTec–DINOv2	$+0.242$ (3/3)	$+0.046$ (3/3)	$+0.521$ (3/3)	$+0.139$ (3/3)
VisA–WRN50	$+1.071$ (3/3)	$+0.037$ (2/3)	$+0.875$ (3/3)	$+0.301$ (3/3)
VisA–DINOv2	$+0.104$ (2/3)	$+0.028$ (2/3)	$+2.293$ (3/3)	$+0.120$ (2/3)

Appendix

Table A5: Change from R50 to R80. Values are differences of three-fold category-macro means; parentheses give the number of improving folds.

Figure A2: Normal-source expansion. The expansion control increases normal-source representation by approximately 21 percentage points without improving P-AP.

Candidate rule	Selection unit	Experimental scope	Result
OOB-NN patch ranking	patch	six-category auxiliary experiment	lower local ranking and P-AP than projection-residual patch ranking
Projection-residual patch ranking	patch	six-category auxiliary experiment	$+1.233$ P-AP over OOB-NN, $-0.371$ relative to whole-image gating
Oracle normal-region selection	patch	ground-truth mask setting	removes anomalous regions and retains only normal regions as candidates
Normal-source expansion	image + patch	15 MVTec categories, three No-Overlap folds	$-0.137$ P-AP and $-0.215$ AUPRO relative to whole-image gating
Adaptive image retention	image	Exact-Final, seed 0	$+0.601$ P-AP at $5\%$ and $-2.095$ at $10\%$ relative to whole-image gating

Appendix

Table A6: Alternative candidate interventions.

Mixed-train contamination	Builder	$p_{\mathrm{mem}}$	I-AUROC	P-AP	AUPRO
$5\%$	Global FF	$0.0155\%$	99.641	72.848	95.976
	risk-aware	$0.0093\%$	99.596	72.814	95.974
$10\%$	Global FF	$0.8059\%$	99.507	70.415	94.323
	risk-aware	$0.6934\%$	99.519	70.573	94.389

Appendix

Table A7: Risk-aware coverage control.

Dataset	Backbone / ratio	SP-style			CleanCon			$\Delta$ P-AP
		I-AUROC	P-AP	AUPRO	I-AUROC	P-AP	AUPRO
MVTec	WRN50 / 5%	98.241	60.029	93.465	97.539	61.016	93.539	+0.987
MVTec	WRN50 / 10%	98.025	60.013	92.971	97.762	60.691	93.827	+0.679
MVTec	DINOv2 / 5%	98.880	70.570	95.440	99.147	72.255	95.770	+1.686
MVTec	DINOv2 / 10%	98.856	68.890	94.814	99.316	72.052	96.063	+3.162
VisA	WRN50 / 5%	92.810	41.495	93.735	92.596	40.061	93.693	-1.434

Appendix

Table A8: SoftPatch-style control and CleanCon under matched detector scaffolds.

Explore similar work

When More References Hurt: Contamination-Aware DINOv2 Memory Banks for Few-Shot Steel Defect Detection
Aug 22, 2026Hannaneh Kalantary, Javad KhoramdelIndustrial Anomaly Detection Anomaly Detection
University of Padua · University of Vienna
ProCon: Projection-Consistency Memory for Training-Free Anomaly Detection
Jul 6, 2026Joongwon Chae, Lihui Luo, Yang Liu +4Industrial Anomaly Detection Anomaly Localization
Tsinghua University Shenzhen International Graduate School, Shenzhen, China · Affiliated Fifth Hospital, Wenzhou Medical University, Wenzhou, Zhejiang, China · Chinese Medicine Guangdong Laboratory +1
Memory-Distilled Selection for Noise-Robust Anomaly Detection
May 26, 2026Sirojbek Safarov, Jaewoo Park, Yoon Gyo Jung +4Industrial Anomaly Detection Data Selection
AIVEX Inc. · Northeastern University · Mitsubishi Electric Research Laboratories (MERL).

Aug 22, 2026J/K move · Enter open · S save

When More References Hurt: Contamination-Aware DINOv2 Memory Banks for Few-Shot Steel Defect Detection

Hannaneh Kalantary, Javad Khoramdel

University of Padua · University of Vienna

Industrial Anomaly Detection Anomaly Detection Few-Shot Anomaly Detection

Open paper AskPDF arXiv

Patch-memory anomaly detectors assume that their reference bank is normal, an assumption that is difficult to guarantee when additional industrial images are unverified. We study whether a few trusted normal images can safely recover useful normal patches from such references without defect masks. Starting from the DINOv2 patch-memory formulation used by AnomalyDINO, we score candidate patches by distance to a clean seed bank, discard the most suspicious 20%, merge the retained patches with the seed, and enforce a fixed budget by greedy coreset selection. On Severstal, naive additional references contain 9.46% anomalous patches; the proposed trim rejects 78.1% of them and reduces residual contamination to 2.59%. At an equal 51,200-patch development budget, the proposed bank reaches 0.1084 AUPRC versus 0.0950 for naive expansion, 0.0952 for random removal, and 0.1030 for eight clean images. Injecting only 0.5% anomalous patches into a clean bank reduces AUPRC from 0.1030 to 0.0759. On all five completed held-out pairs, the proposed bank improves over naive expansion, with a mean gain of 0.0142 AUPRC. Reference purity is therefore a first-order design variable, and unverified images are useful only when their contribution is filtered explicitly.

Abstract

Explore similar work

When More References Hurt: Contamination-Aware DINOv2 Memory Banks for Few-Shot Steel Defect Detection

ProCon: Projection-Consistency Memory for Training-Free Anomaly Detection

Memory-Distilled Selection for Noise-Robust Anomaly Detection