How Many Categories Are Enough? Distribution-Free Certification Limits for Few-Shot Anomaly Thresholds

Gia Huy Thai; Nguyen Thai Anh

cs.LGSep 23, 2026

How Many Categories Are Enough? Distribution-Free Certification Limits for Few-Shot Anomaly Thresholds

Authors: Gia Huy Thai, Nguyen Thai Anh

Organizations: University of Science, VNU-HCM, Ho Chi Minh City, Vietnam · Faculty of Information Technology, Van Lang School of Technology, Van Lang University, Ho Chi Minh City, Vietnam

Abstract

Few-shot anomaly detectors are judged by ranking metrics, yet deployment requires an alarm threshold with a controlled false-alarm rate (FAR). We ask how much normal evidence, in images or category units, is needed to certify such a threshold for an unseen category. Using a frozen DINOv2 principal component analysis (PCA) residual ranker on 15 MVTec and 12 VisA categories under four corruption types, we show that target-only leave-one-image-out (LOIO) calibration is resolution-limited and shift-fragile: rank values cannot fall below $1/(k+1)$ , and at the attainable level $α=0.20$ , empirical FAR reaches 0.341 on Gaussian-corrupted MVTec at $k=4$ , 1.7 times the nominal level. A category-count feasibility calculus is then derived: even with all-zero category losses and no multiplicity charged, any deterministic, uniformly valid, distribution-free 95% upper confidence bound (UCB) requires at least 14, 29, and 59 independent and identically distributed (iid) category draws at $α=0.20$ , $0.10$ , and $0.05$ ; these counts are necessary but not sufficient. The Cross-category Reliability Estimation with Source Support (CRESS) protocol splits source categories into disjoint reference, proposal, and certification roles. With only three or four certification categories, all 960 frozen configurations return the fail-closed threshold $τ^\star=0$ , and the smallest category-level UCB is 0.950. Image-unit analyses of the same archive select positive thresholds in 36.7% to 60.3% of target cells; these bounds hold for the selected source mixture, not for the marginal risk of a new-category draw. The contribution is a quantitative feasibility boundary and an estimand-aware protocol specifying when source evidence can, and cannot, support a transferable reliability claim.

Figures & tables

Figure 1: Analytic category-count feasibility after observing zero loss in every certification category. The black curve is the multiplicity-free lower bound

1-0.05^{1/n}

for deterministic uniformly valid distribution-free procedures in Proposition 3. The blue curve applies the frozen allocation

\beta=\delta/(2AM)

with

\delta=0.05

A=3

reported levels, and the favorable one-candidate case

M=1

; the orange curve is the corresponding Hoeffding floor. Horizontal lines mark the requested risk levels, and the shaded region marks the three or four distinct certification categories available in the frozen audit; interpreting them as iid units is a modeling assumption. The black circular crossings at the 14, 29, and 59 axis ticks mark optimistic necessary counts without multiplicity, not sufficient sample sizes for CRESS. A curve above a requested

\alpha

rules out a positive threshold only within its stated procedure class, even under all-zero observed category loss.

Figure 2: Target-side scoring and reliability routes. Panel (a) employs frozen DINOv2 patch features and a support-fitted PCA residual scorer to produce the fixed image-ranking score and, optionally, a pixel anomaly map. The metric boxes are used exclusively for evaluation; the pixel map is an inherited output and does not serve as a certification target or a localization claim within this study. Panel (b) applies the scorer within LOIO folds to generate target-normal calibration scores and the target-only rank value $p_{c,\mathrm{LOIO}}(x)$ . Since calibration scores utilize $k-1$ supports and the test score uses all $k$ , this rank transformation is subject to empirical auditing rather than being assumed to possess automatic finite-sample validity. The minimum attainable value is $1/(k+1)$ , and the alarm rule does not modify the underlying ranking score.

Figure 3: CRESS source-side threshold certification. Each non-target source category provides normal support images for scorer fitting and support-only normalization statistics, along with held-out known-normal views under specified audit conditions. During source-archive preparation, scores are normalized before source-view selection. When $k=1$ , the declared patch-split support residual replaces LOIO calibration. Eligible source categories are partitioned into disjoint reference ( $\mathcal{R}$ ), proposal ( $\mathcal{P}$ ), and certification ( $\mathcal{C}$ ) roles. The reference set establishes the source-reference map $p_{\mathcal{R}}$ , which is then applied to proposal and certification views. The proposal set determines the candidate family, while the certification set generates category losses for the family-adjusted risk UCB. CRESS returns either the largest passing threshold or the fail-closed value zero. This process yields a source-domain certificate; however, target-category control requires the transfer condition specified in Corollary 1.

rule / candidates	$\alpha=.20$	$\alpha=.10$	$\alpha=.05$
Distribution-free, no multiplicity	14	29	59
Distribution-free, $M=1$	22	46	94
Distribution-free, $M=5$	29	61	125
Distribution-free, $M=20$	35	74	152
Hoeffding, $M=1$	60	240	958
Hoeffding, $M=5$	80	320	1,280

Table 1: The minimum iid certification-category counts required by the all-zero calculations at

\delta=0.05

. The distribution-free rows indicate the necessary lower bounds for deterministic, uniformly valid procedures as described in Proposition 3, first without multiplicity (

\beta=\delta

), and then with

\beta=\delta/(2AM)

A=3

, and realized family size

M

. The ’Hoeffding’ row corresponds to Proposition 2. All rows assume zero observed loss; satisfying a listed count does not guarantee a positive CRESS threshold.

dataset	$k$	floor	nominal $\alpha$	attainable	FAR	power
VisA (full)	4	0.200	0.01	0.000	0.000	0.000
			0.05	0.000	0.000	0.000
			0.10	0.000	0.000	0.000
			0.20	0.200	0.142	0.595
	8	0.111	0.01	0.000	0.000	0.000
			0.05	0.000	0.000	0.000

Table 2: Target-only LOIO operating points. The floor represents the smallest rank value,

1/(k{+}1)

, while the term ’attainable’ refers to the largest rank-value grid point not exceeding the nominal

\alpha

, or zero if no such point exists. For each corruption, FAR and detection power are pooled across categories and seeds. The table subsequently averages the four corruption-specific rates.

dataset	corruption	$k$	FAR	power	precision
VisA (full)	blur	4	0.144	0.606	0.825
	blur	8	0.094	0.566	0.870
	bright/contr.	4	0.146	0.596	0.821
	bright/contr.	8	0.098	0.557	0.864
	Gaussian noise	4	0.133	0.575	0.829
	Gaussian noise	8	0.081	0.538	0.881

Table 3: Target-only LOIO FAR, detection power, and alarm precision at the first reported attainable level,

\alpha{=}0.20

. The results utilize five support-sampling seeds with support sets nested across

k

, and are pooled over images within each dataset, corruption, and

k

. All normal and anomalous test images are retained. Alarms are triggered when the rank value satisfies

p\leq\alpha

, following the comparison tolerance specified in the text.

Figure 4: Empirical CDF of normal-image LOIO rank values at each attainable point

j/(k{+}1)

, compared to the tie-free discrete-uniform benchmark (dashed diagonal), using five support-sampling seeds. Valid ties may cause the population CDF to fall below this benchmark. Points above the diagonal indicate an excess of empirical false alarms relative to the benchmark. The lower-right insets and connector rays provide an enlarged view of the first attainable grid region.

source $\rightarrow$ target	certification unit	units/cell	min UCB	nonzero frac.	interpretation
MVTec $\rightarrow$ MVTec	category	4	0.950	0.000	category-unit gate fails
VisA $\rightarrow$ VisA	category	3	1.000	0.000	category-unit gate fails
MVTec $\rightarrow$ VisA	category	4	0.961	0.000	category-unit gate fails
MVTec $\rightarrow$ MPDD	category	4	0.986	0.000	category-unit gate fails
MVTec $\rightarrow$ MVTec	source image	75 to 191	0.039	0.367	iid-image sensitivity
VisA $\rightarrow$ VisA	source image	150 to 180	0.042	0.603	iid-image sensitivity

Table 4: Frozen strict CRESS audit over 960 job

\times

source-view

\times

k

\times

\alpha

\times

condition configurations, 240 per source-to-target job. Each configuration includes 75 MVTec, 60 VisA, or 30 MPDD target-category/seed cells; thus, the ’nonzero frac.’ metric aggregates 18,000, 14,400, or 7,200 cells per respective job. ’Min UCB’ denotes the smallest observed candidate bound. Category and image rows address distinct estimands; image rows operate under the idealized iid source-image model.

source $\rightarrow$ target	$\alpha$	FAR	power
MVTec $\rightarrow$ MVTec	0.05	0.075	0.404
	0.10	0.131	0.566
	0.20	0.242	0.754
VisA $\rightarrow$ VisA	0.05	0.060	0.320
	0.10	0.111	0.466
	0.20	0.208	0.639

Table 5: Direct pooled-source conformal under matched-condition selection. Reported entries represent unweighted means of within-target-cell rates across categories,

k\in\{1,2,4,8\}

, five support seeds, and five conditions. The complete selected source pool applies

\tau=\alpha

without a reference/proposal/certification split; this is an empirical, uncertified baseline.

Dataset	$k$	Method	AUROC $\uparrow$	AP $\uparrow$	MiB $\downarrow$
MVTec	1	AnomalyDINO-S (reported)	0.965	0.981	–
		WinCLIP (reported)	0.931	N/C	N/R
		Controlled DINOv2 NN	0.914	0.952	2.005
		PCA64 (controlled)	0.904	0.951	0.095
	4	AnomalyDINO-S (reported)	0.976	0.984	–
		WinCLIP (reported)	0.952	N/C	N/R

Table 6: Clean image-level ranking performance and category-specific ranker state in mebibytes (MiB). The controlled DINOv2 NN and PCA64/PCA128 methods utilize a shared feature set, data split, evaluation metric, and storage-accounting protocol, with results averaged over categories and five random seeds. AnomalyDINO-S (448) and WinCLIP values are reported in [ 7 , 16 ] under their respective protocols. Bold marks the best value only among common-protocol rows within each dataset/

k

block. The notation ’–’ indicates not evaluated, N/C denotes not directly comparable, and N/R signifies not reported.

dataset	$k$	max	top-0.5%	top-1%	top-2%	top-5%
MVTec	4	0.9571	0.9638	0.9667	0.9693	0.9621
MVTec	8	0.9578	0.9685	0.9740	0.9759	0.9727
VisA	4	0.9039	0.9101	0.9038	0.8944	0.8806
VisA	8	0.9101	0.9200	0.9147	0.9071	0.8941

Table 7: Aggregation-sensitivity ablation for clean image AUROC under max and top-

\rho

mean pooling. Results average three MVTec categories (bottle, cable, and hazelnut) or four VisA categories (candle, cashew, pcb1, and pipe fryum) over seeds from 0 to 2. Bold marks the best tested aggregator in each row. This subset analysis diagnoses sensitivity within the displayed grid; it is not a full-dataset ranking comparison.

Few-Shot Anomaly Detection

Few-shot anomaly detection uses a small set of examples to identify and localize unusual patterns in new data or domains.

See all 40 papers

Flownomaly: Single-Step Conditional-Mean Recovery for Few-Shot Anomaly Detection
Oct 23, 2026Bernardo Perrone Ribeiro, Erkut Akdag, Egor BondarevFew-Shot Anomaly Detection Industrial Anomaly Detection
AIMS Group, Department of Electrical Engineering, Eindhoven University of Technology
SPARC: Subspace Position-Aware Robust Few-Shot Calibration for Distribution-Shifted Industrial Anomaly Detection
Aug 19, 2026Seokhee Han, Seungjun Chu, Mateusz Nowak +1Industrial Anomaly Detection Anomaly Detection
Korea University
Distribution-free false-alarm calibration and chance-corrected spatial evaluation for industrial anomaly detection
Aug 15, 2026Jie DengIndustrial Anomaly Detection Anomaly Detection
Tongji University

Oct 23, 2026J/K move · Enter open · S save

Flownomaly: Single-Step Conditional-Mean Recovery for Few-Shot Anomaly Detection

Bernardo Perrone Ribeiro, Erkut Akdag, Egor Bondarev

AIMS Group, Department of Electrical Engineering, Eindhoven University of Technology

Few-Shot Anomaly Detection Industrial Anomaly Detection Anomaly Localization Reconstruction-Based Anomaly Detection

Open paper AskPDF arXiv

Few-shot industrial anomaly detection requires modeling category-specific normality from only a handful of defect-free images. Common approaches tie inference to stored exemplars, impose rigid linear structure, or risk reproducing anomalous content. Flownomaly instead casts the problem as normal-counterpart recovery in frozen DINOv2 feature space. Given normal supports, a velocity field is learned through conditional flow matching over frozen patch tokens to transport Gaussian noise toward normal features. Along straight paths, a single recovery step using one evaluation of the optimal field yields the posterior mean of the clean normal token without trajectory integration. The operating time governs how strongly recovery modifies a query token. With no anomalous validation samples available for tuning, a normal-only Jacobian gain-to-noise criterion selects this time from the supports alone. Squared residuals between query tokens and their recovered counterparts yield anomaly signals for both dense localization and image-level detection. Across 1-, 2-, and 4-shot benchmarks on MVTec-AD and VisA, together with sample-shot evaluations on Real-IAD, Flownomaly achieves state-of-the-art performance against recent category-specific methods. In the 1-shot setting, image- and pixel-level AUROC reach 97.8% and 98.0% on MVTec-AD and 95.3% and 98.7% on VisA, with a 2.1-point image-level AUROC improvement over the strongest baseline on VisA. Code is available at https://github.com/b-rbmp/Flownomaly.

Abstract

Few-Shot Anomaly Detection

Explore similar work

Flownomaly: Single-Step Conditional-Mean Recovery for Few-Shot Anomaly Detection

SPARC: Subspace Position-Aware Robust Few-Shot Calibration for Distribution-Shifted Industrial Anomaly Detection

Distribution-free false-alarm calibration and chance-corrected spatial evaluation for industrial anomaly detection