Mitigating Convergence Collapse in Fixed-Target Anomaly Detectors via Kernel-Anchored Locality Regularization

José Lucas De Melo Costa; Fabrice Popineau; Arpad Rimmel; Bich-Liên Doan

cs.LGOct 1, 2026

Mitigating Convergence Collapse in Fixed-Target Anomaly Detectors via Kernel-Anchored Locality Regularization

Authors: José Lucas De Melo Costa, Fabrice Popineau, Arpad Rimmel, Bich-Liên Doan

Organizations: Université Paris-Saclay, CNRS, CentraleSupélec, LISN UMR 9015 Orsay, France · Université Paris-Saclay, CNRS, CentraleSupélec, Laboratoire Interdisciplinaire des Sciences du Numérique Orsay, France

Abstract

A family of tabular anomaly detectors trains a neural map toward a fixed target under squared-error loss and scores anomalies by the test-time residual; contraction matching, one-step rectified flow, and reconstruction autoencoders all fit this template. We characterize a convergence collapse: better optimization makes the detector worse. At convergence, the learned map tracks the target even off-distribution, so the residual signal vanishes on anomalies as well as on normal data. These detectors therefore rely on implicit non-convergence (early stopping, capacity caps) to retain signal. We argue this is structural: effective anomaly detection requires a locality constraint that blocks unconstrained extrapolation. Classical detectors (kNN, KDE, isolation forests, LOF) enforce locality explicitly; fixed-target neural detectors do not. We formalize the connection by showing that the kernel-regression analog of a fixed-target detector is a finite-bandwidth Nadaraya-Watson smoother, which we call Kernel Contraction Matching (KCM). KCM is closed-form, training-free, and CPU-efficient, yet matches established neural baselines on ADBench. Building on this bridge, we introduce the Kernel-Anchored Regularizer (KAR), which penalizes deviation of the neural prediction from a kernel-weighted average of training targets. Across collapse-prone ADBench datasets and three backbones, KAR mitigates collapse and improves AUROC under prolonged training.

Figures & tables

Figure 1. Convergence collapse across fixed-target anomaly detectors. Top: the anomaly-score field flattens along training, until the anomalies (stars) are no longer separated from the normal cluster (dots). Bottom left: on ADBench ( Han et al., 2022 ) 7_Cardiotocography , the methods reach an early peak (

\star

) and then decay toward convergence (

\triangledown

). Bottom right: the peak-to-converged AUROC drops are

0.128

0.051

, and

0.017

. A four-panel figure showing that fixed-target anomaly detectors improve early in training and then collapse as training continues; the anomaly scores flatten, and the AUROC drops while the training loss keeps decreasing.

Family	Off-data score is bounded by	Collapse?
T1. Explicit locality	Nearby training points; far queries score high.	No
T2. Density-coupled target	A $p_{\mathrm{train}}$ -dependent target; off-manifold scores stay non-trivial.	No
T3. Fixed target	An unconstrained residual to a fixed target; convergence can erase it.	Yes

Table 1. A taxonomy of anomaly detectors by what constrains their off-data scores. Fixed-target methods (T3) can collapse.

Detector	vert.	annth.	Pima	Ionos.	WDBC	cardio
Denoising autoencoder ( Ruff et al., 2021 )	$.022$	$.050$	$.039$	$.004$	$.007$	$.070$
Targeted-collapse autoencoder ( Ghafourian et al., 2025 )	$.000$	$.008$	$.018$	$.005$	$.007$	$.065$
Deep SVDD ( Ruff et al., 2018 )	$.019$	$.119$	$.019$	$.003$	$.005$	$.034$
Random-distance prediction ( Wang et al., 2020 )	$.106$	$.030$	$.048$	$.011$	$.013$	$.011$
Rectified-flow correction ( Beizaee et al., 2025 )	$.018$	$.110$	$.091$	$.004$	$.005$	$.018$
Flow-matching detector ( Li et al., 2025a )	$.218$	$.047$	$.042$	$.072$	$.002$	$.015$

Table 2. Peak-to-last-epoch AUROC gap, averaged over

3

seeds after

100

epochs of vanilla training. Positive values mean that training past the early peak hurts detection.

Figure 2. kcm at a query

q

: the prediction

\hat{x}_{h}(q)=\sum_{n}w_{n}(q)\,x_{n}

with

w_{n}\!\propto\!e^{-\|q-x_{n}\|^{2}/2h^{2}}

is a kernel-weighted average of the training normals (point size and colour scale with the weight). The reconstruction lies in

\operatorname{conv}(X)

(Proposition 2 ), and the anomaly score

s_{\textsc{kcm}}(q)=\|q-\hat{x}_{h}(q)\|

is the residual arrow. Schematic of KCM at a single query: training points sit inside their convex hull (shaded polygon); point sizes and colours encode their kernel weight to a query (red X) outside the hull; a blue star inside the hull marks the kernel-weighted-average reconstruction; an arrow from the query to the reconstruction is labelled with the KCM score, the residual norm.

Figure 3. Per-dataset AUROC of the

46

label-free detectors over the

47

ADBench datasets, ordered by mean (diamond). kcm (red) has the highest mean AUROC, level with DTE-NP and LUNAR (the only two methods within

0.012

). Per-method AUROC distribution over the 47 ADBench datasets, methods ordered left-to-right by mean AUROC. Each method is a box-and-whisker with the mean as a diamond. KCM, drawn red on the far left, has the highest mean AUROC; DTE-NP and LUNAR are next; the bulk of classical detectors (grey) and deep detectors (blue) follow.

Figure 4. Nemenyi critical-difference diagram, top twelve methods (

\mathrm{CD}=2.43

); methods joined by a bar are not significantly different. kcm shares the leading group with DTE-NP and LUNAR. Nemenyi critical-difference diagram over the top-twelve detectors on the 47 ADBench datasets, with critical difference 2.43. Methods are arranged along a horizontal mean-rank axis; methods whose ranks differ by less than the critical difference are joined by a thick horizontal bar. KCM, DTE-NP and LUNAR are in the leading group.

Method	Fit (s)	Infer. (s)	Worst (s)
kNN	$0.021$	$0.014$	$280$
IForest	$0.062$	$0.009$	$2.60$
ECOD	$0.156$	$0.009$	$38.0$
DTE-NP	$0.001$	$0.302$	$771$
KDE	$0.152$	$0.213$	$1046$
KCM	$0.331$	$0.060$	$38.5$

Table 3. Per-detector wall-clock over the

47

ADBench datasets, one CPU core: median fit, median inference, worst-case total; cells shaded

\propto\log

within each column. KDE and DTE-NP did not complete on

5

and

1

datasets (medians cover completed runs). All timings are author measurements.

		kar (anchored)			reg. removed
Dataset	kcm	$\rho^{\star}$	AUROC	gap	AUROC	gap
annthyroid	$0.929$	$0.5$	$\mathbf{0.948}$	$+0.000$	$0.905$	$+0.044$
Pima	$0.701$	$0.5$	$\mathbf{0.730}$	$+0.004$	$0.676$	$+0.042$
Ionosphere	$0.974$	$1.0$	$\mathbf{0.976}$	$+0.001$	$0.959$	$+0.003$
WDBC	$0.993$	$1.0$	$0.993$	$+0.001$	$0.976$	$+0.017$
cardio	$0.963$	$0.5$	$0.960$	$+0.002$	$0.938$	$+0.014$

Table 4. kar anchor-radius ablation, six collapse-prone datasets (mean of three seeds,

100

epochs).

\rho^{\star}

, the smallest positive radius approaching the best final AUROC, is an oracle upper bound . Last columns: regularizer removed (

\rho=100

\mathrm{gap}=\mathrm{peak}-\mathrm{final}

Figure 5. Convergence-collapse gap on the

47

ADBench datasets, kar at

\rho=0.5

vs the regularizer removed (

\rho=100

); each series sorted independently, dotted line at the

0.02

threshold. Mean of three seeds,

846

runs. Per-dataset convergence-collapse gap across the 47 ADBench datasets. Two lines, each sorted ascending: KAR at radius rho equals 0.5 stays at or below a 0.02 collapse threshold on 42 of 47 datasets, while the regularizer-removed limit exceeds the threshold on 14 datasets.

Figure 6. Test AUROC over

500

epochs (4 of 6 datasets). kar at

\rho=0.5

(red) stays flat; with the regularizer removed (

\rho=100

, grey) the backbone drifts. Dotted: kcm . Mean of three seeds, shaded

\pm 1

s.d. Four side-by-side line plots, one per dataset (annthyroid, Pima, cardio, WDBC), showing test AUROC over 500 training epochs. The red curve (KAR with rho equals 0.5) stays high and flat; the grey curve (regularizer removed, rho equals 100) drifts downward as training continues; a black dotted line marks the KCM reference. Shaded bands are one standard deviation over three seeds.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Dataset	via TCCM	via AE	via Deep SVDD	spread
annthyroid	$0.944$	$0.944$	$0.945$	$1.4\!\times\!10^{-3}$
Pima	$0.728$	$0.740$	$0.728$	$2.2\!\times\!10^{-2}$
Ionosphere	$0.976$	$0.975$	$0.976$	$2.4\!\times\!10^{-3}$
WDBC	$0.993$	$0.993$	$0.993$	$5.6\!\times\!10^{-4}$
cardio	$0.958$	$0.955$	$0.958$	$5.5\!\times\!10^{-3}$
vertebral	$0.455$	$0.472$	$0.455$	$6.2\!\times\!10^{-2}$

Appendix

Table 5. kar through three fixed-target backbones (

\rho=1

, mean of three seeds,

100

epochs); spread = maximum, over seeds, of the per-seed pairwise gap of the three final AUROCs. Table with one row per of six datasets and four numeric columns: AUROC via TCCM, via AE, via Deep SVDD, and the cross-backbone spread. On five of the six datasets the three AUROCs differ by less than 2.2e-2; on vertebral the spread reaches 6.2e-2 because the kernel anchor is near random.

Anomaly Detection

Anomaly detection finds rare or unexpected patterns in data, from defective products and system faults to fraud and security threats.

See all 655 papers

ProCon: Projection-Consistency Memory for Training-Free Anomaly Detection
Jul 6, 2026Joongwon Chae, Lihui Luo, Yang Liu +4Industrial Anomaly Detection Anomaly Localization
Tsinghua University Shenzhen International Graduate School, Shenzhen, China · Affiliated Fifth Hospital, Wenzhou Medical University, Wenzhou, Zhejiang, China · Chinese Medicine Guangdong Laboratory +1
Between Resolution Collapse and Variance Inflation: Weighted Conformal Anomaly Detection in Low-Data Regimes
Mar 24, 2026Oliver Hennhöfer, Christine PreisachAnomaly Detection Weighted Conformal Prediction
Intelligent Systems Research Group (ISRG), Karlsruhe University of Applied Sciences, Germany
Contrast to Detect: Dynamic Graph Contrastive Regularization for Unsupervised Anomaly Detection in Multivariate Time Series
May 22, 2026Yunhua Pei, Zixing Song, Jin Zheng +1Dynamic Graph Learning Unsupervised Anomaly Detection
School of Computer Science, University of Bristol, Bristol, UK · School of Engineering Mathematics, University of Bristol, Bristol, UK

Jul 6, 2026J/K move · Enter open · S save

ProCon: Projection-Consistency Memory for Training-Free Anomaly Detection

Joongwon Chae, Lihui Luo, Yang Liu, Dongmei Yu+3

Tsinghua University Shenzhen International Graduate School, Shenzhen, China · Affiliated Fifth Hospital, Wenzhou Medical University, Wenzhou, Zhejiang, China · Chinese Medicine Guangdong Laboratory+1

Industrial Anomaly Detection Anomaly Localization Reconstruction-Based Anomaly Detection Anomaly Detection

Open paper AskPDF arXiv

Memory-based anomaly detection is attractive because it localizes defects from normal images without training a decoder or synthesizing pseudo anomalies. However, most memory methods still use the memory bank as a nearest-neighbor lookup table: a test patch is treated as normal if it has one nearby normal anchor. This hard retrieval view is vulnerable to false-normal matches and does not test whether the patch is consistently supported by a local normal neighborhood. We propose ProCon, a training-free framework that turns memory retrieval into decoder-free reconstruction. ProCon softly projects each test patch onto nearby normal memory vectors and uses the projection residual as anomaly evidence. To stabilize this residual, it constructs seed-perturbed layer-wise memories, aggregates bank residuals by a median, and fuses depth-specific residual maps by layer consensus. ProCon requires no decoder training, backbone fine-tuning, learned fusion weights, or pseudo-anomaly supervision. Across MVTec-AD, VisA, and Real-IAD under the single-category evaluation protocol, ProCon achieves strong image- and pixel-level performance under seven standard metrics, including image AUROC scores of 99.8%, 99.2%, and 93.2%, respectively. Ablations show that the gains come from replacing hard retrieval with soft normal projection and stabilizing the residuals through memory and depth consensus. The code is available at https://github.com/jw-chae/Procon

Abstract

Anomaly Detection

Explore similar work

ProCon: Projection-Consistency Memory for Training-Free Anomaly Detection

Between Resolution Collapse and Variance Inflation: Weighted Conformal Anomaly Detection in Low-Data Regimes

Contrast to Detect: Dynamic Graph Contrastive Regularization for Unsupervised Anomaly Detection in Multivariate Time Series