Sparse, self-organizing ensembles of local kernels detect rare statistical anomalies

Gaia Grosso; Sai Sumedh R. Hindupur; Thomas Fel; Samuel Bright-Thonney; Philip Harris; Demba Ba

cs.LGNov 5, 2025

Sparse, self-organizing ensembles of local kernels detect rare statistical anomalies

Authors: Gaia Grosso, Sai Sumedh R. Hindupur, Thomas Fel, Samuel Bright-Thonney, Philip Harris, Demba Ba

Organizations: NSF AI Institute for Artificial Intelligence and Fundamental Interactions, Cambridge, 02139, MA · Laboratory for Nuclear Science, Massachusetts Institute of Technology, Cambridge, 02139, MA · School of Engineering and Applied Sciences, Harvard University, Allston, 02134, MA · Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University, Allston, 02134, MA

Abstract

Modern artificial intelligence has revolutionized how we extract representations from scientific data, yet the statistical properties of these representations remain poorly controlled, causing misspecified anomaly detection methods to falter. The hardest anomalies to detect are the rare, weakly separable ones hiding within the nominal distribution-a regime that grows in importance as models mature and easily separable signals are exhausted. We identify structural desiderata for detection in this regime under minimal prior information: sparsity, to enforce parsimony; locality, to preserve geometric sensitivity; and competition, to promote efficient allocation of model capacity. These principles define a class of self-organizing local kernels that adaptively partition the representation space around regions of statistical imbalance. As an instantiation, we introduce SparKer, a sparse ensemble of Gaussian kernels trained in a semi-supervised Neyman-Pearson framework to locally model the likelihood ratio between a sample that may contain anomalies and an anomaly-free reference. We provide theoretical insights into the mechanisms driving detection and self-organization, and demonstrate the approach on realistic high-dimensional problems in scientific discovery, open-world novelty detection, intrusion detection, and generative-model validation. Ensembles of only a handful of kernels identify statistically significant anomalies in representation spaces of thousands of dimensions while remaining sensitive across regimes, underscoring the interpretability, efficiency, and scalability of the approach.

Figures & tables

Fig. 1 : Detecting rare in-distribution anomalies with SparKer . (a) Cartoon diagram illustrating the regimes of anomaly detection; problems are characterized by anomaly separability ( $x$ -axis) and anomaly fraction ( $y$ -axis). Detection becomes harder near the origin, where anomalies are rare and overlap with normal data. The right panel pictures the different corners of the space qualitatively. The left panel highlights the region SparKer is designed for, i.e. low anomaly fraction and separability. The stars indicate the suite of applications from natural and computer science that we showcase in Sections IV and V . (b) SparKer ’s anomaly detection pipeline: source data are embedded to extract domain-specific features; SparKer compares the distribution of the inspected data to anomaly-free data in feature space by means of a self-organizing ensemble of local kernels, and extracts an anomaly score ; (c) Sparsity, locality and competition, which underlie the SparKer model, make it possible to decompose the anomaly score into distinct, interpretable components, $\{f_{i}\}$ , pointing at different anomalous regions in the feature space, thus enabling a geometry-aware analysis of the anomalies.

Fig. 2 : Local layers enable geometric interpretation of activation patterns. Examples of 2D log-density ratio approximated by a 1-layer model with ReLU activation (top left grid) and a mixture of M Gaussian kernels (bottom left grid) subject to different sparsity constraints. The local activation characterizing Gaussian kernels enables efficient selection of convex regions, whereas linear activation requires the interplay of multiple parameters, undermining interpretability. We report on the right side the ground truth of the log-density ratio function targeted by the model, as well as the density of the sample with injected anomalies.

S_{0}=\{\{{\bm{x}}_{i}\}_{i=1}^{{\rm{N}}_{\cal{D}}}\mid{\bm{x}}\sim p(x|{\rm H_{0}})\}

Algorithm 1 SparKer algorithm

Fig. 9 : Inspecting anomalous particle jets. (a) Distribution of the anomaly score $c(x)$ over the inspected data (black histogram), compared with the average expected distribution in the absence of anomalies (error bars: standard deviation over 100 realizations). (b) Activation pattern over SparKer ’s components for the most anomalous points; $f_{4}$ and $f_{2}$ are the most positively activated components at high anomaly score. (c) Correlation of the two maximally-activated components, color coded by anomaly score: anomalous points are pushed to the edges of the distribution. (d) Correlation of $f_{4}$ with the jet invariant mass, a physics-motivated observable. Without prior physics knowledge, the model highlights jet masses around 125 GeV, matching the Higgs boson mass used as the anomalous signal here.

Fig. 10 : Inspecting detected open-world novelties. (a) Distribution of the anomaly score $c(x)$ over the inspected data (black histogram), compared with the average expected distribution in the absence of anomalies (error bars: standard deviation over 100 realizations). (b) Activation pattern over SparKer ’s components for the most anomalous points; $f_{3}$ is the most positively active. (c) Images of the most anomalous points, distinguishing true positives (“Striped” class in Textures), false positives, and reference events (ImageNet test set). (d) Predicted labels of the most anomalous points: though localized around $f_{3}$ , they span different ImageNet classes, indicating a non-trivial geometric signature.

Fig. 11 : Inspecting detected intrusions in text lines. (a) 2D PCA projection of the data, where the anomalous points, highlighted in yellow, lie on the support of the normal data. (b) 2D projection obtained using the two maximally-activated features found by SparKer ; in contrast to PCA, SparKer finds low-dimensional representations that single out the most anomalous regions of the data. (c) Distribution of SparKer ’s anomaly score over the inspected data (black histogram), compared to the average expected distribution in the absence of anomalies (the light blue bars measure the standard deviation over 100 anomaly-free realizations). (d) Activation pattern of SparKer ’s components for the most anomalous data points. Two components, $f_{0}$ and $f_{4}$ , exhibit the most correlation with the anomaly score. (e) Most anomalous lines colored by their activation pattern.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

	label	$\mu$	$\sigma$	N(S)/N(R)
Gaussian bumps	bulk	1.6	0.16	$4.5\cdot 10^{-2}$
	tail	6.4	0.16	$5\cdot 10^{-3}$
	extreme tail	9	0.16	$2.5\cdot 10^{-3}$
Tail excess	excess			$4.5\cdot 10^{-2}$

Appendix

TABLE I : Summary of the 1D signal benchmarks.

	label	$\mu$	$\sigma$	N(S)/N(R)
Gaussian bumps	bulk	(-6, -6)	1	$1\cdot 10^{-2}$
	ood	(0, 0)	1	$1\cdot 10^{-3}$
Scale distortion	smearing		2.2
	squeezing		1.8

Appendix

TABLE II : Summary of the 2D signal benchmarks.

Fig. 13 : 2D benchmark ground truth. Log-likelihood ratio between the alternative and the reference hypotheses in the four 2D benchmarks considered in this work, shown on a common color scale (clipped at $\pm 5$ ); the black contours are iso-density level curves of the reference mixture, delimiting its bulk and tails. The out-of-distribution cluster (second panel) produces a large log-ratio in the empty central region, and the two scale distortions (third and fourth panels) deform the whole dominance region of the modified component; in contrast, the over-density in the bulk (first panel) is barely visible on this scale, as the signal sits where the reference density is highest and the log-ratio remains close to zero—making it the hardest benchmark of the four, and the closest to the rare, in-distribution regime targeted in this work.

	mass	N(S)/N(R)*
resonant signal	Z’ ( $m=200$ GeV)	$7.5\cdot 10^{-3}$
	Z’ ( $m=300$ GeV)	$3.75\cdot 10^{-3}$
	Z’ ( $m=600$ GeV)	$1.25\cdot 10^{-3}$
non-resonant signal	$c_{W}=1.5\,\text{TeV}^{-2}$	$2.65\cdot 10^{-3}$
*after physics cuts applied

Appendix

TABLE III : Summary of the 5D di-muon signal benchmarks.

Distance-based two-sample tests
Method	bulk	tail	extreme tail	tail excess
Energy distance	$0.67\,[0.62,0.72]$	$0.08\,[0.05,0.12]$	$0.10\,[0.07,0.14]$	$0.90\,[0.86,0.93]$
Nyström-MMD ( $M{=}10$ )	$\mathbf{0.95\,[0.92,0.96]}$	$0.17\,[0.13,0.20]$	$0.07\,[0.05,0.09]$	$\mathbf{0.98\,[0.95,0.99]}$
Nyström-MMD ( $M{=}200$ )	$\mathbf{1.00\,[0.98,1.00]}$	$0.08\,[0.05,0.10]$	$0.07\,[0.05,0.09]$	$\mathbf{1.00\,[0.98,1.00]}$
$k$ -NN two-sample balanced (aggregated)	$0.41\,[0.36,0.46]$	$0.22\,[0.18,0.27]$	$0.18\,[0.14,0.23]$	$0.16\,[0.12,0.21]$
$k$ -NN distance (aggregated)	$0.09\,[0.06,0.13]$	$0.55\,[0.50,0.60]$	$\mathbf{0.82\,[0.77,0.86]}$	$0.69\,[0.64,0.74]$

Appendix

TABLE IV : Two-sample test (1D). Detection power at

5\%

false-positive rate on the 1D benchmark, for each signal type. Values are power with

68\%

Clopper–Pearson intervals, estimated over

n=100

independent realizations; all methods share the identical signal definitions (Table I ) and empirical calibration (Algorithm 1 of the main text). Best entries per column in bold .

Distance-based two-sample tests
Method	bulk	ood	smearing	squeezing
Energy distance	$0.08\,[0.05,0.12]$	$0.06\,[0.04,0.09]$	$0.04\,[0.02,0.07]$	$0.10\,[0.07,0.14]$
Nyström-MMD ( $M{=}15$ )	$0.23\,[0.19,0.27]$	$0.20\,[0.16,0.24]$	$0.19\,[0.15,0.22]$	$0.76\,[0.71,0.80]$
Nyström-MMD ( $M{=}60$ )	$\mathbf{0.28\,[0.23,0.32]}$	$0.18\,[0.14,0.21]$	$0.21\,[0.17,0.25]$	$\mathbf{0.91\,[0.87,0.93]}$
$k$ -NN two-sample balanced (aggregated)	$0.11\,[0.08,0.15]$	$0.20\,[0.16,0.25]$	$0.60\,[0.55,0.65]$	$0.26\,[0.21,0.31]$
$k$ -NN distance (aggregated)	$0.08\,[0.05,0.12]$	$\mathbf{0.76\,[0.71,0.80]}$	$\mathbf{0.96\,[0.93,0.98]}$	$0.00\,[0.00,0.02]$

Appendix

TABLE V : Two-sample test (2D). Detection power at

5\%

false-positive rate on the 2D benchmark, for each signal type. Values are power with

68\%

Clopper–Pearson intervals, estimated over

n=100

independent realizations; all methods share the identical signal definitions (Table II ) and empirical calibration (Algorithm 1 of the main text). Best entries per column in bold .

Distance-based two-sample tests
Method	Z’200 (bulk)	Z’300 (tail)	Z’600 (extreme tail)	EFT (non-resonant)
Nyström-MMD ( $M{=}200$ )	$0.08\,[0.05,0.10]$	$0.07\,[0.05,0.09]$	$0.07\,[0.05,0.09]$	$0.09\,[0.06,0.11]$
Nyström-MMD ( $M{=}1000$ )	$0.08\,[0.05,0.10]$	$0.07\,[0.05,0.09]$	$0.07\,[0.05,0.09]$	$0.09\,[0.06,0.11]$
$k$ -NN two-sample balanced (aggregated)	$0.12\,[0.09,0.16]$	$0.20\,[0.16,0.25]$	$0.13\,[0.10,0.17]$	$0.12\,[0.09,0.16]$
$k$ -NN distance (aggregated)	$0.36\,[0.31,0.41]$	$0.35\,[0.30,0.40]$	$0.40\,[0.35,0.45]$	$0.85\,[0.81,0.89]$
Density-based methods (unsup.)

Appendix

TABLE VI : Two-sample test (5D). Detection power at

5\%

false-positive rate on the 5D benchmark, for each signal type. Values are power with

68\%

Clopper–Pearson intervals, estimated over

n=100

independent realizations; all methods share the identical signal definitions (Table III ) and empirical calibration (Algorithm 1 of the main text). Best entries per column in bold .

Unsupervised point-wise detectors
Method	bulk	tail	extreme_tail	tail_excess
ECOD	$0.00\,[0.00,0.00]$	$\mathbf{1.00\,[1.00,1.00]}$	$\mathbf{1.00\,[1.00,1.00]}$	$0.27\,[0.27,0.28]$
Isolation Forest	$0.00\,[0.00,0.00]$	$\mathbf{1.00\,[1.00,1.00]}$	$\mathbf{1.00\,[1.00,1.00]}$	$0.42\,[0.41,0.43]$
$k$ -NN distance	$0.00\,[0.00,0.00]$	$\mathbf{1.00\,[1.00,1.00]}$	$\mathbf{1.00\,[1.00,1.00]}$	$\mathbf{0.45\,[0.45,0.46]}$
SOM quant. error ( $M{=}200$ )	$0.00\,[0.00,0.00]$	$\mathbf{1.00\,[1.00,1.00]}$	$\mathbf{1.00\,[1.00,1.00]}$	$0.25\,[0.24,0.26]$
Semi-supervised

Appendix

TABLE VII : Point-wise detection (1D). True-positive rate at a

5\%

per-point false-positive rate (

\mathrm{TPR}@5\%

), on the 1D benchmarks where point-level labels exist. The decision threshold targets a

5\%

per-point false-positive rate and is set on anomaly-free data: on the frozen score distribution over the reference

\mathcal{R}

for the unsupervised detectors, and on held-out anomaly-free data via cross-fitting for SparKer (see text). Values are

\mathrm{TPR}

with

68\%

Clopper–Pearson intervals over

n=100

realizations. Note that the per-point false-positive rate is distinct from the collective power@

5\%

of Tables IV – V , and the two should not be compared directly.

Unsupervised point-wise detectors
Method	bulk	ood
ECOD	$0.00\,[0.00,0.00]$	$0.00\,[0.00,0.00]$
Isolation Forest	$0.00\,[0.00,0.00]$	$\mathbf{1.00\,[1.00,1.00]}$
$k$ -NN distance	$0.05\,[0.05,0.05]$	$\mathbf{1.00\,[1.00,1.00]}$
SOM quant. error ( $M{=}200$ )	$0.06\,[0.05,0.06]$	$\mathbf{1.00\,[1.00,1.00]}$
Semi-supervised

Appendix

TABLE VIII : Point-wise detection (2D). True-positive rate at a

5\%

per-point false-positive rate (

\mathrm{TPR}@5\%

), on the 2D benchmarks where point-level labels exist. The decision threshold targets a

5\%

per-point false-positive rate and is set on anomaly-free data: on the frozen score distribution over the reference

\mathcal{R}

for the unsupervised detectors, and on held-out anomaly-free data via cross-fitting for SparKer (see text). Values are

\mathrm{TPR}

with

68\%

Clopper–Pearson intervals over

n=100

realizations. Note that the per-point false-positive rate is distinct from the collective power@

5\%

of Tables IV – V , and the two should not be compared directly.

1D benchmarks
Method	bulk	tail	extreme tail	tail excess
SOM quant. error ( $M{=}5$ )	$0.05\,[0.03,0.08]$	$0.36\,[0.31,0.41]$	$0.32\,[0.27,0.37]$	$0.66\,[0.61,0.71]$
SOM quant. error ( $M{=}20$ )	$0.08\,[0.05,0.12]$	$0.51\,[0.46,0.56]$	$0.58\,[0.53,0.63]$	$0.69\,[0.64,0.74]$
SOM quant. error ( $M{=}100$ )	$0.06\,[0.04,0.09]$	$0.51\,[0.46,0.56]$	$0.59\,[0.54,0.64]$	$0.55\,[0.50,0.60]$
SOM quant. error ( $M{=}200$ )	$0.08\,[0.05,0.12]$	$0.58\,[0.53,0.63]$	$0.56\,[0.51,0.61]$	$0.52\,[0.47,0.57]$
SOM quant. error ( $M$ -aggregated)	$0.10\,[0.07,0.14]$	$0.64\,[0.59,0.69]$	$0.63\,[0.58,0.68]$	$0.75\,[0.70,0.80]$

Appendix

TABLE IX : SOM-based methods: summary of the results. Detection power at

5\%

false-positive rate on the 1D, 2D, and 5D benchmarks, for each signal type, for the two SOM-based detectors across four choices of the number of prototypes

M

used to train the SOM. The

M

-aggregated rows combine the per-

M

p

-values via the same combination rule used for the kernel bandwidths (Eq. 63 ), paying the associated trial factor explicitly. Values are power with

68\%

Clopper–Pearson intervals, estimated over

n=100

NP	$\displaystyle\mathcal{L}_{\mathrm{NP}}[f]=w_{{\cal{R}}}\sum_{x\in{\cal{R}}}\big(e^{f(x)}-1\big)-\sum_{x\in\mathcal{D}}f(x)$
	$\displaystyle\partial_{\theta}\mathcal{L}_{\mathrm{NP}}[f]=w_{{\cal{R}}}\sum_{x\in{\cal{R}}}e^{f(x)}\,\partial_{\theta}f(x)-\sum_{x\in{\cal{D}}}\partial_{\theta}f(x)$
BCE	$\displaystyle\mathcal{L}_{\mathrm{BCE}}[f]=-w_{{\cal{R}}}\sum_{x\in{\cal{R}}}\log\!\big[1-\sigma(f(x))\big]-\sum_{x\in{\cal{D}}}\log\!\big[\sigma(f(x))\big]$
	$\displaystyle\partial_{\theta}\mathcal{L}_{\mathrm{BCE}}[f]=w_{{\cal{R}}}\sum_{x\in{\cal{R}}}\sigma(f(x))\,\partial_{\theta}f(x)-\sum_{x\in{\cal{D}}}[1-\sigma(f(x))]\,\partial_{\theta}f(x)$
MSE	$\displaystyle\mathcal{L}_{\mathrm{MSE}}[f]=w_{{\cal{R}}}\sum_{x\in{\cal{R}}}[\sigma(f(x))]^{2}+\sum_{x\in{\cal{D}}}[1-\sigma(f(x))]^{2}$
	$\displaystyle\partial_{\theta}\mathcal{L}_{\mathrm{MSE}}[f]=w_{{\cal{R}}}\sum_{x\in{\cal{R}}}2\,\sigma(f(x))^{2}[1-\sigma(f(x))]\,\partial_{\theta}f(x)-\sum_{x\in{\cal{D}}}2\,\sigma(f(x))[1-\sigma(f(x))]^{2}\,\partial_{\theta}f(x)$

Appendix

TABLE X : Comparison of loss functions and their gradients. We study the Neyman-Pearson (NP) loss, the Binary Cross Entropy (BCE) loss and the Mean Squared Error (MSE) loss. Each loss defines a different optimization landscape and gradient behavior.

Loss function	MSE	BCE	NP
Power@5%	0.61	0.68	0.86

Appendix

TABLE XI : Detection power at 5% false positive rate for various choices of the loss function: mean squared error (MSE), binary cross-entropy (BCE), and Neyman-Pearson (NP). The NP loss reaches the highest performance.

Model	w/o ${\rm SoftMax}$	w/ ${\rm SoftMax}$
Power@5%	0.72	0.86

Appendix

TABLE XII : Detection power at 5% false positive rate for SparKer model with and without

{\rm SoftMax}

activation layer.

{\rm SoftMax}

activation leads to higher performance.

Benchmark	Signal	$M{=}2$	$M{=}5$	$M{=}20$	$M{=}50$
1D exponential	bulk	0.94 [0.91, 0.96]	0.92 [0.88, 0.94]	0.83 [0.79,0.86]	0.85 [0.81, 0.88]
	tail	0.67 [0.62, 0.71]	0.80 [0.75, 0.83]	0.74 [0.69,0.78]	0.76 [0.71, 0.80]
	extreme tail	0.62 [0.57, 0.66]	0.86 [0.82, 0.89]	0.82 [0.78,0.85]	0.86 [0.82, 0.89]
	tail excess	0.98 [0.95, 0.99]	0.94 [0.91, 0.96]	0.92 [0.88,0.94]	0.95 [0.92, 0.96]
2D mixture	bulk	0.25 [0.20, 0.29]	0.27 [0.22, 0.31]	0.31 [0.18, 0.40]	–
	ood	0.58 [0.53, 0.63]	0.50 [0.44, 0.54]	0.22 [0.18, 0.26]	–

Appendix

TABLE XIII : Ablation of the ensemble size

M

. Detection power at

5\%

false-positive rate of SparKer across the controlled benchmarks, for the ensemble sizes run in this work (dashes: configuration not run). Values are power with

68\%

Clopper–Pearson intervals over

n=100

signal realizations, calibrated on

300

anomaly-free realizations, sharing the signal definitions and calibration pipeline of Section 3. We highlight in bold the best power in each row. Power does not increase with capacity: beyond the anomaly multiplicity, additional kernels inflate the null distribution and cost sensitivity (Section 1.3.1 of the main text).

Application	$\bm{n_{\rm steps}}$	L2 coeff.	Amplitude clip
Gravitational waves detection	100k	1	100
New particle jets discovery	100k	$10^{-3}$	100
Hybrid butterfly discovery	100k	$10^{-4}$	200
intrusion detection in Dante’s text	100k	$10^{-3}$	500
OOD in ImageNet	40k	0	10k
CIFAR-10 vs. CIFAR-5m	40k	0	10k

Appendix

TABLE XIV : SparKer training details. Number of training epochs, L2 regularization coefficient and amplitude clipping for various applications considered in the paper.

Fig. 19 : Extracting geometric interpretation from SparKer . The left side of the figure shows an example of data analysis in the hybrid butterfly detection problem. From panel (b) we identify three SparKer components ( $f_{1}$ , $f_{2}$ and $f_{4}$ ) highly activated by the most anomalous points in the dataset. In panel (c) we report the correlation plot between the positively activated components, color coded by the anomaly score, and we distinguish groups of correctly identified hybrids according to their location in this plane. The right side of the figure shows the results of our experiments on the CIFAR generative model validation problem. Panel (d) shows a consistent excess in the right tail of the anomaly score distribution, indicating that the generative model (used as the reference) fails to adequately represent a subspace of the data. More precisely, it under-represents it. Panel (e) informs us about the main regions of model misspecification; three components ( $f_{1}$ , $f_{2}$ and $f_{3}$ ) highly activate to the anomalous points. Panel (f) shows a 3D scatter plot over these components, color coded by anomaly score, and identifies three orthogonal subgroups of anomalies, aligning with the ImageNet classes of “rule” ( $f_{1}$ ), “Pekinese” ( $f_{2}$ ), and “crossword puzzle” ( $f_{3}$ ).

Explore similar work

Oct 15, 2025cs.LG

Isolation-based Spherical Ensemble Representations for Tabular Anomaly Detection

Unsupervised tabular anomaly detection is a critical task with applications spanning offensive language detection, network security, and quality control. Despite extensive research, existing unsupervised anomaly detection methods still face fundamental challenges including conflicting distributional assumptions, computational inefficiency, and difficulty handling different anomaly types. To address these problems, we propose ISER (Isolation-based Spherical Ensemble Representations) that extends existing isolation-based methods by using hypersphere radii as a monotonic transformation of local density characteristics while maintaining linear time and constant space complexity w.r.t. the dataset size. ISER constructs ensemble representations where hypersphere radii encode local sparsity through a monotonic transformation of density: smaller radii correspond to dense regions while larger radii correspond to sparse regions. We introduce a novel similarity-based scoring method that measures pattern consistency by comparing ensemble representations against a theoretical anomaly reference pattern. Additionally, we enhance the performance of Isolation Forest by using ISER and adapting the scoring function to address axis-parallel bias and local anomaly detection limitations. Comprehensive experiments on 20 real-world datasets demonstrate ISER's competitive performance over 12 SOTA methods.

Yang Cao, Sikun Yang, Hao Tian +5

Tsinghua Shenzhen International Graduate School, Tsinghua University, China · School of Computing and Information Technology, Great Bay University, China · School of Compuer Science, Nanjing University, China +3

Graph Anomaly Detection Interpretable Anomaly Detection Unsupervised Ensemble

Ask PDF

May 7, 2026cs.LG

Kurtosis-Guided Denoising Score Matching for Tabular Anomaly Detection

Denoising score matching (DSM) provides a way to learn data distributions by training a neural network to recover the score function, defined as the gradient of the log density, from noise-corrupted samples. Once trained, the score magnitude at a test point reflects how consistent that point is with the learned distribution, making it a natural anomaly signal. The key practical challenge is selecting the perturbation scale: too little noise yields unstable score estimates in sparse regions, while too much erases local structure and weakens anomaly sensitivity. This is compounded by the difficulty of hyperparameter tuning when anomalies are unknown and no validation set is available. We introduce kurtosis-based noise scaling (K-DSM), a per-feature scheme that sets noise levels from the shape of each marginal distribution, improving coverage of low-density regions and precision in high-density regions without extra model complexity. Contrary to prior claims that multi-scale or noise-conditioned training is necessary, we find that a carefully trained single-scale model is already a strong anomaly detector. On standard tabular anomaly detection benchmarks, K-DSM achieves state-of-the-art performance in the semi-supervised setting. When combined with a lightweight EMA-teacher filtering rule that removes low-density training points before each gradient step, it also achieves strong performance in the fully unsupervised (contaminated) setting, suggesting that simple, data-adaptive noise scaling enables robust anomaly detection while reducing reliance on hyperparameter tuning.

Victor Livernoche, Jie Zan, Reihaneh Rabbany

McGill University, Mila, Montréal, Canada

Graph Anomaly Detection Interpretable Anomaly Detection

Ask PDF

Sep 22, 2026cs.LG

Can We Predict Anomaly Detection Performance from Embedding-Space Geometry?

Anomaly detection systems are often trained using normal data alone, while model selection and evaluation typically require labeled anomalies. We study whether anomaly detection performance can be predicted without access to anomalous data. For kNN-based detectors, we derive a lower bound on the area under the ROC curve (AUC) that relates detection performance to the separation between inlier and outlier scores and to their respective variances. Under a local scaling model, we use this bound to characterize how density variation, intrinsic-dimensional heterogeneity, and cross-domain mismatch contribute to score variability. We then investigate anomaly-free model selection and show that inlier score variance alone does not reliably predict performance across different representations. To address this limitation, we introduce simple pseudo-anomaly probes that provide a reference for estimating relative score separation. Experiments on the DCASE 2022-2025 benchmarks, spanning four embedding models and 208 candidate systems, show that pseudo-anomaly-based estimators substantially improve anomaly-free model selection. In particular, diverse pseudo-anomalies enable anomaly-free model selection to outperform conventional development-set selection under domain shift. These results show that embedding-space geometry contains predictive information about anomaly detection performance while also highlighting the representation-dependent nature of inlier-only performance estimates.

Kevin Wilkinghoff, Zheng-Hua Tan

Department of Electronic Systems, Aalborg University, Denmark · Pioneer Centre for Artificial Intelligence, Denmark

Area Under The Receiver Operating Characteristic Curve Detection Performance Outliers Model Selection Spherical Latent Space

Ask PDF