Bounded AUKS: A Novel Evaluation Metric for Anomaly Detection via Bounded Area Under the Kolmogorov--Smirnov Curve
Authors: Yanshuo Zhang, Juan Du
Organizations: Smart Manufacturing Thrust, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, 100190, China · Data Science and Analytics Thrust, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, 100190, China · Department of Mechanical and Aerospace Engineering, The Hong Kong University of Science and Technology, Hong Kong SAR, 999077, China
Recent literature widely used global metrics such as AUROC and AUPRO to quantify the detection performance. AUROC and AUPRO scores are approaching 100% in some public datasets such as MVTec AD dataset and VisA dataset, which misleadingly suggests that the pixel-level anomaly detection task has already been solved. Threshold-independent metrics represented by AUROC inherently ignore the threshold selection process, which is inevitable in practical applications. Although AUPRO restricts the False Positive rates (FPRs) to an acceptable range, extremely low FPRs can lead to increased false negative rates (FNRs). We argue that there is a lack of metrics capable of quantifying the separation between the anomaly score distributions of normal and anomalous samples, which hinders advancements in anomaly detection. In response, we propose the Bounded Area Under the Kolmogorov--Smirnov curve (Bounded AUKS), a partial integral metric that integrates the difference between TPR and FPR within a data-driven bounded range. Specifically, we introduce AUKS from KS integral as a direct measure of separability between the anomaly score distributions. The bounded range is determined only by score distributions of normal test samples. Lower bound excludes the uninformative high-FPR region. Upper bound avoids evaluation in extreme regions that aggressively push FPR towards zero, which result in a very low recall. Our experiments on 9 models across MVTec AD and VisA datasets show that Bounded AUKS can address the limitations of existing metrics and has a better performance in evaluating unsupervised anomaly detection models.
Figures & tables
Model
First Publication
MVTec AD AUROC
VisA AUROC
Dinomaly ( Guo et al. 2025 )
2025-06
98.6
99.1
HETMM ( Chen et al. 2025b )
2025-05
99.0
99.1
URD ( Liu et al. 2025 )
2025-02
99.0
99.1
GLASS ( Chen et al. 2025a )
2024-11
99.3
98.8
GLAD ( Yao et al. 2025 )
2024-10
98.6
98.6
Table 1: Recent high-performing anomaly detection models with near-perfect AUROC scores on MVTec AD and VisA benchmarks. Pixel-level scores are shown.
Figure 1: Comparison of ideal score distributions under different metrics. (a) A narrow margin under AUROC = 1.0. (b) A wide margin under AUROC = 1.0. Both examples in (a) and (b) achieve the same AUROC, but they differ fundamentally in robustness of threshold selection. (c) The strict requirement for AUKS = 1.0, in which normal-scores are 0 and anomalous-scores are 1, underscores that a distinct separation between distributions makes the threshold easier to determine. (d) Bounded AUKS = 1.0. A perfect value requires the normal distribution to collapse to a constant c and all anomalies to exceed c .
Figure 2: Impact of bounding parameters on segmentation bounds and metric curves for the MVTec AD Pill. (a) Corresponding anomaly map. (b) Lower bounds derived using different α . (c) Upper bounds configured using varying EVT risk levels q . (d) Alignment of these bounds along the corresponding curves.
Tail shape
α
γ
0.2
1.0
2.0
3.0
4.0
5.0
4 (Light)
37.8%
25.0%
10.9%
5.3%
2.8%
1.6%
2 (Typical)
37.0%
25.0%
13.5%
9.2%
5.1%
3.5%
1 (Extreme)
36.6%
25.0%
15.6%
10.8%
8.0%
6.2%
Table 2: Mapping α to FPR under Fréchet Distribution.
Figure 3: Case comparison on VisA (Cashew). (a) test images and ground truth masks. (b)–(d) Anomaly heatmaps and predicted mask segmentations at the F1-max optimal point overlaid on the ground-truth mask for (b) Reverse Distillation, (c) EfficientAD-S, and (d) FastFlow.
Figure 4: Comparison between Dinomaly and UniNet on the grid category of MVTec AD dataset. (a) Original RGB test images and ground truth masks. (b) (c) Anomaly maps and predicted mask segmentations overlaid on the ground truth mask for (b) Dinomaly and (c) UniNet. (d) Corresponding ROC, PRO, PDF, CDF, and KS curves for Dinomaly (left) and UniNet (right). The segmentation threshold for each model is set at the F1-max optimal point.
Figure 5: Pixel-level metric scatter summaries across categories on MVTec AD and VisA. (a) AUROC. (b) AUPRO@5% vs AUPRO@30%. (c) AUKS@100%. (d) Bounded AUKS( q=0.01 , α=2 ). Points denote per-category scores and mean markers lie on each model baseline; marker shape distinguishes datasets.
Anomaly detection systems are often trained using normal data alone, while model selection and evaluation typically require labeled anomalies. We study whether anomaly detection performance can be predicted without access to anomalous data. For kNN-based detectors, we derive a lower bound on the area under the ROC curve (AUC) that relates detection performance to the separation between inlier and outlier scores and to their respective variances. Under a local scaling model, we use this bound to characterize how density variation, intrinsic-dimensional heterogeneity, and cross-domain mismatch contribute to score variability. We then investigate anomaly-free model selection and show that inlier score variance alone does not reliably predict performance across different representations. To address this limitation, we introduce simple pseudo-anomaly probes that provide a reference for estimating relative score separation. Experiments on the DCASE 2022-2025 benchmarks, spanning four embedding models and 208 candidate systems, show that pseudo-anomaly-based estimators substantially improve anomaly-free model selection. In particular, diverse pseudo-anomalies enable anomaly-free model selection to outperform conventional development-set selection under domain shift. These results show that embedding-space geometry contains predictive information about anomaly detection performance while also highlighting the representation-dependent nature of inlier-only performance estimates.
Kevin Wilkinghoff, Zheng-Hua Tan
Department of Electronic Systems, Aalborg University, Denmark · Pioneer Centre for Artificial Intelligence, Denmark
Anomalies are rare, and anomalous data are often unavailable during development, making it difficult to determine which anomaly detection models and configurations will generalize to unseen anomalies. Recent approaches address this challenge by generating pseudo-anomalies and using bounds on the achievable area under the ROC curve (AUC) to select the optimal configuration from a finite set of candidates. Instead, we use the AUC bound as a differentiable, anomaly-free objective for directly optimizing continuous parameters of anomaly detection systems. We demonstrate this framework by optimizing ensemble weights and introducing a learnable score-rescaling mechanism that adapts pseudo-anomaly scores, enabling optimization beyond a predefined candidate set. Experiments across multiple datasets and embedding models show that AUC-bound optimization achieves significant performance gains over conventional model selection and prior development-set-based parameter selection. The results further show that direct optimization is less sensitive to the choice of pseudo-anomaly construction.
Kevin Wilkinghoff, Zheng-Hua Tan
Department of Electronic Systems, Aalborg University, Denmark · Pioneer Centre for AI, Denmark
Anomaly detection is inherently characterised by severe class imbalance, making the interpretation of evaluation metrics challenging. Although metrics such as AUROC, AUPR, F1-score, and MCC are widely used, their values convey different meanings depending on the anomaly ratio. In this work, we analyse the behaviour of those four common anomaly detection metrics under varying levels of imbalance. We focus on the study of metric landscapes, visualisations that relate metric values to true positive and true negative rates, providing an intuitive view of metric preferences and stability. Our analysis offers practical guidance for interpreting and comparing anomaly detection results across datasets with different imbalance ratios.
Romain Hermary, Nesryne Mejri, Djamila Aouada
CVI2, University of Luxembourg Luxembourg, Luxembourg