Anomaly detection (AD) is a critical for trustworthy machine learning and scientific discovery. Existing methods typically define anomalies through local density, isolation heuristics, or distance from a fitted distribution, making them sensitive to neighborhood scale, partitioning choices, or restrictive distributional assumptions. In this work, we introduce a new paradigm by formulating anomaly detection from a diversity perspective. We propose the Vendi Anomaly Score (VAS), which detects anomalies by quantifying how much a sample changes the diversity of the dataset, as measured by the Vendi score, when removed. VAS captures both local redundancy and global data structure without relying on density estimation or partitioning heuristics. Moreover, VAS requires no hyperparameter tuning, instead adapting automatically to the spectral structure of the data. VAS is non-parametric and scales linearly with dataset size. Across 148 benchmark embedding datasets derived from 10 source datasets and 4 embedding architectures, VAS achieves state-of-the-art AD performance. We further validate VAS on large-scale ImageNet experiments, where it remains robust as both contamination rate and dataset scale increase.
Figures & tables
Figure 1: Overview of VAS. Left: A multimodal dataset contaminated with an anomaly. Right: Removing a rare anomalous point induces a larger change in the eigenvalue spectrum than removing a normal sample. VAS detects these shifts by estimating the change in Vendi Score, derived from the Rényi entropy of order q of the spectrum (Eq. 1 ).
Dataset
Encoder
VAS
ADERH
DIF
IForest
LUNAR
DN2
LODA
LOF
KPCA
SLAD
GMD
DTE
CIFAR10
ResNet18
0.745
0.633
0.684
0.627
0.657
0.740
0.635
0.696
0.751
0.735
0.732
0.673
CIFAR10
ViT
0.941
0.924
0.860
0.876
0.837
0.855
0.856
0.519
0.905
0.942
0.898
0.590
FashionMNIST
ResNet18
0.911
0.865
0.868
0.831
0.852
0.858
0.829
0.605
0.915
0.904
0.912
0.848
FashionMNIST
ViT
0.930
0.910
0.856
0.890
0.815
0.860
0.862
0.519
0.917
0.919
0.925
0.657
MNIST-C
ResNet18
0.793
0.763
0.744
0.734
0.780
0.763
0.724
0.628
0.765
0.782
0.793
0.759
MNIST-C
ViT
0.739
0.723
0.704
0.720
0.695
0.704
0.689
0.555
0.719
0.732
0.723
0.568
Table 1: AUROC across dataset families and encoders. For multi-split datasets, we report average AUROC. Orange marks best, blue marks 2nd. Bottom-row p-values are paired Wilcoxon tests: VAS vs. each baseline over all 148 dataset-split/encoder instances.
Figure 2: VAS provides accurate and efficient diversity influence measurements. Left: VAS diversity influence compared against the exact leave-one-out Vendi Score change for multiple orders of q . Spearman rank correlation ρ is shown for each choice of q . Right: Varying dataset sizes shows that VAS provides orders of magnitude speedup over the exact approach.
Figure 3: VAS achieves accurate ImageNet AD. We construct contaminated datasets by sampling normal and anomalous images from disjoint ImageNet classes. Left: AUROC as the anomaly rate varies with 25,000 normal images from 25 classes. Right: AUROC as dataset size increases at a fixed 20% anomaly rate, adding normal samples in increments of 10,000 . VAS performs best across both sweeps, showing robustness to increasing contamination and scale. Results are averaged over 5 replications, with standard deviation bars included.
Figure 4: VAS supports efficient AD. Left: Latency of methods when increasing dataset size at a fixed dimensionality of 128 . Right: Latency of methods with increasing dimensionality at a fixed dataset size of 5000 . Results are averaged over n=3 repetitions.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Distribution of principal eigenvalue (left) and the selected q orders (right) on all tested ADBENCH benchmarks. The geometry of CV datasets is significantly different than those from NLP datasets, so we select different q orders to account for this.
Dataset
Encoder
Approx. AUROC
Exact AUROC
Δ AUROC (%)
Spearman ρ
CIFAR-10
ResNet18
0.745
0.745
-0.01
1.000
CIFAR-10
ViT
0.941
0.941
-0.02
1.000
FashionMNIST
ResNet18
0.911
0.911
-0.02
1.000
FashionMNIST
ViT
0.930
0.929
0.00
1.000
MNIST-C
ResNet18
0.793
0.793
0.00
1.000
MNIST-C
ViT
0.739
0.739
0.01
1.000
Appendix
Table 2: Comparison of exact and approximate VAS across 148 ADBENCH benchmark instances. Results are averaged across dataset splits for each dataset–encoder combination. Δ AUROC (%) is the percentage change from approximate to exact VAS. Spearman’s ρ measures the rank correlation between exact and approximate anomaly scores. The approximation preserves both anomaly rankings and downstream detection performance across datasets.
Dataset
Encoder
VAS
ADERH
DIF
IForest
LUNAR
DN2
LODA
LOF
KPCA
SLAD
GMD
DTE
CIFAR10
ResNet18
0.153
0.093
0.112
0.088
0.104
0.163
0.091
0.122
0.166
0.145
0.141
0.103
CIFAR10
ViT
0.659
0.541
0.326
0.360
0.389
0.386
0.365
0.072
0.486
0.670
0.491
0.109
FashionMNIST
ResNet18
0.449
0.370
0.395
0.325
0.319
0.337
0.321
0.128
0.495
0.450
0.436
0.335
FashionMNIST
ViT
0.546
0.508
0.325
0.423
0.333
0.346
0.381
0.085
0.451
0.508
0.511
0.115
MNIST-C
ResNet18
0.237
0.232
0.223
0.173
0.192
0.198
0.208
0.123
0.214
0.256
0.235
0.227
MNIST-C
ViT
0.226
0.269
0.220
0.242
0.192
0.216
0.200
0.090
0.200
0.231
0.201
0.091
Appendix
Table 3: AP across dataset families and encoders. For multi-split datasets, we report average AP. Orange marks best, blue marks 2nd. Bottom-row p-values are paired Wilcoxon tests: VAS vs. each baseline over all 148 dataset-split/encoder instances.
Dataset
Encoder
VAS
ADERH
DIF
IForest
LUNAR
DN2
LODA
LOF
KPCA
SLAD
GMD
DTE
CIFAR10
ResNet18
0.745
0.632
0.631
0.633
0.663
0.739
0.651
0.689
0.734
0.736
0.732
0.655
CIFAR10
ViT
0.941
0.924
0.788
0.892
0.876
0.831
0.900
0.444
0.610
0.946
0.898
0.578
FashionMNIST
ResNet18
0.911
0.869
0.832
0.839
0.867
0.838
0.853
0.576
0.895
0.904
0.912
0.823
FashionMNIST
ViT
0.930
0.911
0.776
0.896
0.820
0.837
0.893
0.493
0.900
0.923
0.925
0.656
MNIST-C
ResNet18
0.793
0.772
0.708
0.742
0.791
0.752
0.750
0.599
0.791
0.780
0.793
0.727
MNIST-C
ViT
0.739
0.721
0.682
0.723
0.705
0.697
0.718
0.543
0.732
0.733
0.723
0.566
Appendix
Table 4: AUROC across dataset families and encoders using each method’s best hyperparameter setting. For multi-split datasets, we report average AUROC. Orange marks best, blue marks 2nd. Bottom-row p-values are paired Wilcoxon tests: VAS vs. each baseline over all 148 dataset-split/encoder instances.
Dataset
Encoder
VAS
ADERH
DIF
IForest
LUNAR
DN2
LODA
LOF
KPCA
SLAD
GMD
DTE
CIFAR10
ResNet18
0.153
0.094
0.088
0.091
0.105
0.160
0.094
0.116
0.117
0.143
0.141
0.093
CIFAR10
ViT
0.659
0.544
0.221
0.405
0.441
0.341
0.427
0.054
0.075
0.689
0.491
0.108
FashionMNIST
ResNet18
0.449
0.372
0.315
0.329
0.327
0.310
0.359
0.102
0.383
0.459
0.436
0.264
FashionMNIST
ViT
0.546
0.510
0.205
0.433
0.329
0.317
0.435
0.073
0.393
0.537
0.511
0.129
MNIST-C
ResNet18
0.237
0.237
0.169
0.183
0.196
0.190
0.226
0.109
0.220
0.255
0.235
0.169
MNIST-C
ViT
0.226
0.273
0.167
0.243
0.195
0.209
0.235
0.082
0.219
0.238
0.201
0.100
Appendix
Table 5: AP across dataset families and encoders using each method’s best hyperparameter setting. For multi-split datasets, we report average AP. Orange marks best, blue marks 2nd. Bottom-row p-values are paired Wilcoxon tests: VAS vs. each baseline over all 148 dataset-split/encoder instances.
Dataset
Encoder
Adaptive
Slope-1
Slope-5
VS1/D
Top-5
Top-10
CIFAR-10
ResNet18
0.745
0.732
0.725
0.711
0.737
0.729
CIFAR-10
ViT
0.941
0.914
0.943
0.894
0.919
0.911
FashionMNIST
ResNet18
0.911
0.908
0.876
0.897
0.907
0.903
FashionMNIST
ViT
0.930
0.928
0.910
0.926
0.929
0.928
MNIST-C
ResNet18
0.793
0.794
0.755
0.791
0.794
0.793
MNIST-C
ViT
0.739
0.741
0.737
0.743
0.740
0.741
Appendix
Table 6: Sensitivity of VAS to alternative adaptive rules for selecting q across 148 ADBENCH benchmark instances. AUROC values are averaged across dataset splits for each dataset–encoder combination. Mean AUROC is computed across all benchmark instances. The bottom row reports unadjusted, one-sided paired Wilcoxon tests comparing our default adaptive rule against each alternative, with the alternative hypothesis that our default achieves higher AUROC.
Dataset
Encoder
VAS
GMD
DN2
KPCA
LOF
SLAD
CIFAR-10
ResNet18
0.557
0.561
0.571
0.574
0.544
0.551
CIFAR-10
ViT
0.733
0.723
0.719
0.721
0.659
0.727
SVHN
ResNet18
0.610
0.610
0.617
0.617
0.579
0.600
SVHN
ViT
0.552
0.543
0.535
0.543
0.539
0.543
20News
BERT
0.555
0.560
0.526
0.525
0.525
0.548
20News
RoBERTa
0.604
0.560
0.560
0.561
0.536
0.538
Appendix
Table 7: AUROC for local class-confusable anomalies constructed from genuine ADBENCH embeddings. Results are averaged across assigned-class subsets and five independent trials. Anomaly rates vary with class overlap. Average ranks are computed across dataset–encoder combinations (lower is better).
Dataset
Encoder
VAS
GMD
DN2
KPCA
LOF
SLAD
CIFAR-10
ResNet18
0.578
0.561
0.188
0.433
0.517
0.530
CIFAR-10
ViT
0.526
0.531
0.194
0.215
0.521
0.488
SVHN
ResNet18
0.669
0.558
0.289
0.554
0.530
0.560
SVHN
ViT
0.956
0.585
0.272
0.607
0.572
0.586
20News
BERT
0.996
0.719
0.727
0.746
0.919
0.924
20News
RoBERTa
1.000
0.995
1.000
1.000
0.956
1.000
Appendix
Table 8: AUROC for global directional anomalies constructed by perturbing embeddings outside the span of normal-class centroids. Results use α=0.1 and approximately 10% contamination, averaged across five trials. Average ranks are computed across dataset–encoder combinations (lower is better).
Dataset
Encoder
VAS
GMD
DN2
KPCA
LOF
SLAD
CIFAR-10
ResNet18
0.825
0.789
0.844
0.864
0.894
0.747
CIFAR-10
ViT
0.828
0.748
0.793
0.825
0.847
0.669
SVHN
ResNet18
0.858
0.778
0.851
0.841
0.879
0.791
SVHN
ViT
0.959
0.694
0.799
0.818
0.835
0.716
20News
BERT
0.922
0.685
0.814
0.744
0.895
0.775
20News
RoBERTa
0.985
0.549
0.728
0.779
0.762
0.628
Appendix
Table 9: AUROC for feature-dependency anomalies constructed by swapping blocks of orthogonally rotated embeddings. Results use approximately 10% contamination and are averaged across semantic classes and five trials. Average ranks are computed across dataset–encoder combinations (lower is better).
Anomaly detection systems are often trained using normal data alone, while model selection and evaluation typically require labeled anomalies. We study whether anomaly detection performance can be predicted without access to anomalous data. For kNN-based detectors, we derive a lower bound on the area under the ROC curve (AUC) that relates detection performance to the separation between inlier and outlier scores and to their respective variances. Under a local scaling model, we use this bound to characterize how density variation, intrinsic-dimensional heterogeneity, and cross-domain mismatch contribute to score variability. We then investigate anomaly-free model selection and show that inlier score variance alone does not reliably predict performance across different representations. To address this limitation, we introduce simple pseudo-anomaly probes that provide a reference for estimating relative score separation. Experiments on the DCASE 2022-2025 benchmarks, spanning four embedding models and 208 candidate systems, show that pseudo-anomaly-based estimators substantially improve anomaly-free model selection. In particular, diverse pseudo-anomalies enable anomaly-free model selection to outperform conventional development-set selection under domain shift. These results show that embedding-space geometry contains predictive information about anomaly detection performance while also highlighting the representation-dependent nature of inlier-only performance estimates.
Kevin Wilkinghoff, Zheng-Hua Tan
Department of Electronic Systems, Aalborg University, Denmark · Pioneer Centre for Artificial Intelligence, Denmark
Anomaly detection aims to identify observations that deviate from normal patterns. Recent work has increasingly used pretrained vision--language models (VLMs) for training-free image and video anomaly detection without task-specific retraining. Anomaly detection is commonly evaluated by how well anomaly scores rank anomalous images or video frames above normal ones. Generative VLMs, however, assign probabilities to possible answers and then decode a single answer. This decoding step may discard ordering information. We call this loss decoded-answer rank compression. To isolate this effect, we compare two ways of scoring the same VLM output: one uses only the decoded answer, while the other computes a probability-weighted score over all possible answers. Probability-weighted scoring consistently outperforms decoded-answer scoring across image and video anomaly detection benchmarks, using different VLMs and answer scales. The mean gains range from 7.66 to 19.95 points on the primary metrics. Most of this gap comes from decoded-answer ties. Breaking decoded-answer ties with answer probabilities recovers at least 95% of the average performance gap on every benchmark. We further investigate how these ties affect evaluation. We find that evaluating tied scores one input at a time makes the reported AUROC depend on input order, and that linear PR interpolation can inflate reported performance. These results reveal that both how VLM outputs are scored and how discrete anomaly scores are evaluated affect reported anomaly-detection performance.
Denoising score matching (DSM) provides a way to learn data distributions by training a neural network to recover the score function, defined as the gradient of the log density, from noise-corrupted samples. Once trained, the score magnitude at a test point reflects how consistent that point is with the learned distribution, making it a natural anomaly signal. The key practical challenge is selecting the perturbation scale: too little noise yields unstable score estimates in sparse regions, while too much erases local structure and weakens anomaly sensitivity. This is compounded by the difficulty of hyperparameter tuning when anomalies are unknown and no validation set is available. We introduce kurtosis-based noise scaling (K-DSM), a per-feature scheme that sets noise levels from the shape of each marginal distribution, improving coverage of low-density regions and precision in high-density regions without extra model complexity. Contrary to prior claims that multi-scale or noise-conditioned training is necessary, we find that a carefully trained single-scale model is already a strong anomaly detector. On standard tabular anomaly detection benchmarks, K-DSM achieves state-of-the-art performance in the semi-supervised setting. When combined with a lightweight EMA-teacher filtering rule that removes low-density training points before each gradient step, it also achieves strong performance in the fully unsupervised (contaminated) setting, suggesting that simple, data-adaptive noise scaling enables robust anomaly detection while reducing reliance on hyperparameter tuning.