Many practical anomaly detection problems come with only a handful of confirmed anomalies, and the unreviewed remainder of the data cannot be trusted to be normal. We study this setting under a curated-label protocol, in which unconfirmed anomalies are removed from training rather than relabelled as normal, and propose TripletAD, a weakly supervised detector that learns a triplet-margin embedding and calibrates a linear scoring head with two soft penalties: a partial cross-entropy on labelled anomalies and a relative-contrast hinge that separates the mean scores of anomalous and normal samples. Because every mined triplet contains a labelled anomaly, the few labels are reused across the entire training set instead of being diluted by it. On 24 ADBench datasets whose inputs are pre-trained embeddings (text, image and speech vectors), TripletAD outperforms established state-of-the-art weakly supervised detectors (DevNet, PReNet and DeepSAD) and gradient-boosted trees when anomalies are rare. At a 1% anomaly rate it has the highest mean AUCPR of six methods, 20% above DevNet; on the naturally rare Speech dataset it doubles the best baseline (0.394 vs. 0.196 AUCPR); and on MVTec-AD it is best when only a quarter or half of the anomalies are labelled. Pooled over 70 rare- or sparse-label settings, it has the best average rank (2.21 of 6) and outperforms each baseline under Wilcoxon signed-rank tests with Holm correction. When anomalies are more frequent it performs on par with these methods. The method needs no per-dataset tuning and is insensitive to its loss weights and training length; on raw tabular features, gradient-boosted trees remain the stronger choice.
Figures & tables
Dataset
N
D
Anomalies
Rate (%)
Best 1-feature AUROC
Embedding (text: BERT; image: ResNet-18)
20News
3,090
768
154
4.98
0.708
AGNews
10,000
768
500
5.00
0.730
Amazon
10,000
768
500
5.00
0.713
IMDB
10,000
768
500
5.00
0.698
Yelp
10,000
768
500
5.00
0.753
Table 1: Datasets. “Best 1-feature AUROC” is the highest AUROC obtained by ranking on a single input coordinate (either sign).
Dataset
TripletAD
DeepSAD
DevNet
PReNet
CatBoost
RF
Anomaly rate 1%
20News
0.341
0.212
0.341
0.329
0.152
0.106
AGNews
0.325
0.262
0.175
0.171
0.159
0.082
Amazon
0.111
0.095
0.099
0.107
0.057
0.023
IMDB
0.126
0.118
0.095
0.081
0.069
0.022
Yelp
0.181
0.196
0.151
0.150
0.117
0.047
Table 2: Anomaly-rarity sweep: AUCPR (mean of five seeds, with the anomaly subsample redrawn for each seed) when anomalies make up 1%, 2% or 5% (original) of each dataset. Best per row in bold.
Figure 1: Anomaly-rarity sweep on eight text and image embedding datasets. Left: mean AUCPR over datasets and five seeds as the anomaly rate falls from 5% to 1%. Right: the same, divided by DevNet’s score. TripletAD’s lead grows to 20% over DevNet at 1%, while the tree ensembles fall further behind as anomalies become rarer.
Labelled anomalies
TripletAD
DeepSAD
DevNet
PReNet
CatBoost
RF
100%
0.394
0.073
0.178
0.165
0.196
0.070
75%
0.317
0.060
0.180
0.181
0.152
0.068
50%
0.252
0.050
0.157
0.163
0.113
0.071
25%
0.093
0.032
0.106
0.109
0.053
0.036
Table 3: Speech (1.65% anomalies): AUCPR as the fraction of labelled training anomalies varies under the curated protocol.
Labels (categories)
TripletAD
DeepSAD
DevNet
PReNet
CatBoost
RF
TripletAD best
25% (13)
0.704
0.625
0.671
0.683
0.698
0.691
6/13
50% (15)
0.734
0.644 †
0.714
0.710
0.732
0.719
6/15
75% (15)
0.742
0.665 †
0.735
0.726
0.748
0.733
7/15
100% (15)
0.745
0.676 †
0.749
0.737
0.758
0.742
4/15
Table 4: MVTec-AD (ResNet-18 embeddings, 13–33% anomalies): mean AUCPR over categories under curated sparse labels, and the number of categories on which TripletAD is the best of the six methods. † DeepSAD mean over the categories where it ran.
TripletAD vs.
Cells
W / L
Mean AUCPR gain
Holm p
DeepSAD
68
62 / 6
+0.071
2×10−10
Random forest
69
57 / 12
+0.100
4×10−9
CatBoost
69
54 / 15
+0.053
6×10−7
DevNet
69
44 / 24
+0.021
0.020
PReNet
69
44 / 24
+0.023
0.015
Table 5: Pooled comparison over 70 rare- or sparse-label embedding cells (AUCPR averaged over five seeds per cell). W/L: cells where TripletAD is better/worse. p : two-sided Wilcoxon signed-rank, Holm-corrected over the five baselines. A few cells are missing for DeepSAD; one DevNet and one PReNet cell are tied.
Variant
Speech
Fraud
Cover
Full objective
0.394 ± 0.083
0.773 ± 0.040
0.995 ± 0.001
w/o PCE ( λ1=0 )
0.391 ± 0.086
0.761 ± 0.048
0.994 ± 0.004
w/o RC ( λ2=0 )
0.380 ± 0.073
0.514 ± 0.074
0.847 ± 0.237
Triplet only
0.129 ± 0.115
0.157 ± 0.342
0.072 ± 0.150
w/o triplet pretraining
0.380 ± 0.075
0.762 ± 0.044
0.990 ± 0.005
Table 6: Ablation: AUCPR (mean ± s.d. over five seeds), full labels.
Figure 2: Sensitivity. Left: mean test AUCPR over nine embedding datasets for each (λ1,λ2) ; the colour scale spans ±0.05 around the mean so the small spread is shown at true scale; the red box is the default. Right: test AUCPR against the number of joint-training epochs (dashed line: default of 100). The percentage after each dataset name is its label ratio.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
TripletAD
DeepSAD
DevNet
PReNet
CatBoost
RF
20News
0.579
0.623
0.623
0.618
0.586
0.449
AGNews
0.608
0.620
0.462
0.453
0.541
0.339
Amazon
0.324
0.346
0.335
0.320
0.333
0.161
IMDB
0.340
0.436
0.304
0.292
0.335
0.183
Yelp
0.515
0.564
0.488
0.462
0.499
0.276
CIFAR-10
0.468
0.397
0.469
0.487
0.423
0.359
Appendix
Table 7: Text and image embedding datasets at the original 5% anomaly rate with full labels: AUCPR (mean of five seeds).
Dataset
TripletAD
DeepSAD
DevNet
PReNet
CatBoost
RF
Anomaly rate 1%
20News
0.287
0.218
0.230
0.249
0.152
0.103
AGNews
0.201
0.195
0.128
0.126
0.103
0.070
Amazon
0.124
0.114
0.081
0.074
0.069
0.021
IMDB
0.175
0.191
0.104
0.080
0.067
0.031
Yelp
0.290
0.254
0.190
0.168
0.122
0.043
Appendix
Table 8: Anomaly-rarity sweep with a single fixed anomaly subsample (drawn once with seed 0 and reused for all five split seeds). The conclusions match Table 2 : TripletAD has the best mean at 2% (0.350) and 1% (0.230), and its lead over DevNet grows from 4% to 27%.
Labelled anomalies
TripletAD
DeepSAD
DevNet
PReNet
CatBoost
RF
25% (8 sets)
0.445
0.402
0.448
0.439
0.369
0.258
10% (8 sets)
0.357
0.277
0.358
0.353
0.249
0.194
5% (5 sets)
0.319
0.240
0.334
0.326
0.232
0.183
TripletAD wins
–
20/21
9/21
10/21
20/21
21/21
Appendix
Table 9: Eight text and image embedding datasets at their original 5% anomaly rate, with curated sparse labels: mean AUCPR. Cells with fewer than ten labelled anomalies are skipped, so the 5% row covers five datasets.
Dataset
TripletAD
DeepSAD
DevNet
PReNet
CatBoost
RF
Backdoor
0.974
–
0.882
–
0.998
0.996
Cover
0.995
–
0.956
–
1.000
0.999
Fraud
0.773
–
0.743
–
0.866
0.849
HTTP
0.998
–
0.997
–
0.999
0.999
Waveform
0.505
0.538
0.209
0.234
0.605
0.550
Yeast
0.588
0.536
0.487
0.490
0.662
0.655
Appendix
Table 10: Raw tabular datasets with full labels: AUCPR (mean of five seeds). PReNet and DeepSAD were not run on Backdoor, Cover, Fraud and HTTP.
Dataset
Labels
TripletAD
DeepSAD
DevNet
PReNet
CatBoost
RF
20News
25%
0.468
0.416
0.544
0.511
0.300
0.264
AGNews
25%
0.523
0.527
0.463
0.445
0.447
0.223
Amazon
25%
0.310
0.244
0.314
0.303
0.245
0.109
IMDB
25%
0.309
0.305
0.280
0.259
0.245
0.113
Yelp
25%
0.486
0.454
0.468
0.448
0.376
0.185
CIFAR-10
25%
0.318
0.254
0.343
0.362
0.319
0.247
Appendix
Table 11: Text and image embedding datasets with curated sparse labels: AUCPR (mean of five seeds).
Category
Labels
TripletAD
DeepSAD
DevNet
PReNet
CatBoost
RF
bottle
100%
0.947
0.948
0.951
0.930
0.958
0.947
bottle
50%
0.963
0.953
0.927
0.942
0.954
0.949
bottle
25%
0.947
0.948
0.920
0.920
0.948
0.948
cable
100%
0.534
0.545
0.540
0.555
0.543
0.571
cable
50%
0.582
0.492
0.497
0.488
0.526
0.539
cable
25%
0.568
0.495
0.452
0.455
0.548
0.562
Appendix
Table 12: MVTec-AD per category: AUCPR (mean of five seeds). “–”: DeepSAD’s data loader failed.
Weakly supervised anomaly detection (WSAD) has developed in three primary directions: incomplete, inexact, and inaccurate supervision. However, these directions remain isolated, lacking a unified framework to assess whether they address unique challenges or share fundamental mechanisms. This paper introduces WSADBench, the first benchmark that unifies evaluation across distinct weakly supervised scenarios, benchmarking diverse approaches from specialized WSAD methods to advanced tabular foundation models. WSADBench establishes standardized protocols to evaluate 36 algorithms across 4 modalities by systematically varying label quantity, granularity, and quality, revealing the performance boundaries of various methods. Based on over 700K experiments, WSADBench reveals four critical insights: (i) Strong intrinsic correlations exist between these weak supervision scenarios, challenging the isolation of current research directions. (ii) Specialized WSAD algorithms excel only in extreme label-scarcity regimes but are quickly dominated by tabular foundation models and general classification methods as supervision increases or in OOD scenarios. (iii) Unlabeled data shows inconsistent utility across settings, with marginal gains compared to label refinement. (iv) Models exhibit asymmetric sensitivity to different types of label noise. We release WSADBench as an open-source benchmark with code and datasets to facilitate future WSAD research: https://github.com/SUFE-AILAB/WSADBench.
Xu Yao, Siyuan Zhou, Zhenbo Wu +6
Shanghai University of Finance and Economics · Ant Group · Key Laboratory of Interdisciplinary Research of Computation and Economics Shanghai University of Finance and Economics
Weakly supervised anomaly detection on tabular data is an essential task in many label-scarce applications, since tabular data is widely encountered and it is commonly observed in practice that only a limited number of labeled anomalies are available alongside abundant unlabeled samples. Most existing methods try to learn features that capture the inherent distribution patterns in the target domain. However, such features often struggle to provide a complete description of possible abnormal characteristics and are highly susceptible to overfitting the extremely scarce labels. To address this challenge, we propose a weakly supervised anomaly detection method based on universal semantic enhancement for tabular data, which fuses the universal knowledge from a pre-trained tabular foundation model with the domain-specific knowledge learned on the target dataset. The two types of knowledge are naturally complementary: universal semantics describe the common patterns of tabular data across domains, while the domain-specific features capture the inherent distributions of normal and anomalous behaviors in the target dataset. A prototype-conditioned modulation mechanism is then designed to effectively integrate the two types of features, allowing them to reinforce each other for improved anomaly detection. Extensive experiments on public datasets show that the proposed method outperforms competitive methods when only eight labeled anomalies are available in the training procedure.
Siyu Zhou, Yuqin Xie, Hao Yuan +1
Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong · College of Computer Science, Sichuan University, China
Semi-supervised anomaly detection, which aims to improve the anomaly detection performance by using a small amount of labeled anomaly data in addition to unlabeled data, has attracted attention. Existing semi-supervised approaches assume that most unlabeled data are normal, and train anomaly detectors by minimizing the anomaly scores for the unlabeled data while maximizing those for the labeled anomaly data. However, in practice, the unlabeled data are often contaminated with anomalies. This weakens the effect of maximizing the anomaly scores for anomalies, and prevents us from improving the detection performance. To solve this, we propose the deep positive-unlabeled anomaly detection framework, which integrates positive-unlabeled learning with deep anomaly detection models such as autoencoders and deep support vector data descriptions. Our approach enables the approximation of anomaly scores for normal data using the unlabeled data and the labeled anomaly data. Therefore, without labeled normal data, our approach can train anomaly detectors by minimizing the anomaly scores for normal data while maximizing those for the labeled anomaly data. We also provide a theoretical analysis establishing a generalization error bound for the proposed objective, guaranteeing that the empirical minimizer converges asymptotically to the ideal minimizer. Our approach achieves better detection performance than existing approaches on various datasets.