TripletAD: Weakly Supervised Anomaly Detection in Embedding Space with Rare Anomalies and Scarce Labels
Organizations: Dell Technologies
Abstract
Many practical anomaly detection problems come with only a handful of confirmed anomalies, and the unreviewed remainder of the data cannot be trusted to be normal. We study this setting under a curated-label protocol, in which unconfirmed anomalies are removed from training rather than relabelled as normal, and propose TripletAD, a weakly supervised detector that learns a triplet-margin embedding and calibrates a linear scoring head with two soft penalties: a partial cross-entropy on labelled anomalies and a relative-contrast hinge that separates the mean scores of anomalous and normal samples. Because every mined triplet contains a labelled anomaly, the few labels are reused across the entire training set instead of being diluted by it. On 24 ADBench datasets whose inputs are pre-trained embeddings (text, image and speech vectors), TripletAD outperforms established state-of-the-art weakly supervised detectors (DevNet, PReNet and DeepSAD) and gradient-boosted trees when anomalies are rare. At a 1% anomaly rate it has the highest mean AUCPR of six methods, 20% above DevNet; on the naturally rare Speech dataset it doubles the best baseline (0.394 vs. 0.196 AUCPR); and on MVTec-AD it is best when only a quarter or half of the anomalies are labelled. Pooled over 70 rare- or sparse-label settings, it has the best average rank (2.21 of 6) and outperforms each baseline under Wilcoxon signed-rank tests with Holm correction. When anomalies are more frequent it performs on par with these methods. The method needs no per-dataset tuning and is insensitive to its loss weights and training length; on raw tabular features, gradient-boosted trees remain the stronger choice.
Figures & tables
| Dataset | Anomalies | Rate (%) | Best 1-feature AUROC | ||
| Embedding (text: BERT; image: ResNet-18) | |||||
| 20News | 3,090 | 768 | 154 | 4.98 | 0.708 |
| AGNews | 10,000 | 768 | 500 | 5.00 | 0.730 |
| Amazon | 10,000 | 768 | 500 | 5.00 | 0.713 |
| IMDB | 10,000 | 768 | 500 | 5.00 | 0.698 |
| Yelp | 10,000 | 768 | 500 | 5.00 | 0.753 |
| Dataset | TripletAD | DeepSAD | DevNet | PReNet | CatBoost | RF |
|---|---|---|---|---|---|---|
| Anomaly rate 1% | ||||||
| 20News | 0.341 | 0.212 | 0.341 | 0.329 | 0.152 | 0.106 |
| AGNews | 0.325 | 0.262 | 0.175 | 0.171 | 0.159 | 0.082 |
| Amazon | 0.111 | 0.095 | 0.099 | 0.107 | 0.057 | 0.023 |
| IMDB | 0.126 | 0.118 | 0.095 | 0.081 | 0.069 | 0.022 |
| Yelp | 0.181 | 0.196 | 0.151 | 0.150 | 0.117 | 0.047 |
| Labelled anomalies | TripletAD | DeepSAD | DevNet | PReNet | CatBoost | RF |
|---|---|---|---|---|---|---|
| 100% | 0.394 | 0.073 | 0.178 | 0.165 | 0.196 | 0.070 |
| 75% | 0.317 | 0.060 | 0.180 | 0.181 | 0.152 | 0.068 |
| 50% | 0.252 | 0.050 | 0.157 | 0.163 | 0.113 | 0.071 |
| 25% | 0.093 | 0.032 | 0.106 | 0.109 | 0.053 | 0.036 |
| Labels (categories) | TripletAD | DeepSAD | DevNet | PReNet | CatBoost | RF | TripletAD best |
|---|---|---|---|---|---|---|---|
| 25% (13) | 0.704 | 0.625 | 0.671 | 0.683 | 0.698 | 0.691 | 6/13 |
| 50% (15) | 0.734 | 0.644 † | 0.714 | 0.710 | 0.732 | 0.719 | 6/15 |
| 75% (15) | 0.742 | 0.665 † | 0.735 | 0.726 | 0.748 | 0.733 | 7/15 |
| 100% (15) | 0.745 | 0.676 † | 0.749 | 0.737 | 0.758 | 0.742 | 4/15 |
| TripletAD vs. | Cells | W / L | Mean AUCPR gain | Holm |
|---|---|---|---|---|
| DeepSAD | 68 | 62 / 6 | ||
| Random forest | 69 | 57 / 12 | ||
| CatBoost | 69 | 54 / 15 | ||
| DevNet | 69 | 44 / 24 | ||
| PReNet | 69 | 44 / 24 |
| Variant | Speech | Fraud | Cover |
|---|---|---|---|
| Full objective | 0.394 0.083 | 0.773 0.040 | 0.995 0.001 |
| w/o PCE ( ) | 0.391 0.086 | 0.761 0.048 | 0.994 0.004 |
| w/o RC ( ) | 0.380 0.073 | 0.514 0.074 | 0.847 0.237 |
| Triplet only | 0.129 0.115 | 0.157 0.342 | 0.072 0.150 |
| w/o triplet pretraining | 0.380 0.075 | 0.762 0.044 | 0.990 0.005 |
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | TripletAD | DeepSAD | DevNet | PReNet | CatBoost | RF |
|---|---|---|---|---|---|---|
| 20News | 0.579 | 0.623 | 0.623 | 0.618 | 0.586 | 0.449 |
| AGNews | 0.608 | 0.620 | 0.462 | 0.453 | 0.541 | 0.339 |
| Amazon | 0.324 | 0.346 | 0.335 | 0.320 | 0.333 | 0.161 |
| IMDB | 0.340 | 0.436 | 0.304 | 0.292 | 0.335 | 0.183 |
| Yelp | 0.515 | 0.564 | 0.488 | 0.462 | 0.499 | 0.276 |
| CIFAR-10 | 0.468 | 0.397 | 0.469 | 0.487 | 0.423 | 0.359 |
| Dataset | TripletAD | DeepSAD | DevNet | PReNet | CatBoost | RF |
|---|---|---|---|---|---|---|
| Anomaly rate 1% | ||||||
| 20News | 0.287 | 0.218 | 0.230 | 0.249 | 0.152 | 0.103 |
| AGNews | 0.201 | 0.195 | 0.128 | 0.126 | 0.103 | 0.070 |
| Amazon | 0.124 | 0.114 | 0.081 | 0.074 | 0.069 | 0.021 |
| IMDB | 0.175 | 0.191 | 0.104 | 0.080 | 0.067 | 0.031 |
| Yelp | 0.290 | 0.254 | 0.190 | 0.168 | 0.122 | 0.043 |
| Labelled anomalies | TripletAD | DeepSAD | DevNet | PReNet | CatBoost | RF |
|---|---|---|---|---|---|---|
| 25% (8 sets) | 0.445 | 0.402 | 0.448 | 0.439 | 0.369 | 0.258 |
| 10% (8 sets) | 0.357 | 0.277 | 0.358 | 0.353 | 0.249 | 0.194 |
| 5% (5 sets) | 0.319 | 0.240 | 0.334 | 0.326 | 0.232 | 0.183 |
| TripletAD wins | – | 20/21 | 9/21 | 10/21 | 20/21 | 21/21 |
| Dataset | TripletAD | DeepSAD | DevNet | PReNet | CatBoost | RF |
|---|---|---|---|---|---|---|
| Backdoor | 0.974 | – | 0.882 | – | 0.998 | 0.996 |
| Cover | 0.995 | – | 0.956 | – | 1.000 | 0.999 |
| Fraud | 0.773 | – | 0.743 | – | 0.866 | 0.849 |
| HTTP | 0.998 | – | 0.997 | – | 0.999 | 0.999 |
| Waveform | 0.505 | 0.538 | 0.209 | 0.234 | 0.605 | 0.550 |
| Yeast | 0.588 | 0.536 | 0.487 | 0.490 | 0.662 | 0.655 |
| Dataset | Labels | TripletAD | DeepSAD | DevNet | PReNet | CatBoost | RF |
|---|---|---|---|---|---|---|---|
| 20News | 25% | 0.468 | 0.416 | 0.544 | 0.511 | 0.300 | 0.264 |
| AGNews | 25% | 0.523 | 0.527 | 0.463 | 0.445 | 0.447 | 0.223 |
| Amazon | 25% | 0.310 | 0.244 | 0.314 | 0.303 | 0.245 | 0.109 |
| IMDB | 25% | 0.309 | 0.305 | 0.280 | 0.259 | 0.245 | 0.113 |
| Yelp | 25% | 0.486 | 0.454 | 0.468 | 0.448 | 0.376 | 0.185 |
| CIFAR-10 | 25% | 0.318 | 0.254 | 0.343 | 0.362 | 0.319 | 0.247 |
| Category | Labels | TripletAD | DeepSAD | DevNet | PReNet | CatBoost | RF |
|---|---|---|---|---|---|---|---|
| bottle | 100% | 0.947 | 0.948 | 0.951 | 0.930 | 0.958 | 0.947 |
| bottle | 50% | 0.963 | 0.953 | 0.927 | 0.942 | 0.954 | 0.949 |
| bottle | 25% | 0.947 | 0.948 | 0.920 | 0.920 | 0.948 | 0.948 |
| cable | 100% | 0.534 | 0.545 | 0.540 | 0.555 | 0.543 | 0.571 |
| cable | 50% | 0.582 | 0.492 | 0.497 | 0.488 | 0.526 | 0.539 |
| cable | 25% | 0.568 | 0.495 | 0.452 | 0.455 | 0.548 | 0.562 |