Deep Positive-Unlabeled Anomaly Detection for Contaminated Unlabeled Data

Hiroshi Takahashi; Tomoharu Iwata; Atsutoshi Kumagai; Yuuki Yamanaka

stat.MLMay 29, 2024

Deep Positive-Unlabeled Anomaly Detection for Contaminated Unlabeled Data

Authors: Hiroshi Takahashi, Tomoharu Iwata, Atsutoshi Kumagai, Yuuki Yamanaka

Organizations: NTT

Abstract

Semi-supervised anomaly detection, which aims to improve the anomaly detection performance by using a small amount of labeled anomaly data in addition to unlabeled data, has attracted attention. Existing semi-supervised approaches assume that most unlabeled data are normal, and train anomaly detectors by minimizing the anomaly scores for the unlabeled data while maximizing those for the labeled anomaly data. However, in practice, the unlabeled data are often contaminated with anomalies. This weakens the effect of maximizing the anomaly scores for anomalies, and prevents us from improving the detection performance. To solve this, we propose the deep positive-unlabeled anomaly detection framework, which integrates positive-unlabeled learning with deep anomaly detection models such as autoencoders and deep support vector data descriptions. Our approach enables the approximation of anomaly scores for normal data using the unlabeled data and the labeled anomaly data. Therefore, without labeled normal data, our approach can train anomaly detectors by minimizing the anomaly scores for normal data while maximizing those for the labeled anomaly data. We also provide a theoretical analysis establishing a generalization error bound for the proposed objective, guaranteeing that the empirical minimizer converges asymptotically to the ideal minimizer. Our approach achieves better detection performance than existing approaches on various datasets.

Figures & tables

Panel of Figure 1 — Figure 1: The comparison of the PU learning, the unsupervised anomaly detector (DAE), the semi-supervised anomaly detector (ABC), and the proposed method on the toy dataset. The unlabeled data in this dataset include both normal and anomaly data points. The yellow and blue in the contour maps represent abnormality and normality, respectively. The purple stars represent the examples of unseen anomalies, which are new types of anomalies unseen during training.

Term	Definition	Split
Unlabeled dataset ( $\mathcal{U}$ )	Contains normal data and seen anomalies without labels	Training
Anomaly dataset ( $\mathcal{A}$ )	Labeled seen anomaly samples	Training
Normal data	Non-anomalous samples	Training ( $\mathcal{U}$ ) + Test
Seen anomalies	Anomalies of the same types as those in $\mathcal{A}$	Training ( $\mathcal{A}$ , $\mathcal{U}$ ) + Test
Unseen anomalies	Anomalies of types not represented in $\mathcal{A}$	Test only

Table 1: Key terminology and dataset correspondence.

	MNIST	FashionMNIST	SVHN	CIFAR10
IF	0.885 $\pm$ 0.062	0.916 $\pm$ 0.077	0.501 $\pm$ 0.014	0.610 $\pm$ 0.095
AE	0.912 $\pm$ 0.042	0.841 $\pm$ 0.102	0.562 $\pm$ 0.040	0.535 $\pm$ 0.120
DeepSVDD	0.937 $\pm$ 0.045	0.921 $\pm$ 0.088	0.582 $\pm$ 0.035	0.709 $\pm$ 0.054
LOE	0.945 $\pm$ 0.033	0.916 $\pm$ 0.094	0.624 $\pm$ 0.054	0.718 $\pm$ 0.062
ABC	0.916 $\pm$ 0.042	0.841 $\pm$ 0.104	0.562 $\pm$ 0.041	0.535 $\pm$ 0.119
DeepSAD	0.942 $\pm$ 0.041	0.928 $\pm$ 0.089	0.652 $\pm$ 0.034	0.726 $\pm$ 0.051

Table 2: Comparison of anomaly detection performance on MNIST, FashionMNIST, SVHN, and CIFAR10.

	CIFAR100	Path	OCT	Tissue
IF	0.604 $\pm$ 0.004	0.809 $\pm$ 0.118	0.714 $\pm$ 0.004	0.472 $\pm$ 0.187
AE	0.589 $\pm$ 0.010	0.605 $\pm$ 0.240	0.860 $\pm$ 0.005	0.468 $\pm$ 0.178
DeepSVDD	0.587 $\pm$ 0.026	0.759 $\pm$ 0.148	0.726 $\pm$ 0.052	0.661 $\pm$ 0.055
LOE	0.576 $\pm$ 0.035	0.721 $\pm$ 0.160	0.783 $\pm$ 0.030	0.635 $\pm$ 0.086
ABC	0.590 $\pm$ 0.010	0.604 $\pm$ 0.241	0.857 $\pm$ 0.001	0.472 $\pm$ 0.177
DeepSAD	0.594 $\pm$ 0.012	0.763 $\pm$ 0.187	0.823 $\pm$ 0.038	0.683 $\pm$ 0.053

Table 3: Comparison of anomaly detection performance on CIFAR100, Path, OCT and Tissue.

Panel of Figure 2 — Figure 2: Relationship between the anomaly detection performance and the hyperparameter $\alpha$ of the PUSVDD and the SOEL on each dataset. We used class 0 as unseen anomaly, class 1 as normal, and the remaining classes as seen anomalies. The semi-transparent area represents standard deviations.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 3 — Figure 3: The example of the dataset in the case of MNIST.

Panel of Figure 4 — Figure 4: Comparison of anomaly detection performance between the PUSVDD and the SOEL with various numbers of unlabeled anomalies on each dataset. We used class 0 as unseen anomaly, class 1 as normal, and the remaining classes as seen anomalies. The semi-transparent area represents standard deviations.

	MNIST	FashionMNIST	SVHN	CIFAR10
IF	0.814 $\pm$ 0.096	0.915 $\pm$ 0.053	0.510 $\pm$ 0.016	0.585 $\pm$ 0.106
AE	0.843 $\pm$ 0.073	0.840 $\pm$ 0.092	0.563 $\pm$ 0.039	0.573 $\pm$ 0.131
DeepSVDD	0.925 $\pm$ 0.056	0.942 $\pm$ 0.040	0.600 $\pm$ 0.033	0.694 $\pm$ 0.069
LOE	0.942 $\pm$ 0.038	0.940 $\pm$ 0.042	0.645 $\pm$ 0.055	0.713 $\pm$ 0.077
ABC	0.852 $\pm$ 0.072	0.841 $\pm$ 0.092	0.564 $\pm$ 0.039	0.572 $\pm$ 0.131
DeepSAD	0.930 $\pm$ 0.052	0.956 $\pm$ 0.032	0.674 $\pm$ 0.031	0.716 $\pm$ 0.074

Appendix

Table 4: Seen anomaly detection performance on MNIST, FashionMNIST, SVHN, and CIFAR10.

	MNIST	FashionMNIST	SVHN	CIFAR10
IF	0.955 $\pm$ 0.031	0.917 $\pm$ 0.111	0.492 $\pm$ 0.018	0.635 $\pm$ 0.095
AE	0.981 $\pm$ 0.013	0.842 $\pm$ 0.130	0.560 $\pm$ 0.045	0.497 $\pm$ 0.111
DeepSVDD	0.950 $\pm$ 0.044	0.900 $\pm$ 0.143	0.564 $\pm$ 0.043	0.724 $\pm$ 0.087
LOE	0.949 $\pm$ 0.034	0.892 $\pm$ 0.152	0.602 $\pm$ 0.058	0.724 $\pm$ 0.099
ABC	0.980 $\pm$ 0.014	0.840 $\pm$ 0.133	0.559 $\pm$ 0.046	0.497 $\pm$ 0.109
DeepSAD	0.954 $\pm$ 0.040	0.900 $\pm$ 0.153	0.631 $\pm$ 0.044	0.735 $\pm$ 0.078

Appendix

Table 5: Unseen anomaly detection performance on MNIST, FashionMNIST, SVHN, and CIFAR10.

	CIFAR100	Path	OCT	Tissue
IF	0.577 $\pm$ 0.005	0.657 $\pm$ 0.211	0.679 $\pm$ 0.005	0.443 $\pm$ 0.189
AE	0.514 $\pm$ 0.013	0.562 $\pm$ 0.253	0.808 $\pm$ 0.005	0.443 $\pm$ 0.188
DeepSVDD	0.623 $\pm$ 0.036	0.769 $\pm$ 0.139	0.702 $\pm$ 0.043	0.692 $\pm$ 0.047
LOE	0.624 $\pm$ 0.035	0.746 $\pm$ 0.167	0.750 $\pm$ 0.031	0.657 $\pm$ 0.084
ABC	0.516 $\pm$ 0.014	0.564 $\pm$ 0.252	0.805 $\pm$ 0.002	0.448 $\pm$ 0.187
DeepSAD	0.628 $\pm$ 0.042	0.772 $\pm$ 0.165	0.798 $\pm$ 0.032	0.715 $\pm$ 0.040

Appendix

Table 6: Seen anomaly detection performance on CIFAR100, Path, OCT and Tissue.

	CIFAR100	Path	OCT	Tissue
IF	0.632 $\pm$ 0.005	0.961 $\pm$ 0.055	0.785 $\pm$ 0.004	0.500 $\pm$ 0.187
AE	0.665 $\pm$ 0.009	0.647 $\pm$ 0.239	0.965 $\pm$ 0.005	0.493 $\pm$ 0.170
DeepSVDD	0.552 $\pm$ 0.033	0.749 $\pm$ 0.200	0.774 $\pm$ 0.072	0.629 $\pm$ 0.069
LOE	0.528 $\pm$ 0.047	0.696 $\pm$ 0.205	0.851 $\pm$ 0.030	0.613 $\pm$ 0.093
ABC	0.665 $\pm$ 0.009	0.644 $\pm$ 0.241	0.961 $\pm$ 0.002	0.496 $\pm$ 0.168
DeepSAD	0.561 $\pm$ 0.024	0.755 $\pm$ 0.242	0.874 $\pm$ 0.049	0.651 $\pm$ 0.075

Appendix

Table 7: Unseen anomaly detection performance on CIFAR100, Path, OCT and Tissue.

	PUSVDD	SOEL	WSAD-DT
all	0.793 $\pm$ 0.016	0.669 $\pm$ 0.021	0.641 $\pm$ 0.029
seen	0.799 $\pm$ 0.013	0.688 $\pm$ 0.021	0.642 $\pm$ 0.030
unseen	0.787 $\pm$ 0.020	0.649 $\pm$ 0.027	0.640 $\pm$ 0.028

Appendix

Table 8: Comparison of anomaly detection performance on MVTec AD.

Figure 5: Relationship between the anomaly detection performance and the hyperparameter

\alpha

of the PUSVDD and the SOEL on MVTec AD. The semi-transparent area represents standard deviations.

Dataset	$\hat{\alpha}$
MNIST	0.048
FashionMNIST	0.056
SVHN	0.350
CIFAR-10	0.325

Appendix

Table 9: CPE estimates of

\alpha

compared with the true value (

\alpha=0.1

Class	SOEL	PUSVDD
1	0.995 $\pm$ 0.001	0.999 $\pm$ 0.000
2	0.927 $\pm$ 0.047	0.988 $\pm$ 0.003
3	0.986 $\pm$ 0.009	0.996 $\pm$ 0.001
4	0.982 $\pm$ 0.013	0.997 $\pm$ 0.001
5	0.935 $\pm$ 0.024	0.989 $\pm$ 0.009
6	0.964 $\pm$ 0.005	0.967 $\pm$ 0.013

Appendix

Table 10: Per-class AUROC on MNIST. Class 0 (digit 0) is the unseen anomaly.

Class	SOEL	PUSVDD
1	0.983 $\pm$ 0.003	0.993 $\pm$ 0.002
2	0.930 $\pm$ 0.019	0.952 $\pm$ 0.010
3	0.911 $\pm$ 0.007	0.930 $\pm$ 0.015
4	0.920 $\pm$ 0.008	0.945 $\pm$ 0.014
5	0.985 $\pm$ 0.004	0.993 $\pm$ 0.002
6	0.750 $\pm$ 0.026	0.763 $\pm$ 0.019

Appendix

Table 11: Per-class AUROC on FashionMNIST. Class 0 (T-shirt/top) is the unseen anomaly.

Class	SOEL	PUSVDD
1	0.783 $\pm$ 0.040	0.850 $\pm$ 0.048
2	0.760 $\pm$ 0.014	0.801 $\pm$ 0.027
3	0.698 $\pm$ 0.032	0.682 $\pm$ 0.029
4	0.769 $\pm$ 0.033	0.788 $\pm$ 0.025
5	0.765 $\pm$ 0.012	0.799 $\pm$ 0.022
6	0.705 $\pm$ 0.013	0.690 $\pm$ 0.018

Appendix

Table 12: Per-class AUROC on SVHN. Class 0 (digit 0) is the unseen anomaly.

Class	SOEL	PUSVDD
1	0.793 $\pm$ 0.066	0.824 $\pm$ 0.030
2	0.712 $\pm$ 0.012	0.724 $\pm$ 0.010
3	0.724 $\pm$ 0.054	0.795 $\pm$ 0.026
4	0.731 $\pm$ 0.096	0.792 $\pm$ 0.015
5	0.790 $\pm$ 0.029	0.843 $\pm$ 0.025
6	0.856 $\pm$ 0.016	0.852 $\pm$ 0.023

Appendix

Table 13: Per-class AUROC on CIFAR10. Class 0 (airplane) is the unseen anomaly.

Class	SOEL	PUSVDD
1	0.965 $\pm$ 0.051	0.996 $\pm$ 0.005
2	0.526 $\pm$ 0.126	0.554 $\pm$ 0.123
3	0.950 $\pm$ 0.035	0.957 $\pm$ 0.007
4	0.737 $\pm$ 0.031	0.733 $\pm$ 0.068
5	0.741 $\pm$ 0.035	0.751 $\pm$ 0.096
6	0.812 $\pm$ 0.082	0.859 $\pm$ 0.086

Appendix

Table 14: Per-class AUROC on Path. Class 0 is the unseen anomaly.

Class	SOEL	PUSVDD
1	0.623 $\pm$ 0.017	0.622 $\pm$ 0.010
2	0.788 $\pm$ 0.052	0.843 $\pm$ 0.029
3	0.739 $\pm$ 0.029	0.806 $\pm$ 0.030
4	0.693 $\pm$ 0.016	0.749 $\pm$ 0.018
5	0.686 $\pm$ 0.035	0.722 $\pm$ 0.040
6	0.739 $\pm$ 0.026	0.716 $\pm$ 0.011

Appendix

Table 15: Per-class AUROC on Tissue. Class 0 is the unseen anomaly.

Panel of Figure 6 — Figure 6: The anomaly detection performance of the PUAE and the SOEL as the fraction $\beta$ of unseen anomalies among the unlabeled anomalies increases, with the total contamination ratio fixed at $\alpha=0.1$ . Both methods use the same DAE as the base anomaly detector, so that only the training objective differs. The three panels show the AUROC on the whole test set, on the seen anomalies, and on the unseen anomalies, respectively. The semi-transparent area represents standard deviations.

Explore similar work

Deep Convolutional Large-Margin $\ell_p$ -SVDD for Visual Anomaly Detection
Jul 24, 2026Alireza Dastmalchi Saei, Shervin Rahimzadeh ArashlooUnsupervised Detection Singular Value Decomposition
Department of Computer Engineering, Bilkent University, 06800 Ankara, Türkiye
Fast and Accurate Anomaly Detection in Time Series
Jul 2, 2026Emanuele Mele, Massimo Cafaro, Angelo Coluccia +1Abnormality Unsupervised
University of Salento, Via per Monteroni, Lecce, 73100, , Italy
Beyond Normal References: Discriminative Few-Shot Anomaly Detection
May 22, 2026Huan Wang, Jun Shen, Jun Yan +1Multimodal Anomaly Detection Norm
Singapore Management University, Singapore · University of Wollongong, Australia

Jul 24, 2026, cs.CVJ/K move · Enter open · S save

Deep Convolutional Large-Margin $\ell_p$ -SVDD for Visual Anomaly Detection

Alireza Dastmalchi Saei, Shervin Rahimzadeh Arashloo

Department of Computer Engineering, Bilkent University, 06800 Ankara, Türkiye

Unsupervised Detection Singular Value Decomposition

Open paper AskPDF arXiv

Visual anomaly detection requires adaptive representations and reliable decision boundaries, particularly when anomalous training samples are scarce and class distributions are highly imbalanced. Classical kernel-based methods yield principled geometric decision regions but typically operate on fixed features, while deep detectors learn task-specific representations but often fail to provide an explicit margin-aware kernel boundary. In this study, we propose DLM-SVDD, a deep large-margin novelty-detection framework that jointly learns convolutional features and an explicit kernel-based decision boundary. By drawing on the large-margin

\ell_p

-Support Vector Data Description (

\ell_p

-SVDD) approach, the proposed method performs explicit margin maximization and nonlinear slack penalization while adapting the representation to the target task. To train the proposed model, we present an optimization scheme that alternates between a Frank--Wolfe--based update of the convex dual boundary and a CNN update step operating on a smooth margin-violation loss induced by the recovered boundary. To improve scalability, we analyze the efficiency--accuracy trade-offs for different kernel approximation strategies, deriving practical propositions for large-scale anomaly detection. Extensive experiments on multiple standard benchmarks show consistent performance improvements over the baseline and strong overall performance compared with state-of-the-art methods while illustrating that the proposed joint representation--boundary learning scheme remains effective under severe imbalanced class distributions.

Abstract

Explore similar work

Deep Convolutional Large-Margin ℓp\ell_pℓp​-SVDD for Visual Anomaly Detection

Fast and Accurate Anomaly Detection in Time Series

Beyond Normal References: Discriminative Few-Shot Anomaly Detection

Deep Convolutional Large-Margin $\ell_p$ -SVDD for Visual Anomaly Detection