Why Do Unlearnable Examples Work: A Novel Perspective of Mutual Information

Yifan Zhu; Yibo Miao; Yinpeng Dong; Xiao-Shan Gao

cs.LGMar 4, 2026

Why Do Unlearnable Examples Work: A Novel Perspective of Mutual Information

Authors: Yifan Zhu, Yibo Miao, Yinpeng Dong, Xiao-Shan Gao

Organizations: State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences · University of Chinese Academy of Sciences · College of AI, Tsinghua University · Shanghai Qi Zhi Institute

Abstract

The volume of freely scraped data on the Internet has driven the tremendous success of deep learning. Along with this comes the growing concern about data privacy and security. Numerous methods for generating unlearnable examples have been proposed to prevent data from being illicitly learned by unauthorized deep models by impeding generalization. However, the existing approaches primarily rely on empirical heuristics, making it challenging to enhance unlearnable examples with solid explanations. In this paper, we analyze and improve unlearnable examples from a novel perspective: mutual information reduction. We demonstrate that effective unlearnable examples always decrease mutual information between clean features and poisoned features, and when the network gets deeper, the unlearnability goes better together with lower mutual information. Further, we prove from a covariance reduction perspective that minimizing the conditional covariance of intra-class poisoned features reduces the mutual information between distributions. Based on the theoretical results, we propose a novel unlearnable method called Mutual Information Unlearnable Examples (MI-UE) that reduces covariance by maximizing the cosine similarity among intra-class features, thus impeding the generalization effectively. Extensive experiments demonstrate that our approach significantly outperforms the previous methods, even under defense mechanisms.

Figures & tables

Victim ResNet-18	Clean	Random	EM	AP	NTGA	AR	REM	SEM	GUE	TUE	MI-UE
Test Acc(%)	94.45	94.11	24.17	11.21	23.11	17.41	22.94	14.78	12.04	11.25	9.95
Acc Gap(%)	-	0.34	70.28	83.24	71.34	77.04	71.51	79.67	82.41	83.20	84.50
MI	0.7122	0.6747	0.6400	0.5871	0.6126	0.5622	0.6290	0.5747	0.5895	0.6094	0.4969
MI Gap	-	0.0375	0.0722	0.1251	0.0996	0.1500	0.0832	0.1375	0.1227	0.1028	0.2153

Table 1: The Mutual Information (MI) estimation between clean and poison features on the histogram-based estimator and test accuracy for different UEs on CIFAR-10, along with their gaps from the results for clean data. Compared to random noises, MI for all UEs are significantly reduced.

Panel of Figure 1 — Figure 1: The estimation of MI between clean and unlearnable poisoned features on different MI estimators. (a): MI metrics under histogram-based estimator and kernel density estimator (KDE). (b): MI metrics under histogram-based estimator and k-NN estimator. (c): MI metrics under histogram-based method and mutual information neural estimator (MINE). Green triangles represent clean or ineffective UEs, blue circles mean existing effective UEs, orange square denotes our MI-UE. It demonstrates that although different estimation methods show different quantitative results, the effectiveness of UEs is always positively related with the MI between clean and poisoned features.

Panel of Figure 2 — Figure 2: The drop of test accuracy (Acc Gap) and the reduction of MI (MI Gap) of various UEs on different models, including linear, 2-NN, 3-NN, LeNet-5(LN5), VGG-11(V11), ResNet-18(RN18). The results indicate that as the depth and complexity of the models increase, both the drop of test accuracy and the reduction of MI become more pronounced.

Figure 3: Feature covariance of different unlearnable examples. Results show that all unlearnable methods reduce both MI and covariance, with our MI-UE achieving the lowest values.

Dataset/Method	Clean	EM	AP	NTGA	REM	SEM	TUE	MI-UE (ours)
CIFAR-10	94.45	24.17	11.21	23.11	22.94	14.78	11.25	9.95
CIFAR-100	76.65	2.09	3.73	3.08	7.52	6.29	1.34	1.17
ImageNet-subset	80.43	1.26	9.10	8.42	13.74	4.10	4.95	1.03

Table 2: Quantitative results(%) of baseline methods and our MI-UE for ResNet-18 on three benchmark datasets. Our MI-UE achieves the lowest test accuracy compared to other unlearnable examples, indicating excellent poisoning effectiveness.

Model/Method	Clean	EM	AP	NTGA	AR	REM	SEM	GUE	TUE	MI-UE (ours)
ResNet-18	94.45	24.17	11.21	23.11	17.41	22.94	14.78	12.04	11.25	9.95
ResNet-50	95.16	23.57	11.66	19.01	15.28	23.33	13.61	12.99	10.01	9.98
DenseNet-121	94.91	24.87	11.80	19.83	16.50	21.87	15.19	12.46	11.41	9.93
WRN34-10	96.03	24.25	11.28	21.92	14.62	21.64	13.64	13.22	12.11	10.68
ViT-B	90.92	27.35	24.21	43.55	24.16	21.67	25.52	17.72	35.54	15.51
LeNet-5	80.68	26.30	31.38	44.06	73.33	29.97	22.94	13.30	28.37	10.80

Table 3: Quantitative results(%) of baseline methods and our MI-UE on transferability across different models on CIFAR-10. All of unlearnable examples are generated by ResNet-18. Above the dashline represents modern deep networks, while below the dashline represents shallow networks. Our MI-UE consistently results in the lowest test accuracy across all network architectures.

Method	AT-8	AT-6	AT-4	AT-2	ST
Clean	85.10	87.54	89.77	91.95	94.45
EM	84.57	85.42	84.29	52.81	24.17
AP	82.70	85.48	88.14	22.48	11.21
NTGA	84.22	86.27	88.36	87.87	23.11
AR	84.54	87.09	89.81	92.45	17.41
SEM	85.99	86.82	29.77	19.41	14.78

Table 4: Quantitative results(%) of baseline methods and our MI-UE under adversarial training with different budget on CIFAR-10. AT-

i

means AT with budget

i/255

, ST means standard training. Our MI-UE achieves state-of-the-art performance, particularly achieving an impressive 45.55% at AT-6.

Defense	Cutout	Cutmix	Mixup	UER	ISS	OP	AVA	D-VAE	LE
Clean	95.53	96.43	95.83	93.28	82.71	88.82	89.15	93.29	92.32
EM	22.90	24.08	27.22	91.41	82.78	71.70	86.62	91.42	90.93
NTGA	17.65	25.53	19.04	93.39	80.84	78.14	85.13	89.21	87.31
AR	12.84	16.20	16.24	93.32	82.79	84.69	88.38	91.11	89.75
REM	26.49	24.44	18.74	69.63	82.59	29.61	86.28	86.38	90.14
SEM	14.25	15.39	15.06	70.53	81.86	23.72	87.30	88.55	88.25

Table 5: Quantitative results(%) of baseline methods and MI-UE under various defense on CIFAR-10. Our MI-UE achieves the lowest test accuracy under the majority of defense methods.

Table 9

Table 10

Quality	6	8	10	12	15	20	30	Average
Clean	78.52	81.35	83.12	83.97	85.41	86.57	88.17	83.87
EM	77.57	80.96	82.94	83.63	84.91	85.67	86.30	83.14
NTGA	76.39	78.63	79.38	79.64	81.68	81.59	81.50	79.83
AR	78.89	81.55	82.83	84.18	85.39	86.68	88.41	83.99
REM	78.04	81.30	82.20	83.23	84.50	84.60	85.24	82.73
GUE	78.27	80.69	83.00	83.97	85.39	86.46	88.06	83.69

Table 11: Quantitative results(%) of baseline methods and MI-UE under various JPEG compression quality in ISS defense ( Liu et al., 2023 ) . “Average” means the average performance across these compression quality.

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Defense	Method	Clean	Random	EM	AP	NTGA	AR	REM	SEM	GUE	TUE	MI-UE
AT	Acc(%)	85.10	85.07	84.57	82.70	84.22	84.54	85.99	85.99	84.37	84.10	70.56
	Acc Gap(%)	-	0.03	0.53	2.40	0.88	0.56	-0.79	-0.79	0.73	1.00	14.54
	MI	0.7040	0.6516	0.6309	0.6348	0.6330	0.6377	0.6396	0.6323	0.6388	0.6400	0.6125
	MI Gap	-	0.0524	0.0731	0.0692	0.0710	0.0663	0.0644	0.0717	0.0652	0.0640	0.0915
Cutout	Acc(%)	95.53	95.57	22.90	11.30	17.65	12.84	26.49	14.25	13.98	11.01	10.13
	Acc Gap(%)	-	-0.04	72.63	84.23	77.88	82.69	69.04	81.28	81.55	84.52	85.40

Appendix

Table 12: Test accuracy and MI estimation on various UEs compared with clean CIFAR-10 dataset under defense mechanism including AT, Cutout, UER and ISS.

Method	Model	Linear	2-NN	3-NN	LeNet-5	VGG-11	ResNet-18
Clean	Acc(%)	39.13	56.15	62.41	80.68	91.44	94.45
Clean	MI	0.6682	0.6850	0.7197	0.7222	0.7583	0.7122
EM	Acc(%)	32.09	32.5	29.63	26.30	26.34	24.17
	Acc Gap(%)	-7.04	-23.65	-32.78	-54.38	-65.10	-70.28
	MI	0.6310	0.6396	0.6718	0.6647	0.6865	0.6400
	MI Gap	-0.0372	-0.0454	-0.0479	-0.0575	-0.0718	-0.0722

Appendix

Table 13: Test accuracy and MI estimation between EM, REM, GUE and MI-UE unlearnable examples compared with clean CIFAR-10 dataset under victim linear classifiers (Linear), two-layer neural network (2-NN), three-layer neural network (3-NN), LeNet-5, VGG-11 and ResNet18.

Method	Model	ResNet-18	ResNet-34	ResNet-50	ResNet-101	ResNet-152
Clean	Acc(%)	94.45	94.63	95.16	95.43	95.55
Clean	MI	0.7122	0.7067	0.7078	0.7022	0.7035
EM	Acc(%)	24.17	23.96	23.57	23.75	23.32
	Acc Gap(%)	-70.28	-70.67	-71.59	-71.68	-72.23
	MI	0.6400	0.6318	0.6285	0.614	0.6149
	MI Gap	-0.0722	-0.0749	-0.0793	-0.0876	-0.0886

Appendix

Table 14: Test accuracy and MI estimation between EM, REM, GUE and MI-UE unlearnable examples compared with clean CIFAR-10 dataset under ResNet-18, ResNet-34, ResNet-50, ResNet-101 and ResNet-152.

Method	Model	ViT-Base	ViT-Large	ViT-Huge
Clean	Acc(%)	90.92	91.35	91.93
Clean	MI	0.7285	0.7297	0.7274
EM	Acc(%)	27.35	26.26	25.87
	Acc Gap(%)	-63.57	-65.09	-66.06
	MI	0.6531	0.6449	0.6370
	MI Gap	0.0754	-0.0848	-0.0904

Appendix

Table 15: Test accuracy and MI estimation between EM, REM, GUE and MI-UE unlearnable examples compared with clean CIFAR-10 dataset under ViT-Base, ViT-Large and ViT-Huge.

Test Acc(%)	Unlearnable Class	Other Classes
Clean	95.9	94.3
EM	2.2	95.4
AP	0.2	95.3
NTGA	4.0	95.1
AR	0.7	94.9
SEM	0.3	94.9

Appendix

Table 16: The unlearnable class accuracy and other classes accuracy of various one-class UEs on CIFAR-10.

Test Acc(%)	Unlearnable Class	Other Classes
Clean	88	76.6
EM	8	76.7
AP	3	76.2
NTGA	26	77.0
AR	90	77.1
SEM	18	75.9

Appendix

Table 17: The unlearnable class accuracy and other classes accuracy of various one-class UEs on CIFAR-100.

Test Acc(%)	Unlearnable Class	Other Classes
Clean	92	80.31
MI-UE	16	72.61

Appendix

Table 18: The unlearnable class accuracy and other classes accuracy of our MI-UE on ImageNet-subset.

Figure 4: Visualization of MI-UE unlearnable noises and their corresponding clean and poisoned images on CIFAR-10. The first row is the clean images, the second row is the MI-UE noises, the last row is the poisoned images.

Figure 5: Visualization of MI-UE unlearnable noises and their corresponding clean and poisoned images on ImageNet-subset. The first row is the clean images, the second row is the MI-UE noises, the last row is the poisoned images.

Model/Method	Clean	EM	AP	NTGA	AR	REM	SEM	GUE	TUE	MI-UE (ours)
ResNet-18	76.65	2.09	3.73	3.08	6.19	7.52	6.29	22.79	1.34	1.17
ResNet-50	78.25	2.14	4.51	5.21	9.05	7.63	4.55	23.51	3.91	1.72
DenseNet-121	77.78	2.69	3.90	6.04	5.26	7.63	4.54	24.35	2.10	1.11
WRN34-10	80.46	2.45	3.16	3.37	5.21	5.77	5.00	31.21	4.64	1.48
ViT-B	66.54	4.25	3.23	11.52	24.10	8.44	8.80	24.01	9.42	2.62

Appendix

Table 19: Test accuracy(%) of various UEs under different victim models on CIFAR-100. Our MI-UE achieves the lowest test accuracy compared to other UEs, indicating excellent poisoning effectiveness.

SimCLR	Clean	EM	TUE	A-MI-UE
Test Acc(%)	91.71	89.54	56.32	52.84

Appendix

Table 20: The performance of our A-MI-UE method compared with UE and TUE on SimCLR contrastive learning paradigm.

Training Acc(%)	Unlearnable Noises	Unlearnable Datasets
Clean	—	47.99
EM	99.32	99.37
AP	86.53	56.96
NTGA	99.94	95.02
AR	42.09	48.13
SEM	96.66	83.01

Appendix

Table 21: The training accuracy of unlearnable noises and unlearnable datasets by the linear network.

Step size	CIFAR-10	CIFAR-100
0.1/255	9.93	1.13
0.2/255 (default)	9.95	1.17
0.4/255	10.00	1.14
0.8/255	9.96	1.09

Appendix

Table 22: Quantitative results(%) of MI-UE with different poison generation step sizes.

Epochs	Test Acc(%)
50 (baseline)	1.03
30	1.09
15	1.95

Appendix

Table 23: Test accuracy of MI-UE with different poisoning epochs for ImageNet-subset.

Figure 6: Learning process of standard training on EM, AP, REM, SEM and our MI-UE unlearnable examples, the solid lines represent the training accuracy, while the hollow lines represent the test accuracy.

Figure 7: Learning process of adversarial training on EM, AP, REM, SEM and our MI-UE unlearnable examples with the perturbation budget be

8/255

, the solid lines represent the training accuracy, while the hollow lines represent the test accuracy.

Figure 8: Learning process of adversarial training on EM, AP, REM, SEM and our MI-UE unlearnable examples with the perturbation budget be

6/255

, the solid lines represent the training accuracy, while the hollow lines represent the test accuracy.

Figure 9: Learning process of adversarial training on EM, AP, REM, SEM and our MI-UE unlearnable examples with the perturbation budget be

4/255

, the solid lines represent the training accuracy, while the hollow lines represent the test accuracy.

Figure 10: Learning process of adversarial training on EM, AP, REM, SEM and our MI-UE unlearnable examples with the perturbation budget be

2/255

, the solid lines represent the training accuracy, while the hollow lines represent the test accuracy.

Unlearnable Examples

Unlearnable examples use data perturbations to prevent machine-learning models from learning protected images, text, graphs, or other data.

See all 17 papers

Reversible Unlearnable Examples: Towards the Copyright Protection in Deep Learning Era
Aug 6, 2026Binze Wang, Jinyu Tian, Xingrun Wang +2Dataset Watermarking Unlearnable Examples
School of Computer Science and Engineering, Faculty of Innovation Engineering, Macau University of Science and Technology, Macau, China · School of Computer Science and Artificial Intelligence of FoShan University, FoShan 528225, China · Faculty of Applied Sciences of the Macao Polytechnic University, Macau, China
Instance-Level Difficulty: A Missing Perspective in Machine Unlearning
Oct 3, 2024Hammad Rizwan, Mahtab Sarvmaili, Hassan Sajjad +1Machine Unlearning Machine Unlearning Evaluation
Department of Computer Science, Dalhousie University, Canada.
Machine Unlearning via Information Theoretic Regularization
Feb 8, 2025Shizhou Xu, Thomas StrohmerMachine Unlearning
Department of Mathematics University of California Davis Davis, CA 95616-5270, USA · Department of Mathematics Center of Data Science and Artificial Intelligence Research University of California Davis Davis, CA 95616-5270, USA

Aug 6, 2026J/K move · Enter open · S save

Reversible Unlearnable Examples: Towards the Copyright Protection in Deep Learning Era

Binze Wang, Jinyu Tian, Xingrun Wang, Xiaochen Yuan+1

School of Computer Science and Engineering, Faculty of Innovation Engineering, Macau University of Science and Technology, Macau, China · School of Computer Science and Artificial Intelligence of FoShan University, FoShan 528225, China · Faculty of Applied Sciences of the Macao Polytechnic University, Macau, China

Dataset Watermarking Unlearnable Examples

Open paper AskPDF arXiv

Significant advancements in deep learning have been made possible by the utilization of large datasets, underscoring the critical importance of copyright protection. Adding meticulously designed perturbations to examples, making them unlearnable has become a crucial approach for safeguarding data copyright. Existing methods for creating unlearnable examples overlook the risk of data leakage, which can threaten data ownership. Thus, copyright protection in deep learning faces two main threats: illegal model training and malicious data leakage. We investigate that these two threats cannot be solved by straightforwardly combining existing availability attacks and watermarking techniques as their negative interaction effects. Therefore, in this paper, we propose a novel copyright protection mechanism for the aforementioned security concerns. Considering that the prevention of unauthorized model training requires powerful generalizability of unlearnable perturbations, we generate perturbations to induce the model to learn uncorrelated features of input images. It works by minimizing the mutual information of the input and output of the model. On the other hand, to eliminate the side impact of unlearnable perturbations on the watermark extraction, we design a dual extraction strategy by using two distinct watermark extractors. Extensive experiments on the image datasets {ImageNet, CIFAR10, and Pets} show that our proposed method could provide comprehensive copyright protection to images. The code is available at {https://github.com/Yeah21/ReversibleUnlearnableExamples}.

Abstract

Unlearnable Examples

Explore similar work

Reversible Unlearnable Examples: Towards the Copyright Protection in Deep Learning Era

Instance-Level Difficulty: A Missing Perspective in Machine Unlearning

Machine Unlearning via Information Theoretic Regularization