When Priors Backfire: On the Vulnerability of Unlearnable Examples to Pretraining

Zhihao Li; Gezheng Xu; Jiale Cai; Ruiyi Fang; Di Wu; Qicheng Lao; Charles Ling; Boyu Wang

cs.LGMar 5, 2026

When Priors Backfire: On the Vulnerability of Unlearnable Examples to Pretraining

Authors: Zhihao Li, Gezheng Xu, Jiale Cai, Ruiyi Fang, Di Wu, Qicheng Lao, Charles Ling, Boyu Wang

Organizations: Western University · Concordia University · Beijing University of Posts and Telecommunications · Vector Institute

Abstract

Unlearnable Examples (UEs) serve as a data protection strategy that generates imperceptible perturbations to mislead models into learning spurious correlations instead of underlying semantics. In this paper, we uncover a fundamental vulnerability of UEs that emerges when learning starts from a pretrained model. Crucially, our empirical analysis shows that even when data are protected by carefully crafted perturbations, pretraining priors still furnish rich semantic representations that allow the model to circumvent the shortcuts introduced by UEs and capture genuine features, thereby nullifying unlearnability. To address this, we propose BAIT (Binding Artificial perturbations to Incorrect Targets), a novel bi-level optimization formulation. Specifically, the inner level aims at associating the perturbed samples with real labels to simulate standard data-label alignment, while the outer level actively disrupts this alignment by enforcing a mislabel-perturbation binding that maps samples to designated incorrect targets. This mechanism effectively overrides the semantic guidance of priors, forcing the model to rely on the injected perturbations and consequently preventing the acquisition of true semantics. Extensive experiments on standard benchmarks and multiple pretrained backbones demonstrate that BAIT effectively mitigates the influence of pretraining priors and maintains data unlearnability.

Figures & tables

Panel of Figure 1 — Figure 1: Empirical analysis of the vulnerability of UEs to pretraining priors. All experiments are conducted on CIFAR-10 with ResNet-18. (a) Existing UE methods suffer severe unlearnability degradation when applied to pretrained (PT) backbones instead of train-from-scratch (TS) models. (b) We progressively replace layers of a four-layer ImageNet pretrained ResNet-18 with randomly initialized layers until obtaining a train-from-scratch model. The resistance to UEs steadily diminishes as pretraining priors are removed. (c) We report the normalized model parameter updates when training on clean data and UEs (details in Appendix B.1 ). We observe that effective perturbations result in minimal parameter updates. In contrast, pretrained models bypass the spurious correlations induced by EMN ( Huang et al., 2021 ) and remain fully optimized. (d) We plot the learning curve starting from a pretrained backbone. The concurrent rise in the training and test accuracy of EMN indicates that such substantial parameter updates drive the model to acquire real semantics rather than the injected shortcuts, thereby nullifying the unlearnability.

Dataset	Backbone	Clean	EMN	TUE	REM	LSP	GUE	14A	Ours
CIFAR-10	ResNet-18	84.10	61.82	82.72	74.46	54.20	23.17	65.70	14.40
	ResNet-50	86.48	57.54	85.68	85.06	65.00	71.42	72.81	14.82
	VGG-11	88.39	80.90	87.67	63.32	55.63	45.77	50.80	22.14
	DenseNet-121	87.17	63.31	86.30	61.05	62.66	72.31	74.89	20.32
CIFAR-100	ResNet-18	55.73	55.53	55.20	55.22	36.98	54.09	33.67	20.77
	ResNet-50	60.73	45.78	59.54	58.69	42.84	53.14	36.58	28.02

Table 1: Test accuracy (%)

\downarrow

evaluation against ImageNet-pretrained backbones.

Method	CIFAR-10	CIFAR-100	SVHN
EMN ∗	59.84	45.20	32.33
TUE ∗	82.65	55.28	47.71
REM ∗	56.08	34.30	55.26
Ours	14.40	20.77	14.37

Table 2: Test accuracy (%)

\downarrow

comparison against re-implemented baselines with pretrained surrogates on an ImageNet-pretrained ResNet-18.

Dataset	Pretraining Prior	EMN	TUE	REM	LSP	GUE	Ours
CIFAR-10	CIFAR-100	48.58	82.26	60.88	60.64	27.05	20.58
CIFAR-10	SVHN	28.33	49.00	37.91	38.53	15.55	9.75
CIFAR-100	CIFAR-10	33.67	61.05	24.77	35.08	66.93	22.89
CIFAR-100	SVHN	62.57	62.18	16.77	26.14	59.10	11.21
SVHN	CIFAR-10	19.13	12.80	70.43	26.73	29.35	11.27
SVHN	CIFAR-100	28.75	19.47	64.42	47.71	24.14	17.49

Table 3: Transferability across pretrained backbones with different priors. An ImageNet-pretrained ResNet-18 is used as the surrogate model for perturbation optimization, and the resulting unlearnable examples are evaluated against prior knowledge from CIFAR-10, CIFAR-100, and SVHN.

Dataset	Pretraining Prior	EMN	TUE	REM	LSP	GUE	Ours
Flowers	ImageNet	44.36	46.63	43.91	47.76	45.18	24.91
ImageNet ∗	ImageNet †	51.98	59.00	59.12	59.44	37.72	24.22

Table 4: Test accuracy (%)

\downarrow

evaluation against pretrained backbones on more complex datasets. Note that ImageNet ∗ and ImageNet † denote two randomly selected 100-class subsets of ImageNet with no overlapping classes.

Figure 2: Evaluation of unlearnability transferability (test accuracy (%)

\downarrow

) across architectures, where UEs are generated using a ResNet-18 surrogate and evaluated against diverse pretrained backbones (including CNNs and ViTs).

Stage1	Stage2	Stage3	CIFAR-10	CIFAR-100	SVHN
✓			23.68	35.75	21.75
✓	✓		20.60	23.94	16.58
✓	✓	✓	14.40	20.77	14.37

Table 5: Ablation study of target label selection on ResNet-18. Stages 1, 2, and 3 denote selecting target labels from Hard Negative, Random, and Most Dissimilar Classes, respectively.

Method	ResNet-18	ResNet-50	VGG-11	DenseNet-121
EMN	16.42	13.45	16.93	14.71
TUE	10.67	12.23	12.79	17.60
REM	15.18	25.57	47.68	41.54
LSP	13.54	15.94	17.15	17.47
GUE	13.25	12.97	23.89	13.71
14A	41.34	24.85	39.87	48.97

Table 6: Test accuracy (%)

\downarrow

evaluation against randomly initialized backbones on CIFAR-10.

Method	w/o	Cutout	CutMix	Mixup
Clean	84.10	83.25	83.06	84.14
EMN	61.82	51.60	54.71	68.70
TUE	82.72	81.54	81.35	82.69
REM	74.46	65.39	73.71	78.08
LSP	54.20	44.68	51.53	59.44
GUE	23.17	29.31	28.30	34.48

Table 7: Test accuracy (%)

\downarrow

evaluation against an ImageNet-pretrained ResNet-18 under additional defenses, where CIFAR-10 is utilized as the downstream dataset.

JPEG Compression	90	80	70	60	50	40	30	20	10
Clean	82.64	82.13	81.08	80.58	80.03	79.19	77.77	75.64	73.95
EMN	68.22	69.09	69.23	69.13	69.91	71.32	73.47	74.18	74.46
TUE	81.21	80.41	79.91	79.62	79.70	79.26	78.91	77.81	75.11
REM	77.15	77.29	78.08	78.17	78.34	78.55	78.33	78.14	75.33
LSP	55.47	57.24	59.56	61.43	63.48	63.84	64.86	68.53	71.00
GUE	60.94	65.59	66.73	66.86	68.39	69.18	70.11	72.33	72.88

Table 8: Test accuracy (%)

\downarrow

comparison against an ImageNet-pretrained ResNet-18 with JPEG compression defense on CIFAR-10.

Figure 3: Perturbed examples on CIFAR-10.

Figure 4: t-SNE visualization of the last layer features. Classifiers are trained on the perturbed training set and tested on the clean test set. The top row displays models trained in a train-from-scratch manner, whereas the bottom row shows models utilizing pretraining priors.

Figure 5: Training accuracy curve on CIFAR-10, illustrating that BAIT successfully misleads the ImageNet-pretrained surrogate model during perturbation optimization.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Learning curves illustration during standard UE evaluation on CIFAR-10 with ResNet-18, where training starts from a randomly initialized backbone.

Block Name	Layer	Number
Down-sampling layers
Conv	Conv (3 × 3)	× 6
	InstanceNorm
	ReLU
Bottleneck layers
Residual	ReflectionPad	× 8

Appendix

Table 9: The architecture of the perturbation generator.

Figure 7: Effect of each pretraining layer of ResNet-18 against unlearnability on CIFAR-10.

Finetuning Dataset	Accuracy Variation ( $\Delta\text{Acc}$ )
Finetuning Dataset	Epoch 1	Epoch 2	$\cdots$	Epoch 60
Clean CIFAR-10	-36.45	-53.83	$\cdots$	-55.28
Unlearnable CIFAR-10 (Ours)	-40.66	-55.11	$\cdots$	-55.35

Appendix

Table 10: Test accuracy change (

\Delta\text{Acc}

) on the original 1000-class ImageNet validation set when the ImageNet-pretrained model is finetuned on CIFAR-10.

Dataset	EMN	TUE	REM	LSP	GUE	14A	Ours
CIFAR-10	61.82	82.72	74.46	54.20	23.17	65.70	15.97
CIFAR-100	55.53	55.20	55.22	36.98	54.09	33.67	22.83
SVHN	37.55	41.15	76.45	38.91	39.68	81.47	17.93

Appendix

Table 11: Test accuracy (%)

\downarrow

evaluation against ImageNet-pretrained backbones, where BAIT utilizes a randomly initialized surrogate instead of a pretrained surrogate for perturbation optimization.

Method	Pascal VOC 2012 Semantic (mIoU (%) $\downarrow$ )
Method	Aeroplane	Bicycle	Bird	Boat	Bottle
Clean	80.9	35.6	84.4	65.8	74.7
UnSeg	30.5	16.4	58.6	19.1	40.4
LSP	41.3	50.2	63.3	46.9	57.1
Ours	40.0	49.0	60.4	44.6	53.5

Appendix

Table 12: Evaluation of cross-task transferability on Pascal VOC 2012.

Poison Ratio $r$	10%	20%	30%	40%	50%	60%	70%	80%	90%	100%
Clean ( $1-r$ )	83.13	82.97	82.20	81.40	80.02	78.40	76.96	73.88	67.57	$-$
EMN	83.58	83.41	83.32	83.08	83.04	82.37	81.24	80.52	77.28	61.82
TUE	83.87	83.83	83.81	83.79	83.54	83.43	83.39	83.03	82.97	82.72
REM	84.18	83.79	83.63	83.05	82.69	82.90	81.96	81.09	79.81	74.46
LSP	83.56	83.51	82.53	82.49	81.48	80.29	79.68	77.97	74.91	54.20
GUE	83.94	83.58	83.48	82.48	81.79	81.14	79.88	78.18	74.50	23.17

Appendix

Table 13: Test accuracy (%)

\downarrow

with different poisoning ratios against a ImageNet-pretrained ResNet-18 on CIFAR-10.

Unlearnable Examples

Unlearnable examples use data perturbations to prevent machine-learning models from learning protected images, text, graphs, or other data.

See all 17 papers

Channel-Level Semantic Perturbations: Unlearnable Examples for Diverse Training Paradigms
Apr 18, 2026Bo Wang, Jia Ni, Mengnan Zhao +2Adversarial Examples Privacy-Preserving ML
School of Information and Communication Engineering, Dalian University of Technology, Dalian 116024, China · School of Computer Science and Technology, Anhui University, Hefei 230601, China · School of Computer Science and Technology, Zhejiang University, Hangzhou 310058, China
Dual-branch Robust Unlearnable Examples
May 3, 2026Xianlong Wang, Hangtao Zhang, Wenbo Pan +4Adversarial Robustness Adversarial Examples
Department of Computer Science, City University of Hong Kong, Hong Kong SAR, China · University of Pennsylvania, Philadelphia, USA · Huazhong University of Science and Technology, Wuhan, China +2
Full spectrum Unlearnable Examples via Spectral Equalization
Jun 25, 2026Jiale Cai, Gezheng Xu, Zhihao Li +6Adversarial Examples Adversarial Perturbations
Department of Computer Science, Western University, London, ON, Canada · School of Statistics and Data Science, Southeast University, Nanjing, China · Department of Mechanical, Industrial and Aerospace Engineering, Concordia University +2

Apr 18, 2026J/K move · Enter open · S save

Channel-Level Semantic Perturbations: Unlearnable Examples for Diverse Training Paradigms

Bo Wang, Jia Ni, Mengnan Zhao, Zhan Qin+1

School of Information and Communication Engineering, Dalian University of Technology, Dalian 116024, China · School of Computer Science and Technology, Anhui University, Hefei 230601, China · School of Computer Science and Technology, Zhejiang University, Hangzhou 310058, China

Adversarial Examples Privacy-Preserving ML Adversarial Perturbations Unlearnable Examples

Open paper AskPDF arXiv

The unauthorized use of personal data in model training has emerged as a growing privacy threat. Unlearnable examples (UEs) address this issue by embedding imperceptible perturbations into benign examples to obstruct feature learning. However, existing studies mainly evaluate UEs under from-scratch training settings, leaving their behavior under the widely adopted pretraining-finetuning (PF) paradigm largely unexplored. In this work, we provide the first systematic investigation of unlearnable examples across diverse training paradigms. Our analysis reveals that loading and freezing pretrained weights significantly weakens the effectiveness of existing UEs methods. We further explain these findings through semantic filtering: while UEs tend to induce models to overfit non-semantic noise, thereby weakening their semantic extraction capabilities, under the PF paradigm, frozen shallow layers preserve data semantics, effectively filtering out distracting information like unlearnable noise. Guided by these insights, we propose a hierarchical deception strategy, Shallow Semantic Camouflage (SSC), that confines the generation process to a semantically valid subspace, aiming to bypass the semantic suppression introduced by pretrained weights. Extensive experiments demonstrate that our method consistently preserves data unlearnability even under challenging training paradigms, such as shallow-layer freezing and semantic-focused pretraining (SF-Pretrain), bridging the critical gap in pretrain-based unlearnable learning.

Abstract

Unlearnable Examples

Explore similar work

Channel-Level Semantic Perturbations: Unlearnable Examples for Diverse Training Paradigms

Dual-branch Robust Unlearnable Examples

Full spectrum Unlearnable Examples via Spectral Equalization