Learning to Unlearn: Machine Unlearning via Learning the Unlearning Behaviors

Hang Zhang; Kaifeng Zhang; Yixiao Ma; Weijie Xu; Ye Zhu; Kai Ming Ting

cs.LGAug 17, 2026

Learning to Unlearn: Machine Unlearning via Learning the Unlearning Behaviors

Authors: Hang Zhang, Kaifeng Zhang, Yixiao Ma, Weijie Xu, Ye Zhu, Kai Ming Ting

Organizations: State Key Laboratory for Novel Software Technology Nanjing University Nanjing, China · Centre for Cyber Resilience and Trust Deakin University Burwood, VIC, Australia

Abstract

Various machine unlearning techniques have been developed in response to privacy legislation requirements, enabling individuals to exercise their legal right to have their data $D_f$ removed from a machine learning model. This process is typically accomplished via the use of an unlearning function denoted as $U$ . Existing methods focus on designing an intricate $U$ to unlearn $D_f \subset D$ from a previous model $A(D)$ , so that the unlearned model performs as closely as possible to the retrained model $A(D \setminus D_f)$ . However, these methods often suffer from high computational costs when dealing with massive training data, as the complex structures of $U$ become a bottleneck even for models with fewer parameters. Inspired by Learning to Optimize, we introduce the first learning-based model-agnostic approach, Learning-to-UnLearn (L2UL). Our core insight is to shift from manually designing $U$ to learning the unlearning behaviors from a distribution perspective, thereby acquiring a simple and efficient $U$ via learning. Our experimental results demonstrate that the accuracy achieved by L2UL is comparable to that of retraining while exhibiting impressive efficiency, particularly in data-intensive scenarios. Furthermore, we validate the performance and scalability of our method on larger models ResNet.

Figures & tables

Symbols	Definition
$\mathcal{X}$	Sample space
$\mathcal{Y}$	Label space
$D$	Training set
$D_{f}$	Forget set
$A(\cdot)$	Learning algorithm
$U(\cdot)$	Unlearning algorithm

Table 1. Key notations used in the paper.

Figure 1. The Machine unlearning workflow and the framework of our proposed method L2UL. The model $A(D)$ is learned on dataset $D$ (which includes $D_{f}$ ) by $A$ . An unlearning algorithm $U$ unlearns $D_{f}$ from $A(D)$ to obtain a revised model, which is expected to be equal to the retrained model $A(D\setminus D_{f})$ .

	Magic	Adult	Sepsus	Skin	Covetype	SUSY	HIGGS
n	19020	32562	40328	245057	581012	5000000	11000000
d	10	14	89	3	54	18	28

Table 2. Dataset Summary. n=no.instances, d=no.attributes.

Panel of Figure 2 — Figure 2. Results of unlearning Logistic Regression.

Panel of Figure 3 — Figure 3. Results of unlearning MLP ( $|D_{f}|=1$ ).

Panel of Figure 4 — Figure 4. Results of unlearning MLP ( $|D_{f}|=100$ ).

Panel of Figure 5 — Figure 5. Results of unlearning MLP ( $|D_{f}|=1000$ ).

	MIA			Accuracy on $D_{f}$
Datasets	Retrain	FYEMU	L2UL	Retrain	SISA	FYEMU	L2UL
Magic	0.79	0.65	0.75	0.85	0.79	0.71	0.78
Adult	0.80	0.68	0.74	0.81	0.78	0.75	0.76
Skin	1.00	1.00	1.00	1.00	1.00	1.00	1.00
Covetype	0.73	0.68	0.67	0.82	0.76	0.76	0.71
SUSY	0.72	0.69	0.69	0.73	0.71	0.70	0.71

Table 3. Results in terms of Membership Inference Attack and accuracy on unlearn data.

Panel of Figure 6 — Figure 6. Artificial dataset with clean data and contaminated data, and the results of LR and MLP.

Panel of Figure 7 — Figure 7. Parameter sensitivity analysis of $\psi$ (top), $s$ (middle), $m$ (bottom) on SUSY (left) and HIGGS (right) for LR.

Panel of Figure 8 — Figure 8. Parameter sensitivity analysis of $\psi$ (top), $s$ (middle), $m$ (bottom) on SUSY (left) and HIGGS (right) for MLP.

$U$	$D$ -or	$D_{f}$ -or	$A(D)$ -or	L2UL
Factor	$A\&D$	$D_{f}$	$D_{f}\&\ \theta$	$D_{f}$

Table 4. Factor of time complexity (

D

-or:

D

-oriented,

D_{f}

-or:

D_{f}

-oriented,

A(D)

-or:

A(D)

-oriented).

		10	100	1000	3000
ToW( $\uparrow$ )	Retrain	1.00	1.00	1.00	1.00
	ADV+IMP	0.32	0.13	0.01	0.01
	RUM	0.60	0.80	0.83	0.83
	L2UL(Ours)	0.80	1.00	0.88	0.99
MIA gap( $\downarrow$ )	Retrain	0.00	0.00	0.00	0.00
	ADV+IMP	0.30	0.13	0.17	0.14

Table 5. Results of unlearning ResNet-18 on CIFAR-10 dataset.

Machine Unlearning

Machine unlearning removes the influence of selected data or concepts from trained models while preserving useful behavior and avoiding full retraining.

See all 349 papers

Efficient Unlearning with Privacy Guarantees
Jul 7, 2025Josep Domingo-Ferrer, Najeeb Jebreel, David SánchezPrivacy-Preserving ML Machine Unlearning
1CYBERCAT (Center for Cybersecurity Research of2026 Catalonia), ComSCIAM-Center for Computational Science and Applied Mathematics, Department of Computer EngineeringJun and Mathematics, Universitat Rovira i Virgili, Av. Pa¨ısos26 Catalans 26, E-43007 Tarragona, Catalonia
Efficient machine unlearning with minimax optimality
Apr 7, 2026Jingyi Xie, Linjun Zhang, Sai LiMachine Unlearning Statistical Learning Theory
Institute of Statistics and Big Data, Renmin University of China, Beijing 100872, China · Department of Statistics, Rutgers University, Piscataway, NJ 08854, USA · Department of Statistics and Data Science, Tsinghua University, Beijing 100084, China
Auditing Machine Unlearning: A Systematic Research on Whether Models Truly Forget
Jun 15, 2026Dayong Ye, Tianqing Zhu, Ruiding Huang +5Machine Unlearning LLM Unlearning

Jul 7, 2025J/K move · Enter open · S save

Efficient Unlearning with Privacy Guarantees

Josep Domingo-Ferrer, Najeeb Jebreel, David Sánchez

1CYBERCAT (Center for Cybersecurity Research of2026 Catalonia), ComSCIAM-Center for Computational Science and Applied Mathematics, Department of Computer EngineeringJun and Mathematics, Universitat Rovira i Virgili, Av. Pa¨ısos26 Catalans 26, E-43007 Tarragona, Catalonia

Privacy-Preserving ML Machine Unlearning

Open paper AskPDF arXiv

Privacy protection laws, such as the GDPR, grant individuals the right to request the forgetting of their personal data not only from databases but also from machine learning (ML) models trained on them. Machine unlearning has emerged as a practical means to facilitate model forgetting of data instances seen during training. Although some existing machine unlearning methods guarantee exact forgetting, they are typically costly in computational terms. On the other hand, more affordable methods do not offer forgetting guarantees and are applicable only to specific ML models. In this paper, we present efficient unlearning with privacy guarantees (EUPG), a novel machine unlearning framework that offers formal privacy guarantees to individuals whose data are being unlearned. EUPG involves pre-training ML models on data protected using privacy models, and it enables {\em efficient unlearning with the privacy guarantees offered by the privacy models in use}. Through empirical evaluation on four heterogeneous data sets protected with

k

-anonymity and

ε

-differential privacy as privacy models, our approach demonstrates utility and forgetting effectiveness comparable to those of exact unlearning methods, while significantly reducing computational and storage costs. Our code is available at https://github.com/najeebjebreel/EUPG.

Abstract

Machine Unlearning

Explore similar work

Efficient Unlearning with Privacy Guarantees

Efficient machine unlearning with minimax optimality

Auditing Machine Unlearning: A Systematic Research on Whether Models Truly Forget