Learning to Unlearn: Machine Unlearning via Learning the Unlearning Behaviors

Hang Zhang; Kaifeng Zhang; Yixiao Ma; Weijie Xu; Ye Zhu; Kai Ming Ting

cs.LGAug 17, 2026

Learning to Unlearn: Machine Unlearning via Learning the Unlearning Behaviors

Authors: Hang Zhang, Kaifeng Zhang, Yixiao Ma, Weijie Xu, Ye Zhu, Kai Ming Ting

Organizations: State Key Laboratory for Novel Software Technology Nanjing University Nanjing, China · Centre for Cyber Resilience and Trust Deakin University Burwood, VIC, Australia

Abstract

Various machine unlearning techniques have been developed in response to privacy legislation requirements, enabling individuals to exercise their legal right to have their data $D_f$ removed from a machine learning model. This process is typically accomplished via the use of an unlearning function denoted as $U$ . Existing methods focus on designing an intricate $U$ to unlearn $D_f \subset D$ from a previous model $A(D)$ , so that the unlearned model performs as closely as possible to the retrained model $A(D \setminus D_f)$ . However, these methods often suffer from high computational costs when dealing with massive training data, as the complex structures of $U$ become a bottleneck even for models with fewer parameters. Inspired by Learning to Optimize, we introduce the first learning-based model-agnostic approach, Learning-to-UnLearn (L2UL). Our core insight is to shift from manually designing $U$ to learning the unlearning behaviors from a distribution perspective, thereby acquiring a simple and efficient $U$ via learning. Our experimental results demonstrate that the accuracy achieved by L2UL is comparable to that of retraining while exhibiting impressive efficiency, particularly in data-intensive scenarios. Furthermore, we validate the performance and scalability of our method on larger models ResNet.

Figures & tables

Symbols	Definition
$\mathcal{X}$	Sample space
$\mathcal{Y}$	Label space
$D$	Training set
$D_{f}$	Forget set
$A(\cdot)$	Learning algorithm
$U(\cdot)$	Unlearning algorithm

Table 1. Key notations used in the paper.

Figure 1. The Machine unlearning workflow and the framework of our proposed method L2UL. The model $A(D)$ is learned on dataset $D$ (which includes $D_{f}$ ) by $A$ . An unlearning algorithm $U$ unlearns $D_{f}$ from $A(D)$ to obtain a revised model, which is expected to be equal to the retrained model $A(D\setminus D_{f})$ .

	Magic	Adult	Sepsus	Skin	Covetype	SUSY	HIGGS
n	19020	32562	40328	245057	581012	5000000	11000000
d	10	14	89	3	54	18	28

Table 2. Dataset Summary. n=no.instances, d=no.attributes.

Panel of Figure 2 — Figure 2. Results of unlearning Logistic Regression.

Panel of Figure 3 — Figure 3. Results of unlearning MLP ( $|D_{f}|=1$ ).

Panel of Figure 4 — Figure 4. Results of unlearning MLP ( $|D_{f}|=100$ ).

Panel of Figure 5 — Figure 5. Results of unlearning MLP ( $|D_{f}|=1000$ ).

	MIA			Accuracy on $D_{f}$
Datasets	Retrain	FYEMU	L2UL	Retrain	SISA	FYEMU	L2UL
Magic	0.79	0.65	0.75	0.85	0.79	0.71	0.78
Adult	0.80	0.68	0.74	0.81	0.78	0.75	0.76
Skin	1.00	1.00	1.00	1.00	1.00	1.00	1.00
Covetype	0.73	0.68	0.67	0.82	0.76	0.76	0.71
SUSY	0.72	0.69	0.69	0.73	0.71	0.70	0.71

Table 3. Results in terms of Membership Inference Attack and accuracy on unlearn data.

Panel of Figure 6 — Figure 6. Artificial dataset with clean data and contaminated data, and the results of LR and MLP.

Panel of Figure 7 — Figure 7. Parameter sensitivity analysis of $\psi$ (top), $s$ (middle), $m$ (bottom) on SUSY (left) and HIGGS (right) for LR.

Panel of Figure 8 — Figure 8. Parameter sensitivity analysis of $\psi$ (top), $s$ (middle), $m$ (bottom) on SUSY (left) and HIGGS (right) for MLP.

$U$	$D$ -or	$D_{f}$ -or	$A(D)$ -or	L2UL
Factor	$A\&D$	$D_{f}$	$D_{f}\&\ \theta$	$D_{f}$

Table 4. Factor of time complexity (

D

-or:

D

-oriented,

D_{f}

-or:

D_{f}

-oriented,

A(D)

-or:

A(D)

-oriented).

		10	100	1000	3000
ToW( $\uparrow$ )	Retrain	1.00	1.00	1.00	1.00
	ADV+IMP	0.32	0.13	0.01	0.01
	RUM	0.60	0.80	0.83	0.83
	L2UL(Ours)	0.80	1.00	0.88	0.99
MIA gap( $\downarrow$ )	Retrain	0.00	0.00	0.00	0.00
	ADV+IMP	0.30	0.13	0.17	0.14

Table 5. Results of unlearning ResNet-18 on CIFAR-10 dataset.

Machine Unlearning

Machine unlearning removes the influence of selected data or concepts from trained models while preserving useful behavior and avoiding full retraining.

See all 351 papers

Jul 7, 2025

Efficient Unlearning with Privacy Guarantees

Privacy protection laws, such as the GDPR, grant individuals the right to request the forgetting of their personal data not only from databases but also from machine learning (ML) models trained on them. Machine unlearning has emerged as a practical means to facilitate model forgetting of data instances seen during training. Although some existing machine unlearning methods guarantee exact forgetting, they are typically costly in computational terms. On the other hand, more affordable methods do not offer forgetting guarantees and are applicable only to specific ML models. In this paper, we present efficient unlearning with privacy guarantees (EUPG), a novel machine unlearning framework that offers formal privacy guarantees to individuals whose data are being unlearned. EUPG involves pre-training ML models on data protected using privacy models, and it enables {\em efficient unlearning with the privacy guarantees offered by the privacy models in use}. Through empirical evaluation on four heterogeneous data sets protected with

k

-anonymity and

ε

-differential privacy as privacy models, our approach demonstrates utility and forgetting effectiveness comparable to those of exact unlearning methods, while significantly reducing computational and storage costs. Our code is available at https://github.com/najeebjebreel/EUPG.

Josep Domingo-Ferrer, Najeeb Jebreel, David Sánchez

1CYBERCAT (Center for Cybersecurity Research of2026 Catalonia), ComSCIAM-Center for Computational Science and Applied Mathematics, Department of Computer EngineeringJun and Mathematics, Universitat Rovira i Virgili, Av. Pa¨ısos26 Catalans 26, E-43007 Tarragona, Catalonia

Privacy-Preserving ML Machine Unlearning

Ask PDF

Apr 7, 2026

Efficient machine unlearning with minimax optimality

There is a growing demand for efficient data removal to comply with regulations like the GDPR and to mitigate the influence of biased or corrupted data. This has motivated the field of machine unlearning, which aims to eliminate the influence of specific data subsets without the cost of full retraining. In this work, we propose a statistical framework for machine unlearning with generic loss functions and establish theoretical guarantees. For squared loss, especially, we develop Unlearning Least Squares (ULS) and establish its minimax optimality for estimating the model parameter of remaining data when only the pre-trained estimator, forget samples, and a small subsample of the remaining data are available. Our results reveal that the estimation error decomposes into an oracle term and an unlearning cost determined by the forget proportion and the forget model bias. We further establish asymptotically valid inference procedures without requiring full retraining. Numerical experiments and real-data applications demonstrate that the proposed method achieves performance close to retraining while requiring substantially less data access.

Jingyi Xie, Linjun Zhang, Sai Li

Institute of Statistics and Big Data, Renmin University of China, Beijing 100872, China · Department of Statistics, Rutgers University, Piscataway, NJ 08854, USA · Department of Statistics and Data Science, Tsinghua University, Beijing 100084, China

Machine Unlearning Statistical Learning Theory Minimax Estimation

Ask PDF

Jun 15, 2026

Auditing Machine Unlearning: A Systematic Research on Whether Models Truly Forget

Machine unlearning has been extensively studied in response to growing privacy concerns and regulatory requirements. However, auditing whether unlearning algorithms have truly erased the influence of specific data remains an open challenge. The lack of reliable and practical auditing mechanisms can lead to critical privacy risks, such as residual information leakage. This paper initiates a systematic investigation into whether existing unlearning algorithms can truly forget the designated data. We propose the first practical and general-purpose auditing framework for machine unlearning, inspired by the concept of proof of ignorance. Our framework addresses the key practicality limitations of existing methods by eliminating the need for retraining-from-scratch baselines, avoiding the training of large numbers of shadow models, and requiring no intrusive intervention in the original training process. To evaluate the effectiveness of our framework, we first conduct validation experiments to verify its soundness and completeness. We then perform comprehensive experiments across six datasets and ten representative unlearning methods. The results demonstrate that our framework reliably distinguishes between successful and failed unlearning. In particular, we observe that retraining-based and fine-tuning-based methods can achieve effective unlearning, even when the target data remain in the original dataset. In contrast, de-optimization-based methods fail to achieve true unlearning and instead degrade the model's performance. Fisher/Hessian-based methods also fail to unlearn requested data, even formal certification is provided. Moreover, we show that our framework is robust against fake unlearning attempts and generalizes well to large language models.

Dayong Ye, Tianqing Zhu, Ruiding Huang +5

Machine Unlearning LLM Unlearning Machine Unlearning Evaluation

Ask PDF

Abstract

Machine Unlearning

Explore similar work

Efficient Unlearning with Privacy Guarantees

Efficient machine unlearning with minimax optimality

Auditing Machine Unlearning: A Systematic Research on Whether Models Truly Forget