Approximate Unlearning in Prototype-based-classifiers

Sreejita Ghosh; Mohammad Mohammadi

cs.LGOct 25, 2026

Approximate Unlearning in Prototype-based-classifiers

Authors: Sreejita Ghosh, Mohammad Mohammadi

Organizations: UMC Groningen, Groningen, The Netherlands · Technical University of Eindhoven, Eindhoven, The Netherlands · Tilburg University, Tilburg, The Netherlands

Abstract

The recent data and AI regulations and governance needs have highlighted the need for removing specific data even from models trained on or with them. While retraining the entire model is a theoretically trivial solution, it is computationally expensive, thereby motivating research in Machine Unlearning. This paper introduces a robust approximate unlearning technique for the family of prototype-based classifiers called Generalized Learning Vector Quantization, aimed at removal of effects of specified samples, either due to deletion request or due to them being identified as noisy or corrupt. We first show the impact of the different sizes of the forget set, as well as the characteristics of the data in this set, on the unlearned model. To address the sources of instabilities detected in this unlearning method we propose an adapted version of this technique, such that the unlearning is robust to the characteristics of the forget set. Thereafter using the intrinsic interpretability property of GLVQ and appropriate metrics defined in unlearning literature, we show the efficacy (approximately 0 deviation of parameters and performance of unlearned model relative to retrained model) and efficiency (6X for smallest dataset to 10^(5)X in the largest dataset) of the proposed unlearning techniques compared to naive retrained, on six benchmark datasets for tabular data recommended by literature. Although in this work we investigated only binary class problems, the proposed robust unlearning strategy can be readily applied for removing sample effects from any trained GLVQ model, i.e., it does not require any modification in the training scheme of GLVQ.

Figures & tables

Figure 1: Unlearning vs retraining a model within the ML lifecycle.

Figure 2: Preventing model deterioration in unlearning through ‘enforce’ samples (

\mathcal{D}^{e}

Dataset	$\mathbf{N_{training}}$	$\mathbf{N_{test}}$	Class imbalance	$\#_{features}$	samples in $\mathbf{\mathcal{D}^{f}}$
Breast cancer	381	188	145/236 ( $\approx 0.614$ )	30	[2, 5, 20, 30, 50, 100]
Surgical	11708	2927	3000/8708 ( $\approx 0.34$ )	18	[2, 12, 118, 586, 1171, 2342]
Adult	32561	16281	7841/24720 ( $\approx 0.31$ )	10	[4, 33, 326, 1629, 3257, 6513]
Banking	32950	8238	3705/29245 ( $\approx 0.13$ )	18	[4, 33, 330, 1648, 3295, 6590]
Diabetes	81412	20354	37500/43912 ( $\approx 0.85$ )	38	[9, 82, 815, 4071, 8142, 16283]
Criteo	11463263	2516328	538681/10924582 ( $\approx 0.05$ )	13	[1147, 114633, 1146327]

Table 1: Characteristics of the benchmark datasets in this paper

Panel of Figure 3 — Figure 3: Comparison of the unlearned and retrained models in terms of speed-up and relative deviation from original model.

Panel of Figure 4 — Figure 4: Comparison of the retrained model and unlearning-with enforcer in terms of speed-up and relative deviation from original model.

Panel of Figure 5 — Figure 5: Impact on unlearning vs retraining on their respective prototypes, and their distribution with respect to the ‘remembered’ and ‘forget’ sets of data.

Panel of Figure 6 — Figure 6: Impact of location of samples in $\mathcal{D}^{f}$ on unlearning( $\mathcal{M}_{0}\rightarrow\mathcal{M}_{2}$ ), without and with ‘enforcer’ set (the hexagon represents the ‘enforcer’ sample.).

Panel of Figure 7 — Figure 7: Performance of the retrained and unlearned models on the samples from $\mathcal{D}^{r}$ , test set, and $\mathcal{D}^{f}$ of the ‘Breast cancer’ dataset, in terms of predictions retained, lost and gained with respect to the original trained model.

ppc	N( $\mathcal{D}^{f}$ )	Subset	Model	% Bal. Acc	% Gained	% Lost	% Retained
1	2	retraining	M1	95.81 (0.16)	0.0 (0.0)	0.11 (0.15)	99.89 (0.15)
			M2	95.88 (0.21)	0.0 (0.0)	0.05 (0.12)	99.95 (0.12)
		test	M1	95.03 (0.0)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
			M2	95.03 (0.0)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
		unlearned	M1	80.0 (27.39)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
			M2	80.0 (27.39)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)

Table 2: Comparison performance of the retrained and unlearned models across different batch sizes of

\mathcal{D}^{f}

(N(

\mathcal{D}^{f}

)) in the ‘Breast cancer’ dataset, reported as mean (std) over 5 iterations.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

	Breast cancer	Surgical	Banking	Adult	Diabetes	Criteo
$\beta_{ppc=1}$	5	15	18	15	10	10
$\beta_{ppc=2}$	5	5	20	10	10	10
$\alpha$	0.05	0.05	0.05	0.05	0.05	0.075
max_runs	5	5	5	5	5	3

Appendix

Table A1 :

\beta

in the Swish activation function and learning rate (

alpha

) and max runs in Steepest Gradient descent across different datasets.

ppc	N( $\mathcal{D}^{f}$ )	Subset	Model	% Bal. Acc	% Gained	% Lost	% Retained
1	2	retraining	M1	67.91 (0.07)	0.24 (0.06)	0.18 (0.03)	99.82 (0.03)
			M2	67.9 (0.03)	0.01 (0.01)	0.03 (0.02)	99.97 (0.02)
		test	M1	67.62 (0.2)	0.3 (0.11)	0.16 (0.14)	99.84 (0.14)
			M2	67.55 (0.02)	0.03 (0.02)	0.03 (0.02)	99.97 (0.02)
		unlearned	M1	80.0 (27.39)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
			M2	80.0 (27.39)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)

Appendix

Table A2 : Comparison performance of the retrained and unlearned models across different batch sizes of

\mathcal{D}^{f}

(N(

\mathcal{D}^{f}

)) in the ‘Surgical’ dataset, reported as mean (std) over 5 iterations.

ppc	N( $\mathcal{D}^{f}$ )	Subset	Model	% Bal. Acc	% Gained	% Lost	% Retained
1	4	retraining	M1	62.49 (0.23)	0.13 (0.03)	0.1 (0.04)	99.9 (0.04)
			M2	62.43 (0.04)	0.01 (0.01)	0.01 (0.01)	99.99 (0.01)
		test	M1	61.85 (0.25)	0.15 (0.04)	0.16 (0.06)	99.84 (0.06)
			M2	61.81 (0.03)	0.01 (0.01)	0.02 (0.01)	99.98 (0.01)
		unlearned	M1	80.0 (27.39)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
			M2	80.0 (27.39)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)

Appendix

Table A3 : Comparison of performance of the retrained and unlearned models across different batch sizes of

\mathcal{D}^{f}

(N(

\mathcal{D}^{f}

)) in the ‘Banking’ dataset, reported as mean (std) over 5 iterations.

ppc	N( $\mathcal{D}^{f}$ )	Subset	Model	% Bal. Acc	% Gained	% Lost	% Retained
1	4	retraining	M1	65.38 (0.7)	0.67 (0.14)	0.9 (0.12)	99.1 (0.12)
			M2	66.04 (0.17)	0.1 (0.06)	0.14 (0.05)	99.86 (0.05)
		test	M1	64.79 (0.45)	0.73 (0.13)	0.86 (0.15)	99.14 (0.15)
			M2	65.34 (0.15)	0.1 (0.06)	0.17 (0.06)	99.83 (0.06)
		unlearned	M1	66.67 (23.57)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
			M2	66.67 (23.57)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)

Appendix

Table A4 : Comparison of performance of the retrained and unlearned models across different batch sizes of

\mathcal{D}^{f}

(N(

\mathcal{D}^{f}

)) in the ‘Adult’ dataset, reported as mean (std) over 5 iterations.

Panel of Figure 0 — Figure 0.D.1 : Performances on the Banking, Adult, Diabetes and Criteo datasets.

ppc	N( $\mathcal{D}^{f}$ )	Subset	Model	% Bal. Acc	% Gained	% Lost	% Retained
1	2	retraining	M1	95.86 (0.1)	0.0 (0.0)	0.05 (0.12)	99.95 (0.12)
			M2	95.93 (0.15)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
		test	M1	95.03 (0.0)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
			M2	95.03 (0.0)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
		unlearned	M1	80.0 (27.39)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
			M2	80.0 (27.39)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)

Appendix

Table A5 : Comparison of performance of the retrained and unlearned (with enforcer) models across different batch sizes of

\mathcal{D}^{f}

(N(

\mathcal{D}^{f}

)) in the ‘Breast cancer’ dataset, reported as mean (std) over 5 iterations.

ppc	N( $\mathcal{D}^{f}$ )	Subset	Model	% Bal. Acc	% Gained	% Lost	% Retained
1	2	retraining	M1	67.93 (0.06)	0.23 (0.05)	0.19 (0.03)	99.81 (0.03)
			M2	67.94 (0.01)	0.01 (0.02)	0.01 (0.01)	99.99 (0.01)
		test	M1	67.62 (0.18)	0.29 (0.1)	0.16 (0.14)	99.84 (0.14)
			M2	67.55 (0.02)	0.02 (0.02)	0.03 (0.02)	99.97 (0.02)
		unlearned	M1	80.0 (27.39)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
			M2	80.0 (27.39)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)

Appendix

Table A6 : Comparison of performance of the retrained and unlearned (with enforcer) models across different batch sizes of

\mathcal{D}^{f}

(N(

\mathcal{D}^{f}

)) in the ‘Surgical’ dataset, reported as mean (std) over 5 iterations.

ppc	N( $\mathcal{D}^{f}$ )	Subset	Model	% Bal. Acc	% Gained	% Lost	% Retained
1	2	retraining	M1	67.93 (0.06)	0.23 (0.05)	0.19 (0.03)	99.81 (0.03)
			M2	67.94 (0.01)	0.01 (0.02)	0.01 (0.01)	99.99 (0.01)
		test	M1	67.62 (0.18)	0.29 (0.1)	0.16 (0.14)	99.84 (0.14)
			M2	67.55 (0.02)	0.02 (0.02)	0.03 (0.02)	99.97 (0.02)
		unlearned	M1	80.0 (27.39)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
			M2	80.0 (27.39)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)

Appendix

Table A7 : Comparison of performance of the retrained and unlearned (with enforcer) models across different batch sizes of

\mathcal{D}^{f}

(N(

\mathcal{D}^{f}

)) in the ‘Banking’ dataset, reported as mean (std) over 5 iterations.

ppc	N( $\mathcal{D}^{f}$ )	Subset	Model	% Bal. Acc	% Gained	% Lost	% Retained
1	4	retraining	M1	65.37 (0.69)	0.67 (0.14)	0.9 (0.12)	99.1 (0.12)
			M2	66.16 (2.68)	2.3 (1.56)	3.46 (3.1)	96.54 (3.1)
		test	M1	64.79 (0.46)	0.73 (0.14)	0.86 (0.15)	99.14 (0.15)
			M2	65.52 (2.4)	2.29 (1.62)	3.49 (3.04)	96.51 (3.04)
		unlearned	M1	66.67 (23.57)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
			M2	66.67 (23.57)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)

Appendix

Table A8 : Comparison of performance of the retrained and unlearned (with enforcer) models across different batch sizes of

\mathcal{D}^{f}

(N(

\mathcal{D}^{f}

)) in the ‘Adult’ dataset, reported as mean (std) over 5 iterations.

ppc	N( $\mathcal{D}^{f}$ )	Subset	Model	% Bal. Acc	% Gained	% Lost	% Retained
1	9	retraining	M1	61.68 (0.08)	1.44 (0.09)	1.33 (0.19)	98.67 (0.19)
			M2	61.59 (0.0)	0.01 (0.0)	0.01 (0.01)	99.99 (0.01)
		test	M1	61.73 (0.05)	1.42 (0.1)	1.26 (0.16)	98.74 (0.16)
			M2	61.62 (0.01)	0.02 (0.01)	0.01 (0.01)	99.99 (0.01)
		unlearned	M1	62.12 (19.14)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
			M2	62.12 (19.14)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)

Appendix

Table A9 : Comparison of performance of the retrained and unlearned (with enforcer) models across different batch sizes of

\mathcal{D}^{f}

(N(

\mathcal{D}^{f}

)) in the ‘Diabetes’ dataset, reported as mean (std) over 5 iterations.

ppc	N( $\mathcal{D}^{f}$ )	Subset	Model	% Bal. Acc	% Gained	% Lost	% Retained
1	1147	retraining	M1	55.41 (0.35)	0.04 (0.02)	0.06 (0.03)	99.94 (0.03)
			M2	55.95 (0.01)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
		test	M1	55.47 (0.36)	0.04 (0.02)	0.06 (0.03)	99.94 (0.03)
			M2	56.0 (0.01)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
		unlearned	M1	52.86 (0.91)	0.03 (0.05)	0.06 (0.1)	99.94 (0.1)
			M2	53.65 (1.28)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)

Appendix

Table A10 : Comparison of performance of the retrained and unlearned (with enforcer) models across different batch sizes of

\mathcal{D}^{f}

(N(

\mathcal{D}^{f}

)) in the ‘Criteo’ dataset, reported as mean (std) over 5 iterations.

Machine Unlearning

Machine unlearning removes the influence of selected data or concepts from trained models while preserving useful behavior and avoiding full retraining.

See all 356 papers

May 11, 2026

Behavioral Guarantees for Proxy-Based Unlearning

This paper proposes a framework generalizing recent proxy-based unlearning methods and proves theoretical guarantees about the behavior of the resulting unlearned model: upper bounds on its Kullback-Leibler divergence to the ideal posterior distribution of the retain data. We model approximate unlearning as a constrained optimization problem and interpret a family of solutions as introducing a scaled unlearning signal in the output space. The unlearning signal arises from proxies of the posterior data distributions. Its scale is adapted to the proxies to ensure the behavioral upper bounds. This framework relies on the structure of the data distributions in order to create proxies. If need be, the target serves as a teacher to distill the update in the weights. Our approach is experimentally validated over two forgetting scenarios as reaching the closest classifier to the model retrained from scratch.

Virgile Dine, Teddy Furon

Centre Inria de l’Université de Rennes France

Machine Unlearning

Ask PDF

Apr 7, 2026

Efficient machine unlearning with minimax optimality

There is a growing demand for efficient data removal to comply with regulations like the GDPR and to mitigate the influence of biased or corrupted data. This has motivated the field of machine unlearning, which aims to eliminate the influence of specific data subsets without the cost of full retraining. In this work, we propose a statistical framework for machine unlearning with generic loss functions and establish theoretical guarantees. For squared loss, especially, we develop Unlearning Least Squares (ULS) and establish its minimax optimality for estimating the model parameter of remaining data when only the pre-trained estimator, forget samples, and a small subsample of the remaining data are available. Our results reveal that the estimation error decomposes into an oracle term and an unlearning cost determined by the forget proportion and the forget model bias. We further establish asymptotically valid inference procedures without requiring full retraining. Numerical experiments and real-data applications demonstrate that the proposed method achieves performance close to retraining while requiring substantially less data access.

Jingyi Xie, Linjun Zhang, Sai Li

Institute of Statistics and Big Data, Renmin University of China, Beijing 100872, China · Department of Statistics, Rutgers University, Piscataway, NJ 08854, USA · Department of Statistics and Data Science, Tsinghua University, Beijing 100084, China

Machine Unlearning Statistical Learning Theory Minimax Estimation

Ask PDF

Aug 17, 2026

Learning to Unlearn: Machine Unlearning via Learning the Unlearning Behaviors

Various machine unlearning techniques have been developed in response to privacy legislation requirements, enabling individuals to exercise their legal right to have their data

D_f

removed from a machine learning model. This process is typically accomplished via the use of an unlearning function denoted as

U

. Existing methods focus on designing an intricate

U

to unlearn

D_f \subset D

from a previous model

A(D)

, so that the unlearned model performs as closely as possible to the retrained model

A(D \setminus D_f)

. However, these methods often suffer from high computational costs when dealing with massive training data, as the complex structures of

U

become a bottleneck even for models with fewer parameters. Inspired by Learning to Optimize, we introduce the first learning-based model-agnostic approach, Learning-to-UnLearn (L2UL). Our core insight is to shift from manually designing

U

to learning the unlearning behaviors from a distribution perspective, thereby acquiring a simple and efficient

U

via learning. Our experimental results demonstrate that the accuracy achieved by L2UL is comparable to that of retraining while exhibiting impressive efficiency, particularly in data-intensive scenarios. Furthermore, we validate the performance and scalability of our method on larger models ResNet.

Hang Zhang, Kaifeng Zhang, Yixiao Ma +3

State Key Laboratory for Novel Software Technology Nanjing University Nanjing, China · Centre for Cyber Resilience and Trust Deakin University Burwood, VIC, Australia

Machine Unlearning Privacy-Preserving ML

Ask PDF

Abstract

Machine Unlearning

Explore similar work

Behavioral Guarantees for Proxy-Based Unlearning

Efficient machine unlearning with minimax optimality

Learning to Unlearn: Machine Unlearning via Learning the Unlearning Behaviors