Approximate Unlearning in Prototype-based-classifiers

Sreejita Ghosh; Mohammad Mohammadi

cs.LGOct 25, 2026

Approximate Unlearning in Prototype-based-classifiers

Authors: Sreejita Ghosh, Mohammad Mohammadi

Organizations: UMC Groningen, Groningen, The Netherlands · Technical University of Eindhoven, Eindhoven, The Netherlands · Tilburg University, Tilburg, The Netherlands

Abstract

The recent data and AI regulations and governance needs have highlighted the need for removing specific data even from models trained on or with them. While retraining the entire model is a theoretically trivial solution, it is computationally expensive, thereby motivating research in Machine Unlearning. This paper introduces a robust approximate unlearning technique for the family of prototype-based classifiers called Generalized Learning Vector Quantization, aimed at removal of effects of specified samples, either due to deletion request or due to them being identified as noisy or corrupt. We first show the impact of the different sizes of the forget set, as well as the characteristics of the data in this set, on the unlearned model. To address the sources of instabilities detected in this unlearning method we propose an adapted version of this technique, such that the unlearning is robust to the characteristics of the forget set. Thereafter using the intrinsic interpretability property of GLVQ and appropriate metrics defined in unlearning literature, we show the efficacy (approximately 0 deviation of parameters and performance of unlearned model relative to retrained model) and efficiency (6X for smallest dataset to 10^(5)X in the largest dataset) of the proposed unlearning techniques compared to naive retrained, on six benchmark datasets for tabular data recommended by literature. Although in this work we investigated only binary class problems, the proposed robust unlearning strategy can be readily applied for removing sample effects from any trained GLVQ model, i.e., it does not require any modification in the training scheme of GLVQ.

Figures & tables

Figure 1: Unlearning vs retraining a model within the ML lifecycle.

Figure 2: Preventing model deterioration in unlearning through ‘enforce’ samples (

\mathcal{D}^{e}

Dataset	$\mathbf{N_{training}}$	$\mathbf{N_{test}}$	Class imbalance	$\#_{features}$	samples in $\mathbf{\mathcal{D}^{f}}$
Breast cancer	381	188	145/236 ( $\approx 0.614$ )	30	[2, 5, 20, 30, 50, 100]
Surgical	11708	2927	3000/8708 ( $\approx 0.34$ )	18	[2, 12, 118, 586, 1171, 2342]
Adult	32561	16281	7841/24720 ( $\approx 0.31$ )	10	[4, 33, 326, 1629, 3257, 6513]
Banking	32950	8238	3705/29245 ( $\approx 0.13$ )	18	[4, 33, 330, 1648, 3295, 6590]
Diabetes	81412	20354	37500/43912 ( $\approx 0.85$ )	38	[9, 82, 815, 4071, 8142, 16283]
Criteo	11463263	2516328	538681/10924582 ( $\approx 0.05$ )	13	[1147, 114633, 1146327]

Table 1: Characteristics of the benchmark datasets in this paper

Panel of Figure 3 — Figure 3: Comparison of the unlearned and retrained models in terms of speed-up and relative deviation from original model.

Panel of Figure 4 — Figure 4: Comparison of the retrained model and unlearning-with enforcer in terms of speed-up and relative deviation from original model.

Panel of Figure 5 — Figure 5: Impact on unlearning vs retraining on their respective prototypes, and their distribution with respect to the ‘remembered’ and ‘forget’ sets of data.

Panel of Figure 6 — Figure 6: Impact of location of samples in $\mathcal{D}^{f}$ on unlearning( $\mathcal{M}_{0}\rightarrow\mathcal{M}_{2}$ ), without and with ‘enforcer’ set (the hexagon represents the ‘enforcer’ sample.).

Panel of Figure 7 — Figure 7: Performance of the retrained and unlearned models on the samples from $\mathcal{D}^{r}$ , test set, and $\mathcal{D}^{f}$ of the ‘Breast cancer’ dataset, in terms of predictions retained, lost and gained with respect to the original trained model.

ppc	N( $\mathcal{D}^{f}$ )	Subset	Model	% Bal. Acc	% Gained	% Lost	% Retained
1	2	retraining	M1	95.81 (0.16)	0.0 (0.0)	0.11 (0.15)	99.89 (0.15)
			M2	95.88 (0.21)	0.0 (0.0)	0.05 (0.12)	99.95 (0.12)
		test	M1	95.03 (0.0)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
			M2	95.03 (0.0)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
		unlearned	M1	80.0 (27.39)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
			M2	80.0 (27.39)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)

Table 2: Comparison performance of the retrained and unlearned models across different batch sizes of

\mathcal{D}^{f}

(N(

\mathcal{D}^{f}

)) in the ‘Breast cancer’ dataset, reported as mean (std) over 5 iterations.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

	Breast cancer	Surgical	Banking	Adult	Diabetes	Criteo
$\beta_{ppc=1}$	5	15	18	15	10	10
$\beta_{ppc=2}$	5	5	20	10	10	10
$\alpha$	0.05	0.05	0.05	0.05	0.05	0.075
max_runs	5	5	5	5	5	3

Appendix

Table A1 :

\beta

in the Swish activation function and learning rate (

alpha

) and max runs in Steepest Gradient descent across different datasets.

ppc	N( $\mathcal{D}^{f}$ )	Subset	Model	% Bal. Acc	% Gained	% Lost	% Retained
1	2	retraining	M1	67.91 (0.07)	0.24 (0.06)	0.18 (0.03)	99.82 (0.03)
			M2	67.9 (0.03)	0.01 (0.01)	0.03 (0.02)	99.97 (0.02)
		test	M1	67.62 (0.2)	0.3 (0.11)	0.16 (0.14)	99.84 (0.14)
			M2	67.55 (0.02)	0.03 (0.02)	0.03 (0.02)	99.97 (0.02)
		unlearned	M1	80.0 (27.39)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
			M2	80.0 (27.39)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)

Appendix

Table A2 : Comparison performance of the retrained and unlearned models across different batch sizes of

\mathcal{D}^{f}

(N(

\mathcal{D}^{f}

)) in the ‘Surgical’ dataset, reported as mean (std) over 5 iterations.

ppc	N( $\mathcal{D}^{f}$ )	Subset	Model	% Bal. Acc	% Gained	% Lost	% Retained
1	4	retraining	M1	62.49 (0.23)	0.13 (0.03)	0.1 (0.04)	99.9 (0.04)
			M2	62.43 (0.04)	0.01 (0.01)	0.01 (0.01)	99.99 (0.01)
		test	M1	61.85 (0.25)	0.15 (0.04)	0.16 (0.06)	99.84 (0.06)
			M2	61.81 (0.03)	0.01 (0.01)	0.02 (0.01)	99.98 (0.01)
		unlearned	M1	80.0 (27.39)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
			M2	80.0 (27.39)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)

Appendix

Table A3 : Comparison of performance of the retrained and unlearned models across different batch sizes of

\mathcal{D}^{f}

(N(

\mathcal{D}^{f}

)) in the ‘Banking’ dataset, reported as mean (std) over 5 iterations.

ppc	N( $\mathcal{D}^{f}$ )	Subset	Model	% Bal. Acc	% Gained	% Lost	% Retained
1	4	retraining	M1	65.38 (0.7)	0.67 (0.14)	0.9 (0.12)	99.1 (0.12)
			M2	66.04 (0.17)	0.1 (0.06)	0.14 (0.05)	99.86 (0.05)
		test	M1	64.79 (0.45)	0.73 (0.13)	0.86 (0.15)	99.14 (0.15)
			M2	65.34 (0.15)	0.1 (0.06)	0.17 (0.06)	99.83 (0.06)
		unlearned	M1	66.67 (23.57)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
			M2	66.67 (23.57)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)

Appendix

Table A4 : Comparison of performance of the retrained and unlearned models across different batch sizes of

\mathcal{D}^{f}

(N(

\mathcal{D}^{f}

)) in the ‘Adult’ dataset, reported as mean (std) over 5 iterations.

Panel of Figure 0 — Figure 0.D.1 : Performances on the Banking, Adult, Diabetes and Criteo datasets.

ppc	N( $\mathcal{D}^{f}$ )	Subset	Model	% Bal. Acc	% Gained	% Lost	% Retained
1	2	retraining	M1	95.86 (0.1)	0.0 (0.0)	0.05 (0.12)	99.95 (0.12)
			M2	95.93 (0.15)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
		test	M1	95.03 (0.0)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
			M2	95.03 (0.0)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
		unlearned	M1	80.0 (27.39)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
			M2	80.0 (27.39)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)

Appendix

Table A5 : Comparison of performance of the retrained and unlearned (with enforcer) models across different batch sizes of

\mathcal{D}^{f}

(N(

\mathcal{D}^{f}

)) in the ‘Breast cancer’ dataset, reported as mean (std) over 5 iterations.

ppc	N( $\mathcal{D}^{f}$ )	Subset	Model	% Bal. Acc	% Gained	% Lost	% Retained
1	2	retraining	M1	67.93 (0.06)	0.23 (0.05)	0.19 (0.03)	99.81 (0.03)
			M2	67.94 (0.01)	0.01 (0.02)	0.01 (0.01)	99.99 (0.01)
		test	M1	67.62 (0.18)	0.29 (0.1)	0.16 (0.14)	99.84 (0.14)
			M2	67.55 (0.02)	0.02 (0.02)	0.03 (0.02)	99.97 (0.02)
		unlearned	M1	80.0 (27.39)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
			M2	80.0 (27.39)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)

Appendix

Table A6 : Comparison of performance of the retrained and unlearned (with enforcer) models across different batch sizes of

\mathcal{D}^{f}

(N(

\mathcal{D}^{f}

)) in the ‘Surgical’ dataset, reported as mean (std) over 5 iterations.

ppc	N( $\mathcal{D}^{f}$ )	Subset	Model	% Bal. Acc	% Gained	% Lost	% Retained
1	2	retraining	M1	67.93 (0.06)	0.23 (0.05)	0.19 (0.03)	99.81 (0.03)
			M2	67.94 (0.01)	0.01 (0.02)	0.01 (0.01)	99.99 (0.01)
		test	M1	67.62 (0.18)	0.29 (0.1)	0.16 (0.14)	99.84 (0.14)
			M2	67.55 (0.02)	0.02 (0.02)	0.03 (0.02)	99.97 (0.02)
		unlearned	M1	80.0 (27.39)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
			M2	80.0 (27.39)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)

Appendix

Table A7 : Comparison of performance of the retrained and unlearned (with enforcer) models across different batch sizes of

\mathcal{D}^{f}

(N(

\mathcal{D}^{f}

)) in the ‘Banking’ dataset, reported as mean (std) over 5 iterations.

ppc	N( $\mathcal{D}^{f}$ )	Subset	Model	% Bal. Acc	% Gained	% Lost	% Retained
1	4	retraining	M1	65.37 (0.69)	0.67 (0.14)	0.9 (0.12)	99.1 (0.12)
			M2	66.16 (2.68)	2.3 (1.56)	3.46 (3.1)	96.54 (3.1)
		test	M1	64.79 (0.46)	0.73 (0.14)	0.86 (0.15)	99.14 (0.15)
			M2	65.52 (2.4)	2.29 (1.62)	3.49 (3.04)	96.51 (3.04)
		unlearned	M1	66.67 (23.57)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
			M2	66.67 (23.57)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)

Appendix

Table A8 : Comparison of performance of the retrained and unlearned (with enforcer) models across different batch sizes of

\mathcal{D}^{f}

(N(

\mathcal{D}^{f}

)) in the ‘Adult’ dataset, reported as mean (std) over 5 iterations.

ppc	N( $\mathcal{D}^{f}$ )	Subset	Model	% Bal. Acc	% Gained	% Lost	% Retained
1	9	retraining	M1	61.68 (0.08)	1.44 (0.09)	1.33 (0.19)	98.67 (0.19)
			M2	61.59 (0.0)	0.01 (0.0)	0.01 (0.01)	99.99 (0.01)
		test	M1	61.73 (0.05)	1.42 (0.1)	1.26 (0.16)	98.74 (0.16)
			M2	61.62 (0.01)	0.02 (0.01)	0.01 (0.01)	99.99 (0.01)
		unlearned	M1	62.12 (19.14)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
			M2	62.12 (19.14)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)

Appendix

Table A9 : Comparison of performance of the retrained and unlearned (with enforcer) models across different batch sizes of

\mathcal{D}^{f}

(N(

\mathcal{D}^{f}

)) in the ‘Diabetes’ dataset, reported as mean (std) over 5 iterations.

ppc	N( $\mathcal{D}^{f}$ )	Subset	Model	% Bal. Acc	% Gained	% Lost	% Retained
1	1147	retraining	M1	55.41 (0.35)	0.04 (0.02)	0.06 (0.03)	99.94 (0.03)
			M2	55.95 (0.01)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
		test	M1	55.47 (0.36)	0.04 (0.02)	0.06 (0.03)	99.94 (0.03)
			M2	56.0 (0.01)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)
		unlearned	M1	52.86 (0.91)	0.03 (0.05)	0.06 (0.1)	99.94 (0.1)
			M2	53.65 (1.28)	0.0 (0.0)	0.0 (0.0)	100.0 (0.0)

Appendix

Table A10 : Comparison of performance of the retrained and unlearned (with enforcer) models across different batch sizes of

\mathcal{D}^{f}

(N(

\mathcal{D}^{f}

)) in the ‘Criteo’ dataset, reported as mean (std) over 5 iterations.

Machine Unlearning

Machine unlearning removes the influence of selected data or concepts from trained models while preserving useful behavior and avoiding full retraining.

See all 356 papers

Behavioral Guarantees for Proxy-Based Unlearning
May 11, 2026Virgile Dine, Teddy FuronMachine Unlearning
Centre Inria de l’Université de Rennes France
Efficient machine unlearning with minimax optimality
Apr 7, 2026Jingyi Xie, Linjun Zhang, Sai LiMachine Unlearning Statistical Learning Theory
Institute of Statistics and Big Data, Renmin University of China, Beijing 100872, China · Department of Statistics, Rutgers University, Piscataway, NJ 08854, USA · Department of Statistics and Data Science, Tsinghua University, Beijing 100084, China
Learning to Unlearn: Machine Unlearning via Learning the Unlearning Behaviors
Aug 17, 2026Hang Zhang, Kaifeng Zhang, Yixiao Ma +3Machine Unlearning Privacy-Preserving ML
State Key Laboratory for Novel Software Technology Nanjing University Nanjing, China · Centre for Cyber Resilience and Trust Deakin University Burwood, VIC, Australia

May 11, 2026J/K move · Enter open · S save

Behavioral Guarantees for Proxy-Based Unlearning

Virgile Dine, Teddy Furon

Centre Inria de l’Université de Rennes France

Machine Unlearning

Open paper AskPDF arXiv

This paper proposes a framework generalizing recent proxy-based unlearning methods and proves theoretical guarantees about the behavior of the resulting unlearned model: upper bounds on its Kullback-Leibler divergence to the ideal posterior distribution of the retain data. We model approximate unlearning as a constrained optimization problem and interpret a family of solutions as introducing a scaled unlearning signal in the output space. The unlearning signal arises from proxies of the posterior data distributions. Its scale is adapted to the proxies to ensure the behavioral upper bounds. This framework relies on the structure of the data distributions in order to create proxies. If need be, the target serves as a teacher to distill the update in the weights. Our approach is experimentally validated over two forgetting scenarios as reaching the closest classifier to the model retrained from scratch.

Abstract

Machine Unlearning

Explore similar work

Behavioral Guarantees for Proxy-Based Unlearning

Efficient machine unlearning with minimax optimality

Learning to Unlearn: Machine Unlearning via Learning the Unlearning Behaviors