Models Designed to Forget: Machine Unlearning via Key Deletion

Sonia Laguna; Jorge da Silva Goncalves; Moritz Vandenhirtz; Alain Ryser; Irene Cannistraci; Julia E. Vogt

cs.LGMar 16, 2026

Models Designed to Forget: Machine Unlearning via Key Deletion

Authors: Sonia Laguna, Jorge da Silva Goncalves, Moritz Vandenhirtz, Alain Ryser, Irene Cannistraci, Julia E. Vogt

Organizations: Department of Computer Science, ETH Zurich, Switzerland

Abstract

Machine unlearning for vision models is rapidly becoming a practical requirement, driven by privacy regulations, data errors, and the need to remove harmful or corrupted training images. Despite this, most existing approximate unlearning methods tackle the problem from a post-hoc perspective. They attempt to erase the influence of targeted samples through parameter updates that typically require access to the full training data. This creates a mismatch with real deployment scenarios where unlearning requests can be anticipated, revealing a fundamental limitation of post-hoc approaches. We motivate unlearning by design, a novel paradigm for approximate methods in which models are directly trained to support forgetting as an inherent architectural capability. We instantiate this idea with Machine UNlearning via KEY deletion (MUNKEY), a memory-augmented transformer that decouples instance-specific memorization from model weights. Here, unlearning corresponds to removing the instance-identifying key, enabling zero-shot forgetting without weight updates or access to the original samples or labels. Across natural image benchmarks, fine-grained visual recognition, and medical datasets, MUNKEY outperforms all post-hoc baselines. Our results establish that unlearning by design enables fast, deployment-oriented unlearning while preserving predictive performance.

Figures & tables

Figure 1 : Overview of MUNKEY. (1) Training: The learnable exemplar token ${\bm{v}}_{i}$ from the memory bank $\mathcal{M}$ is concatenated to image tokens ${\bm{z}}_{i}$ to predict the label ${\bm{y}}_{i}$ . (2) Unlearning results in a simple instance-specific key deletion in $\mathcal{M}$ . (3) Inference: For a query $x_{q}$ , the model retrieves $K$ -nearest neighbor tokens from the updated memory $\mathcal{M}_{u}$ and ensembles their predictions for the final output.

	Random Forget (10%)					Random Forget (2%)
Method	TA	RA	FA	MIA	Avg Gap $\downarrow$	TA	RA	FA	MIA	Avg Gap $\downarrow$
DermaMNIST
Retrain (T)	$76.46_{\scalebox{0.72}{$ \pm,0.22 $}}$	$100.0_{\scalebox{0.72}{$ \pm,0.00 $}}$	$74.86_{\scalebox{0.72}{$ \pm,1.77 $}}$	$48.88_{\scalebox{0.72}{$ \pm,0.69 $}}$	–	$76.49_{\scalebox{0.72}{$ \pm,0.26 $}}$	$100.0_{\scalebox{0.72}{$ \pm,0.00 $}}$	$75.71_{\scalebox{0.72}{$ \pm,2.02 $}}$	$50.72_{\scalebox{0.72}{$ \pm,1.60 $}}$	–
Retrain (M)	$81.25_{\scalebox{0.72}{$ \pm,0.66 $}}$	$99.70_{\scalebox{0.72}{$ \pm,0.08 $}}$	$80.29_{\scalebox{0.72}{$ \pm,1.41 $}}$	$50.04_{\scalebox{0.72}{$ \pm,1.05 $}}$	–	$81.80_{\scalebox{0.72}{$ \pm,0.04 $}}$	$99.75_{\scalebox{0.72}{$ \pm,0.05 $}}$	$81.43_{\scalebox{0.72}{$ \pm,2.10 $}}$	$49.36_{\scalebox{0.72}{$ \pm,2.86 $}}$	–
KNN	$74.60_{\scalebox{0.72}{$ \pm,0.24 $}}$	$85.54_{\scalebox{0.72}{$ \pm,0.14 $}}$	$74.81_{\scalebox{0.72}{$ \pm,1.23 $}}$	$49.94_{\scalebox{0.72}{$ \pm,0.90 $}}$	–	$75.01_{\scalebox{0.72}{$ \pm,0.07 $}}$	$85.84_{\scalebox{0.72}{$ \pm,0.09 $}}$	$74.52_{\scalebox{0.72}{$ \pm,0.67 $}}$	$50.32_{\scalebox{0.72}{$ \pm,3.30 $}}$	–
CF	$75.28_{\scalebox{0.72}{$ \pm,0.98 $}}$	$97.50_{\scalebox{0.72}{$ \pm,1.55 $}}$	$94.48_{\scalebox{0.72}{$ \pm,2.23 $}}$	$64.41_{\scalebox{0.72}{$ \pm,1.96 $}}$	$9.71_{\scalebox{0.72}{$ \pm,0.99 $}}$	$76.38_{\scalebox{0.72}{$ \pm,0.28 $}}$	$97.50_{\scalebox{0.72}{$ \pm,1.16 $}}$	$95.48_{\scalebox{0.72}{$ \pm,1.21 $}}$	$61.20_{\scalebox{0.72}{$ \pm,1.80 $}}$	$8.22_{\scalebox{0.72}{$ \pm,0.90 $}}$

Table 1 : Comparison of machine unlearning performance across random forget rates (10% and 2%) on DermaMNIST and CIFAR-10. We present the accuracy on the test (TA), retain (RA), and forget (FA) sets, as well as the membership inference attack (MIA) and average gap values. Across all tables in the manuscript, when applicable, we bold the lowest Avg Gap and underline the second lowest.

Method	Cost	DermaMNIST	CIFAR-10
Standard ViT	Train/Ep.	$24.7$ s	$174.6$ s
MUNKEY	Train/Ep.	$25.9$ s	$178.5$ s
Fastest baseline	Unlearn	$23.1$ s	$105.8$ s
Slowest baseline	Unlearn	$313.4$ s	$1940.5$ s
MUNKEY	Unlearn	$\mathbf{\approx 0}$ s	$\mathbf{\approx 0}$ s

Table 2 : Runtime efficiency, full timings in Table 4 . Unlearn is post-training deletion time. Train/Ep. denotes per-epoch training time for the base models.

Panel of Figure 3 — Figure 3 : Retrieval visualizations on DermaMNIST on a test (left) and a forget (right) sample. Rows show the original and unlearned behavior, and their retrieved counterparts. Labels: new neighbors , correct matches , wrong-class neighbors , forgotten samples , correct and incorrect predictions.

	DermaMNIST		CIFAR-10
Strategy	$P_{s}$ ( $\downarrow$ )	Avg Gap ( $\downarrow$ )	$P_{s}$ ( $\downarrow$ )	Avg Gap ( $\downarrow$ )
Direct Key	$0.212_{\pm.012}$	$8.23_{\pm.70}$	$0.239_{\pm.007}$	$3.34_{\pm.18}$
Deep Prompt	$0.172_{\pm.006}$	$1.79_{\pm.69}$	$0.188_{\pm.006}$	$0.92_{\pm.52}$
Cross Attn.	$0.208_{\pm.005}$	$\mathbf{1.02}_{\pm.46}$	$0.292_{\pm.005}$	$\mathbf{0.29}_{\pm.13}$
Ensemble	$0.169_{\pm.004}$	1.52 ±.56	$0.184_{\pm.006}$	0.58 ±.22
Softmax	$0.168_{\pm.006}$	$2.01_{\pm.89}$	$0.186_{\pm.006}$	$0.71_{\pm.23}$

Table 3 : Ablation of aggregation strategies on DermaMNIST and CIFAR-10.

P_{s}

denotes Pathway Sensitivity (lower is more balanced), Avg Gap denotes unlearning faithfulness (lower is better).

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

	DermaMNIST		CIFAR-10
Model	Unlearn	Train/Ep.	Unlearn	Train/Ep.
KNN	$\approx 0$	$\approx 0$	$\approx 0$	$\approx 0$
CF	$114.1_{\pm 1.9}$	$24.7_{\pm 0.2}$	$669.5_{\pm 1.6}$	$174.6_{\pm 1.6}$
CF-k	$58.5_{\pm 2.8}$	$24.7_{\pm 0.2}$	$311.4_{\pm 1.6}$	$174.6_{\pm 1.6}$
EU-k	$58.2_{\pm 0.9}$	$24.7_{\pm 0.2}$	$311.1_{\pm 1.1}$	$174.6_{\pm 1.6}$
$\ell_{1}$ -sparse	$114.3_{\pm 0.8}$	$24.7_{\pm 0.2}$	$694.2_{\pm 0.9}$	$174.6_{\pm 1.6}$

Appendix

Table 4 : Run-Time Efficiency comparison (seconds) on DermaMNIST and CIFAR-10. Unlearn is the time to remove data post-training. Train/Ep. is the per-epoch training cost of base models.

Model	TA	RA	FA	MIA	Avg Gap $\downarrow$
Retrain (T)	$96.88_{\pm\text{\tiny 0.16}}$	$99.99_{\pm\text{\tiny 0.00}}$	$96.43_{\pm\text{\tiny 0.42}}$	$49.60_{\pm\text{\tiny 0.28}}$	–
Retrain (M)	$97.55_{\pm\text{\tiny 0.15}}$	$99.49_{\pm\text{\tiny 0.13}}$	$97.21_{\pm\text{\tiny 0.22}}$	$49.60_{\pm\text{\tiny 0.41}}$	–
KNN	$93.05_{\pm\text{\tiny 0.05}}$	$95.23_{\pm\text{\tiny 0.04}}$	$92.25_{\pm\text{\tiny 0.58}}$	$49.80_{\pm\text{\tiny 0.61}}$	–
CF	$96.65_{\pm\text{\tiny 0.21}}$	$99.74_{\pm\text{\tiny 0.12}}$	$99.50_{\pm\text{\tiny 0.41}}$	$54.10_{\pm\text{\tiny 0.12}}$	$2.01_{\pm\text{\tiny 0.18}}$
CF-k	$96.93_{\pm\text{\tiny 0.25}}$	$99.96_{\pm\text{\tiny 0.04}}$	$100.00_{\pm\text{\tiny 0.00}}$	$52.50_{\pm\text{\tiny 0.47}}$	$1.64_{\pm\text{\tiny 0.19}}$
EU-k	$96.94_{\pm\text{\tiny 0.26}}$	$99.94_{\pm\text{\tiny 0.07}}$	$99.97_{\pm\text{\tiny 0.04}}$	$52.90_{\pm\text{\tiny 0.42}}$	$1.74_{\pm\text{\tiny 0.18}}$

Appendix

Table 5 : Unlearning performance on a random forget set of 10% of the training data on BloodMNIST.

Model	TA	RA	FA	MIA	Avg Gap $\downarrow$
Retrain (T)	$98.64_{\pm\text{\tiny 0.01}}$	$100.00_{\pm\text{\tiny 0.00}}$	$98.53_{\pm\text{\tiny 0.05}}$	$49.95_{\pm\text{\tiny 0.15}}$	–
Retrain (M)	$98.96_{\pm\text{\tiny 0.07}}$	$100.00_{\pm\text{\tiny 0.00}}$	$98.78_{\pm\text{\tiny 0.17}}$	$50.09_{\pm\text{\tiny 0.24}}$	–
KNN	$96.69_{\pm\text{\tiny 0.08}}$	$97.10_{\pm\text{\tiny 0.01}}$	$96.81_{\pm\text{\tiny 0.17}}$	$50.27_{\pm\text{\tiny 0.17}}$	–
CF	$98.30_{\pm\text{\tiny 0.08}}$	$99.87_{\pm\text{\tiny 0.08}}$	$99.77_{\pm\text{\tiny 0.12}}$	$51.31_{\pm\text{\tiny 0.27}}$	$0.77_{\pm\text{\tiny 0.09}}$
CF-k	$98.56_{\pm\text{\tiny 0.08}}$	$99.99_{\pm\text{\tiny 0.00}}$	$100.00_{\pm\text{\tiny 0.01}}$	$51.89_{\pm\text{\tiny 0.24}}$	$0.88_{\pm\text{\tiny 0.07}}$
EU-k	$98.54_{\pm\text{\tiny 0.04}}$	$99.99_{\pm\text{\tiny 0.00}}$	$100.00_{\pm\text{\tiny 0.01}}$	$51.92_{\pm\text{\tiny 0.26}}$	$0.89_{\pm\text{\tiny 0.08}}$

Appendix

Table 6 : Unlearning performance on a random forget set of 10% of the training data on PathMNIST.

Model	TA	RA	FA	MIA	Avg Gap $\downarrow$
Retrain (T)	$47.17_{\pm\text{\tiny 0.61}}$	$87.37_{\pm\text{\tiny 0.41}}$	$47.00_{\pm\text{\tiny 0.39}}$	$49.88_{\pm\text{\tiny 0.26}}$	–
Retrain (M)	$76.19_{\pm\text{\tiny 0.48}}$	$99.84_{\pm\text{\tiny 0.05}}$	$76.05_{\pm\text{\tiny 0.17}}$	$49.91_{\pm\text{\tiny 0.30}}$	–
KNN	$71.86_{\pm\text{\tiny 0.31}}$	$81.67_{\pm\text{\tiny 0.07}}$	$71.65_{\pm\text{\tiny 0.36}}$	$49.88_{\pm\text{\tiny 0.35}}$	–
CF	$46.69_{\pm\text{\tiny 1.04}}$	$90.68_{\pm\text{\tiny 0.67}}$	$76.94_{\pm\text{\tiny 0.95}}$	$68.79_{\pm\text{\tiny 0.17}}$	$13.16_{\pm\text{\tiny 0.45}}$
CF-k	$49.03_{\pm\text{\tiny 0.48}}$	$94.71_{\pm\text{\tiny 0.19}}$	$94.51_{\pm\text{\tiny 0.44}}$	$77.88_{\pm\text{\tiny 0.45}}$	$21.18_{\pm\text{\tiny 0.30}}$
EU-k	$49.04_{\pm\text{\tiny 0.56}}$	$92.75_{\pm\text{\tiny 0.26}}$	$92.61_{\pm\text{\tiny 0.41}}$	$77.58_{\pm\text{\tiny 0.42}}$	$20.14_{\pm\text{\tiny 0.30}}$

Appendix

Table 7 : Unlearning performance on a random forget set of 10% of the training data on CIFAR-100.

Model	TA	RA	FA	MIA	Avg Gap $\downarrow$
Retrain (T)	$36.74_{\pm\text{\tiny 0.76}}$	$76.78_{\pm\text{\tiny 0.29}}$	$36.30_{\pm\text{\tiny 0.49}}$	$49.87_{\pm\text{\tiny 0.23}}$	–
Retrain (M)	$70.63_{\pm\text{\tiny 0.52}}$	$99.94_{\pm\text{\tiny 0.00}}$	$70.62_{\pm\text{\tiny 0.43}}$	$50.10_{\pm\text{\tiny 0.42}}$	–
KNN	$66.56_{\pm\text{\tiny 0.31}}$	$77.46_{\pm\text{\tiny 0.08}}$	$66.78_{\pm\text{\tiny 0.52}}$	$49.99_{\pm\text{\tiny 0.24}}$	–
CF	$35.46_{\pm\text{\tiny 0.37}}$	$88.20_{\pm\text{\tiny 0.57}}$	$68.24_{\pm\text{\tiny 0.97}}$	$72.71_{\pm\text{\tiny 0.25}}$	$16.87_{\pm\text{\tiny 0.39}}$
CF-k	$36.98_{\pm\text{\tiny 0.55}}$	$92.58_{\pm\text{\tiny 0.32}}$	$92.26_{\pm\text{\tiny 0.54}}$	$83.12_{\pm\text{\tiny 0.36}}$	$26.31_{\pm\text{\tiny 0.33}}$
EU-k	$36.84_{\pm\text{\tiny 0.40}}$	$88.64_{\pm\text{\tiny 0.33}}$	$88.01_{\pm\text{\tiny 0.60}}$	$81.54_{\pm\text{\tiny 0.09}}$	$23.84_{\pm\text{\tiny 0.32}}$

Appendix

Table 8 : Unlearning performance on a random forget set of 10% of the training data on Tiny ImageNet.

	DermaMNIST					CIFAR-10
Model	TA	RA	FA	MIA	Avg Gap $\downarrow$	TA	RA	FA	MIA	Avg Gap $\downarrow$
Retrain (T)	$73.60_{\scalebox{0.72}{$ \pm,0.47 $}}$	$99.98_{\scalebox{0.72}{$ \pm,0.03 $}}$	$73.39_{\scalebox{0.72}{$ \pm,0.46 $}}$	$50.09_{\scalebox{0.72}{$ \pm,0.61 $}}$	–	$71.62_{\scalebox{0.72}{$ \pm,0.27 $}}$	$87.57_{\scalebox{0.72}{$ \pm,0.71 $}}$	$71.16_{\scalebox{0.72}{$ \pm,0.44 $}}$	$49.80_{\scalebox{0.72}{$ \pm,0.28 $}}$	–
Retrain (M)	$78.84_{\scalebox{0.72}{$ \pm,0.51 $}}$	$99.33_{\scalebox{0.72}{$ \pm,0.38 $}}$	$77.70_{\scalebox{0.72}{$ \pm,0.46 $}}$	$49.86_{\scalebox{0.72}{$ \pm,0.36 $}}$	–	$91.32_{\scalebox{0.72}{$ \pm,0.13 $}}$	$99.76_{\scalebox{0.72}{$ \pm,0.05 $}}$	$91.41_{\scalebox{0.72}{$ \pm,0.02 $}}$	$49.73_{\scalebox{0.72}{$ \pm,0.18 $}}$	–
KNN	$73.82_{\scalebox{0.72}{$ \pm,0.00 $}}$	$83.19_{\scalebox{0.72}{$ \pm,0.00 $}}$	$71.77_{\scalebox{0.72}{$ \pm,0.00 $}}$	$50.22_{\scalebox{0.72}{$ \pm,0.00 $}}$	–	$90.57_{\scalebox{0.72}{$ \pm,0.04 $}}$	$94.18_{\scalebox{0.72}{$ \pm,0.09 $}}$	$90.85_{\scalebox{0.72}{$ \pm,0.06 $}}$	$50.36_{\scalebox{0.72}{$ \pm,0.13 $}}$	–
CF	$77.19_{\scalebox{0.72}{$ \pm,0.60 $}}$	$100.0_{\scalebox{0.72}{$ \pm,0.00 $}}$	$99.75_{\scalebox{0.72}{$ \pm,0.08 $}}$	$67.02_{\scalebox{0.72}{$ \pm,0.15 $}}$	$11.73_{\scalebox{0.72}{$ \pm,0.27 $}}$	$79.82_{\scalebox{0.72}{$ \pm,0.31 $}}$	$97.08_{\scalebox{0.72}{$ \pm,0.43 $}}$	$93.38_{\scalebox{0.72}{$ \pm,0.54 $}}$	$59.54_{\scalebox{0.72}{$ \pm,1.33 $}}$	$12.42_{\scalebox{0.72}{$ \pm,0.45 $}}$
CF-k	$77.04_{\scalebox{0.72}{$ \pm,0.42 $}}$	$99.77_{\scalebox{0.72}{$ \pm,0.17 $}}$	$99.83_{\scalebox{0.72}{$ \pm,0.12 $}}$	$67.28_{\scalebox{0.72}{$ \pm,0.13 $}}$	$11.82_{\scalebox{0.72}{$ \pm,0.26 $}}$	$79.44_{\scalebox{0.72}{$ \pm,0.66 $}}$	$94.13_{\scalebox{0.72}{$ \pm,0.54 $}}$	$94.15_{\scalebox{0.72}{$ \pm,0.64 $}}$	$60.36_{\scalebox{0.72}{$ \pm,1.29 $}}$	$11.98_{\scalebox{0.72}{$ \pm,0.48 $}}$

Appendix

Table 9 : Comparison of MUNKEY and studied baselines using ViT-Tiny backbone on a random forget set of 50% and three seeds on DermaMNIST and CIFAR-10.

Category	Model Variant	TA	RA	FA	MIA	$P_{s}$	Avg Gap $\downarrow$
Retrained	Direct Key	$81.83_{\pm\text{\tiny 0.68}}$	$99.25_{\pm\text{\tiny 0.22}}$	$80.05_{\pm\text{\tiny 1.10}}$	$48.30_{\pm\text{\tiny 0.98}}$	–	–
	Deep Prompt	$79.57_{\pm\text{\tiny 0.43}}$	$93.85_{\pm\text{\tiny 1.30}}$	$79.10_{\pm\text{\tiny 0.44}}$	$50.34_{\pm\text{\tiny 0.84}}$	–	–
	Cross Attention	$80.62_{\pm\text{\tiny 0.13}}$	$97.23_{\pm\text{\tiny 0.05}}$	$80.33_{\pm\text{\tiny 1.17}}$	$50.31_{\pm\text{\tiny 0.65}}$	–	–
	Ensembling	$80.80_{\pm\text{\tiny 0.54}}$	$97.47_{\pm\text{\tiny 0.15}}$	$80.10_{\pm\text{\tiny 0.99}}$	$50.53_{\pm\text{\tiny 0.89}}$	–	–
	Softmax	$79.70_{\pm\text{\tiny 0.82}}$	$93.37_{\pm\text{\tiny 1.75}}$	$78.95_{\pm\text{\tiny 0.18}}$	$50.44_{\pm\text{\tiny 0.77}}$	–	–
	Rank	$79.32_{\pm\text{\tiny 0.58}}$	$88.34_{\pm\text{\tiny 1.37}}$	$78.57_{\pm\text{\tiny 0.35}}$	$48.90_{\pm\text{\tiny 0.93}}$	–	–

Appendix

Table 10 : Results on the effect of pathway aggregation and late-fusion of neighbors on DermaMNIST.

Category	Model Variant	TA	RA	FA	MIA	$P_{s}$	Avg Gap $\downarrow$
Retrained	Direct Key	$92.41_{\pm\text{\tiny 0.16}}$	$99.56_{\pm\text{\tiny 0.07}}$	$92.46_{\pm\text{\tiny 0.10}}$	$50.11_{\pm\text{\tiny 0.58}}$	–	–
	Deep Prompt	$91.56_{\pm\text{\tiny 1.02}}$	$99.76_{\pm\text{\tiny 0.14}}$	$91.32_{\pm\text{\tiny 1.05}}$	$50.22_{\pm\text{\tiny 0.53}}$	–	–
	Cross Attention	$92.79_{\pm\text{\tiny 0.02}}$	$100.0_{\pm\text{\tiny 0.00}}$	$92.71_{\pm\text{\tiny 0.19}}$	$50.13_{\pm\text{\tiny 0.19}}$	–	–
	Ensembling	$92.93_{\pm\text{\tiny 0.16}}$	$99.93_{\pm\text{\tiny 0.02}}$	$92.76_{\pm\text{\tiny 0.45}}$	$49.97_{\pm\text{\tiny 0.25}}$	–	–
	Softmax	$92.53_{\pm\text{\tiny 0.14}}$	$99.85_{\pm\text{\tiny 0.05}}$	$92.29_{\pm\text{\tiny 0.46}}$	$49.95_{\pm\text{\tiny 0.20}}$	–	–
	Rank	$92.36_{\pm\text{\tiny 0.03}}$	$95.69_{\pm\text{\tiny 0.07}}$	$92.12_{\pm\text{\tiny 0.33}}$	$49.86_{\pm\text{\tiny 0.14}}$	–	–

Appendix

Table 11 : Results on the effect of pathway aggregation and late-fusion of neighbors on CIFAR-10.

Category	Model Variant	TA	RA	FA	MIA	$P_{s}$	Avg Gap $\downarrow$
Retrained	Direct Key	$97.97_{\pm\text{\tiny 0.12}}$	$99.93_{\pm\text{\tiny 0.03}}$	$98.38_{\pm\text{\tiny 0.48}}$	$49.25_{\pm\text{\tiny 0.66}}$	–	–
	Deep Prompt	$97.52_{\pm\text{\tiny 0.27}}$	$99.55_{\pm\text{\tiny 0.18}}$	$97.10_{\pm\text{\tiny 0.26}}$	$49.29_{\pm\text{\tiny 1.04}}$	–	–
	Cross Attention	$97.36_{\pm\text{\tiny 0.04}}$	$98.57_{\pm\text{\tiny 0.12}}$	$97.27_{\pm\text{\tiny 0.42}}$	$49.73_{\pm\text{\tiny 0.25}}$	–	–
	Ensembling	$97.55_{\pm\text{\tiny 0.15}}$	$99.49_{\pm\text{\tiny 0.13}}$	$97.21_{\pm\text{\tiny 0.22}}$	$49.60_{\pm\text{\tiny 0.41}}$	–	–
	Softmax	$97.62_{\pm\text{\tiny 0.21}}$	$99.51_{\pm\text{\tiny 0.19}}$	$97.49_{\pm\text{\tiny 0.24}}$	$49.63_{\pm\text{\tiny 0.72}}$	–	–
	Rank	$97.56_{\pm\text{\tiny 0.10}}$	$99.62_{\pm\text{\tiny 0.15}}$	$97.27_{\pm\text{\tiny 0.10}}$	$49.53_{\pm\text{\tiny 0.49}}$	–	–

Appendix

Table 12 : Results on the effect of pathway aggregation and late-fusion of neighbors on BloodMNIST.

Category	Model Variant	TA	RA	FA	MIA	$P_{s}$	Avg Gap $\downarrow$
Retrained	Direct Key	$98.98_{\pm\text{\tiny 0.04}}$	$100.0_{\pm\text{\tiny 0.00}}$	$98.94_{\pm\text{\tiny 0.11}}$	$49.88_{\pm\text{\tiny 0.30}}$	–	–
	Deep Prompt	$99.04_{\pm\text{\tiny 0.03}}$	$100.0_{\pm\text{\tiny 0.00}}$	$98.90_{\pm\text{\tiny 0.11}}$	$49.98_{\pm\text{\tiny 0.25}}$	–	–
	Cross Attention	$98.68_{\pm\text{\tiny 0.07}}$	$99.97_{\pm\text{\tiny 0.01}}$	$98.55_{\pm\text{\tiny 0.12}}$	$49.98_{\pm\text{\tiny 0.07}}$	–	–
	Ensembling	$98.96_{\pm\text{\tiny 0.07}}$	$100.00_{\pm\text{\tiny 0.00}}$	$98.78_{\pm\text{\tiny 0.17}}$	$50.09_{\pm\text{\tiny 0.24}}$	–	–
	Softmax	$99.24_{\pm\text{\tiny 0.08}}$	$100.0_{\pm\text{\tiny 0.00}}$	$99.23_{\pm\text{\tiny 0.05}}$	$49.74_{\pm\text{\tiny 0.11}}$	–	–
	Rank	$99.05_{\pm\text{\tiny 0.04}}$	$100.0_{\pm\text{\tiny 0.00}}$	$98.91_{\pm\text{\tiny 0.10}}$	$49.86_{\pm\text{\tiny 0.22}}$	–	–

Appendix

Table 13 : Results on the effect of pathway aggregation and late-fusion of neighbors on PathMNIST.

Category	Model Variant	TA	RA	FA	MIA	$P_{s}$	Avg Gap $\downarrow$
Retrained	Direct Key	$75.40_{\pm\text{\tiny 0.47}}$	$99.48_{\pm\text{\tiny 0.33}}$	$74.77_{\pm\text{\tiny 0.53}}$	$49.44_{\pm\text{\tiny 0.66}}$	–	–
	Deep Prompt	$73.31_{\pm\text{\tiny 0.78}}$	$99.77_{\pm\text{\tiny 0.05}}$	$73.44_{\pm\text{\tiny 0.69}}$	$49.91_{\pm\text{\tiny 0.27}}$	–	–
	Cross Attention	$76.23_{\pm\text{\tiny 0.35}}$	$99.92_{\pm\text{\tiny 0.01}}$	$76.17_{\pm\text{\tiny 0.30}}$	$50.02_{\pm\text{\tiny 0.27}}$	–	–
	Ensembling	$76.19_{\pm\text{\tiny 0.48}}$	$99.84_{\pm\text{\tiny 0.05}}$	$76.05_{\pm\text{\tiny 0.17}}$	$49.91_{\pm\text{\tiny 0.30}}$	–	–
	Softmax	$73.28_{\pm\text{\tiny 0.55}}$	$99.76_{\pm\text{\tiny 0.07}}$	$73.24_{\pm\text{\tiny 0.43}}$	$49.88_{\pm\text{\tiny 0.03}}$	–	–
	Rank	$72.94_{\pm\text{\tiny 0.50}}$	$87.11_{\pm\text{\tiny 0.46}}$	$72.58_{\pm\text{\tiny 0.40}}$	$49.85_{\pm\text{\tiny 0.21}}$	–	–

Appendix

Table 14 : Results on the effect of pathway aggregation and late-fusion of neighbors on CIFAR-100.

Category	Model Variant	TA	RA	FA	MIA	$P_{s}$	Avg Gap $\downarrow$
Retrained	Direct Key	$68.98_{\pm\text{\tiny 0.29}}$	$99.98_{\pm\text{\tiny 0.00}}$	$68.34_{\pm\text{\tiny 0.70}}$	$49.93_{\pm\text{\tiny 0.51}}$	–	–
	Deep Prompt	$67.75_{\pm\text{\tiny 0.53}}$	$99.84_{\pm\text{\tiny 0.02}}$	$67.68_{\pm\text{\tiny 0.59}}$	$49.83_{\pm\text{\tiny 0.20}}$	–	–
	Cross Attention	$70.41_{\pm\text{\tiny 0.16}}$	$99.94_{\pm\text{\tiny 0.00}}$	$69.95_{\pm\text{\tiny 0.47}}$	$49.93_{\pm\text{\tiny 0.20}}$	–	–
	Ensembling	$70.63_{\pm\text{\tiny 0.52}}$	$99.94_{\pm\text{\tiny 0.00}}$	$70.62_{\pm\text{\tiny 0.43}}$	$50.10_{\pm\text{\tiny 0.42}}$	–	–
	Softmax	$67.70_{\pm\text{\tiny 0.37}}$	$99.83_{\pm\text{\tiny 0.01}}$	$67.68_{\pm\text{\tiny 0.60}}$	$49.82_{\pm\text{\tiny 0.16}}$	–	–
	Rank	$67.46_{\pm\text{\tiny 0.47}}$	$78.75_{\pm\text{\tiny 0.37}}$	$67.35_{\pm\text{\tiny 0.67}}$	$49.81_{\pm\text{\tiny 0.55}}$	–	–

Appendix

Table 15 : Results on the effect of pathway aggregation and late-fusion of neighbors on Tiny ImageNet.

Dataset	Model Variant	TA	RA	FA	MIA	Avg Gap $\downarrow$
CIFAR-10	Retrain (CLIP)	$92.61_{\pm\text{\tiny 0.04}}$	$98.23_{\pm\text{\tiny 0.02}}$	$93.19_{\pm\text{\tiny 0.15}}$	$50.82_{\pm\text{\tiny 0.04}}$	–
	Retrain (DINO-v2)	$95.32_{\pm\text{\tiny 0.02}}$	$99.96_{\pm\text{\tiny 0.01}}$	$95.68_{\pm\text{\tiny 0.22}}$	$50.71_{\pm\text{\tiny 0.34}}$	–
	Retrain (ViT/B)	$92.93_{\pm\text{\tiny 0.16}}$	$99.93_{\pm\text{\tiny 0.02}}$	$92.76_{\pm\text{\tiny 0.45}}$	$49.97_{\pm\text{\tiny 0.25}}$	–
	Key Encoder CLIP	$92.69_{\pm\text{\tiny 0.08}}$	$98.25_{\pm\text{\tiny 0.09}}$	$93.84_{\pm\text{\tiny 0.16}}$	$51.85_{\pm\text{\tiny 0.06}}$	0.45 ${}_{\pm\text{\tiny 0.07}}$
	Key Encoder DINO-v2	$95.34_{\pm\text{\tiny 0.08}}$	$99.95_{\pm\text{\tiny 0.02}}$	$96.04_{\pm\text{\tiny 0.34}}$	$51.98_{\pm\text{\tiny 0.12}}$	0.42 ${}_{\pm\text{\tiny 0.14}}$
	Key Encoder Pre-trained-ViT/B	$93.12_{\pm\text{\tiny 0.08}}$	$99.94_{\pm\text{\tiny 0.01}}$	$93.41_{\pm\text{\tiny 0.58}}$	$51.45_{\pm\text{\tiny 0.39}}$	$0.58_{\pm\text{\tiny 0.22}}$

Appendix

Table 16 : Performance under varying key encoders

g_{\phi}

for the memory bank. Results on CIFAR-10 and DermaMNIST.

Dataset	Model Variant	TA	RA	FA	MIA	Avg Gap $\downarrow$
CIFAR-10	Retrain (20% $\mathcal{D}$ ViT/T)	$76.72_{\pm\text{\tiny 1.11}}$	$99.40_{\pm\text{\tiny 0.05}}$	$76.10_{\pm\text{\tiny 0.73}}$	$50.34_{\pm\text{\tiny 0.23}}$	–
	Retrain (ViT/B)	$92.14_{\pm\text{\tiny 0.09}}$	$99.96_{\pm\text{\tiny 0.01}}$	$92.07_{\pm\text{\tiny 0.70}}$	$50.18_{\pm\text{\tiny 0.22}}$	–
	Key Encoder 20% $\mathcal{D}$ ViT/T	$76.42_{\pm\text{\tiny 0.73}}$	$99.34_{\pm\text{\tiny 0.07}}$	$79.11_{\pm\text{\tiny 0.09}}$	$51.88_{\pm\text{\tiny 0.27}}$	$1.23_{\pm\text{\tiny 0.39}}$
	Key Encoder Pre-trained-ViT/B	$92.28_{\pm\text{\tiny 0.16}}$	$99.95_{\pm\text{\tiny 0.01}}$	$92.64_{\pm\text{\tiny 0.72}}$	$51.39_{\pm\text{\tiny 0.56}}$	$0.48_{\pm\text{\tiny 0.30}}$
DermaMNIST	Retrain (20% $\mathcal{D}$ ) ViT/T	$74.11_{\pm\text{\tiny 0.43}}$	$92.96_{\pm\text{\tiny 0.23}}$	$73.87_{\pm\text{\tiny 0.22}}$	$50.37_{\pm\text{\tiny 0.58}}$	–
	Retrain (ViT/B)	$81.05_{\pm\text{\tiny 0.25}}$	$99.81_{\pm\text{\tiny 0.04}}$	$79.82_{\pm\text{\tiny 0.76}}$	$49.68_{\pm\text{\tiny 0.80}}$	–

Appendix

Table 17 : Performance on encoder

g_{\phi}

trained from scratch. We compare the default pre-trained encoder against a key encoder trained on only 20% of the original training data, with MUNKEY trained on the remaining 80%. Results are shown on CIFAR-10 and DermaMNIST.

Figure 4 : Performance sensitivity to the number of neighbors on DermaMNIST and CIFAR-10. We report Test Accuracy (%) (left) and Avg Gap (right) across different neighborhood sizes.

Figure 5 : Performance of MUNKEY under varying exemplar token size. The impact of the exemplar token dimensionality (

d_{e}\in\{64,128,256\}

) on model utility and unlearning efficacy is illustrated through Test Accuracy, Forget Accuracy, and Avg Gap. Results on DermaMNIST and CIFAR-10 evaluated on three seeds with a forget fraction of 10%.

Model	Token Size	TA	RA	FA	MIA	Avg Gap $\downarrow$
Retrain (M)	64	$80.23_{\pm\text{\tiny 0.80}}$	$99.76_{\pm\text{\tiny 0.08}}$	$79.95_{\pm\text{\tiny 1.81}}$	$49.90_{\pm\text{\tiny 1.08}}$	–
Retrain (M)	128	$81.25_{\pm\text{\tiny 0.66}}$	$99.70_{\pm\text{\tiny 0.08}}$	$80.29_{\pm\text{\tiny 1.41}}$	$50.04_{\pm\text{\tiny 1.05}}$	–
Retrain (M)	256	$80.98_{\pm\text{\tiny 0.55}}$	$99.87_{\pm\text{\tiny 0.03}}$	$80.48_{\pm\text{\tiny 1.52}}$	$50.08_{\pm\text{\tiny 0.97}}$	–
MUNKEY	64	$81.16_{\pm\text{\tiny 0.66}}$	$99.73_{\pm\text{\tiny 0.04}}$	$87.95_{\pm\text{\tiny 2.19}}$	$53.52_{\pm\text{\tiny 1.91}}$	$3.15_{\pm\text{\tiny 0.93}}$
MUNKEY	128	$81.06_{\pm\text{\tiny 0.44}}$	$99.71_{\pm\text{\tiny 0.08}}$	$84.00_{\pm\text{\tiny 1.22}}$	$51.19_{\pm\text{\tiny 1.19}}$	$1.27_{\pm\text{\tiny 0.64}}$
MUNKEY	256	$81.03_{\pm\text{\tiny 0.29}}$	$99.84_{\pm\text{\tiny 0.05}}$	$83.10_{\pm\text{\tiny 1.31}}$	$50.84_{\pm\text{\tiny 1.05}}$	$0.87_{\pm\text{\tiny 0.86}}$

Appendix

Table 18 : Extended comparison of MUNKEY across exemplar token size with random forget rate 10% on DermaMNIST. We present the accuracy on the test (TA), retain (RA), and forget (FA) sets, as well as membership inference attack (MIA) and average gap values across three seeds.

Model	Token Size	TA	RA	FA	MIA	Avg Gap $\downarrow$
Retrain (M)	64	$92.55_{\pm\text{\tiny 0.18}}$	$99.82_{\pm\text{\tiny 0.05}}$	$92.06_{\pm\text{\tiny 0.55}}$	$49.93_{\pm\text{\tiny 0.35}}$	–
Retrain (M)	128	$92.38_{\pm\text{\tiny 0.06}}$	$99.93_{\pm\text{\tiny 0.03}}$	$92.05_{\pm\text{\tiny 0.23}}$	$49.90_{\pm\text{\tiny 0.34}}$	–
Retrain (M)	256	$92.28_{\pm\text{\tiny 0.10}}$	$99.99_{\pm\text{\tiny 0.00}}$	$91.86_{\pm\text{\tiny 0.21}}$	$49.84_{\pm\text{\tiny 0.22}}$	–
MUNKEY	64	$92.69_{\pm\text{\tiny 0.15}}$	$99.82_{\pm\text{\tiny 0.03}}$	$92.99_{\pm\text{\tiny 0.40}}$	$51.73_{\pm\text{\tiny 0.48}}$	$0.72_{\pm\text{\tiny 0.23}}$
MUNKEY	128	$92.63_{\pm\text{\tiny 0.20}}$	$99.94_{\pm\text{\tiny 0.01}}$	$92.53_{\pm\text{\tiny 0.52}}$	$51.40_{\pm\text{\tiny 0.44}}$	$0.56_{\pm\text{\tiny 0.21}}$
MUNKEY	256	$92.32_{\pm\text{\tiny 0.19}}$	$100.00_{\pm\text{\tiny 0.00}}$	$92.12_{\pm\text{\tiny 0.25}}$	$50.57_{\pm\text{\tiny 0.19}}$	$0.26_{\pm\text{\tiny 0.96}}$

Appendix

Table 19 : Extended comparison of MUNKEY across exemplar token size with random forget rate 10% on CIFAR-10. We present the accuracy on the test (TA), retain (RA), and forget (FA) sets, as well as membership inference attack (MIA) and average gap values across three seeds.

Model	# Layers	# Parameters	TA	RA	FA	MIA	Avg Gap $\downarrow$
Retrain (M)	10	4.6M	$81.15_{\pm\text{\tiny 0.86}}$	$99.91_{\pm\text{\tiny 0.01}}$	$80.38_{\pm\text{\tiny 1.30}}$	$50.49_{\pm\text{\tiny 0.74}}$	–
Retrain (M)	11	5.1M	$81.33_{\pm\text{\tiny 0.36}}$	$99.93_{\pm\text{\tiny 0.01}}$	$80.86_{\pm\text{\tiny 1.69}}$	$50.55_{\pm\text{\tiny 0.89}}$	–
Retrain (M)	12 (ViT-T)	5.5M	$81.25_{\pm\text{\tiny 0.66}}$	$99.70_{\pm\text{\tiny 0.08}}$	$80.29_{\pm\text{\tiny 1.41}}$	$50.04_{\pm\text{\tiny 1.05}}$	–
Retrain (M)	13	6.0M	$81.25_{\pm\text{\tiny 0.55}}$	$99.92_{\pm\text{\tiny 0.02}}$	$80.29_{\pm\text{\tiny 0.73}}$	$50.50_{\pm\text{\tiny 0.45}}$	–
Retrain (M)	14	6.4M	$81.01_{\pm\text{\tiny 0.12}}$	$99.89_{\pm\text{\tiny 0.03}}$	$80.38_{\pm\text{\tiny 1.59}}$	$50.32_{\pm\text{\tiny 1.22}}$	–
MUNKEY	10	4.6M	$81.80_{\pm\text{\tiny 0.45}}$	$99.78_{\pm\text{\tiny 0.03}}$	$84.86_{\pm\text{\tiny 1.53}}$	$51.53_{\pm\text{\tiny 0.90}}$	$1.58_{\pm\text{\tiny 0.63}}$

Appendix

Table 20 : Comparison of MUNKEY across number of hidden layers with random forget rate 10% on DermaMNIST. We present the accuracy on the test (TA), retain (RA), and forget (FA) sets, membership inference attack (MIA), and average gap values across three seeds.

Model	# Layers	# Parameters	TA	RA	FA	MIA	Avg Gap $\downarrow$
Retrain (M)	10	4.6M	$91.95_{\pm\text{\tiny 0.08}}$	$100.00_{\pm\text{\tiny 0.00}}$	$91.52_{\pm\text{\tiny 0.13}}$	$50.08_{\pm\text{\tiny 0.31}}$	–
Retrain (M)	11	5.1M	$91.94_{\pm\text{\tiny 0.06}}$	$100.00_{\pm\text{\tiny 0.00}}$	$91.53_{\pm\text{\tiny 0.33}}$	$49.89_{\pm\text{\tiny 0.22}}$	–
Retrain (M)	12 (ViT-T)	5.5M	$92.38_{\pm\text{\tiny 0.06}}$	$99.93_{\pm\text{\tiny 0.03}}$	$92.05_{\pm\text{\tiny 0.23}}$	$49.90_{\pm\text{\tiny 0.34}}$	–
Retrain (M)	13	6.0M	$91.91_{\pm\text{\tiny 0.05}}$	$100.00_{\pm\text{\tiny 0.00}}$	$91.57_{\pm\text{\tiny 0.13}}$	$50.02_{\pm\text{\tiny 0.08}}$	–
Retrain (M)	14	6.4M	$91.91_{\pm\text{\tiny 0.10}}$	$100.00_{\pm\text{\tiny 0.00}}$	$91.61_{\pm\text{\tiny 0.10}}$	$50.13_{\pm\text{\tiny 0.11}}$	–
MUNKEY	10	4.6M	$92.46_{\pm\text{\tiny 0.08}}$	$99.96_{\pm\text{\tiny 0.01}}$	$92.51_{\pm\text{\tiny 0.34}}$	$51.11_{\pm\text{\tiny 0.42}}$	$0.64_{\pm\text{\tiny 0.16}}$

Appendix

Table 21 : Comparison of MUNKEY across number of hidden layers with random forget rate 10% on CIFAR-10. We present the accuracy on the test (TA), retain (RA), and forget (FA) sets, membership inference attack (MIA), and average gap values across three seeds.

	DermaMNIST					CIFAR-10
Model	TA	RA	FA	MIA	Avg Gap $\downarrow$	TA	RA	FA	MIA	Avg Gap $\downarrow$
Retrain (B)	$76.74_{\scalebox{0.72}{$ \pm,0.20 $}}$	$100.0_{\scalebox{0.72}{$ \pm,0.00 $}}$	$75.71_{\scalebox{0.72}{$ \pm,0.84 $}}$	$49.54_{\scalebox{0.72}{$ \pm,0.22 $}}$	–	$75.58_{\scalebox{0.72}{$ \pm,0.19 $}}$	$99.64_{\scalebox{0.72}{$ \pm,0.02 $}}$	$75.22_{\scalebox{0.72}{$ \pm,0.14 $}}$	$50.30_{\scalebox{0.72}{$ \pm,0.07 $}}$	–
Retrain (M)	$79.25_{\scalebox{0.72}{$ \pm,0.75 $}}$	$99.01_{\scalebox{0.72}{$ \pm,0.26 $}}$	$78.05_{\scalebox{0.72}{$ \pm,2.12 $}}$	$49.75_{\scalebox{0.72}{$ \pm,0.93 $}}$	–	$91.99_{\scalebox{0.72}{$ \pm,0.06 $}}$	$99.95_{\scalebox{0.72}{$ \pm,0.02 $}}$	$92.16_{\scalebox{0.72}{$ \pm,0.50 $}}$	$50.02_{\scalebox{0.72}{$ \pm,0.49 $}}$	–
KNN	$74.60_{\scalebox{0.72}{$ \pm,0.24 $}}$	$85.54_{\scalebox{0.72}{$ \pm,0.14 $}}$	$74.81_{\scalebox{0.72}{$ \pm,1.23 $}}$	$49.94_{\scalebox{0.72}{$ \pm,0.90 $}}$	–	$91.06_{\scalebox{0.72}{$ \pm,0.05 $}}$	$94.56_{\scalebox{0.72}{$ \pm,0.06 $}}$	$91.02_{\scalebox{0.72}{$ \pm,0.13 $}}$	$50.16_{\scalebox{0.72}{$ \pm,0.10 $}}$	–
CF	$75.99_{\scalebox{0.72}{$ \pm,0.82 $}}$	$98.11_{\scalebox{0.72}{$ \pm,0.73 $}}$	$95.00_{\scalebox{0.72}{$ \pm,1.85 $}}$	$64.09_{\scalebox{0.72}{$ \pm,1.53 $}}$	$9.12_{\scalebox{0.72}{$ \pm,0.70 $}}$	$74.26_{\scalebox{0.72}{$ \pm,0.83 $}}$	$95.55_{\scalebox{0.72}{$ \pm,0.15 $}}$	$88.49_{\scalebox{0.72}{$ \pm,0.58 $}}$	$59.22_{\scalebox{0.72}{$ \pm,0.38 $}}$	$6.90_{\scalebox{0.72}{$ \pm,0.28 $}}$
CF-k	$77.64_{\scalebox{0.72}{$ \pm,0.52 $}}$	$100.0_{\scalebox{0.72}{$ \pm,0.00 $}}$	$100.0_{\scalebox{0.72}{$ \pm,0.00 $}}$	$64.44_{\scalebox{0.72}{$ \pm,0.47 $}}$	$10.02_{\scalebox{0.72}{$ \pm,0.28 $}}$	$77.22_{\scalebox{0.72}{$ \pm,0.30 $}}$	$99.70_{\scalebox{0.72}{$ \pm,0.05 $}}$	$99.63_{\scalebox{0.72}{$ \pm,0.13 $}}$	$69.50_{\scalebox{0.72}{$ \pm,0.23 $}}$	$11.33_{\scalebox{0.72}{$ \pm,0.12 $}}$

Appendix

Table 22 : Comparison of MUNKEY and studied baselines using ViT-B/16 backbone (85.8M parameters) on a random forget set of 10% and three seeds on DermaMNIST and CIFAR-10.

Dataset	Model	TA	RA	FA	MIA
DermaMNIST	Retrain (M)	$81.25_{\pm\text{\tiny 0.66}}$	$99.70_{\pm\text{\tiny 0.08}}$	$80.29_{\pm\text{\tiny 1.41}}$	$50.04_{\pm\text{\tiny 1.05}}$
	Retrain (M) w/o $\mathcal{M}$	$33.25_{\pm\text{\tiny 6.73}}$	$35.41_{\pm\text{\tiny 6.96}}$	$33.57_{\pm\text{\tiny 5.94}}$	$50.23_{\pm\text{\tiny 1.68}}$
	MUNKEY	$81.06_{\pm\text{\tiny 0.44}}$	$99.71_{\pm\text{\tiny 0.08}}$	$84.00_{\pm\text{\tiny 1.22}}$	$51.19_{\pm\text{\tiny 1.19}}$
	MUNKEY w/o $\mathcal{M}$	$37.68_{\pm\text{\tiny 8.40}}$	$39.34_{\pm\text{\tiny 8.20}}$	$38.29_{\pm\text{\tiny 8.52}}$	$50.81_{\pm\text{\tiny 0.51}}$
CIFAR-10	Retrain (M)	$92.38_{\pm\text{\tiny 0.06}}$	$99.93_{\pm\text{\tiny 0.03}}$	$92.05_{\pm\text{\tiny 0.23}}$	$49.90_{\pm\text{\tiny 0.34}}$
	Retrain (M) w/o $\mathcal{M}$	$34.43_{\pm\text{\tiny 7.37}}$	$35.84_{\pm\text{\tiny 7.76}}$	$34.74_{\pm\text{\tiny 7.74}}$	$50.26_{\pm\text{\tiny 0.21}}$

Appendix

Table 23 : Membership inference on isolated model subcomponents: Standard MUNKEY and removing retrieval from the memory bank (w/o

\mathcal{M}

) across three seeds.

Figure 6 : Overview of the performance metrics under varying image and token dropout probabilities in MUNKEY training in CIFAR-10 for visualization of model selection.

Figure 7 : Overview of the resulting performance metrics under varying image and token dropout probabilities in MUNKEY training in DermaMNIST for visualization of model selection.

Explore similar work

Unmerge: Efficient Machine Unlearning via Task Arithmetic
Sep 30, 2026Haoran Tang, Andrew Tan, Rajiv KhannaMachine Unlearning Retraining
Department of Computer Science Purdue University
ICED: Concept-level Machine Unlearning via Interpretable Concept Decomposition
May 14, 2026Shen Lin, Jing Lin, Junhao Dong +2Large Language Model Unlearning Machine Unlearning
1Fujian Normal University · 2Nanyang Technological University · University of New South Wales +1
CATA: Continual Machine Unlearning via Conflict-Averse Task Arithmetic
May 18, 2026Shen Lin, Junhao Dong, Rongjie Chen +3Large Language Model Unlearning Fidelity
1Fujian Normal University · 2Nanyang Technological University · 3Xidian University

Sep 30, 2026, cs.LGJ/K move · Enter open · S save

Unmerge: Efficient Machine Unlearning via Task Arithmetic

Haoran Tang, Andrew Tan, Rajiv Khanna

Department of Computer Science Purdue University

Machine Unlearning Retraining Merge Task Arithmetic Approximation

Open paper AskPDF arXiv

Approximate machine unlearning seeks to remove the influence of a forget set from a trained model without full retraining. Existing gradient-based methods require data-dependent hyperparameter search, struggle when forget and retain knowledge are entangled, and offer little insight into where unlearning actually happens inside the network. We recast unlearning through the lens of task arithmetic: if finetuning produces a merged task vector

τ_m

that combines learning on forget and retain sets, unlearning is the inverse operation that subtracts a learned forget component

τ_F

to recover the retain task vector

τ_R

. The forget signal is concentrated: at every layer, forget activations lie in a subspace spanned by a handful of dominant directions, so we factorize

τ_F

in a low-rank forget basis, which is faithful up to a small tail-eigenvalue residual and limits how far the correction can perturb retain. We then optimize three intuitive goals (match the merged vector inside the forget span, suppress leakage into the retain span, and bound the correction size) that provably bound forget leakage and retain damage in activation space. The resulting algorithm, Unmerge, is fast and powerful: on class-level unlearning with ResNet-50 on CIFAR-100 and Tiny ImageNet, it improves Tug-of-War by up to ~24% over a baseline of comparable runtime and by up to ~18% over stronger baselines that run ~5x slower, keeps membership-inference exposure at the level of retraining, and shrinks the feature-distribution gap to the retrained model, where relabeling methods leave forget features cleanly separable. Further studies show that Unmerge also applies to ViT-S/16 and scales to Llama-3.2-3B. The per-layer basis geometry that drives the algorithm also serves as a layerwise diagnostic for when and where unlearning becomes structurally hard.

Abstract

Explore similar work

Unmerge: Efficient Machine Unlearning via Task Arithmetic

ICED: Concept-level Machine Unlearning via Interpretable Concept Decomposition

CATA: Continual Machine Unlearning via Conflict-Averse Task Arithmetic