Models Designed to Forget: Machine Unlearning via Key Deletion

Sonia Laguna; Jorge da Silva Goncalves; Moritz Vandenhirtz; Alain Ryser; Irene Cannistraci; Julia E. Vogt

cs.LGMar 16, 2026

Models Designed to Forget: Machine Unlearning via Key Deletion

Authors: Sonia Laguna, Jorge da Silva Goncalves, Moritz Vandenhirtz, Alain Ryser, Irene Cannistraci, Julia E. Vogt

Organizations: Department of Computer Science, ETH Zurich, Switzerland

Abstract

Machine unlearning for vision models is rapidly becoming a practical requirement, driven by privacy regulations, data errors, and the need to remove harmful or corrupted training images. Despite this, most existing approximate unlearning methods tackle the problem from a post-hoc perspective. They attempt to erase the influence of targeted samples through parameter updates that typically require access to the full training data. This creates a mismatch with real deployment scenarios where unlearning requests can be anticipated, revealing a fundamental limitation of post-hoc approaches. We motivate unlearning by design, a novel paradigm for approximate methods in which models are directly trained to support forgetting as an inherent architectural capability. We instantiate this idea with Machine UNlearning via KEY deletion (MUNKEY), a memory-augmented transformer that decouples instance-specific memorization from model weights. Here, unlearning corresponds to removing the instance-identifying key, enabling zero-shot forgetting without weight updates or access to the original samples or labels. Across natural image benchmarks, fine-grained visual recognition, and medical datasets, MUNKEY outperforms all post-hoc baselines. Our results establish that unlearning by design enables fast, deployment-oriented unlearning while preserving predictive performance.

Figures & tables

Figure 1 : Overview of MUNKEY. (1) Training: The learnable exemplar token ${\bm{v}}_{i}$ from the memory bank $\mathcal{M}$ is concatenated to image tokens ${\bm{z}}_{i}$ to predict the label ${\bm{y}}_{i}$ . (2) Unlearning results in a simple instance-specific key deletion in $\mathcal{M}$ . (3) Inference: For a query $x_{q}$ , the model retrieves $K$ -nearest neighbor tokens from the updated memory $\mathcal{M}_{u}$ and ensembles their predictions for the final output.

	Random Forget (10%)					Random Forget (2%)
Method	TA	RA	FA	MIA	Avg Gap $\downarrow$	TA	RA	FA	MIA	Avg Gap $\downarrow$
DermaMNIST
Retrain (T)	$76.46_{\scalebox{0.72}{$ \pm,0.22 $}}$	$100.0_{\scalebox{0.72}{$ \pm,0.00 $}}$	$74.86_{\scalebox{0.72}{$ \pm,1.77 $}}$	$48.88_{\scalebox{0.72}{$ \pm,0.69 $}}$	–	$76.49_{\scalebox{0.72}{$ \pm,0.26 $}}$	$100.0_{\scalebox{0.72}{$ \pm,0.00 $}}$	$75.71_{\scalebox{0.72}{$ \pm,2.02 $}}$	$50.72_{\scalebox{0.72}{$ \pm,1.60 $}}$	–
Retrain (M)	$81.25_{\scalebox{0.72}{$ \pm,0.66 $}}$	$99.70_{\scalebox{0.72}{$ \pm,0.08 $}}$	$80.29_{\scalebox{0.72}{$ \pm,1.41 $}}$	$50.04_{\scalebox{0.72}{$ \pm,1.05 $}}$	–	$81.80_{\scalebox{0.72}{$ \pm,0.04 $}}$	$99.75_{\scalebox{0.72}{$ \pm,0.05 $}}$	$81.43_{\scalebox{0.72}{$ \pm,2.10 $}}$	$49.36_{\scalebox{0.72}{$ \pm,2.86 $}}$	–
KNN	$74.60_{\scalebox{0.72}{$ \pm,0.24 $}}$	$85.54_{\scalebox{0.72}{$ \pm,0.14 $}}$	$74.81_{\scalebox{0.72}{$ \pm,1.23 $}}$	$49.94_{\scalebox{0.72}{$ \pm,0.90 $}}$	–	$75.01_{\scalebox{0.72}{$ \pm,0.07 $}}$	$85.84_{\scalebox{0.72}{$ \pm,0.09 $}}$	$74.52_{\scalebox{0.72}{$ \pm,0.67 $}}$	$50.32_{\scalebox{0.72}{$ \pm,3.30 $}}$	–
CF	$75.28_{\scalebox{0.72}{$ \pm,0.98 $}}$	$97.50_{\scalebox{0.72}{$ \pm,1.55 $}}$	$94.48_{\scalebox{0.72}{$ \pm,2.23 $}}$	$64.41_{\scalebox{0.72}{$ \pm,1.96 $}}$	$9.71_{\scalebox{0.72}{$ \pm,0.99 $}}$	$76.38_{\scalebox{0.72}{$ \pm,0.28 $}}$	$97.50_{\scalebox{0.72}{$ \pm,1.16 $}}$	$95.48_{\scalebox{0.72}{$ \pm,1.21 $}}$	$61.20_{\scalebox{0.72}{$ \pm,1.80 $}}$	$8.22_{\scalebox{0.72}{$ \pm,0.90 $}}$

Table 1 : Comparison of machine unlearning performance across random forget rates (10% and 2%) on DermaMNIST and CIFAR-10. We present the accuracy on the test (TA), retain (RA), and forget (FA) sets, as well as the membership inference attack (MIA) and average gap values. Across all tables in the manuscript, when applicable, we bold the lowest Avg Gap and underline the second lowest.

Method	Cost	DermaMNIST	CIFAR-10
Standard ViT	Train/Ep.	$24.7$ s	$174.6$ s
MUNKEY	Train/Ep.	$25.9$ s	$178.5$ s
Fastest baseline	Unlearn	$23.1$ s	$105.8$ s
Slowest baseline	Unlearn	$313.4$ s	$1940.5$ s
MUNKEY	Unlearn	$\mathbf{\approx 0}$ s	$\mathbf{\approx 0}$ s

Table 2 : Runtime efficiency, full timings in Table 4 . Unlearn is post-training deletion time. Train/Ep. denotes per-epoch training time for the base models.

Panel of Figure 3 — Figure 3 : Retrieval visualizations on DermaMNIST on a test (left) and a forget (right) sample. Rows show the original and unlearned behavior, and their retrieved counterparts. Labels: new neighbors , correct matches , wrong-class neighbors , forgotten samples , correct and incorrect predictions.

	DermaMNIST		CIFAR-10
Strategy	$P_{s}$ ( $\downarrow$ )	Avg Gap ( $\downarrow$ )	$P_{s}$ ( $\downarrow$ )	Avg Gap ( $\downarrow$ )
Direct Key	$0.212_{\pm.012}$	$8.23_{\pm.70}$	$0.239_{\pm.007}$	$3.34_{\pm.18}$
Deep Prompt	$0.172_{\pm.006}$	$1.79_{\pm.69}$	$0.188_{\pm.006}$	$0.92_{\pm.52}$
Cross Attn.	$0.208_{\pm.005}$	$\mathbf{1.02}_{\pm.46}$	$0.292_{\pm.005}$	$\mathbf{0.29}_{\pm.13}$
Ensemble	$0.169_{\pm.004}$	1.52 ±.56	$0.184_{\pm.006}$	0.58 ±.22
Softmax	$0.168_{\pm.006}$	$2.01_{\pm.89}$	$0.186_{\pm.006}$	$0.71_{\pm.23}$

Table 3 : Ablation of aggregation strategies on DermaMNIST and CIFAR-10.

P_{s}

denotes Pathway Sensitivity (lower is more balanced), Avg Gap denotes unlearning faithfulness (lower is better).

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

	DermaMNIST		CIFAR-10
Model	Unlearn	Train/Ep.	Unlearn	Train/Ep.
KNN	$\approx 0$	$\approx 0$	$\approx 0$	$\approx 0$
CF	$114.1_{\pm 1.9}$	$24.7_{\pm 0.2}$	$669.5_{\pm 1.6}$	$174.6_{\pm 1.6}$
CF-k	$58.5_{\pm 2.8}$	$24.7_{\pm 0.2}$	$311.4_{\pm 1.6}$	$174.6_{\pm 1.6}$
EU-k	$58.2_{\pm 0.9}$	$24.7_{\pm 0.2}$	$311.1_{\pm 1.1}$	$174.6_{\pm 1.6}$
$\ell_{1}$ -sparse	$114.3_{\pm 0.8}$	$24.7_{\pm 0.2}$	$694.2_{\pm 0.9}$	$174.6_{\pm 1.6}$

Appendix

Table 4 : Run-Time Efficiency comparison (seconds) on DermaMNIST and CIFAR-10. Unlearn is the time to remove data post-training. Train/Ep. is the per-epoch training cost of base models.

Model	TA	RA	FA	MIA	Avg Gap $\downarrow$
Retrain (T)	$96.88_{\pm\text{\tiny 0.16}}$	$99.99_{\pm\text{\tiny 0.00}}$	$96.43_{\pm\text{\tiny 0.42}}$	$49.60_{\pm\text{\tiny 0.28}}$	–
Retrain (M)	$97.55_{\pm\text{\tiny 0.15}}$	$99.49_{\pm\text{\tiny 0.13}}$	$97.21_{\pm\text{\tiny 0.22}}$	$49.60_{\pm\text{\tiny 0.41}}$	–
KNN	$93.05_{\pm\text{\tiny 0.05}}$	$95.23_{\pm\text{\tiny 0.04}}$	$92.25_{\pm\text{\tiny 0.58}}$	$49.80_{\pm\text{\tiny 0.61}}$	–
CF	$96.65_{\pm\text{\tiny 0.21}}$	$99.74_{\pm\text{\tiny 0.12}}$	$99.50_{\pm\text{\tiny 0.41}}$	$54.10_{\pm\text{\tiny 0.12}}$	$2.01_{\pm\text{\tiny 0.18}}$
CF-k	$96.93_{\pm\text{\tiny 0.25}}$	$99.96_{\pm\text{\tiny 0.04}}$	$100.00_{\pm\text{\tiny 0.00}}$	$52.50_{\pm\text{\tiny 0.47}}$	$1.64_{\pm\text{\tiny 0.19}}$
EU-k	$96.94_{\pm\text{\tiny 0.26}}$	$99.94_{\pm\text{\tiny 0.07}}$	$99.97_{\pm\text{\tiny 0.04}}$	$52.90_{\pm\text{\tiny 0.42}}$	$1.74_{\pm\text{\tiny 0.18}}$

Appendix

Table 5 : Unlearning performance on a random forget set of 10% of the training data on BloodMNIST.

Model	TA	RA	FA	MIA	Avg Gap $\downarrow$
Retrain (T)	$98.64_{\pm\text{\tiny 0.01}}$	$100.00_{\pm\text{\tiny 0.00}}$	$98.53_{\pm\text{\tiny 0.05}}$	$49.95_{\pm\text{\tiny 0.15}}$	–
Retrain (M)	$98.96_{\pm\text{\tiny 0.07}}$	$100.00_{\pm\text{\tiny 0.00}}$	$98.78_{\pm\text{\tiny 0.17}}$	$50.09_{\pm\text{\tiny 0.24}}$	–
KNN	$96.69_{\pm\text{\tiny 0.08}}$	$97.10_{\pm\text{\tiny 0.01}}$	$96.81_{\pm\text{\tiny 0.17}}$	$50.27_{\pm\text{\tiny 0.17}}$	–
CF	$98.30_{\pm\text{\tiny 0.08}}$	$99.87_{\pm\text{\tiny 0.08}}$	$99.77_{\pm\text{\tiny 0.12}}$	$51.31_{\pm\text{\tiny 0.27}}$	$0.77_{\pm\text{\tiny 0.09}}$
CF-k	$98.56_{\pm\text{\tiny 0.08}}$	$99.99_{\pm\text{\tiny 0.00}}$	$100.00_{\pm\text{\tiny 0.01}}$	$51.89_{\pm\text{\tiny 0.24}}$	$0.88_{\pm\text{\tiny 0.07}}$
EU-k	$98.54_{\pm\text{\tiny 0.04}}$	$99.99_{\pm\text{\tiny 0.00}}$	$100.00_{\pm\text{\tiny 0.01}}$	$51.92_{\pm\text{\tiny 0.26}}$	$0.89_{\pm\text{\tiny 0.08}}$

Appendix

Table 6 : Unlearning performance on a random forget set of 10% of the training data on PathMNIST.

Model	TA	RA	FA	MIA	Avg Gap $\downarrow$
Retrain (T)	$47.17_{\pm\text{\tiny 0.61}}$	$87.37_{\pm\text{\tiny 0.41}}$	$47.00_{\pm\text{\tiny 0.39}}$	$49.88_{\pm\text{\tiny 0.26}}$	–
Retrain (M)	$76.19_{\pm\text{\tiny 0.48}}$	$99.84_{\pm\text{\tiny 0.05}}$	$76.05_{\pm\text{\tiny 0.17}}$	$49.91_{\pm\text{\tiny 0.30}}$	–
KNN	$71.86_{\pm\text{\tiny 0.31}}$	$81.67_{\pm\text{\tiny 0.07}}$	$71.65_{\pm\text{\tiny 0.36}}$	$49.88_{\pm\text{\tiny 0.35}}$	–
CF	$46.69_{\pm\text{\tiny 1.04}}$	$90.68_{\pm\text{\tiny 0.67}}$	$76.94_{\pm\text{\tiny 0.95}}$	$68.79_{\pm\text{\tiny 0.17}}$	$13.16_{\pm\text{\tiny 0.45}}$
CF-k	$49.03_{\pm\text{\tiny 0.48}}$	$94.71_{\pm\text{\tiny 0.19}}$	$94.51_{\pm\text{\tiny 0.44}}$	$77.88_{\pm\text{\tiny 0.45}}$	$21.18_{\pm\text{\tiny 0.30}}$
EU-k	$49.04_{\pm\text{\tiny 0.56}}$	$92.75_{\pm\text{\tiny 0.26}}$	$92.61_{\pm\text{\tiny 0.41}}$	$77.58_{\pm\text{\tiny 0.42}}$	$20.14_{\pm\text{\tiny 0.30}}$

Appendix

Table 7 : Unlearning performance on a random forget set of 10% of the training data on CIFAR-100.

Model	TA	RA	FA	MIA	Avg Gap $\downarrow$
Retrain (T)	$36.74_{\pm\text{\tiny 0.76}}$	$76.78_{\pm\text{\tiny 0.29}}$	$36.30_{\pm\text{\tiny 0.49}}$	$49.87_{\pm\text{\tiny 0.23}}$	–
Retrain (M)	$70.63_{\pm\text{\tiny 0.52}}$	$99.94_{\pm\text{\tiny 0.00}}$	$70.62_{\pm\text{\tiny 0.43}}$	$50.10_{\pm\text{\tiny 0.42}}$	–
KNN	$66.56_{\pm\text{\tiny 0.31}}$	$77.46_{\pm\text{\tiny 0.08}}$	$66.78_{\pm\text{\tiny 0.52}}$	$49.99_{\pm\text{\tiny 0.24}}$	–
CF	$35.46_{\pm\text{\tiny 0.37}}$	$88.20_{\pm\text{\tiny 0.57}}$	$68.24_{\pm\text{\tiny 0.97}}$	$72.71_{\pm\text{\tiny 0.25}}$	$16.87_{\pm\text{\tiny 0.39}}$
CF-k	$36.98_{\pm\text{\tiny 0.55}}$	$92.58_{\pm\text{\tiny 0.32}}$	$92.26_{\pm\text{\tiny 0.54}}$	$83.12_{\pm\text{\tiny 0.36}}$	$26.31_{\pm\text{\tiny 0.33}}$
EU-k	$36.84_{\pm\text{\tiny 0.40}}$	$88.64_{\pm\text{\tiny 0.33}}$	$88.01_{\pm\text{\tiny 0.60}}$	$81.54_{\pm\text{\tiny 0.09}}$	$23.84_{\pm\text{\tiny 0.32}}$

Appendix

Table 8 : Unlearning performance on a random forget set of 10% of the training data on Tiny ImageNet.

	DermaMNIST					CIFAR-10
Model	TA	RA	FA	MIA	Avg Gap $\downarrow$	TA	RA	FA	MIA	Avg Gap $\downarrow$
Retrain (T)	$73.60_{\scalebox{0.72}{$ \pm,0.47 $}}$	$99.98_{\scalebox{0.72}{$ \pm,0.03 $}}$	$73.39_{\scalebox{0.72}{$ \pm,0.46 $}}$	$50.09_{\scalebox{0.72}{$ \pm,0.61 $}}$	–	$71.62_{\scalebox{0.72}{$ \pm,0.27 $}}$	$87.57_{\scalebox{0.72}{$ \pm,0.71 $}}$	$71.16_{\scalebox{0.72}{$ \pm,0.44 $}}$	$49.80_{\scalebox{0.72}{$ \pm,0.28 $}}$	–
Retrain (M)	$78.84_{\scalebox{0.72}{$ \pm,0.51 $}}$	$99.33_{\scalebox{0.72}{$ \pm,0.38 $}}$	$77.70_{\scalebox{0.72}{$ \pm,0.46 $}}$	$49.86_{\scalebox{0.72}{$ \pm,0.36 $}}$	–	$91.32_{\scalebox{0.72}{$ \pm,0.13 $}}$	$99.76_{\scalebox{0.72}{$ \pm,0.05 $}}$	$91.41_{\scalebox{0.72}{$ \pm,0.02 $}}$	$49.73_{\scalebox{0.72}{$ \pm,0.18 $}}$	–
KNN	$73.82_{\scalebox{0.72}{$ \pm,0.00 $}}$	$83.19_{\scalebox{0.72}{$ \pm,0.00 $}}$	$71.77_{\scalebox{0.72}{$ \pm,0.00 $}}$	$50.22_{\scalebox{0.72}{$ \pm,0.00 $}}$	–	$90.57_{\scalebox{0.72}{$ \pm,0.04 $}}$	$94.18_{\scalebox{0.72}{$ \pm,0.09 $}}$	$90.85_{\scalebox{0.72}{$ \pm,0.06 $}}$	$50.36_{\scalebox{0.72}{$ \pm,0.13 $}}$	–
CF	$77.19_{\scalebox{0.72}{$ \pm,0.60 $}}$	$100.0_{\scalebox{0.72}{$ \pm,0.00 $}}$	$99.75_{\scalebox{0.72}{$ \pm,0.08 $}}$	$67.02_{\scalebox{0.72}{$ \pm,0.15 $}}$	$11.73_{\scalebox{0.72}{$ \pm,0.27 $}}$	$79.82_{\scalebox{0.72}{$ \pm,0.31 $}}$	$97.08_{\scalebox{0.72}{$ \pm,0.43 $}}$	$93.38_{\scalebox{0.72}{$ \pm,0.54 $}}$	$59.54_{\scalebox{0.72}{$ \pm,1.33 $}}$	$12.42_{\scalebox{0.72}{$ \pm,0.45 $}}$
CF-k	$77.04_{\scalebox{0.72}{$ \pm,0.42 $}}$	$99.77_{\scalebox{0.72}{$ \pm,0.17 $}}$	$99.83_{\scalebox{0.72}{$ \pm,0.12 $}}$	$67.28_{\scalebox{0.72}{$ \pm,0.13 $}}$	$11.82_{\scalebox{0.72}{$ \pm,0.26 $}}$	$79.44_{\scalebox{0.72}{$ \pm,0.66 $}}$	$94.13_{\scalebox{0.72}{$ \pm,0.54 $}}$	$94.15_{\scalebox{0.72}{$ \pm,0.64 $}}$	$60.36_{\scalebox{0.72}{$ \pm,1.29 $}}$	$11.98_{\scalebox{0.72}{$ \pm,0.48 $}}$

Appendix

Table 9 : Comparison of MUNKEY and studied baselines using ViT-Tiny backbone on a random forget set of 50% and three seeds on DermaMNIST and CIFAR-10.

Category	Model Variant	TA	RA	FA	MIA	$P_{s}$	Avg Gap $\downarrow$
Retrained	Direct Key	$81.83_{\pm\text{\tiny 0.68}}$	$99.25_{\pm\text{\tiny 0.22}}$	$80.05_{\pm\text{\tiny 1.10}}$	$48.30_{\pm\text{\tiny 0.98}}$	–	–
	Deep Prompt	$79.57_{\pm\text{\tiny 0.43}}$	$93.85_{\pm\text{\tiny 1.30}}$	$79.10_{\pm\text{\tiny 0.44}}$	$50.34_{\pm\text{\tiny 0.84}}$	–	–
	Cross Attention	$80.62_{\pm\text{\tiny 0.13}}$	$97.23_{\pm\text{\tiny 0.05}}$	$80.33_{\pm\text{\tiny 1.17}}$	$50.31_{\pm\text{\tiny 0.65}}$	–	–
	Ensembling	$80.80_{\pm\text{\tiny 0.54}}$	$97.47_{\pm\text{\tiny 0.15}}$	$80.10_{\pm\text{\tiny 0.99}}$	$50.53_{\pm\text{\tiny 0.89}}$	–	–
	Softmax	$79.70_{\pm\text{\tiny 0.82}}$	$93.37_{\pm\text{\tiny 1.75}}$	$78.95_{\pm\text{\tiny 0.18}}$	$50.44_{\pm\text{\tiny 0.77}}$	–	–
	Rank	$79.32_{\pm\text{\tiny 0.58}}$	$88.34_{\pm\text{\tiny 1.37}}$	$78.57_{\pm\text{\tiny 0.35}}$	$48.90_{\pm\text{\tiny 0.93}}$	–	–

Appendix

Table 10 : Results on the effect of pathway aggregation and late-fusion of neighbors on DermaMNIST.

Category	Model Variant	TA	RA	FA	MIA	$P_{s}$	Avg Gap $\downarrow$
Retrained	Direct Key	$92.41_{\pm\text{\tiny 0.16}}$	$99.56_{\pm\text{\tiny 0.07}}$	$92.46_{\pm\text{\tiny 0.10}}$	$50.11_{\pm\text{\tiny 0.58}}$	–	–
	Deep Prompt	$91.56_{\pm\text{\tiny 1.02}}$	$99.76_{\pm\text{\tiny 0.14}}$	$91.32_{\pm\text{\tiny 1.05}}$	$50.22_{\pm\text{\tiny 0.53}}$	–	–
	Cross Attention	$92.79_{\pm\text{\tiny 0.02}}$	$100.0_{\pm\text{\tiny 0.00}}$	$92.71_{\pm\text{\tiny 0.19}}$	$50.13_{\pm\text{\tiny 0.19}}$	–	–
	Ensembling	$92.93_{\pm\text{\tiny 0.16}}$	$99.93_{\pm\text{\tiny 0.02}}$	$92.76_{\pm\text{\tiny 0.45}}$	$49.97_{\pm\text{\tiny 0.25}}$	–	–
	Softmax	$92.53_{\pm\text{\tiny 0.14}}$	$99.85_{\pm\text{\tiny 0.05}}$	$92.29_{\pm\text{\tiny 0.46}}$	$49.95_{\pm\text{\tiny 0.20}}$	–	–
	Rank	$92.36_{\pm\text{\tiny 0.03}}$	$95.69_{\pm\text{\tiny 0.07}}$	$92.12_{\pm\text{\tiny 0.33}}$	$49.86_{\pm\text{\tiny 0.14}}$	–	–

Appendix

Table 11 : Results on the effect of pathway aggregation and late-fusion of neighbors on CIFAR-10.

Category	Model Variant	TA	RA	FA	MIA	$P_{s}$	Avg Gap $\downarrow$
Retrained	Direct Key	$97.97_{\pm\text{\tiny 0.12}}$	$99.93_{\pm\text{\tiny 0.03}}$	$98.38_{\pm\text{\tiny 0.48}}$	$49.25_{\pm\text{\tiny 0.66}}$	–	–
	Deep Prompt	$97.52_{\pm\text{\tiny 0.27}}$	$99.55_{\pm\text{\tiny 0.18}}$	$97.10_{\pm\text{\tiny 0.26}}$	$49.29_{\pm\text{\tiny 1.04}}$	–	–
	Cross Attention	$97.36_{\pm\text{\tiny 0.04}}$	$98.57_{\pm\text{\tiny 0.12}}$	$97.27_{\pm\text{\tiny 0.42}}$	$49.73_{\pm\text{\tiny 0.25}}$	–	–
	Ensembling	$97.55_{\pm\text{\tiny 0.15}}$	$99.49_{\pm\text{\tiny 0.13}}$	$97.21_{\pm\text{\tiny 0.22}}$	$49.60_{\pm\text{\tiny 0.41}}$	–	–
	Softmax	$97.62_{\pm\text{\tiny 0.21}}$	$99.51_{\pm\text{\tiny 0.19}}$	$97.49_{\pm\text{\tiny 0.24}}$	$49.63_{\pm\text{\tiny 0.72}}$	–	–
	Rank	$97.56_{\pm\text{\tiny 0.10}}$	$99.62_{\pm\text{\tiny 0.15}}$	$97.27_{\pm\text{\tiny 0.10}}$	$49.53_{\pm\text{\tiny 0.49}}$	–	–

Appendix

Table 12 : Results on the effect of pathway aggregation and late-fusion of neighbors on BloodMNIST.

Category	Model Variant	TA	RA	FA	MIA	$P_{s}$	Avg Gap $\downarrow$
Retrained	Direct Key	$98.98_{\pm\text{\tiny 0.04}}$	$100.0_{\pm\text{\tiny 0.00}}$	$98.94_{\pm\text{\tiny 0.11}}$	$49.88_{\pm\text{\tiny 0.30}}$	–	–
	Deep Prompt	$99.04_{\pm\text{\tiny 0.03}}$	$100.0_{\pm\text{\tiny 0.00}}$	$98.90_{\pm\text{\tiny 0.11}}$	$49.98_{\pm\text{\tiny 0.25}}$	–	–
	Cross Attention	$98.68_{\pm\text{\tiny 0.07}}$	$99.97_{\pm\text{\tiny 0.01}}$	$98.55_{\pm\text{\tiny 0.12}}$	$49.98_{\pm\text{\tiny 0.07}}$	–	–
	Ensembling	$98.96_{\pm\text{\tiny 0.07}}$	$100.00_{\pm\text{\tiny 0.00}}$	$98.78_{\pm\text{\tiny 0.17}}$	$50.09_{\pm\text{\tiny 0.24}}$	–	–
	Softmax	$99.24_{\pm\text{\tiny 0.08}}$	$100.0_{\pm\text{\tiny 0.00}}$	$99.23_{\pm\text{\tiny 0.05}}$	$49.74_{\pm\text{\tiny 0.11}}$	–	–
	Rank	$99.05_{\pm\text{\tiny 0.04}}$	$100.0_{\pm\text{\tiny 0.00}}$	$98.91_{\pm\text{\tiny 0.10}}$	$49.86_{\pm\text{\tiny 0.22}}$	–	–

Appendix

Table 13 : Results on the effect of pathway aggregation and late-fusion of neighbors on PathMNIST.

Category	Model Variant	TA	RA	FA	MIA	$P_{s}$	Avg Gap $\downarrow$
Retrained	Direct Key	$75.40_{\pm\text{\tiny 0.47}}$	$99.48_{\pm\text{\tiny 0.33}}$	$74.77_{\pm\text{\tiny 0.53}}$	$49.44_{\pm\text{\tiny 0.66}}$	–	–
	Deep Prompt	$73.31_{\pm\text{\tiny 0.78}}$	$99.77_{\pm\text{\tiny 0.05}}$	$73.44_{\pm\text{\tiny 0.69}}$	$49.91_{\pm\text{\tiny 0.27}}$	–	–
	Cross Attention	$76.23_{\pm\text{\tiny 0.35}}$	$99.92_{\pm\text{\tiny 0.01}}$	$76.17_{\pm\text{\tiny 0.30}}$	$50.02_{\pm\text{\tiny 0.27}}$	–	–
	Ensembling	$76.19_{\pm\text{\tiny 0.48}}$	$99.84_{\pm\text{\tiny 0.05}}$	$76.05_{\pm\text{\tiny 0.17}}$	$49.91_{\pm\text{\tiny 0.30}}$	–	–
	Softmax	$73.28_{\pm\text{\tiny 0.55}}$	$99.76_{\pm\text{\tiny 0.07}}$	$73.24_{\pm\text{\tiny 0.43}}$	$49.88_{\pm\text{\tiny 0.03}}$	–	–
	Rank	$72.94_{\pm\text{\tiny 0.50}}$	$87.11_{\pm\text{\tiny 0.46}}$	$72.58_{\pm\text{\tiny 0.40}}$	$49.85_{\pm\text{\tiny 0.21}}$	–	–

Appendix

Table 14 : Results on the effect of pathway aggregation and late-fusion of neighbors on CIFAR-100.

Category	Model Variant	TA	RA	FA	MIA	$P_{s}$	Avg Gap $\downarrow$
Retrained	Direct Key	$68.98_{\pm\text{\tiny 0.29}}$	$99.98_{\pm\text{\tiny 0.00}}$	$68.34_{\pm\text{\tiny 0.70}}$	$49.93_{\pm\text{\tiny 0.51}}$	–	–
	Deep Prompt	$67.75_{\pm\text{\tiny 0.53}}$	$99.84_{\pm\text{\tiny 0.02}}$	$67.68_{\pm\text{\tiny 0.59}}$	$49.83_{\pm\text{\tiny 0.20}}$	–	–
	Cross Attention	$70.41_{\pm\text{\tiny 0.16}}$	$99.94_{\pm\text{\tiny 0.00}}$	$69.95_{\pm\text{\tiny 0.47}}$	$49.93_{\pm\text{\tiny 0.20}}$	–	–
	Ensembling	$70.63_{\pm\text{\tiny 0.52}}$	$99.94_{\pm\text{\tiny 0.00}}$	$70.62_{\pm\text{\tiny 0.43}}$	$50.10_{\pm\text{\tiny 0.42}}$	–	–
	Softmax	$67.70_{\pm\text{\tiny 0.37}}$	$99.83_{\pm\text{\tiny 0.01}}$	$67.68_{\pm\text{\tiny 0.60}}$	$49.82_{\pm\text{\tiny 0.16}}$	–	–
	Rank	$67.46_{\pm\text{\tiny 0.47}}$	$78.75_{\pm\text{\tiny 0.37}}$	$67.35_{\pm\text{\tiny 0.67}}$	$49.81_{\pm\text{\tiny 0.55}}$	–	–

Appendix

Table 15 : Results on the effect of pathway aggregation and late-fusion of neighbors on Tiny ImageNet.

Dataset	Model Variant	TA	RA	FA	MIA	Avg Gap $\downarrow$
CIFAR-10	Retrain (CLIP)	$92.61_{\pm\text{\tiny 0.04}}$	$98.23_{\pm\text{\tiny 0.02}}$	$93.19_{\pm\text{\tiny 0.15}}$	$50.82_{\pm\text{\tiny 0.04}}$	–
	Retrain (DINO-v2)	$95.32_{\pm\text{\tiny 0.02}}$	$99.96_{\pm\text{\tiny 0.01}}$	$95.68_{\pm\text{\tiny 0.22}}$	$50.71_{\pm\text{\tiny 0.34}}$	–
	Retrain (ViT/B)	$92.93_{\pm\text{\tiny 0.16}}$	$99.93_{\pm\text{\tiny 0.02}}$	$92.76_{\pm\text{\tiny 0.45}}$	$49.97_{\pm\text{\tiny 0.25}}$	–
	Key Encoder CLIP	$92.69_{\pm\text{\tiny 0.08}}$	$98.25_{\pm\text{\tiny 0.09}}$	$93.84_{\pm\text{\tiny 0.16}}$	$51.85_{\pm\text{\tiny 0.06}}$	0.45 ${}_{\pm\text{\tiny 0.07}}$
	Key Encoder DINO-v2	$95.34_{\pm\text{\tiny 0.08}}$	$99.95_{\pm\text{\tiny 0.02}}$	$96.04_{\pm\text{\tiny 0.34}}$	$51.98_{\pm\text{\tiny 0.12}}$	0.42 ${}_{\pm\text{\tiny 0.14}}$
	Key Encoder Pre-trained-ViT/B	$93.12_{\pm\text{\tiny 0.08}}$	$99.94_{\pm\text{\tiny 0.01}}$	$93.41_{\pm\text{\tiny 0.58}}$	$51.45_{\pm\text{\tiny 0.39}}$	$0.58_{\pm\text{\tiny 0.22}}$

Appendix

Table 16 : Performance under varying key encoders

g_{\phi}

for the memory bank. Results on CIFAR-10 and DermaMNIST.

Dataset	Model Variant	TA	RA	FA	MIA	Avg Gap $\downarrow$
CIFAR-10	Retrain (20% $\mathcal{D}$ ViT/T)	$76.72_{\pm\text{\tiny 1.11}}$	$99.40_{\pm\text{\tiny 0.05}}$	$76.10_{\pm\text{\tiny 0.73}}$	$50.34_{\pm\text{\tiny 0.23}}$	–
	Retrain (ViT/B)	$92.14_{\pm\text{\tiny 0.09}}$	$99.96_{\pm\text{\tiny 0.01}}$	$92.07_{\pm\text{\tiny 0.70}}$	$50.18_{\pm\text{\tiny 0.22}}$	–
	Key Encoder 20% $\mathcal{D}$ ViT/T	$76.42_{\pm\text{\tiny 0.73}}$	$99.34_{\pm\text{\tiny 0.07}}$	$79.11_{\pm\text{\tiny 0.09}}$	$51.88_{\pm\text{\tiny 0.27}}$	$1.23_{\pm\text{\tiny 0.39}}$
	Key Encoder Pre-trained-ViT/B	$92.28_{\pm\text{\tiny 0.16}}$	$99.95_{\pm\text{\tiny 0.01}}$	$92.64_{\pm\text{\tiny 0.72}}$	$51.39_{\pm\text{\tiny 0.56}}$	$0.48_{\pm\text{\tiny 0.30}}$
DermaMNIST	Retrain (20% $\mathcal{D}$ ) ViT/T	$74.11_{\pm\text{\tiny 0.43}}$	$92.96_{\pm\text{\tiny 0.23}}$	$73.87_{\pm\text{\tiny 0.22}}$	$50.37_{\pm\text{\tiny 0.58}}$	–
	Retrain (ViT/B)	$81.05_{\pm\text{\tiny 0.25}}$	$99.81_{\pm\text{\tiny 0.04}}$	$79.82_{\pm\text{\tiny 0.76}}$	$49.68_{\pm\text{\tiny 0.80}}$	–

Appendix

Table 17 : Performance on encoder

g_{\phi}

trained from scratch. We compare the default pre-trained encoder against a key encoder trained on only 20% of the original training data, with MUNKEY trained on the remaining 80%. Results are shown on CIFAR-10 and DermaMNIST.

Figure 4 : Performance sensitivity to the number of neighbors on DermaMNIST and CIFAR-10. We report Test Accuracy (%) (left) and Avg Gap (right) across different neighborhood sizes.

Figure 5 : Performance of MUNKEY under varying exemplar token size. The impact of the exemplar token dimensionality (

d_{e}\in\{64,128,256\}

) on model utility and unlearning efficacy is illustrated through Test Accuracy, Forget Accuracy, and Avg Gap. Results on DermaMNIST and CIFAR-10 evaluated on three seeds with a forget fraction of 10%.

Model	Token Size	TA	RA	FA	MIA	Avg Gap $\downarrow$
Retrain (M)	64	$80.23_{\pm\text{\tiny 0.80}}$	$99.76_{\pm\text{\tiny 0.08}}$	$79.95_{\pm\text{\tiny 1.81}}$	$49.90_{\pm\text{\tiny 1.08}}$	–
Retrain (M)	128	$81.25_{\pm\text{\tiny 0.66}}$	$99.70_{\pm\text{\tiny 0.08}}$	$80.29_{\pm\text{\tiny 1.41}}$	$50.04_{\pm\text{\tiny 1.05}}$	–
Retrain (M)	256	$80.98_{\pm\text{\tiny 0.55}}$	$99.87_{\pm\text{\tiny 0.03}}$	$80.48_{\pm\text{\tiny 1.52}}$	$50.08_{\pm\text{\tiny 0.97}}$	–
MUNKEY	64	$81.16_{\pm\text{\tiny 0.66}}$	$99.73_{\pm\text{\tiny 0.04}}$	$87.95_{\pm\text{\tiny 2.19}}$	$53.52_{\pm\text{\tiny 1.91}}$	$3.15_{\pm\text{\tiny 0.93}}$
MUNKEY	128	$81.06_{\pm\text{\tiny 0.44}}$	$99.71_{\pm\text{\tiny 0.08}}$	$84.00_{\pm\text{\tiny 1.22}}$	$51.19_{\pm\text{\tiny 1.19}}$	$1.27_{\pm\text{\tiny 0.64}}$
MUNKEY	256	$81.03_{\pm\text{\tiny 0.29}}$	$99.84_{\pm\text{\tiny 0.05}}$	$83.10_{\pm\text{\tiny 1.31}}$	$50.84_{\pm\text{\tiny 1.05}}$	$0.87_{\pm\text{\tiny 0.86}}$

Appendix

Table 18 : Extended comparison of MUNKEY across exemplar token size with random forget rate 10% on DermaMNIST. We present the accuracy on the test (TA), retain (RA), and forget (FA) sets, as well as membership inference attack (MIA) and average gap values across three seeds.

Model	Token Size	TA	RA	FA	MIA	Avg Gap $\downarrow$
Retrain (M)	64	$92.55_{\pm\text{\tiny 0.18}}$	$99.82_{\pm\text{\tiny 0.05}}$	$92.06_{\pm\text{\tiny 0.55}}$	$49.93_{\pm\text{\tiny 0.35}}$	–
Retrain (M)	128	$92.38_{\pm\text{\tiny 0.06}}$	$99.93_{\pm\text{\tiny 0.03}}$	$92.05_{\pm\text{\tiny 0.23}}$	$49.90_{\pm\text{\tiny 0.34}}$	–
Retrain (M)	256	$92.28_{\pm\text{\tiny 0.10}}$	$99.99_{\pm\text{\tiny 0.00}}$	$91.86_{\pm\text{\tiny 0.21}}$	$49.84_{\pm\text{\tiny 0.22}}$	–
MUNKEY	64	$92.69_{\pm\text{\tiny 0.15}}$	$99.82_{\pm\text{\tiny 0.03}}$	$92.99_{\pm\text{\tiny 0.40}}$	$51.73_{\pm\text{\tiny 0.48}}$	$0.72_{\pm\text{\tiny 0.23}}$
MUNKEY	128	$92.63_{\pm\text{\tiny 0.20}}$	$99.94_{\pm\text{\tiny 0.01}}$	$92.53_{\pm\text{\tiny 0.52}}$	$51.40_{\pm\text{\tiny 0.44}}$	$0.56_{\pm\text{\tiny 0.21}}$
MUNKEY	256	$92.32_{\pm\text{\tiny 0.19}}$	$100.00_{\pm\text{\tiny 0.00}}$	$92.12_{\pm\text{\tiny 0.25}}$	$50.57_{\pm\text{\tiny 0.19}}$	$0.26_{\pm\text{\tiny 0.96}}$

Appendix

Table 19 : Extended comparison of MUNKEY across exemplar token size with random forget rate 10% on CIFAR-10. We present the accuracy on the test (TA), retain (RA), and forget (FA) sets, as well as membership inference attack (MIA) and average gap values across three seeds.

Model	# Layers	# Parameters	TA	RA	FA	MIA	Avg Gap $\downarrow$
Retrain (M)	10	4.6M	$81.15_{\pm\text{\tiny 0.86}}$	$99.91_{\pm\text{\tiny 0.01}}$	$80.38_{\pm\text{\tiny 1.30}}$	$50.49_{\pm\text{\tiny 0.74}}$	–
Retrain (M)	11	5.1M	$81.33_{\pm\text{\tiny 0.36}}$	$99.93_{\pm\text{\tiny 0.01}}$	$80.86_{\pm\text{\tiny 1.69}}$	$50.55_{\pm\text{\tiny 0.89}}$	–
Retrain (M)	12 (ViT-T)	5.5M	$81.25_{\pm\text{\tiny 0.66}}$	$99.70_{\pm\text{\tiny 0.08}}$	$80.29_{\pm\text{\tiny 1.41}}$	$50.04_{\pm\text{\tiny 1.05}}$	–
Retrain (M)	13	6.0M	$81.25_{\pm\text{\tiny 0.55}}$	$99.92_{\pm\text{\tiny 0.02}}$	$80.29_{\pm\text{\tiny 0.73}}$	$50.50_{\pm\text{\tiny 0.45}}$	–
Retrain (M)	14	6.4M	$81.01_{\pm\text{\tiny 0.12}}$	$99.89_{\pm\text{\tiny 0.03}}$	$80.38_{\pm\text{\tiny 1.59}}$	$50.32_{\pm\text{\tiny 1.22}}$	–
MUNKEY	10	4.6M	$81.80_{\pm\text{\tiny 0.45}}$	$99.78_{\pm\text{\tiny 0.03}}$	$84.86_{\pm\text{\tiny 1.53}}$	$51.53_{\pm\text{\tiny 0.90}}$	$1.58_{\pm\text{\tiny 0.63}}$

Appendix

Table 20 : Comparison of MUNKEY across number of hidden layers with random forget rate 10% on DermaMNIST. We present the accuracy on the test (TA), retain (RA), and forget (FA) sets, membership inference attack (MIA), and average gap values across three seeds.

Model	# Layers	# Parameters	TA	RA	FA	MIA	Avg Gap $\downarrow$
Retrain (M)	10	4.6M	$91.95_{\pm\text{\tiny 0.08}}$	$100.00_{\pm\text{\tiny 0.00}}$	$91.52_{\pm\text{\tiny 0.13}}$	$50.08_{\pm\text{\tiny 0.31}}$	–
Retrain (M)	11	5.1M	$91.94_{\pm\text{\tiny 0.06}}$	$100.00_{\pm\text{\tiny 0.00}}$	$91.53_{\pm\text{\tiny 0.33}}$	$49.89_{\pm\text{\tiny 0.22}}$	–
Retrain (M)	12 (ViT-T)	5.5M	$92.38_{\pm\text{\tiny 0.06}}$	$99.93_{\pm\text{\tiny 0.03}}$	$92.05_{\pm\text{\tiny 0.23}}$	$49.90_{\pm\text{\tiny 0.34}}$	–
Retrain (M)	13	6.0M	$91.91_{\pm\text{\tiny 0.05}}$	$100.00_{\pm\text{\tiny 0.00}}$	$91.57_{\pm\text{\tiny 0.13}}$	$50.02_{\pm\text{\tiny 0.08}}$	–
Retrain (M)	14	6.4M	$91.91_{\pm\text{\tiny 0.10}}$	$100.00_{\pm\text{\tiny 0.00}}$	$91.61_{\pm\text{\tiny 0.10}}$	$50.13_{\pm\text{\tiny 0.11}}$	–
MUNKEY	10	4.6M	$92.46_{\pm\text{\tiny 0.08}}$	$99.96_{\pm\text{\tiny 0.01}}$	$92.51_{\pm\text{\tiny 0.34}}$	$51.11_{\pm\text{\tiny 0.42}}$	$0.64_{\pm\text{\tiny 0.16}}$

Appendix

Table 21 : Comparison of MUNKEY across number of hidden layers with random forget rate 10% on CIFAR-10. We present the accuracy on the test (TA), retain (RA), and forget (FA) sets, membership inference attack (MIA), and average gap values across three seeds.

	DermaMNIST					CIFAR-10
Model	TA	RA	FA	MIA	Avg Gap $\downarrow$	TA	RA	FA	MIA	Avg Gap $\downarrow$
Retrain (B)	$76.74_{\scalebox{0.72}{$ \pm,0.20 $}}$	$100.0_{\scalebox{0.72}{$ \pm,0.00 $}}$	$75.71_{\scalebox{0.72}{$ \pm,0.84 $}}$	$49.54_{\scalebox{0.72}{$ \pm,0.22 $}}$	–	$75.58_{\scalebox{0.72}{$ \pm,0.19 $}}$	$99.64_{\scalebox{0.72}{$ \pm,0.02 $}}$	$75.22_{\scalebox{0.72}{$ \pm,0.14 $}}$	$50.30_{\scalebox{0.72}{$ \pm,0.07 $}}$	–
Retrain (M)	$79.25_{\scalebox{0.72}{$ \pm,0.75 $}}$	$99.01_{\scalebox{0.72}{$ \pm,0.26 $}}$	$78.05_{\scalebox{0.72}{$ \pm,2.12 $}}$	$49.75_{\scalebox{0.72}{$ \pm,0.93 $}}$	–	$91.99_{\scalebox{0.72}{$ \pm,0.06 $}}$	$99.95_{\scalebox{0.72}{$ \pm,0.02 $}}$	$92.16_{\scalebox{0.72}{$ \pm,0.50 $}}$	$50.02_{\scalebox{0.72}{$ \pm,0.49 $}}$	–
KNN	$74.60_{\scalebox{0.72}{$ \pm,0.24 $}}$	$85.54_{\scalebox{0.72}{$ \pm,0.14 $}}$	$74.81_{\scalebox{0.72}{$ \pm,1.23 $}}$	$49.94_{\scalebox{0.72}{$ \pm,0.90 $}}$	–	$91.06_{\scalebox{0.72}{$ \pm,0.05 $}}$	$94.56_{\scalebox{0.72}{$ \pm,0.06 $}}$	$91.02_{\scalebox{0.72}{$ \pm,0.13 $}}$	$50.16_{\scalebox{0.72}{$ \pm,0.10 $}}$	–
CF	$75.99_{\scalebox{0.72}{$ \pm,0.82 $}}$	$98.11_{\scalebox{0.72}{$ \pm,0.73 $}}$	$95.00_{\scalebox{0.72}{$ \pm,1.85 $}}$	$64.09_{\scalebox{0.72}{$ \pm,1.53 $}}$	$9.12_{\scalebox{0.72}{$ \pm,0.70 $}}$	$74.26_{\scalebox{0.72}{$ \pm,0.83 $}}$	$95.55_{\scalebox{0.72}{$ \pm,0.15 $}}$	$88.49_{\scalebox{0.72}{$ \pm,0.58 $}}$	$59.22_{\scalebox{0.72}{$ \pm,0.38 $}}$	$6.90_{\scalebox{0.72}{$ \pm,0.28 $}}$
CF-k	$77.64_{\scalebox{0.72}{$ \pm,0.52 $}}$	$100.0_{\scalebox{0.72}{$ \pm,0.00 $}}$	$100.0_{\scalebox{0.72}{$ \pm,0.00 $}}$	$64.44_{\scalebox{0.72}{$ \pm,0.47 $}}$	$10.02_{\scalebox{0.72}{$ \pm,0.28 $}}$	$77.22_{\scalebox{0.72}{$ \pm,0.30 $}}$	$99.70_{\scalebox{0.72}{$ \pm,0.05 $}}$	$99.63_{\scalebox{0.72}{$ \pm,0.13 $}}$	$69.50_{\scalebox{0.72}{$ \pm,0.23 $}}$	$11.33_{\scalebox{0.72}{$ \pm,0.12 $}}$

Appendix

Table 22 : Comparison of MUNKEY and studied baselines using ViT-B/16 backbone (85.8M parameters) on a random forget set of 10% and three seeds on DermaMNIST and CIFAR-10.

Dataset	Model	TA	RA	FA	MIA
DermaMNIST	Retrain (M)	$81.25_{\pm\text{\tiny 0.66}}$	$99.70_{\pm\text{\tiny 0.08}}$	$80.29_{\pm\text{\tiny 1.41}}$	$50.04_{\pm\text{\tiny 1.05}}$
	Retrain (M) w/o $\mathcal{M}$	$33.25_{\pm\text{\tiny 6.73}}$	$35.41_{\pm\text{\tiny 6.96}}$	$33.57_{\pm\text{\tiny 5.94}}$	$50.23_{\pm\text{\tiny 1.68}}$
	MUNKEY	$81.06_{\pm\text{\tiny 0.44}}$	$99.71_{\pm\text{\tiny 0.08}}$	$84.00_{\pm\text{\tiny 1.22}}$	$51.19_{\pm\text{\tiny 1.19}}$
	MUNKEY w/o $\mathcal{M}$	$37.68_{\pm\text{\tiny 8.40}}$	$39.34_{\pm\text{\tiny 8.20}}$	$38.29_{\pm\text{\tiny 8.52}}$	$50.81_{\pm\text{\tiny 0.51}}$
CIFAR-10	Retrain (M)	$92.38_{\pm\text{\tiny 0.06}}$	$99.93_{\pm\text{\tiny 0.03}}$	$92.05_{\pm\text{\tiny 0.23}}$	$49.90_{\pm\text{\tiny 0.34}}$
	Retrain (M) w/o $\mathcal{M}$	$34.43_{\pm\text{\tiny 7.37}}$	$35.84_{\pm\text{\tiny 7.76}}$	$34.74_{\pm\text{\tiny 7.74}}$	$50.26_{\pm\text{\tiny 0.21}}$

Appendix

Table 23 : Membership inference on isolated model subcomponents: Standard MUNKEY and removing retrieval from the memory bank (w/o

\mathcal{M}

) across three seeds.

Figure 6 : Overview of the performance metrics under varying image and token dropout probabilities in MUNKEY training in CIFAR-10 for visualization of model selection.

Figure 7 : Overview of the resulting performance metrics under varying image and token dropout probabilities in MUNKEY training in DermaMNIST for visualization of model selection.

Explore similar work

Sep 30, 2026cs.LG

Unmerge: Efficient Machine Unlearning via Task Arithmetic

Approximate machine unlearning seeks to remove the influence of a forget set from a trained model without full retraining. Existing gradient-based methods require data-dependent hyperparameter search, struggle when forget and retain knowledge are entangled, and offer little insight into where unlearning actually happens inside the network. We recast unlearning through the lens of task arithmetic: if finetuning produces a merged task vector

τ_m

that combines learning on forget and retain sets, unlearning is the inverse operation that subtracts a learned forget component

τ_F

to recover the retain task vector

τ_R

. The forget signal is concentrated: at every layer, forget activations lie in a subspace spanned by a handful of dominant directions, so we factorize

τ_F

in a low-rank forget basis, which is faithful up to a small tail-eigenvalue residual and limits how far the correction can perturb retain. We then optimize three intuitive goals (match the merged vector inside the forget span, suppress leakage into the retain span, and bound the correction size) that provably bound forget leakage and retain damage in activation space. The resulting algorithm, Unmerge, is fast and powerful: on class-level unlearning with ResNet-50 on CIFAR-100 and Tiny ImageNet, it improves Tug-of-War by up to ~24% over a baseline of comparable runtime and by up to ~18% over stronger baselines that run ~5x slower, keeps membership-inference exposure at the level of retraining, and shrinks the feature-distribution gap to the retrained model, where relabeling methods leave forget features cleanly separable. Further studies show that Unmerge also applies to ViT-S/16 and scales to Llama-3.2-3B. The per-layer basis geometry that drives the algorithm also serves as a layerwise diagnostic for when and where unlearning becomes structurally hard.

Haoran Tang, Andrew Tan, Rajiv Khanna

Department of Computer Science Purdue University

Machine Unlearning Retraining Merge Task Arithmetic Approximation

Ask PDF

May 14, 2026cs.CV

ICED: Concept-level Machine Unlearning via Interpretable Concept Decomposition

Machine unlearning in Vision-Language Models (VLMs) is typically performed at the image or instance level, making it difficult to precisely remove target knowledge without affecting unrelated semantics. This issue is especially pronounced since a single image often contains multiple entangled concepts, including both target concepts to be forgotten and contextual information that should be preserved. In this paper, we propose an interpretable concept-level unlearning framework for VLMs, which constructs a compact task-specific concept vocabulary from the forgetting set using a multimodal large language model. In addition to modality alignment, visual representations are decomposed into sparse, nonnegative combinations of semantic concepts, providing an explicit interface for fine-grained knowledge manipulation. Based on this decomposition, our method formulates unlearning as concept-level optimization, where target concepts are selectively suppressed while intra-instance non-target semantics and global cross-modal knowledge are preserved. Extensive experiments across both in-domain and out-of-domain forgetting settings demonstrate that our method enables more comprehensive target forgetting, better preserves non-target knowledge within the same image, and maintains competitive model utility compared with existing VLM unlearning methods.

Shen Lin, Jing Lin, Junhao Dong +2

1Fujian Normal University · 2Nanyang Technological University · University of New South Wales +1

Large Language Model Unlearning Machine Unlearning Cross-Modal Forgetting Decomposition

Ask PDF

May 18, 2026cs.CV

CATA: Continual Machine Unlearning via Conflict-Averse Task Arithmetic

Vision-language models (VLMs) have shown remarkable ability in aligning visual and textual representations, enabling a wide range of multimodal applications. However, their large-scale training data inevitably raises concerns about privacy, copyright, and undesirable content, creating a strong need for machine unlearning. While existing studies mainly focus on single-shot unlearning, practical VLM deployment often involves sequential removal requests over time, giving rise to continual machine unlearning. In this work, we make the first attempt to study continual unlearning for VLMs and identify three key challenges in this setting: effectiveness in removing target knowledge, fidelity in preserving retained model utility, and persistence in preventing knowledge re-emergence under sequential updates. To address these challenges, we propose CATA, a conflict-averse task arithmetic method that represents each forget request as an unlearning task vector. By maintaining historical task vectors and performing sign-aware conflict-averse aggregation, CATA suppresses conflicting update components that may weaken previous forgetting effects. Extensive experiments under both single-shot and continual settings show that CATA outperforms baselines in terms of forgetting effectiveness, model fidelity, and forgetting persistence.

Shen Lin, Junhao Dong, Rongjie Chen +3

1Fujian Normal University · 2Nanyang Technological University · 3Xidian University

Large Language Model Unlearning Fidelity

Ask PDF