Exact Unlearning of Finetuning Data via Model Merging at Scale

Kevin Kuo; Amrith Setlur; Kartik Srinivas; Aditi Raghunathan; Virginia Smith

cs.LGApr 6, 2025

Exact Unlearning of Finetuning Data via Model Merging at Scale

Authors: Kevin Kuo, Amrith Setlur, Kartik Srinivas, Aditi Raghunathan, Virginia Smith

Organizations: Computer Science Department · Machine Learning Department

Abstract

Approximate unlearning has gained popularity as an approach to efficiently update an LLM so that it behaves (roughly) as if it was not trained on a subset of data to begin with. However, existing methods are brittle in practice and can easily be attacked to reveal supposedly unlearned information. To alleviate issues with approximate unlearning, we instead propose SIFT-Masks (SIgn-Fixed Tuning-Masks), an exact unlearning method based on model merging. SIFT-Masks addresses two key limitations of standard model merging: (1) merging a large number of tasks can severely harm utility; and (2) methods that boost utility by sharing extra information across tasks make exact unlearning prohibitively expensive. SIFT-Masks solves these issues by (1) applying local masks to recover task-specific performance; and (2) constraining finetuning to align with a global sign vector as a lightweight approach to determine masks independently before merging. Across four settings where we merge up to 500 models, SIFT-Masks improves accuracy by 5-80% over naive merging and uses up to 250x less compute for exact unlearning compared to other merging baselines.

Figures & tables

Fig. 1: We finetune 500 tasks (corresponding to user tweets) on Sent140 [ 19 ] , and report the accuracy after finetuning (y-axis) as well as the cost to exactly unlearn all tasks (x-axis). To enable efficient exact unlearning without sacrificing accuracy, we propose SIFT-Masks , a method for one-shot finetuning and model merging that can match central training accuracy ( Central FT ) while being as efficient for exact unlearning as naïve merging ( FT + Merge ).

Method	Description	Unlearning Cost	Storage
Central FT	Train a single model on all pooled data	$T$	$M$
(1) Exact unlearning methods
SISA [ 13 ]	Ensemble models trained on random data shards	$T/S$	$MS$
APA [ 20 ]	Merge models trained on data-driven shards	$T$	$MS$
(2) Model merging methods
FT + Merge [ 18 ]	Merge models trained on task-level shards	$1$	$M$

TABLE I: Comparison of various model merging and ensembling methods which can be applied to exact unlearning. First, Central FT naïvely trains a new model on all retain tasks, which is prohibitive in terms of compute. Prior exact unlearning methods (1) based on merging and ensembling reduce unlearning costs by modifying how data is partitioned into tasks. However, these methods store all local models and only ensemble them at inference, which has high storage costs. Model merging methods (2) usually do not consider exact unlearning and degrade as the number of tasks grows. Finally, our work focuses on merging-and-localization methods (3) . While FT + Merge is a cheap baseline, improvements to this method do not consider exact unlearning and thus increase unlearning costs by a factor of

T

T

is the number of retained tasks,

S

is the number of shards (disjoint partition of tasks), and

M

is the number of model parameters. “Unlearning Cost” is the worst-case compute cost of unlearning a single task in terms of the number of tasks we must retrain the model on.

Fig. 2: Overview of our method. SIFT-Masks consists of three steps: finetuning, merging, and localization. We start by running (1) SI gn- F ixed (Fine)- T uning, which initializes a random sign vector $v$ and constrains finetuning to match this sign vector or otherwise be sparse, producing a sparse local model $M_{i}$ with mask $m_{i}$ . We then (2) merge these local models into a global model and only keep the masks $m_{i}$ . When (3) serving task $c_{i}$ , we apply $m_{i}$ to the merged model. Finally, to unlearn task $c_{i}$ , we simply unmerge $M_{i}$ from $\overline{M}$ and discard $m_{i}$ .

Fig. 3: We finetune and merge models on up to 200 tasks (corresponding to TOFU authors) . Naïve merging ( FT + Merge ,

x>1

) degrades performance compared to applying local models (

x=1

); this issue becomes more severe as the number of models increases, potentially reducing performance to zeroshot quality. Our method SIFT-Masks modifies finetuning to produce a merged model and a local mask for each task which can be applied to the merged model to recover task-specific performance.

Dataset	Tasks	Labels	Label Type
TOFU	200	50256	tokens
Sent140	500	2	sentiment
Reddit	500	20	subreddits
StackOverflow	500	10	tags
MATH	35	-	exact match
PubMed	100	50256	tokens

TABLE II: Overview of our datasets. Each dataset is naturally partitioned over a large number of tasks. Sent140, Reddit, and StackOverflow are classification datasets, TOFU and PubMed are language modeling datasets, and MATH is a reasoning dataset.

Fig. 4: SIFT-Masks outperforms FT + Merge across multiple datasets. We finetune on 500 tasks for each dataset (200 for TOFU) and evaluate the performance of SIFT-Masks compared to FT + Merge . Across the 4 datasets, SIFT-Masks improves performance by 5-80%. While merging at scale significantly harms FT + Merge in some settings (TOFU), it can retain reasonable performance in other settings (Sent140) since we initialize finetuning with a strong zero-shot model.

Fig. 5: SIFT-Masks is competitive with TALL-masks and even local models in terms of accuracy. We finetune 500 tasks on on Reddit and StackOverflow, and evaluate the performance immediately after each step of merging-and-localization: finetuning (local models), merging (merged model), and localization (local models). Surprisingly, we find that SIFT-Masks outperforms TALL-masks , despite using a relatively simple technique to construct the masks.

Method	MATH (35 tasks)	PubMed (100 tasks)
FT + Merge	18.8	37.4
TALL-masks	21.8	51.9
SIFT-Masks	22.6	80.7

TABLE III: Pre-unlearning accuracy on MATH and PubMed.

Fig. 6: SIFT reduces the distance between the merged and local models (blue vs. orange) . However, this does not directly improve accuracy due to interference from merging a large number of tasks. Instead, accuracy only improves after applying local masks (green).

Method	L1	Sign	Probability
Method	Distance	Conflicts	Probability
FT + Merge	606K	587M	0.09
Loc.-and-Stitch (merge-only)	561K	95M	0.05
Loc.-and-Stitch (merge-and-localize)	203K	95M	0.24
TALL-masks	549K	0	0.31
SIFT-Masks	223K	0	0.94

TABLE IV: We compare the final served model to the original local model for each task. FT + Merge and Loc.-and-Stitch (merge-only) only produce a single model and perform poorly, while localization slightly improves Loc.-and-Stitch .

Method	Density	FT+Merge	+Mask
Loc.-and-Stitch
– Dataless	0.05	0.01	0.20
– Dataless	0.20	0.05	0.24
– $\lambda$ =100	0.40	0.09	0.13
SIFT-Masks	0.57	0.09	0.94

TABLE V: We vary how local masks are determined in Loc.-and-Stitch . If data is available, the model can directly be optimized using SGD with sparsity regularization (

\lambda=100

). In dataless settings, the mask retains the TopK% of local weights.

Fig. 7: We compare the cost to unlearn 1 task from SIFT-Masks versus Central FT , FT + Merge , and TALL-masks . Depending on the setting, SIFT-Masks can outperform Central FT due to task heterogeneity (e.g. conflicting examples or distinct distributions). Across all settings, SIFT-Masks improves on the tradeoff of efficiency and accuracy compared to varying the number of finetuning steps used for Central FT . Compared to merging-and-localization baselines, SIFT-Masks is not only competitive in terms of accuracy, but also reduces unlearning costs by multiple orders of magnitude.

Fig. 8: We vary the number of steps used for local finetuning. While we use 20 steps for local finetuning on all datasets and tasks, this amount can be carefully reduced to make merging approaches even more efficient.

Fig. 9: We evaluate two post-unlearning metrics: held-out performance on all clients (including unlearned clients) and held-in performance on the retain clients. For unlearned clients, SIFT-Masks applies the merged model without any mask. Held-out performance decreases as more tasks are unlearned, due to the poor performance of the merged model compared to SIFT-Masks . Held-in performance increases as more tasks are unlearned because of less interference in the merged model.

Fig. 10: Training on fewer initial tasks (

X<500

) can reduce the unlearning costs of Central FT , but limits the maximum number of deletion requests the system can support. Furthermore, the total cost of unlearning is still

X/2

X

times larger than that of merging approaches. Y-axis is on a log scale.

Fig. 11: Clustering allows naïve baselines to trade off between accuracy and resource constraints. By paying extra storage cost to maintain multiple models, we can reduce the unlearning cost of Central FT (left) or improve the accuracy of FT + Merge (right). However, storing additional models with these two baselines is unable to match the compute or accuracy benefits of SIFT-Masks .

Method	flan-t5-small	flan-t5-large
Reddit
FT + Merge	0.22	0.59
TALL-masks	0.29	0.74
SIFT-Masks	0.29	0.78
StackOverflow
FT + Merge	0.40	0.66

TABLE VI: We test our approach on two sizes of the FLAN-T5 architecture. Unfortunately, larger pretrained models are critical for strong performance.

Method	LoRA $(r=16)$	Full FT
Reddit
FT + Merge	0.57	0.61
TALL-masks	0.73	0.74
SIFT-Masks	0.76	0.80
StackOverflow
FT + Merge	0.66	0.66

TABLE VII: We combine LoRA with merging approaches. While the gap between merging methods is larger under Full FT, SIFT-Masks retains better performance than the baselines.

Method	Accuracy	Storage Cost
Local LoRAs	0.86	95 GB
FT + Merge	0.59	38.0 MB
Central FT	0.74	38.0 MB
EMR-merging	0.72	3.01 GB
TALL-masks	0.73	3.01 GB
SIFT-Masks	0.82	3.01 GB

TABLE VIII: LoRA enables us to scale to 2500 tasks. Without LoRA, the storage costs would be

\sim 70\times

larger, meaning that storing all local models would cost

\sim

7TB.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

			Labels (per task)		Examples (per task)
Dataset	Model		Max	Total	Min	Max	Total
TOFU	GPT2-XL	200	-	50256	20	20	4000
Sent140	Llama3.2-1B-Instruct	500	2	2	52	440	40504
Reddit	FLAN-T5-Large	500	12	20	4	718	67484
StackOverflow	FLAN-T5-Large	500	10	10	124	632	97037

Appendix

TABLE IX: More details on the datasets in our experiments. Each task besides TOFU has a minimum of 2 labels (TOFU is a next-token prediction task).

	Sent140
Model	Params	Format	Zeroshot	Finetune	Merge
GPT2-XL	1.61B	CLS Head	50.0	85.7	70.8
Llama 3.2-1B	1.24B	CLS Head	50.0	69.4	55.3
		CLS Head + ZS Init	50.0	67.4	53.0
		LM Head + Prompt	68.5	85.5	77.1

Appendix

TABLE X: Merging 100 clients on Sent140. GPT2-XL and Llama 3.2 can achieve similar local accuracy after finetuning, but Llama 3.2 has much better accuracy after merging.

	Reddit
Model	Params	Format	Zeroshot	Finetune	Merge
Llama 3.2-1B	1.24B	LM Head + Prompt	10.0	75.7	24.9
FLAN-T5-Large	783M	CLS Head	3.0	61.3	19.3
		CLS Head + ZS Init	7.1	61.8	16.5
		CLS Head + ZS Init + Prompt	6.7	70.5	21.7
		LM Head + Prompt	29.7	76.5	43.2

Appendix

TABLE XI: Merging 100 clients on Reddit. While Llama 3.2 does well on Sent140, it does poorly on Reddit, indicating that testing multiple model architectures can be helpful. We do not test FLAN-T5 on Sent140 because Sent140 is included in the FLAN dataset. Unlike the Reddit setting presenting in the main paper, here we ran experiments on a setting where each client’s data is subsampled to only hold 50 examples from each of its 2 most common labels.

Fig. 12: We compare TIES-merging to FT + Merge while varying the number of models merged. Scale can disproportionately harm TIES-merging compared to regular FT + Merge .

Fig. 13: We compare TIES-merging to FT + Merge while varying the density of TIES-merging and the number of models merged. First, sparsity (trimming) tends to be more beneficial when merging a larger number of models. However, in most settings, sparsity harms the accuracy of individual models and outweighs the benefits of reducing sign conflicts.

Model	Zeroshot	FT + Merge	TALL-masks	SIFT-Masks
TOFU
GPT2-XL	11.1	11.1	50.3	95.7
FLAN-T5-Large	8.1	8.0	18.7	47.5
Llama3-1B-Instruct	9.8	9.7	41.2	82.4
Sent140
GPT2-XL	0.0	62.5	72.3	70.0

Appendix

TABLE XII: Performance comparison across different methods and datasets

Method	VerbMem	Forget	PrivLeak	Retain
Target (pre-unlearn)	58.4	63.9	-99.8	55.2
Retrain (unlearned)	20.8	33.1	0	55.0
GA_GDR	4.9	31.0	108.1	27.3
NPO_GDR	1.2	54.6	105.8	40.5
FT + Merge	18.7	30.8	0	33.1
TALL-masks	19.1	29.3	0	35.1

Appendix

TABLE XIII: Evaluation on MUSE-News

Reddit	LoRA (r=4)	LoRA (r=16)	LoRA (r=64)	Full FT
TALL-masks	73.7	73.8	74.7	74.2
SIFT-Masks	76.4	76.5	76.8	80.2

Appendix

TABLE XIV: Performance comparison across different LoRA rank

Biased Rademacher $p$	0.5 (default)	0.7	0.9
TOFU Probability	0.965	0.963	0.962

Appendix

TABLE XV: Biased Rademacher Evaluation

Machine Unlearning

Machine unlearning removes the influence of selected data or concepts from trained models while preserving useful behavior and avoiding full retraining.

See all 351 papers

Unmerge: Efficient Machine Unlearning via Task Arithmetic
Sep 30, 2026Haoran Tang, Andrew Tan, Rajiv KhannaMachine Unlearning
Department of Computer Science Purdue University
Fast Unlearning at Scale via Margin Self-Correction
Jun 1, 2026Federico Di Gennaro, Alexander Shevchenko, Fanny YangMachine Unlearning LLM Unlearning
ETH Zürich
Anatomy of Unlearning: The Dual Impact of Fact Salience and Model Fine-Tuning
Feb 23, 2026Anna Borisiuk, Andrey Savchenko, Alexander Panchenko +1Machine Unlearning LLM Unlearning
AIRI · Skoltech · Sber AI Lab +2

Sep 30, 2026J/K move · Enter open · S save

Unmerge: Efficient Machine Unlearning via Task Arithmetic

Haoran Tang, Andrew Tan, Rajiv Khanna

Department of Computer Science Purdue University

Machine Unlearning

Open paper AskPDF arXiv

Approximate machine unlearning seeks to remove the influence of a forget set from a trained model without full retraining. Existing gradient-based methods require data-dependent hyperparameter search, struggle when forget and retain knowledge are entangled, and offer little insight into where unlearning actually happens inside the network. We recast unlearning through the lens of task arithmetic: if finetuning produces a merged task vector

τ_m

that combines learning on forget and retain sets, unlearning is the inverse operation that subtracts a learned forget component

τ_F

to recover the retain task vector

τ_R

. The forget signal is concentrated: at every layer, forget activations lie in a subspace spanned by a handful of dominant directions, so we factorize

τ_F

in a low-rank forget basis, which is faithful up to a small tail-eigenvalue residual and limits how far the correction can perturb retain. We then optimize three intuitive goals (match the merged vector inside the forget span, suppress leakage into the retain span, and bound the correction size) that provably bound forget leakage and retain damage in activation space. The resulting algorithm, Unmerge, is fast and powerful: on class-level unlearning with ResNet-50 on CIFAR-100 and Tiny ImageNet, it improves Tug-of-War by up to ~24% over a baseline of comparable runtime and by up to ~18% over stronger baselines that run ~5x slower, keeps membership-inference exposure at the level of retraining, and shrinks the feature-distribution gap to the retrained model, where relabeling methods leave forget features cleanly separable. Further studies show that Unmerge also applies to ViT-S/16 and scales to Llama-3.2-3B. The per-layer basis geometry that drives the algorithm also serves as a layerwise diagnostic for when and where unlearning becomes structurally hard.

Abstract

Machine Unlearning

Explore similar work

Unmerge: Efficient Machine Unlearning via Task Arithmetic

Fast Unlearning at Scale via Margin Self-Correction

Anatomy of Unlearning: The Dual Impact of Fact Salience and Model Fine-Tuning