Anatomy of Unlearning: The Dual Impact of Fact Salience and Model Fine-Tuning

Anna Borisiuk; Andrey Savchenko; Alexander Panchenko; Elena Tutubalina

cs.CLFeb 23, 2026

Anatomy of Unlearning: The Dual Impact of Fact Salience and Model Fine-Tuning

Authors: Anna Borisiuk, Andrey Savchenko, Alexander Panchenko, Elena Tutubalina

Organizations: AIRI · Skoltech · Sber AI Lab · HSE University ·

Abstract

Machine Unlearning (MU) enables Large Language Models (LLMs) to remove unsafe or outdated information. However, existing work assumes that all facts are equally forgettable and largely ignores whether the forgotten knowledge originates from pretraining or supervised fine-tuning (SFT). In this paper, we introduce DUET (Dual Unlearning Evaluation across Training Stages), a benchmark of 28.6k Wikidata-derived triplets annotated with fact popularity using Wikipedia link counts and LLM-based salience scores. Our experiments show that pretrained and SFT models respond differently to unlearning. An SFT step on the forget data yields smoother forgetting, more stable tuning, and 10-50% higher retention, while direct unlearning on pretrained models remains unstable and prone to relearning or catastrophic forgetting.

Figures & tables

Figure 1: Unlearning landscape across fact popularity and model training type. Existing unlearning work does not account for popularity, implicitly assuming all facts are equal. Most studies evaluate forgetting on pretrained or SFT models without contrasting the two. The underexplored quadrant concerns analyzing how unlearning differs between pretrained and SFT models when fact popularity is taken into account.

Panel of Figure 2 — Figure 2: Overview of the DUET benchmark. (a) Topic distribution across 25 semantic themes (28.6k validated QA pairs), dominated by the places city domain. (b) The filtering and stratification to derive rare and popular forget sets and the retain intersection. (c) Compact retain subsets ( retain 500 and retain 1500 ) drawn from places city , providing resource-efficient yet structurally consistent evaluation settings.

Figure 3: Unlearning results for the city domain and, top $N$ = 5% rare/popular entities. Top: rare; bottom: popular. Left: forget ( city_forget_{rare,popular}_5 ); right: retain ( city_fast_retain_500 ). ROUGE is reported across learning rates. Pretrain is shown as a solid line with circles; SFT as a dashed line with stars.

Train type	Algo	Pop.	Forget $\downarrow$	Retain $\uparrow$	Forget $\downarrow$	Retain $\uparrow$	Forget $\uparrow$	Retain $\uparrow$
			ROUGE-L		Judge Acc.		Judge Flu.
Pretrain	w/o	rare	0.691	0.871	0.584	0.839	0.701	0.840
SFT	w/o	rare	0.987	0.990	0.773	0.862	0.741	0.649
Pretrain	GA	rare	0.476 ( $-$ 0.22)	0.717 ( $-$ 0.15)	0.368 ( $-$ 0.22)	0.670 ( $-$ 0.17)	0.646 ( $-$ 0.06)	0.748 ( $-$ 0.09)
SFT	GA	rare	0.350 ( $-$ 0.64)	0.931 ( $-$ 0.06)	0.507 ( $-$ 0.27)	0.842 ( $-$ 0.02)	0.642 ( $-$ 0.10)	0.615 ( $-$ 0.03)
Pretrain	GD	rare	0.465 ( $-$ 0.23)	0.779 ( $-$ 0.09)	0.358 ( $-$ 0.23)	0.747 ( $-$ 0.09)	0.577 ( $-$ 0.12)	0.627 ( $-$ 0.21)

Table 1: LLaMA-3.1-8B at

N=5\%

lr=2\times 10^{-5}

. ROUGE-L measures lexical forgetting and retention. Judge Accuracy measures factual alignment with the reference answer; Judge Fluency measures linguistic naturalness (DeepSeek v1). Parentheses show change from the matching w/o baseline.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Question	Answer	Pop. score	Label
What is the founded by Welch’s?	Thomas Bramwell Welch	44	rare
What is the instance of Viltolarsen?	antisense oligonucleotide	47	rare
What is the subclass of cowpunk?	punk music	47	rare
What is the capital of Poland?	Warsaw	2435	popular
What is the highest judicial authority of Australia?	High Court of Australia	2443	popular
What is the country of Cayenne?	France	2445	popular

Appendix

Table 2: Examples of DUET question–answer pairs with popularity annotations.

Figure 4: Distributions of popularity scores ( pop_sum ) for the city domain in the new and old benchmark splits. The left panel shows the rare forget 5% subset and the right panel the popular forget 5% subset. Of 482 samples, 87 differ from the previous version, yielding an overall overlap of 81.95%.

Algorithm	LR	Popularity	ROUGE-L Before	ROUGE-L After
GradAscent	2e-5	popular	0.998	0.439
GradAscent	4e-5	popular	0.998	0.008
GradAscent	5e-6	popular	0.998	0.998
GradAscent	2e-5	rare	0.994	0.991
GradAscent	4e-5	rare	0.994	0.369
GradAscent	5e-6	rare	0.994	0.993

Appendix

Table 3: Full-parameter SFT with LoRA unlearning on the city-forget split. ROUGE-L is reported before and after unlearning.

Algorithm	LR	Popularity	ROUGE-L Before	ROUGE-L After
GradAscent	2e-5	popular	0.988	0.000
GradAscent	2e-5	rare	0.964	0.000
GradDiff	2e-5	popular	0.988	0.001
GradDiff	2e-5	rare	0.964	0.001
NPO	2e-5	popular	0.988	0.144
NPO	2e-5	rare	0.964	0.046

Appendix

Table 4: Full-parameter SFT with LoRA unlearning on the multi-domain split at

lr=2\times 10^{-5}

. All algorithms show near-complete output collapse, in contrast to LoRA SFT (Table 1 ).

Figure 5: City, $N=1\%$ . Top: rare; bottom: popular. Left: forget ( city_forget_{rare,popular}_1 ); right: retain ( city_fast_retain_500 ). ROUGE is reported across learning rates. Pretrain is shown as a solid line with circles; SFT as a dashed line with stars. Baselines are solid horizontal lines.

Figure 6: City, $N=10\%$ . Top: rare; bottom: popular. Left: forget ( city_forget_{rare,popular}_1 ); right: retain ( city_fast_retain_500 ). ROUGE is reported across learning rates. Pretrain is shown as a solid line with circles; SFT as a dashed line with stars. Baselines are solid horizontal lines.

Train type	Algo	Pop.	Forget $\downarrow$	Retain $\uparrow$	Forget $\downarrow$	Retain $\uparrow$	Forget $\uparrow$	Retain $\uparrow$
			ROUGE-L		Judge Acc.		Judge Flu.
Pretrain	w/o	rare	0.515	0.445	0.212	0.562	0.973	0.990
SFT	w/o	rare	0.973	0.923	0.770	0.903	0.952	0.915
Pretrain	GA	rare	0.267 ( $-$ 0.25)	0.148 ( $-$ 0.30)	0.369 ( $+$ 0.16)	0.269 ( $-$ 0.29)	0.546 ( $-$ 0.43)	0.830 ( $-$ 0.16)
SFT	GA	rare	0.337 ( $-$ 0.64)	0.869 ( $-$ 0.05)	0.316 ( $-$ 0.45)	0.801 ( $-$ 0.10)	0.765 ( $-$ 0.19)	0.820 ( $-$ 0.10)
Pretrain	GD	rare	0.421 ( $-$ 0.09)	0.633 ( $+$ 0.19)	0.149 ( $-$ 0.06)	0.645 ( $+$ 0.08)	0.620 ( $-$ 0.35)	0.877 ( $-$ 0.11)

Appendix

Table 5: Gemma-7B at

N=5\%

lr=2\times 10^{-5}

. ROUGE-L measures lexical forgetting and retention. Judge Accuracy measures factual alignment; Judge Fluency measures linguistic naturalness (DeepSeek v1). Parentheses show change from the matching w/o baseline.

Figure 7: Evaluation on LLM benchmarks. Left: MMLU accuracy. Right: HellaSwag accuracy (acc_norm). Curves show unlearning algorithms (GradAscent, GradDiff, NPO) on rare and popular forget sets across learning rates; pretrained models are solid, SFT models dashed; horizontal lines indicate the corresponding baselines.

Alg.	Set	Phase	Pop.	$n$	$\Delta\log P$	rank ${}_{\text{base}}$	rank ${}_{\text{unl}}$	$\Delta$ rank
GA	forget	pretrain	popular	482	3.88	34.2	41.8	7.5
GA	forget	pretrain	rare	482	2.07	8.8	230.4	221.6
GA	forget	sft	popular	482	10.94	59.9	55.2	-4.6
GA	forget	sft	rare	482	2.72	111.3	49.7	-61.6
NPO	forget	pretrain	popular	482	0.32	34.2	35.3	1.1
NPO	forget	pretrain	rare	482	0.99	8.8	17.6	8.9

Appendix

Table 6: Token-level probability and rank shifts under unlearning for GradAscent (GA) and NPO. Values are averaged over samples;

\Delta

denotes unlearned minus base.

Alg.	Phase	Unlearn	Pop.	cosine	$\ell_{2}$	cosine (std)	$\ell_{2}$ (std)
GA	pretrain	pop	popular	0.982	3.35	0.006	0.61
GA	pretrain	rare	rare	0.981	2.70	0.008	0.66
GA	sft	pop	popular	0.958	5.47	0.016	1.01
GA	sft	rare	rare	0.966	3.81	0.011	0.81
NPO	pretrain	pop	popular	0.996	1.52	0.001	0.27
NPO	pretrain	rare	rare	0.990	1.98	0.003	0.36

Appendix

Table 7: Hidden-state similarity between base and unlearned models for GradAscent (GA) and NPO. Metrics are averaged across layers and samples.

Train type	Algorithm	Forget set	LR	Forget	Retain
SFT	UNDIAL	popular	1e-6	0.931	0.990
SFT	UNDIAL	popular	2e-5	0.865	0.998
SFT	UNDIAL	popular	5e-5	0.823	1.000
SFT	UNDIAL	rare	1e-6	0.988	0.990
SFT	UNDIAL	rare	2e-5	0.648	0.987
SFT	UNDIAL	rare	5e-5	0.679	0.994

Appendix

Table 8: UNDIAL (self-distillation) unlearning results on DUET across learning rates, evaluated on popular and rare forget splits.

Figure 8: City split ( $N=5\%$ ) evaluated on Gemma 7B. Top: rare; bottom: popular. Left: forget sets ( city_forget_{rare,popular}_5 ); right: retain set ( city_fast_retain_500 ). ROUGE scores are shown across learning rates. Pretrained models are plotted as solid lines with circles, and SFT models are plotted as dashed lines with stars. Solid horizontal lines indicate baselines.

Figure 9: City split ( $N=5\%$ ) evaluated on Qwen-2.5 7B. Top: rare; bottom: popular. Left: forget sets ( city_forget_{rare,popular}_5 ); right: retain set ( city_fast_retain_500 ). ROUGE scores are shown across learning rates. Pretrained models are plotted as solid lines with circles, and SFT models are plotted as dashed lines with stars. Solid horizontal lines indicate baselines.

Train type	Algorithm	Forget set	LR	Forget	Retain
SFT	GradAscent	popular	2e-5	0.461	0.736
SFT	GradAscent	rare	2e-5	0.651	0.729
SFT	GradDiff	popular	2e-5	0.477	0.743
SFT	GradDiff	rare	2e-5	0.654	0.754
SFT	NPO	popular	2e-5	0.482	0.741
SFT	NPO	rare	2e-5	0.668	0.731

Appendix

Table 9: Multi-domain unlearning results on DUET subset. Metrics are reported for learning rate

2\mathrm{e}{-5}

Machine Unlearning

Machine unlearning removes the influence of selected data or concepts from trained models while preserving useful behavior and avoiding full retraining.

See all 343 papers

Forget Narrowly, Retain Broadly: Unlearning as an Asymmetric Generalization Problem
Jul 10, 2026Amit Peleg, Naman Deep Singh, Naama Pearl +2Factual Knowledge in Language Models Machine Unlearning
Tübingen AI Center, University of Tübingen
PreUnlearn: Auditing Collateral Knowledge Damage Before Large Language Model Unlearning
Jun 16, 2026Bo Su, Ankit Shah, Thai LeLLM Auditing Machine Unlearning
Indiana University Bloomington, IN, USA
Making LLMs Truly Forget: Deep Unlearning by Searching, Selecting, and Severing Knowledge Paths
Sep 28, 2026Jialu Wang, Peizhi Niu, Haoteng Yin +3Factual Knowledge in Language Models Machine Unlearning
Tongji University · University of Illinois Urbana-Champaign · Purdue University +1

Jul 10, 2026J/K move · Enter open · S save

Forget Narrowly, Retain Broadly: Unlearning as an Asymmetric Generalization Problem

Amit Peleg, Naman Deep Singh, Naama Pearl, Bibhabasu Mohapatra+1

Tübingen AI Center, University of Tübingen

Factual Knowledge in Language Models Machine Unlearning LLM Unlearning Machine Unlearning Evaluation

Open paper AskPDF arXiv

Machine unlearning in LLMs is the targeted removal of specific knowledge while preserving all other capabilities, critical for privacy and safety. Yet existing benchmarks measure it unreliably. They miss knowledge that resurfaces under paraphrased or indirect queries, a failure we call under-forgetting, and lack the semantic, syntactic, and lexical probes needed to verify that unrelated knowledge is preserved, a failure we call over-forgetting. Both failures reflect an asymmetric generalization problem. Forget evaluation must cover diverse query formulations of the same target facts, testing whether forgetting holds beyond exact training prompts. Retain evaluation must probe a far larger and implicitly defined set, namely every fact disjoint from the forget target. The retain set thus defines the effective forget set, yet current datasets provide no fine-grained annotation of this forget-retain boundary. We address this with SUITE, an evaluation protocol and training corpus that captures forget-retain structure for real-world factual domains. Methods trained on SUITE improve substantially, showing that training data is as important as algorithmic design. Building on the obtained insights, we introduce JensUn++, an unlearning algorithm that achieves the best forget-retain utility trade-off across three LLMs, in both sequential and joint unlearning settings. Code and datasets are available at https://amitpeleg.github.io/forget-narrowly-retain-broadly

Abstract

Machine Unlearning

Explore similar work

Forget Narrowly, Retain Broadly: Unlearning as an Asymmetric Generalization Problem

PreUnlearn: Auditing Collateral Knowledge Damage Before Large Language Model Unlearning

Making LLMs Truly Forget: Deep Unlearning by Searching, Selecting, and Severing Knowledge Paths