The Unlearning Mirage: A Dynamic Framework for Evaluating LLM Unlearning

Raj Sanjay Shah; Jing Huang; Keerthiram Murugesan; Nathalie Baracaldo; Diyi Yang

cs.AIMar 11, 2026

The Unlearning Mirage: A Dynamic Framework for Evaluating LLM Unlearning

Authors: Raj Sanjay Shah, Jing Huang, Keerthiram Murugesan, Nathalie Baracaldo, Diyi Yang

Organizations: Georgia Institute of Technology · Stanford University · IBM Research

Abstract

Unlearning in Large Language Models (LLMs) aims to enhance safety, mitigate biases, and comply with legal mandates, such as the right to be forgotten. However, existing unlearning methods are brittle: minor query modifications, such as multi-hop reasoning and entity aliasing, can recover supposedly forgotten information. As a result, current evaluation metrics often create an illusion of effectiveness, failing to detect these vulnerabilities due to reliance on static, unstructured benchmarks. We propose a dynamic framework that stress tests unlearning robustness using complex structured queries. Our approach first elicits knowledge from the target model (pre-unlearning) and constructs targeted probes, ranging from simple queries to multi-hop chains, allowing precise control over query difficulty. Our experiments show that the framework (1) shows comparable coverage to existing benchmarks by automatically generating semantically equivalent Q&A probes, (2) aligns with prior evaluations, and (3) uncovers new unlearning failures missed by other benchmarks, particularly in multi-hop settings. Furthermore, activation analyses show that single-hop queries typically follow dominant computation pathways, which are more likely to be disrupted by unlearning methods. In contrast, multi-hop queries tend to use alternative pathways that often remain intact, explaining the brittleness of unlearning techniques in multi-hop settings. Our framework enables practical and scalable evaluation of unlearning methods without the need for manual construction of forget test sets, enabling easier adoption for real-world applications. We release the pip package and the code at https://sites.google.com/view/unlearningmirage/home.

Figures & tables

Figure 1: Overview of our framework: Our evaluation framework constructs a knowledge graph from pre-unlearning model outputs, enabling the automatic generation of structured single-hop, multi-hop, and alias-based queries. After applying unlearning, we probe the model to assess residual knowledge. The framework is dynamic, instantiable for any entity, and structured, providing fine-grained control over query complexity.

Forget Assessment
Benchmark	WHP	WMDP	MUSE	TOFU	RWKU	Ours
	( Eldan & Russinovich, 2023 )	( Li et al., 2024 )	( Shi et al., 2024 )	( Maini et al., 2024 )	( Jin et al., 2025 )
# Unlearning Targets	1	2	2	200	200	Any Entity
# Forget Probes	300	4,157	220	4,000	13,131	Dynamic
Forget Corpus	Harry Potter series	PubMed, Github	Books/ News	Syn. QA pairs	N/A	N/A
Retain Corpus	N/A	Wikitext	Fan pages/ News	Syn. QA pairs	N/A	N/A

Table 1: A comparison between existing unlearning benchmarks and our benchmark. Our benchmark allows us to evaluate any entity, allowing us to automatically generate forget probes. Knowledge Memorization probes: these are cloze style probes (“Capital of France is __”); Knowledge Manipulation probes: these are MCQ style Q&A probes; ❍ : While we do not cover Knowledge Manipulation probes, our framework can be easily modified for this probe style. ∗ We cover a subset of adversarial attacks.

	Multi-hop Queries $\downarrow$			Retention criteria $\uparrow$			Multi-hop	Avg.	Overall
Method	1-hop	2-hop	3-hop	1-fact away	2-facts away	Rel. Ret.	Forget Score	Retain	Score
Target model	98.6	97.2	84.1	98.9	98.1	99.1	93.3	98.7	12.5
ICL	14.7	19.2	28.5	34.2	52.5	93.4	20.8	60.0	68.3
GA	19.3	23.8	31.2	44.6	59.3	55.5	24.8	53.1	62.3
GDR	21.8	25.7	32.5	73.8	70.5	76.2	26.7	73.5	73.4
$\textsf{GA}_{\textsf{KLR}}$	22.3	26.2	33.0	74.5	71.2	76.4	27.2	74.0	73.4

Table 2: Scores from our evaluation metric instantiated with the seed entities in RWKU for Llama 3.1-Instruct (8B). Values indicate

\uparrow

means higher is better, and

\downarrow

means lower is better. Methods as described in section 3.2

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

	Multi-hop Queries $\downarrow$			Retention criteria $\uparrow$			Multi-hop	Avg.	Overall
Method	1-hop	2-hop	3-hop	1-fact away	2-facts away	Rel. Ret.	Forget Score	Retain	Score
Target model	94.1	92.6	78.4	94.2	93.8	95.0	88.4	94.3	20.7
ICL	17.7	22.2	31.5	31.2	49.4	89.6	23.8	56.7	65.0
GA	21.6	27.4	34.3	42.1	56.8	52.3	27.8	50.4	59.4
GDR	24.5	29.6	35.6	70.5	67.2	72.5	29.9	70.1	70.1
GKL	24.8	30.2	36.1	71.1	68.3	73.4	30.4	70.9	70.3

Appendix

Table 4: Scores from our evaluation metric instantiated with the seed entities in RWKU for Phi-4-mini-instruct (3.8B). Values indicate

\uparrow

means higher is better, and

\downarrow

means lower is better. Methods as described in section 3.2

	Multi-hop Queries $\downarrow$			Retention criteria $\uparrow$			Multi-hop	Avg.	Overall
Method	1-hop	2-hop	3-hop	1-fact away	2-facts away	Rel. Ret.	Forget Score	Retain	Score
Target model	98.2	96.9	80.4	98.0	97.3	97.5	91.8	97.6	15.1
ICL	13.1	17.6	27.9	35.6	50.3	91.8	19.5	59.2	68.2
GA	18.2	24.7	30.1	43.1	60.8	54.2	24.3	52.7	62.1
GDR	22.6	26.9	30.6	72.2	69.1	74.1	26.7	71.8	72.5
GKL	22.8	27.9	31.4	72.6	69.7	75.2	27.4	72.5	72.6

Appendix

Table 5: Scores from our evaluation metric instantiated with the seed entities in RWKU for IBM Granite 3.2-8B-Instruct. Values indicate

\uparrow

means higher is better, and

\downarrow

means lower is better. Methods as described in section 3.2

	Previous Metrics		Our Metric
Method	Forget Set (All) $\downarrow$	Neighbor Set (All) $\uparrow$	Multi-hop Forget Score $\downarrow$	Avg. Ret. Score $\uparrow$
Target model	77.3	90.7	93.3	98.7
ICL	16.5	55.7	20.8	60.0
GA	39.6	65.5	24.8	53.1
GDR	47.3	74.1	26.7	73.5
$\textsf{GA}_{\textsf{KLR}}$	50.6	70.2	27.2	74.0

Appendix

Table 6: RWKU: Comparison of unlearning effectiveness using previous static metrics vs. our dynamic evaluation framework when using Llama 3.1-Instruct (8B). Spearman’s rank correlation shows strong agreement with prior rankings while revealing new vulnerabilities missed by static benchmarks. Significance levels:

{}^{***}p<0.005

	Multi-hop Queries $\downarrow$			Retention criteria $\uparrow$			Multi-hop	Avg.	Overall
Method	1-hop	2-hop	3-hop	1-fact away	2-facts away	Rel. Ret.	Forget Score	Retain	Score
Target model	92.8	86.3	78.6	94.0	88.7	92.7	85.9	91.8	24.4
ICL	13.8	20.3	30.5	31.5	56.2	87.2	21.5	58.3	66.9
GA	20.5	25.3	28.9	47.9	62.9	59.5	24.8	56.8	64.7
GDR	23.2	24.7	30.7	67.9	74.6	70.6	25.9	71.0	72.5
$\textsf{GA}_{\textsf{KLR}}$	22.0	26.3	30.6	69.7	65.6	70.5	26.3	68.6	71.1

Appendix

Table 7: Our Metric on the TOFU benchmark for Llama 3.1-Instruct (8B). Values indicate

\uparrow

means higher is better, and

\downarrow

means lower is better. Methods: GA – Gradient Ascent; GDR – Gradient Diff (Gradient ascent of forget set with gradient descent of retain set);

\textsf{GA}_{\textsf{KLR}}

– Gradient ascent of forget set with KL Divergence minimization on the retain set; DPO – Direct Preference Optimization;

\textsf{DPO}_{\textsf{GDR}}

– Direct Preference Optimization with Gradient descent retention;

\textsf{DPO}_{\textsf{KLR}}

– Direct Preference Optimization with KL Divergence minimization on the retain set; NPO – Negative Preference Optimization;

\textsf{NPO}_{\textsf{GDR}}

– Negative Preference Optimization with Gradient descent retention;

\textsf{NPO}_{\textsf{KLR}}

– Negative Preference Optimization with KL Divergence minimization on the retain set; ICL – Incontext Unlearning; TV – Task vectors; ULD – Unlearning via Logit Difference.

	Previous Metrics		Our Metric
Method	Forget Quality $\uparrow$	Model Utility $\uparrow$	Multi-hop Forget Score $\downarrow$	Avg. Ret. Score $\uparrow$
Target model	0.00	0.68	85.9	91.8
ICL	NA	NA	21.5	58.3
GA	0.41	0.54	24.8	56.8
GDR	0.26	0.55	25.9	71.0
$\textsf{GA}_{\textsf{KLR}}$	0.45	0.54	26.3	68.6

Appendix

Table 8: TOFU: Comparison of unlearning effectiveness using previous static metrics vs. our dynamic evaluation framework (LLaMa 3.1 (8B)). Spearman’s rank correlation shows strong agreement with prior rankings while highlighting differences missed by static benchmarks. Significance levels:

{}^{**}p<0.01

{}^{***}p<0.005

Benchmark	Benchmark Probe	Equivalent Probe	Answer
TOFU	In which city was Hina Ameen born?	Where was Hina Ameen born?	Karachi, Pakistan
RWKU	Stephen King was born in ___, Maine.	Where was Stephen King born?	Portland

Appendix

Table 9: Examples of semantic equivalence between benchmark probes and our framework. Partial matches are accepted for open-ended queries.

Benchmark	Total probes	# Overlapping probes	Coverage (%)	Avg. graph size
RWKU	13,131	10,256	78.1%	143
TOFU	4,000	2,636	65.9%	36

Appendix

Table 10: Coverage of our evaluation framework with respect to existing unlearning benchmarks. Total Probes refers to the number of queries in the original benchmark. # Overlapping probes counts how many of those probes are semantically matched by our automatically generated probe set. Coverage (%) indicates the proportion of probes reproduced. Avg. graph size is the average number of nodes needed in our knowledge graph to reach maximum overlap with each benchmark.

Methods	Previous Metrics			Our metric									Score
	Removal Crit.		Ret. Crit.	Multi-hop Queries				Fact Ret.		Rel.	Avg.	Avg.
	Vbtim. $\downarrow$	Know. $\downarrow$	Util. $\uparrow$	1-hop $\downarrow$	2-hop $\downarrow$	2-hop - Seq. $\downarrow$	3-hop $\downarrow$	1-hop $\uparrow$	2-hop $\uparrow$	Ret. $\uparrow$	Rem. $\downarrow$	Ret. $\uparrow$	$\uparrow$
Muse-Books
Trgt Model	91.4	59.1	62.2	99.2%	98.5%	97.7%	92.4%	99.4%	98.3%	99.3%	96.9%	99.0%	NA
GA	0	0	0	9.2%	20.5%	20.7%	26.4%	65.8%	69.7%	84.3%	19.2%	73.3%	76.8%
GDR	0	0	10.3	10.8%	22.4%	22.1%	32.3%	71.1%	76.3%	86.2%	21.9%	77.9%	78.0%

Appendix

Table 11: Comparison of Unlearning Methods on Various Metrics on the MUSE-books benchmark Shi et al. (2024) . The target model here is LLama 3.1 - 8B. Methods: GA – Gradient Ascent; GDR – Gradient Diff (Gradient ascent of forget set with gradient descent of retain set);

\textsf{GA}_{\textsf{KLR}}

– Gradient ascent of forget set with KL Divergence minimization on the retain set; DPO – Direct Preference Optimization;

\textsf{DPO}_{\textsf{GDR}}

– Direct Preference Optimization with Gradient descent retention;

\textsf{DPO}_{\textsf{KLR}}

– Direct Preference Optimization with KL Divergence minimization on the retain set; NPO – Negative Preference Optimization;

\textsf{NPO}_{\textsf{GDR}}

–Negative Preference Optimization with Gradient descent retention;

\textsf{NPO}_{\textsf{KLR}}

– Negative Preference Optimization with KL Divergence minimization on the retain set; ICL – Incontext Unlearning; TV – Task vectors; ULD – Unlearning via Logit Difference.

LLM Unlearning

Large language model unlearning studies how to remove specified knowledge or behaviors while preserving a model’s unrelated capabilities.

See all 191 papers

Aug 5, 2026

Leak-Resistant Unlearning: A New Benchmark for Evaluating Multi-Hop Reasoning Consistency and Recovery Robustness

Benchmarking machine unlearning methods is critical to understand whether sensitive knowledge is removed from large language models (LLMs) or not. Current unlearning benchmarks include mainly single-hop questions and a narrow set of multi-hop questions. Although effective, they still face two challenges. (1) Knowledge is not isolated, whereby diverse multi-hop reasoning paths can potentially induce knowledge leakage than normal queries. (2) Unlearning may be fragile: unlearned knowledge can be partially recovered through recovery attacks such as lightweight post-unlearning adaptation, making static evaluation insufficient. Therefore, in this paper, we introduce \unlearning as a novel benchmark to understand robust LLM knowledge removal across diverse reasoning paths and recovery attacks. We experiment with this benchmark on 3 models, 6 unlearning methods, and 2 carefully curated datasets. Results show that existing methods are vulnerable to multi-hop reasoning paths and recovery attacks. We further explore the trade-off among forget quality, robustness, and model utility for LLM unlearning.

Haoting Qian, Qingjie Zhang, Zhicong Huang +2

Tsinghua University, China. · Ant Group, China.

Multi-Hop Reasoning Language Model Robustness LLM Unlearning Machine Unlearning Evaluation

Ask PDF

Oct 7, 2026

Open-MMUnlearning: Unifying Methods and Evaluation for MLLM Unlearning

As multimodal large language models (MLLMs) become more capable and widely deployed, concerns about privacy and safety have become increasingly pressing. Machine unlearning offers one approach to addressing these concerns by removing designated information from trained models while preserving unrelated capabilities. However, fragmented implementations and evaluation protocols, incomplete robustness testing, and limited understanding of metric reliability make progress in MLLM unlearning difficult to assess systematically. We introduce Open-MMUnlearning, an open-source, extensible framework that integrates target-model preparation, multimodal data processing, unlearning, and evaluation through shared interfaces and structured configurations. The framework supports five benchmarks spanning privacy, safety, and copyright, eight MLLMs from four model families, and twelve unlearning methods. Its evaluation suite jointly assesses forgetting effectiveness, retained utility, and robustness to model interventions, adversarial inputs, and membership inference attacks. Using a common evaluation protocol, we compare ten representative unlearning methods. In this comparison, GD and MIP-Editor tie for the highest overall score: GD achieves the highest Forget Quality, while MIP-Editor preserves more Model Utility. We further introduce a metric meta-evaluation protocol that tests faithfulness using models with controlled exposure to target knowledge and robustness under quantization and relearning. Among the thirteen evaluated metrics, BLEU achieves the highest aggregate reliability score. KS-Test attains the highest faithfulness AUC but performs less well on robustness. Together, the framework and these findings support reproducible comparison of MLLM unlearning methods and systematic assessment of evaluation reliability.

Junkai Chen, Yuhao He, Qianshan Wei +19

Institute of Automation, Chinese Academy of Sciences · University of the Chinese Academy of Sciences · ByteDance +9

MLLM Unlearning Machine Unlearning Evaluation Multimodal Model Evaluation

Ask PDF

May 23, 2026

Measuring the Depth of LLM Unlearning via Activation Patching

Large language model (LLM) unlearning has emerged as a crucial post-hoc mechanism for privacy protection and AI safety, yet auditing whether target knowledge is truly erased remains challenging. Existing output-level metrics fail to detect when this knowledge remains recoverable from internal representations. Recent white-box studies reveal such residual knowledge but often rely on auxiliary training or dataset-specific adaptations, leaving no generalizable metric. To address these limitations, we propose the Unlearning Depth Score (UDS), a metric that quantifies the mechanistic depth of unlearning via activation patching. UDS first identifies layers that encode the target knowledge using a retain model baseline, then measures how much of it is erased in the unlearned model on a 0-1 scale. In a meta-evaluation across 20 metrics on 150 unlearned models spanning 8 methods, UDS achieves the highest faithfulness and robustness, confirming our causal approach as the most reliable for unlearning evaluation. Case studies further reveal that white-box metrics can disagree at the layer level and that erasure depth varies across examples. We provide guidelines for integrating UDS into existing benchmarking frameworks and streamlining the evaluation pipeline. Code and data are available at https://github.com/gnueaj/unlearning-depth-score

Jaeung Lee, Dohyun Kim, Jaemin Jo

Sungkyunkwan University Republic of Korea

LLM Auditing LLM Unlearning Machine Unlearning Evaluation

Ask PDF

Abstract

LLM Unlearning

Explore similar work

Leak-Resistant Unlearning: A New Benchmark for Evaluating Multi-Hop Reasoning Consistency and Recovery Robustness

Open-MMUnlearning: Unifying Methods and Evaluation for MLLM Unlearning

Measuring the Depth of LLM Unlearning via Activation Patching