The Unlearning Mirage: A Dynamic Framework for Evaluating LLM Unlearning

Raj Sanjay Shah; Jing Huang; Keerthiram Murugesan; Nathalie Baracaldo; Diyi Yang

cs.AIMar 11, 2026

The Unlearning Mirage: A Dynamic Framework for Evaluating LLM Unlearning

Authors: Raj Sanjay Shah, Jing Huang, Keerthiram Murugesan, Nathalie Baracaldo, Diyi Yang

Organizations: Georgia Institute of Technology · Stanford University · IBM Research

Abstract

Unlearning in Large Language Models (LLMs) aims to enhance safety, mitigate biases, and comply with legal mandates, such as the right to be forgotten. However, existing unlearning methods are brittle: minor query modifications, such as multi-hop reasoning and entity aliasing, can recover supposedly forgotten information. As a result, current evaluation metrics often create an illusion of effectiveness, failing to detect these vulnerabilities due to reliance on static, unstructured benchmarks. We propose a dynamic framework that stress tests unlearning robustness using complex structured queries. Our approach first elicits knowledge from the target model (pre-unlearning) and constructs targeted probes, ranging from simple queries to multi-hop chains, allowing precise control over query difficulty. Our experiments show that the framework (1) shows comparable coverage to existing benchmarks by automatically generating semantically equivalent Q&A probes, (2) aligns with prior evaluations, and (3) uncovers new unlearning failures missed by other benchmarks, particularly in multi-hop settings. Furthermore, activation analyses show that single-hop queries typically follow dominant computation pathways, which are more likely to be disrupted by unlearning methods. In contrast, multi-hop queries tend to use alternative pathways that often remain intact, explaining the brittleness of unlearning techniques in multi-hop settings. Our framework enables practical and scalable evaluation of unlearning methods without the need for manual construction of forget test sets, enabling easier adoption for real-world applications. We release the pip package and the code at https://sites.google.com/view/unlearningmirage/home.

Figures & tables

Figure 1: Overview of our framework: Our evaluation framework constructs a knowledge graph from pre-unlearning model outputs, enabling the automatic generation of structured single-hop, multi-hop, and alias-based queries. After applying unlearning, we probe the model to assess residual knowledge. The framework is dynamic, instantiable for any entity, and structured, providing fine-grained control over query complexity.

Forget Assessment
Benchmark	WHP	WMDP	MUSE	TOFU	RWKU	Ours
	( Eldan & Russinovich, 2023 )	( Li et al., 2024 )	( Shi et al., 2024 )	( Maini et al., 2024 )	( Jin et al., 2025 )
# Unlearning Targets	1	2	2	200	200	Any Entity
# Forget Probes	300	4,157	220	4,000	13,131	Dynamic
Forget Corpus	Harry Potter series	PubMed, Github	Books/ News	Syn. QA pairs	N/A	N/A
Retain Corpus	N/A	Wikitext	Fan pages/ News	Syn. QA pairs	N/A	N/A

Table 1: A comparison between existing unlearning benchmarks and our benchmark. Our benchmark allows us to evaluate any entity, allowing us to automatically generate forget probes. Knowledge Memorization probes: these are cloze style probes (“Capital of France is __”); Knowledge Manipulation probes: these are MCQ style Q&A probes; ❍ : While we do not cover Knowledge Manipulation probes, our framework can be easily modified for this probe style. ∗ We cover a subset of adversarial attacks.

	Multi-hop Queries $\downarrow$			Retention criteria $\uparrow$			Multi-hop	Avg.	Overall
Method	1-hop	2-hop	3-hop	1-fact away	2-facts away	Rel. Ret.	Forget Score	Retain	Score
Target model	98.6	97.2	84.1	98.9	98.1	99.1	93.3	98.7	12.5
ICL	14.7	19.2	28.5	34.2	52.5	93.4	20.8	60.0	68.3
GA	19.3	23.8	31.2	44.6	59.3	55.5	24.8	53.1	62.3
GDR	21.8	25.7	32.5	73.8	70.5	76.2	26.7	73.5	73.4
$\textsf{GA}_{\textsf{KLR}}$	22.3	26.2	33.0	74.5	71.2	76.4	27.2	74.0	73.4

Table 2: Scores from our evaluation metric instantiated with the seed entities in RWKU for Llama 3.1-Instruct (8B). Values indicate

\uparrow

means higher is better, and

\downarrow

means lower is better. Methods as described in section 3.2

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

	Multi-hop Queries $\downarrow$			Retention criteria $\uparrow$			Multi-hop	Avg.	Overall
Method	1-hop	2-hop	3-hop	1-fact away	2-facts away	Rel. Ret.	Forget Score	Retain	Score
Target model	94.1	92.6	78.4	94.2	93.8	95.0	88.4	94.3	20.7
ICL	17.7	22.2	31.5	31.2	49.4	89.6	23.8	56.7	65.0
GA	21.6	27.4	34.3	42.1	56.8	52.3	27.8	50.4	59.4
GDR	24.5	29.6	35.6	70.5	67.2	72.5	29.9	70.1	70.1
GKL	24.8	30.2	36.1	71.1	68.3	73.4	30.4	70.9	70.3

Appendix

Table 4: Scores from our evaluation metric instantiated with the seed entities in RWKU for Phi-4-mini-instruct (3.8B). Values indicate

\uparrow

means higher is better, and

\downarrow

means lower is better. Methods as described in section 3.2

	Multi-hop Queries $\downarrow$			Retention criteria $\uparrow$			Multi-hop	Avg.	Overall
Method	1-hop	2-hop	3-hop	1-fact away	2-facts away	Rel. Ret.	Forget Score	Retain	Score
Target model	98.2	96.9	80.4	98.0	97.3	97.5	91.8	97.6	15.1
ICL	13.1	17.6	27.9	35.6	50.3	91.8	19.5	59.2	68.2
GA	18.2	24.7	30.1	43.1	60.8	54.2	24.3	52.7	62.1
GDR	22.6	26.9	30.6	72.2	69.1	74.1	26.7	71.8	72.5
GKL	22.8	27.9	31.4	72.6	69.7	75.2	27.4	72.5	72.6

Appendix

Table 5: Scores from our evaluation metric instantiated with the seed entities in RWKU for IBM Granite 3.2-8B-Instruct. Values indicate

\uparrow

means higher is better, and

\downarrow

means lower is better. Methods as described in section 3.2

	Previous Metrics		Our Metric
Method	Forget Set (All) $\downarrow$	Neighbor Set (All) $\uparrow$	Multi-hop Forget Score $\downarrow$	Avg. Ret. Score $\uparrow$
Target model	77.3	90.7	93.3	98.7
ICL	16.5	55.7	20.8	60.0
GA	39.6	65.5	24.8	53.1
GDR	47.3	74.1	26.7	73.5
$\textsf{GA}_{\textsf{KLR}}$	50.6	70.2	27.2	74.0

Appendix

Table 6: RWKU: Comparison of unlearning effectiveness using previous static metrics vs. our dynamic evaluation framework when using Llama 3.1-Instruct (8B). Spearman’s rank correlation shows strong agreement with prior rankings while revealing new vulnerabilities missed by static benchmarks. Significance levels:

{}^{***}p<0.005

	Multi-hop Queries $\downarrow$			Retention criteria $\uparrow$			Multi-hop	Avg.	Overall
Method	1-hop	2-hop	3-hop	1-fact away	2-facts away	Rel. Ret.	Forget Score	Retain	Score
Target model	92.8	86.3	78.6	94.0	88.7	92.7	85.9	91.8	24.4
ICL	13.8	20.3	30.5	31.5	56.2	87.2	21.5	58.3	66.9
GA	20.5	25.3	28.9	47.9	62.9	59.5	24.8	56.8	64.7
GDR	23.2	24.7	30.7	67.9	74.6	70.6	25.9	71.0	72.5
$\textsf{GA}_{\textsf{KLR}}$	22.0	26.3	30.6	69.7	65.6	70.5	26.3	68.6	71.1

Appendix

Table 7: Our Metric on the TOFU benchmark for Llama 3.1-Instruct (8B). Values indicate

\uparrow

means higher is better, and

\downarrow

means lower is better. Methods: GA – Gradient Ascent; GDR – Gradient Diff (Gradient ascent of forget set with gradient descent of retain set);

\textsf{GA}_{\textsf{KLR}}

– Gradient ascent of forget set with KL Divergence minimization on the retain set; DPO – Direct Preference Optimization;

\textsf{DPO}_{\textsf{GDR}}

– Direct Preference Optimization with Gradient descent retention;

\textsf{DPO}_{\textsf{KLR}}

– Direct Preference Optimization with KL Divergence minimization on the retain set; NPO – Negative Preference Optimization;

\textsf{NPO}_{\textsf{GDR}}

– Negative Preference Optimization with Gradient descent retention;

\textsf{NPO}_{\textsf{KLR}}

– Negative Preference Optimization with KL Divergence minimization on the retain set; ICL – Incontext Unlearning; TV – Task vectors; ULD – Unlearning via Logit Difference.

	Previous Metrics		Our Metric
Method	Forget Quality $\uparrow$	Model Utility $\uparrow$	Multi-hop Forget Score $\downarrow$	Avg. Ret. Score $\uparrow$
Target model	0.00	0.68	85.9	91.8
ICL	NA	NA	21.5	58.3
GA	0.41	0.54	24.8	56.8
GDR	0.26	0.55	25.9	71.0
$\textsf{GA}_{\textsf{KLR}}$	0.45	0.54	26.3	68.6

Appendix

Table 8: TOFU: Comparison of unlearning effectiveness using previous static metrics vs. our dynamic evaluation framework (LLaMa 3.1 (8B)). Spearman’s rank correlation shows strong agreement with prior rankings while highlighting differences missed by static benchmarks. Significance levels:

{}^{**}p<0.01

{}^{***}p<0.005

Benchmark	Benchmark Probe	Equivalent Probe	Answer
TOFU	In which city was Hina Ameen born?	Where was Hina Ameen born?	Karachi, Pakistan
RWKU	Stephen King was born in ___, Maine.	Where was Stephen King born?	Portland

Appendix

Table 9: Examples of semantic equivalence between benchmark probes and our framework. Partial matches are accepted for open-ended queries.

Benchmark	Total probes	# Overlapping probes	Coverage (%)	Avg. graph size
RWKU	13,131	10,256	78.1%	143
TOFU	4,000	2,636	65.9%	36

Appendix

Table 10: Coverage of our evaluation framework with respect to existing unlearning benchmarks. Total Probes refers to the number of queries in the original benchmark. # Overlapping probes counts how many of those probes are semantically matched by our automatically generated probe set. Coverage (%) indicates the proportion of probes reproduced. Avg. graph size is the average number of nodes needed in our knowledge graph to reach maximum overlap with each benchmark.

Methods	Previous Metrics			Our metric									Score
	Removal Crit.		Ret. Crit.	Multi-hop Queries				Fact Ret.		Rel.	Avg.	Avg.
	Vbtim. $\downarrow$	Know. $\downarrow$	Util. $\uparrow$	1-hop $\downarrow$	2-hop $\downarrow$	2-hop - Seq. $\downarrow$	3-hop $\downarrow$	1-hop $\uparrow$	2-hop $\uparrow$	Ret. $\uparrow$	Rem. $\downarrow$	Ret. $\uparrow$	$\uparrow$
Muse-Books
Trgt Model	91.4	59.1	62.2	99.2%	98.5%	97.7%	92.4%	99.4%	98.3%	99.3%	96.9%	99.0%	NA
GA	0	0	0	9.2%	20.5%	20.7%	26.4%	65.8%	69.7%	84.3%	19.2%	73.3%	76.8%
GDR	0	0	10.3	10.8%	22.4%	22.1%	32.3%	71.1%	76.3%	86.2%	21.9%	77.9%	78.0%

Appendix

Table 11: Comparison of Unlearning Methods on Various Metrics on the MUSE-books benchmark Shi et al. (2024) . The target model here is LLama 3.1 - 8B. Methods: GA – Gradient Ascent; GDR – Gradient Diff (Gradient ascent of forget set with gradient descent of retain set);

\textsf{GA}_{\textsf{KLR}}

– Gradient ascent of forget set with KL Divergence minimization on the retain set; DPO – Direct Preference Optimization;

\textsf{DPO}_{\textsf{GDR}}

– Direct Preference Optimization with Gradient descent retention;

\textsf{DPO}_{\textsf{KLR}}

– Direct Preference Optimization with KL Divergence minimization on the retain set; NPO – Negative Preference Optimization;

\textsf{NPO}_{\textsf{GDR}}

–Negative Preference Optimization with Gradient descent retention;

\textsf{NPO}_{\textsf{KLR}}

– Negative Preference Optimization with KL Divergence minimization on the retain set; ICL – Incontext Unlearning; TV – Task vectors; ULD – Unlearning via Logit Difference.

LLM Unlearning

Large language model unlearning studies how to remove specified knowledge or behaviors while preserving a model’s unrelated capabilities.

See all 191 papers

Leak-Resistant Unlearning: A New Benchmark for Evaluating Multi-Hop Reasoning Consistency and Recovery Robustness
Aug 5, 2026Haoting Qian, Qingjie Zhang, Zhicong Huang +2Multi-Hop Reasoning Language Model Robustness
Tsinghua University, China. · Ant Group, China.
Open-MMUnlearning: Unifying Methods and Evaluation for MLLM Unlearning
Oct 7, 2026Junkai Chen, Yuhao He, Qianshan Wei +19MLLM Unlearning Machine Unlearning Evaluation
Institute of Automation, Chinese Academy of Sciences · University of the Chinese Academy of Sciences · ByteDance +9
Measuring the Depth of LLM Unlearning via Activation Patching
May 23, 2026Jaeung Lee, Dohyun Kim, Jaemin JoLLM Auditing LLM Unlearning
Sungkyunkwan University Republic of Korea

Aug 5, 2026J/K move · Enter open · S save

Leak-Resistant Unlearning: A New Benchmark for Evaluating Multi-Hop Reasoning Consistency and Recovery Robustness

Haoting Qian, Qingjie Zhang, Zhicong Huang, Cheng Hong+1

Tsinghua University, China. · Ant Group, China.

Multi-Hop Reasoning Language Model Robustness LLM Unlearning Machine Unlearning Evaluation

Open paper AskPDF arXiv

Benchmarking machine unlearning methods is critical to understand whether sensitive knowledge is removed from large language models (LLMs) or not. Current unlearning benchmarks include mainly single-hop questions and a narrow set of multi-hop questions. Although effective, they still face two challenges. (1) Knowledge is not isolated, whereby diverse multi-hop reasoning paths can potentially induce knowledge leakage than normal queries. (2) Unlearning may be fragile: unlearned knowledge can be partially recovered through recovery attacks such as lightweight post-unlearning adaptation, making static evaluation insufficient. Therefore, in this paper, we introduce \unlearning as a novel benchmark to understand robust LLM knowledge removal across diverse reasoning paths and recovery attacks. We experiment with this benchmark on 3 models, 6 unlearning methods, and 2 carefully curated datasets. Results show that existing methods are vulnerable to multi-hop reasoning paths and recovery attacks. We further explore the trade-off among forget quality, robustness, and model utility for LLM unlearning.

Abstract

LLM Unlearning

Explore similar work

Leak-Resistant Unlearning: A New Benchmark for Evaluating Multi-Hop Reasoning Consistency and Recovery Robustness

Open-MMUnlearning: Unifying Methods and Evaluation for MLLM Unlearning

Measuring the Depth of LLM Unlearning via Activation Patching