A Comprehensive Evaluation of LLM Unlearning Robustness under Multi-Turn Interaction

Ruihao Pan; Suhang Wang

cs.CLFeb 28, 2026

A Comprehensive Evaluation of LLM Unlearning Robustness under Multi-Turn Interaction

Authors: Ruihao Pan, Suhang Wang

Organizations: Pennsylvania State University

Abstract

Machine unlearning aims to remove the influence of specific training data from pre-trained models without retraining from scratch, and is increasingly important for large language models (LLMs) due to safety, privacy, and legal concerns. Although prior work primarily evaluates unlearning in static, single-turn settings, forgetting robustness under realistic interactive use remains underexplored. In this paper, we study whether unlearning remains stable in interactive environments by examining two common interaction patterns: self-correction and dialogue-conditioned querying. We find that knowledge appearing forgotten in static evaluation can often be recovered through interaction. Although stronger unlearning improves apparent robustness, it often results in behavioral rigidity rather than genuine knowledge erasure. Our findings suggest that static evaluation may overestimate real-world effectiveness and highlight the need for ensuring stable forgetting under interactive settings.

Figures & tables

Panel of Figure 1 — Figure 1: Overview of Unlearning Robustness Evaluation. While unlearning may produce safe behavior under conventional static evaluation, this evaluation paradigm may overestimate the extent of knowledge removal. In interactive settings, where outputs are influenced by user feedback or prior conversational context, previously unlearned knowledge can reappear.

Model	R1 $\downarrow$	R2_S1 $\downarrow$	S1 $\Delta$ Ans $\uparrow$	R2_S2 $\downarrow$	S2 CondAcc $\downarrow$	R2_S3 $\downarrow$	S3 $\Delta$ Ans $\downarrow$
Llama	66.93	83.03	99.76	83.11	48.93	65.51	31.26
Llama–GA	24.98	59.23	58.12	75.65	67.54	41.48	17.75
Llama–GA+GD	25.69	72.03	90.27	75.33	66.81	48.00	25.45
Llama–GA+KL	25.77	71.96	90.26	75.26	66.67	48.15	25.53
Llama–NPO	24.90	69.68	98.64	70.15	60.25	47.76	26.00
Llama–NPO+GD	25.37	70.86	99.89	70.86	60.95	52.79	32.60

Table 1: Forgetting robustness on wmdp-bio under self-correction. R1 denotes Round 1 accuracy. R2_S1, R2_S2, and R2_S3 report Round 2 accuracy under the three self-correction strategies. S1

\Delta

Ans measures the answer change rate on previously incorrect examples under S1 (higher indicates stronger responsiveness to corrective feedback). S2 CondAcc reports conditional accuracy after excluding the Round 1 answer under S2 (lower indicates more thorough knowledge suppression). S3

\Delta

Ans measures the answer change rate between R1 and R2 under S3 (lower indicates stronger stability under introspection).

Panel of Figure 2 — Figure 2: Effect of unlearning strength (learning rate) on self-correction robustness (WMDP-Bio accuracy). We vary the learning rate and report R1 and R2 accuracy under three self-correction strategies (S1/S2/S3).

Panel of Figure 3 — Figure 3: Effect of unlearning strength (learning rate) on $\Delta$ Ans across models. Results are shown for GA- and NPO-based models on Llama and Zephyr, with dashed lines representing the Base model performance.

Panel of Figure 4 — Figure 4: Effect of unlearning strength (learning rate) on dialogue-conditioned forgetting robustness (WMDP-Bio accuracy). Results are shown under Base (single-turn) and Dialogue-conditioned settings, with dialogue histories from either the target (WMDP-Bio) or non-target (MMLU) domain.

Model	Base	WMDP-Bio		MMLU
		MCQ	QA	MCQ	QA
Llama	67.32	73.92	73.13	68.97	70.54
Llama–GA	25.06	25.53	26.16	25.45	25.77
Llama–GA+GD	26.47	30.95	31.03	30.79	30.32
Llama–GA+KL	26.47	31.03	31.11	31.19	30.56
Llama–NPO	25.29	31.26	31.03	31.81	29.38

Table 2: Forgetting robustness under dialogue-conditioned interaction (WMDP-Bio accuracy). Base denotes single-turn evaluation without prior dialogue interaction.

Panel of Figure 5 — Figure 5: Effect of unlearning strength (learning rate) on dialogue-conditioned utility preservation (MMLU accuracy). Results are shown under Base (single-turn) and Dialogue-conditioned settings, with dialogue histories from either the target (WMDP-Bio) or non-target (MMLU) domain.

Model	Base	WMDP-Bio		MMLU
		MCQ	QA	MCQ	QA
Llama	57.36	61.69	61.07	62.46	62.50
Llama–GA	24.81	24.91	24.72	25.77	25.49
Llama–GA+GD	26.84	29.37	29.05	31.02	30.59
Llama–GA+KL	26.88	29.45	29.14	31.10	30.67
Llama–NPO	27.70	32.39	31.84	35.77	34.05

Table 3: Utility preservation (MMLU accuracy) under dialogue-conditioned interaction. Base denotes single-turn evaluation without prior dialogue interaction.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Model	R1 $\downarrow$	R2_S1 $\downarrow$	S1 $\Delta$ Ans $\uparrow$	R2_S2 $\downarrow$	S2 CondAcc $\downarrow$	R2_S3 $\downarrow$	S3 $\Delta$ Ans $\downarrow$
Llama	40.66	65.22	99.83	65.27	41.48	40.92	43.28
Llama–GA	24.76	30.90	8.76	64.22	52.44	27.63	2.97
Llama–GA+GD	24.76	30.95	8.83	64.42	52.71	27.73	3.07
Llama–GA+KL	25.21	33.77	13.19	62.86	50.34	28.03	3.67
Llama–NPO	26.93	56.27	97.25	57.02	41.18	33.27	9.06
Llama–NPO+GD	26.93	55.51	98.76	55.91	39.67	33.82	9.86

Appendix

Table 4: Forgetting robustness on wmdp-cyber under self-correction. R1 denotes Round 1 accuracy. R2_S1, R2_S2, and R2_S3 report Round 2 accuracy under the three self-correction strategies. S1

\Delta

\Delta

Ans measures the answer change rate between R1 and R2 under S3 (lower indicates stronger stability under introspection).

Panel of Figure 6 — Figure 6: Effect of unlearning strength (learning rate) on self-correction robustness (WMDP-Cyber accuracy). We vary the learning rate and report R1 and R2 accuracy under three self-correction strategies (S1/S2/S3).

Panel of Figure 7 — Figure 7: Effect of unlearning strength (learning rate) on $\Delta$ Ans across models. Results are shown for GA- and NPO-baseds model on Llama and Zephyr, with dashed lines representing the Base model performance.

Model	Base	WMDP-Cyber		MMLU
		MCQ	QA	MCQ	QA
Llama	41.42	46.70	45.34	43.63	42.93
Llama–GA	24.71	25.52	24.61	28.23	26.77
Llama–GA+GD	24.71	25.36	24.61	28.18	26.77
Llama–GA+KL	25.06	27.43	24.86	29.54	28.49
Llama–NPO	27.08	31.45	29.69	31.10	29.94

Appendix

Table 5: Forgetting robustness under dialogue-conditioned interaction (WMDP-Cyber accuracy). Base denotes single-turn evaluation without prior dialogue interaction.

Model	Base	WMDP-Cyber		MMLU
		MCQ	QA	MCQ	QA
Llama	57.36	61.79	60.25	62.46	62.50
Llama–GA	30.84	28.98	28.21	35.80	35.77
Llama–GA+GD	30.93	28.75	28.09	35.78	35.72
Llama–GA+KL	32.62	31.65	29.95	40.00	39.43
Llama–NPO	27.97	33.76	31.35	40.93	38.78

Appendix

Table 6: Utility preservation (MMLU accuracy) under dialogue-conditioned interaction. Base denotes single turn evaluation without prior interaction.

Panel of Figure 8 — Figure 8: Effect of unlearning strength (learning rate) on dialogue-conditioned forgetting robustness (WMDP-Cyber accuracy). Results are shown under Base (single-turn) and Dialogue-conditioned settings, with dialogue histories from either the target (WMDP-Cyber) or non-target (MMLU) domain.

Panel of Figure 9 — Figure 9: Effect of unlearning strength (learning rate) on dialogue-conditioned utility preservation (MMLU accuracy). Results are shown under Base (single-turn) and Dialogue-conditioned settings, with dialogue histories from either the target (WMDP-Cyber) or non-target (MMLU) domain.

Model	Base	WMDP-Bio		MMLU
		MCQ	QA	MCQ	QA
Llama	67.32	73.76	74.31	69.13	71.33
Llama–GA	25.06	27.26	25.06	25.84	25.29
Llama–GA+GD	26.47	34.56	29.07	31.97	29.38
Llama–GA+KL	26.47	34.72	29.07	32.05	29.38
Llama–NPO	25.29	35.19	25.77	32.05	26.87

Appendix

Table 7: Forgetting robustness under few-shot conditioning (WMDP-Bio accuracy). Base denotes zero-shot evaluation.

Model	Base	WMDP-Cyber		MMLU
		MCQ	QA	MCQ	QA
Llama	41.42	46.90	44.64	42.98	42.88
Llama–GA	24.71	27.03	24.56	28.59	25.21
Llama–GA+GD	24.71	26.98	24.56	28.59	25.21
Llama–GA+KL	25.06	28.74	24.56	30.25	26.62
Llama–NPO	27.08	33.77	28.84	32.11	28.64

Appendix

Table 8: Forgetting robustness under few-shot conditioning (WMDP-Cyber accuracy). Base denotes zero-shot evaluation.

Model	Base	WMDP-Bio		MMLU
		MCQ	QA	MCQ	QA
Llama	57.36	61.81	61.16	62.98	62.73
Llama–GA	24.81	26.25	24.10	26.71	25.22
Llama–GA+GD	26.84	31.79	26.89	33.62	29.31
Llama–GA+KL	26.88	31.85	26.95	33.73	29.37
Llama–NPO	27.70	34.63	27.50	36.33	30.12

Appendix

Table 9: Utility preservation (MMLU accuracy) under few-shot evaluation. Base denotes zero-shot evaluation.

Model	Base	WMDP-Cyber		MMLU
		MCQ	QA	MCQ	QA
Llama	57.36	61.59	60.53	62.98	62.73
Llama–GA	30.84	30.75	27.25	36.58	31.78
Llama–GA+GD	30.93	30.75	27.21	36.62	31.67
Llama–GA+KL	32.62	33.41	28.29	41.87	34.90
Llama–NPO	27.97	38.36	29.49	42.00	34.68

Appendix

Table 10: Utility preservation (MMLU accuracy) under few-shot evaluation. Base denotes zero-shot evaluation.

LLM Unlearning

Large language model unlearning studies how to remove specified knowledge or behaviors while preserving a model’s unrelated capabilities.

See all 191 papers

Can LLMs Truly Forget? Revealing Unlearning Gaps Through Adversarial Evaluation
Aug 21, 2026Ayush Gupta, Hima Varshini Surisetty, Sreevidya Bollineni +5Machine Unlearning LLM Unlearning
University of Massachusetts Amherst · Microsoft Corporation
The Unlearning Mirage: A Dynamic Framework for Evaluating LLM Unlearning
Mar 11, 2026Raj Sanjay Shah, Jing Huang, Keerthiram Murugesan +2LLM Unlearning Machine Unlearning Evaluation
Georgia Institute of Technology · Stanford University · IBM Research
Leak-Resistant Unlearning: A New Benchmark for Evaluating Multi-Hop Reasoning Consistency and Recovery Robustness
Aug 5, 2026Haoting Qian, Qingjie Zhang, Zhicong Huang +2Multi-Hop Reasoning Language Model Robustness
Tsinghua University, China. · Ant Group, China.

Aug 21, 2026J/K move · Enter open · S save

Can LLMs Truly Forget? Revealing Unlearning Gaps Through Adversarial Evaluation

Ayush Gupta, Hima Varshini Surisetty, Sreevidya Bollineni, Varad Ingale+4

University of Massachusetts Amherst · Microsoft Corporation

Machine Unlearning LLM Unlearning Machine Unlearning Evaluation

Open paper AskPDF arXiv

Machine unlearning aims to remove the influence of targeted training data from a model while preserving its remaining capabilities, but evaluating whether such information has truly become inaccessible remains challenging. Existing benchmarks primarily assess unlearning under clean, non-adversarial queries, leaving open whether information that appears forgotten can still be recovered through strategic prompting. We address this gap through a unified evaluation of prompt-based and fine-tuning-based unlearning methods on TOFU using Llama-3.2-3B-Instruct, followed by an adversarial robustness evaluation of methods that perform strongly under standard metrics. We introduce Attack Success Rate (ASR), an LLM-as-judge metric that measures the fraction of adversarial responses whose leakage score exceeds

0.2

, and evaluate recovery across eight attack suites. Our results reveal a substantial gap between clean-query forgetting and adversarial robustness. Although several fine-tuning-based methods achieve Forget Quality above

0.91

, targeted information remains recoverable with ASRs between

72.8\%

and

84.3\%

, close to the

87.5\%

ASR of the unprotected base model. In contrast, clean multilingual reformulations yield only

2.95\%

measured leakage. A manual audit further finds agreement between binary ASR decisions and human factual assessments in seven of ten cases, indicating that ASR provides a useful, though imperfect, signal of behavioral recoverability. These findings show that strong standard-metric performance alone is insufficient to establish robustness after unlearning and motivate adversarial stress-testing as a complementary component of unlearning evaluation.

Abstract

LLM Unlearning

Explore similar work

Can LLMs Truly Forget? Revealing Unlearning Gaps Through Adversarial Evaluation

The Unlearning Mirage: A Dynamic Framework for Evaluating LLM Unlearning

Leak-Resistant Unlearning: A New Benchmark for Evaluating Multi-Hop Reasoning Consistency and Recovery Robustness