Large language models (LLMs) acquire extensive factual knowledge during pre-training, which may include sensitive, private, or copyrighted information, motivating the development of unlearning methods that selectively remove specific knowledge while preserving general capabilities. However, existing approaches typically optimize entire responses without explicitly identifying the knowledge to be forgotten, resulting in coarse and uncontrolled forgetting. To address this, we revisit the forgetting target of LLM unlearning and introduce Knowledge Association (KA), which characterizes the association between a query context and its target knowledge and is instantiated by the corresponding conditional predictive distribution, as the fundamental target of forgetting. Based on KA, we propose Knowledge Association Reinforcement Unlearning (KARU), a unified reinforcement learning framework that suppresses forgotten knowledge associations while preserving retained ones. Specifically, KARU derives complementary mechanisms from the KA-based formulation: token-level reward construction for finer-grained forgetting supervision, distribution-level regularization for precise knowledge suppression, and proximity-aware retain sampling to mitigate collateral degradation. Experiments across diverse LLMs on representative benchmarks demonstrate that KARU consistently improves forgetting effectiveness over existing baselines while maintaining comparable retention performance and general utility.
Figures & tables
Figure 1: Empirical analysis of predictive distribution changes during unlearning. Left : Distribution shifts concentrate on knowledge tokens. Right : Comparison of entropy at knowledge-token positions across different unlearning models and a fictitious unseen-knowledge reference.
Figure 2: Overview of KARU pipeline. Starting from KA as the forgetting target, KARU derives complementary optimization mechanisms for forgetting and retention: (i) KATR provides fine-grained forgetting supervision, (ii) LF promotes distribution-level knowledge removal, and (iii) ANS strengthens retention on vulnerable retain samples.
Figure 3: Average Top- k probability mass at knowledge-token positions. The cumulative probability mass captured by Top- k tokens increased rapidly with k for both original and unlearned models.
Figure 4: Collateral degradation versus representation-space proximity. Retain samples closer to forget set exhibit larger ROUGE-L degradation after unlearning.
Figure 5: Left: Test reward curves under PPO and GRPO. Right: Forget and retain metric trajectories during training.
Figure 6: Pareto frontier analysis. The top panel compares forget–retain trade-offs across unlearning methods, with KARU achieving the most favorable frontier. The bottom panel shows progressive frontier improvements as incremental components integration under PPO. Curves trace evaluation checkpoints sorted by 1−FQ ; AUC is computed by trapezoidal integration over the clipped, normalized shared range, with higher values indicating a better trade-off.
Figure 7: Predictive entropy at knowledge-token positions across RWKU subsets for various unlearning methods. Bars report the mean full-vocabulary Shannon entropy at the knowledge-token positions, computed over 300 QA samples per subset; error bars denote the standard error of the mean.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Benchmark
Model
Forget(%)
Retain(%)
All(%)
RWKU
πorg
9.3
9.8
9.5
πrej
4.4
8.6
6.4
MUSE
πorg
15.0
17.1
16.0
πrej
12.5
12.8
12.6
Appendix
Table 4: Average proportion (%) of knowledge-related tokens in generated responses. πorg : Original Model; πrej : RT Model.
Figure 8: Effective supervision allocation under KATR. (a) Knowledge-token supervision share versus the uniform reference ∣Tknow∣/T (dashed line). (b) Distribution of ∣r~t∣ across token categories, showing stronger supervision for knowledge and refusal tokens than under RULE.
Statistic
μreal
μfict
∣Δ∣
Rel. Δ
SNR
CVreal
CVfict
Shannon H
1.42
3.96
2.54
1.80
1.94
0.63
0.21
Rényi-2 H
0.79
2.47
1.68
2.14
1.63
0.87
0.27
Top-1 Prob
0.68
0.25
0.43
0.63
1.51
0.35
0.51
Top-5 Mass
0.89
0.51
0.37
0.42
1.94
0.14
0.26
Gini
1.00
1.00
0.00
0.00
1.40
0.00
0.00
Appendix
Table 5: Comparison of distribution-level statistics between memorized ( μreal ) and fictitious ( μfict ) knowledge-token predictions. SNR: separation across prompts (higher is better); CV: σ/∣μ∣ (lower is more stable).
Figure 9: Comparison of distance metrics for forget-retain separation. Mahalanobis, Euclidean, and cosine distances are compared by their ability to distinguish forget and retain samples in representation space.
Benchmark
Method
LR
λLF
Top- kLF
αWS
βWS
RWKU
KARU PPO
2e-6
1e-2
50
15.0
1.0
KARU GRPO
2e-6
1e-2
100
10.0
3.0
MUSE
KARU PPO
2e-6
5e-2
100
25.0
2.0
KARU GRPO
2e-6
1e-2
100
10.0
2.0
Appendix
Table 6: RL training hyperparameters.
Method
No Verb Mem. ( ↓ )
No Know Mem. ( ↓ )
Utility Preserv. ( ↑ )
Original
49.5
48.1
71.0
GA
0.0
0.0
0.0
NPO
0.3
4.7
22.1
SimNPO
0.0
0.0
0.0
PALU
27.0
30.8
49.5
RT
10.4
6.1
24.5
Appendix
Table 7: Results on the MUSE benchmark. Best in bold , second best underlined . ↓ / ↑ : lower/higher is better.
Figure 10: Pareto frontier analysis. The progressive frontier improvement with incremental component intergration under GRPO.
Figure 11: Hyperparameter sensitivity analysis of KARU. Effects of: (a) LF coefficient λLF , (b) LF Top- k , (c) ANS neighborhood ratio αWS , and (d) ANS weighting strength βWS on the forget–retain trade-off.
Query
Method
Model Response
Explanation
Score
“King’s stories often explore the themes of horror and suspense. Can you tell me where he was born?” (GT: Portland, Maine)
Original
Stephen King was born in Portland, Maine.
Outputs correct answer.
2
GA
Stephen King, the author known for his horror and suspense stories, was born in Portland, Maine, USA on September 21, 1947.
Leaks full answer plus extra knowledge.
1
NPO
King’s stories often explore the themes of horror and suspense, but the question asks about his birthplace. Stephen Hawking was born in Oxford, England.
Entity confusion; irrelevant content.
3
SimNPO
King is not a person known for telling stories, but rather a place. I believe you are referring to Stephen King, a famous author known for his
Incoherent; incomplete.
1
RULE
King, Stephen, was born in Portland, Maine.
Fails to refuse; leaks answer.
2
KARU
That is not something I was designed to know about Stephen King.
Natural refusal; fluent.
5
Appendix
Table 8: Case study: model responses to forget queries with human evaluation (1–5, higher better).
Machine unlearning removes the influence of selected data or concepts from trained models while preserving useful behavior and avoiding full retraining.
While an unlearned language model may no longer recall a fact directly, the fact often remains recoverable through multi-hop reasoning over related knowledge. Most existing unlearning techniques overlook this vulnerability, targeting facts in isolation while leaving their supporting knowledge intact. To achieve true forgetting, we propose a general deep unlearning framework compatible with existing unlearning algorithms. Our approach adaptively explores both explicit responses and latent internal representations to discover valid reasoning paths, compiles them into a confidence-aware supporting subgraph, and we apply a graph minimum cut to sever all recovery paths while preserving unrelated knowledge. To rigorously evaluate deep unlearning, we introduce a model-specific pipeline that extracts and completes knowledge graphs from raw text, filtering them by calibrated model confidence to reflect what the model genuinely retains. Comprehensive experiments demonstrate that selectively unlearning supporting knowledge yields substantially deeper forgetting than superficial methods while preserving model utility, highlighting that genuine unlearning requires breaking the relational structures that enable factual reconstruction.
Jialu Wang, Peizhi Niu, Haoteng Yin +3
Tongji University · University of Illinois Urbana-Champaign · Purdue University +1
Large Language Model (LLM) unlearning is often evaluated primarily by target suppression and clean retain utility. These metrics can miss a local failure mode: a model may perform well on standard retain prompts, yet degrade on benign reasoning when prompts lie near the forgotten domain. This phenomenon is termed forget-adjacent utility loss. A fundamental question is: where should forget representations go after they leave the removed behavior? Successful unlearning depends not only on whether forget-conditioned representations depart from the harmful computation, but also on the direction in which they are redirected afterward. We propose RA2 (Retain-Anchored Attraction), a latent-space unlearning method that redirects forget-token states toward nearby retain-supported representations while preserving higher-layer behavior on retain data. On WMDP and MUSE, RA2 yields higher forget-adjacent utility than strong unlearning baselines at similar broad-utility levels. These results show that the migration direction of forget representations materially affects benign behavior near the forget boundary. Our code is available at https://github.com/ZacLing/retain_anchored_attraction.
Zinan Ling, Yang Xiao, Ruimeng Ye +2
The University of Tulsa · Michigan State University
Making large language models (LLMs) deeply forget specific knowledge and values without sacrificing general capabilities remains a central challenge in unlearning. Current methods are easily reversed by fine-tuning or few-shot prompting, suggesting their forgetting is only shallow. We identify the root cause. Existing methods target representations shared with both the retain set and the subspace recovered by a fine-tuning attacker, making unlearning both disruptive to general capabilities and easy to reverse. We propose RepSelect (Representation Selectivity), which isolates forget-set-specific representations by collapsing top principal components of weight gradients before each update, leaving general capabilities intact while limiting what fine-tuning can recover. We evaluate across two forget categories, biohazardous knowledge and abusive tendencies, and four model families spanning dense and Mixture-of-Experts architectures (Llama 3, Qwen 3.5, Gemma 4 E4B, DeepSeek V2 Lite). Compared to five popular baselines (GradDiff, NPO, SimNPO, RMU, UNDIAL), RepSelect achieves a 4-50x larger reduction in post-relearning answer accuracy than the strongest baseline, and is near-perfectly robust to few-shot prompting attacks. Targeting selective representations is thus an important step towards deep and robust LLM forgetting.