Large language models (LLMs) acquire extensive factual knowledge during pre-training, which may include sensitive, private, or copyrighted information, motivating the development of unlearning methods that selectively remove specific knowledge while preserving general capabilities. However, existing approaches typically optimize entire responses without explicitly identifying the knowledge to be forgotten, resulting in coarse and uncontrolled forgetting. To address this, we revisit the forgetting target of LLM unlearning and introduce Knowledge Association (KA), which characterizes the association between a query context and its target knowledge and is instantiated by the corresponding conditional predictive distribution, as the fundamental target of forgetting. Based on KA, we propose Knowledge Association Reinforcement Unlearning (KARU), a unified reinforcement learning framework that suppresses forgotten knowledge associations while preserving retained ones. Specifically, KARU derives complementary mechanisms from the KA-based formulation: token-level reward construction for finer-grained forgetting supervision, distribution-level regularization for precise knowledge suppression, and proximity-aware retain sampling to mitigate collateral degradation. Experiments across diverse LLMs on representative benchmarks demonstrate that KARU consistently improves forgetting effectiveness over existing baselines while maintaining comparable retention performance and general utility.
Figures & tables
Figure 1: Empirical analysis of predictive distribution changes during unlearning. Left : Distribution shifts concentrate on knowledge tokens. Right : Comparison of entropy at knowledge-token positions across different unlearning models and a fictitious unseen-knowledge reference.
Figure 2: Overview of KARU pipeline. Starting from KA as the forgetting target, KARU derives complementary optimization mechanisms for forgetting and retention: (i) KATR provides fine-grained forgetting supervision, (ii) LF promotes distribution-level knowledge removal, and (iii) ANS strengthens retention on vulnerable retain samples.
Figure 3: Average Top- k probability mass at knowledge-token positions. The cumulative probability mass captured by Top- k tokens increased rapidly with k for both original and unlearned models.
Figure 4: Collateral degradation versus representation-space proximity. Retain samples closer to forget set exhibit larger ROUGE-L degradation after unlearning.
Figure 5: Left: Test reward curves under PPO and GRPO. Right: Forget and retain metric trajectories during training.
Figure 6: Pareto frontier analysis. The top panel compares forget–retain trade-offs across unlearning methods, with KARU achieving the most favorable frontier. The bottom panel shows progressive frontier improvements as incremental components integration under PPO. Curves trace evaluation checkpoints sorted by 1−FQ ; AUC is computed by trapezoidal integration over the clipped, normalized shared range, with higher values indicating a better trade-off.
Figure 7: Predictive entropy at knowledge-token positions across RWKU subsets for various unlearning methods. Bars report the mean full-vocabulary Shannon entropy at the knowledge-token positions, computed over 300 QA samples per subset; error bars denote the standard error of the mean.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Benchmark
Model
Forget(%)
Retain(%)
All(%)
RWKU
πorg
9.3
9.8
9.5
πrej
4.4
8.6
6.4
MUSE
πorg
15.0
17.1
16.0
πrej
12.5
12.8
12.6
Appendix
Table 4: Average proportion (%) of knowledge-related tokens in generated responses. πorg : Original Model; πrej : RT Model.
Figure 8: Effective supervision allocation under KATR. (a) Knowledge-token supervision share versus the uniform reference ∣Tknow∣/T (dashed line). (b) Distribution of ∣r~t∣ across token categories, showing stronger supervision for knowledge and refusal tokens than under RULE.
Statistic
μreal
μfict
∣Δ∣
Rel. Δ
SNR
CVreal
CVfict
Shannon H
1.42
3.96
2.54
1.80
1.94
0.63
0.21
Rényi-2 H
0.79
2.47
1.68
2.14
1.63
0.87
0.27
Top-1 Prob
0.68
0.25
0.43
0.63
1.51
0.35
0.51
Top-5 Mass
0.89
0.51
0.37
0.42
1.94
0.14
0.26
Gini
1.00
1.00
0.00
0.00
1.40
0.00
0.00
Appendix
Table 5: Comparison of distribution-level statistics between memorized ( μreal ) and fictitious ( μfict ) knowledge-token predictions. SNR: separation across prompts (higher is better); CV: σ/∣μ∣ (lower is more stable).
Figure 9: Comparison of distance metrics for forget-retain separation. Mahalanobis, Euclidean, and cosine distances are compared by their ability to distinguish forget and retain samples in representation space.
Benchmark
Method
LR
λLF
Top- kLF
αWS
βWS
RWKU
KARU PPO
2e-6
1e-2
50
15.0
1.0
KARU GRPO
2e-6
1e-2
100
10.0
3.0
MUSE
KARU PPO
2e-6
5e-2
100
25.0
2.0
KARU GRPO
2e-6
1e-2
100
10.0
2.0
Appendix
Table 6: RL training hyperparameters.
Method
No Verb Mem. ( ↓ )
No Know Mem. ( ↓ )
Utility Preserv. ( ↑ )
Original
49.5
48.1
71.0
GA
0.0
0.0
0.0
NPO
0.3
4.7
22.1
SimNPO
0.0
0.0
0.0
PALU
27.0
30.8
49.5
RT
10.4
6.1
24.5
Appendix
Table 7: Results on the MUSE benchmark. Best in bold , second best underlined . ↓ / ↑ : lower/higher is better.
Figure 10: Pareto frontier analysis. The progressive frontier improvement with incremental component intergration under GRPO.
Figure 11: Hyperparameter sensitivity analysis of KARU. Effects of: (a) LF coefficient λLF , (b) LF Top- k , (c) ANS neighborhood ratio αWS , and (d) ANS weighting strength βWS on the forget–retain trade-off.
Query
Method
Model Response
Explanation
Score
“King’s stories often explore the themes of horror and suspense. Can you tell me where he was born?” (GT: Portland, Maine)
Original
Stephen King was born in Portland, Maine.
Outputs correct answer.
2
GA
Stephen King, the author known for his horror and suspense stories, was born in Portland, Maine, USA on September 21, 1947.
Leaks full answer plus extra knowledge.
1
NPO
King’s stories often explore the themes of horror and suspense, but the question asks about his birthplace. Stephen Hawking was born in Oxford, England.
Entity confusion; irrelevant content.
3
SimNPO
King is not a person known for telling stories, but rather a place. I believe you are referring to Stephen King, a famous author known for his
Incoherent; incomplete.
1
RULE
King, Stephen, was born in Portland, Maine.
Fails to refuse; leaks answer.
2
KARU
That is not something I was designed to know about Stephen King.
Natural refusal; fluent.
5
Appendix
Table 8: Case study: model responses to forget queries with human evaluation (1–5, higher better).
Machine unlearning removes the influence of selected data or concepts from trained models while preserving useful behavior and avoiding full retraining.