cs.LGOct 14, 2026

RA2^2: Retain-Anchored Attraction Preserves Forget-Adjacent Utility in LLM Unlearning

Authors: Zinan Ling, Yang Xiao, Ruimeng Ye, Sijia Liu, Bo Hui

Organizations: The University of Tulsa · Michigan State University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Large Language Model (LLM) unlearning is often evaluated primarily by target suppression and clean retain utility. These metrics can miss a local failure mode: a model may perform well on standard retain prompts, yet degrade on benign reasoning when prompts lie near the forgotten domain. This phenomenon is termed forget-adjacent utility loss. A fundamental question is: where should forget representations go after they leave the removed behavior? Successful unlearning depends not only on whether forget-conditioned representations depart from the harmful computation, but also on the direction in which they are redirected afterward. We propose RA2^2 (Retain-Anchored Attraction), a latent-space unlearning method that redirects forget-token states toward nearby retain-supported representations while preserving higher-layer behavior on retain data. On WMDP and MUSE, RA2^2 yields higher forget-adjacent utility than strong unlearning baselines at similar broad-utility levels. These results show that the migration direction of forget representations materially affects benign behavior near the forget boundary. Our code is available at https://github.com/ZacLing/retain_anchored_attraction.

Figures & tables

Figure 1: General utility and forget-adjacent utility can separate under aligned unlearning comparisons . Some unlearned models remain close to the original model on a general-utility metric such as MMLU, yet fall more sharply on forget-adjacent evaluations.
MethodPlain questionAfter adding a bio-adjacent prefix
GA [ 66 ]“The ;. ( ; @ @ ; …”“The\n\n\n…”
RMU [ 28 ]“ Chlorophyll , gives plants their green color…”“This is a saaaa…”
RMU-LAT [ 46 ]“ Chlorophyll , specifically chlorophyll a and chlorophyll b…”“Are / In / In / In / T / T / …”
RA 2 (Ours)“ Chlorophyll is the pigment that gives plants their green color…”“Answer: Chlorophyll , a green pigment found in the chloroplasts…”
Table 1: Qualitative bio-prefix example. Target question: “What pigment gives plants their green color?” Expected answer: chlorophyll .
Scored targetContextBehaviorEvaluation
BenignOrdinaryPreserveMMLU, TruthfulQA
BenignForget-relatedPreserveMMLU-P, WMDP →\to MMLU, Bio-MNCP
Forget knowledgeBenign prefixSuppressMMLU →\to WMDP
Forget knowledgeDirect querySuppressWMDP
Table 2: Evaluation settings by target and context. Benign targets are preserved under admissible context transformations; designated forget targets are suppressed.
Figure 2: Cross-layer view of RA 2 . Left colors mark the frozen retain , aligned retain , and reshape layers. Right colors mark retain states , anchors , forget states , and redirected states .
 Direct Forgetting ↓\downarrowGeneral Utility ↑\uparrowForget-Adjacent Utility
ModelWMDP- BioWMDP- CyberMMLUTruthfulQA- ESMMLU-P ↑\uparrowWMDP →\rightarrow MMLU ↑\uparrowA−\mathbf{A^{-}} ↑\uparrowGcue\mathbf{G_{\mathrm{cue}}} ↓\downarrow
Zephyr-7B- β\beta63.744.058.134.861.155.953.10.000
++ GA27.324.927.00.026.2 Δo ⁣−57.1%\Delta_{o}\!-57.1\%25.6 Δo ⁣−54.3%\Delta_{o}\!-54.3\%20.8 Δo ⁣−60.9%\Delta_{o}\!-60.9\%0.072
++ GradDiff27.628.950.80.047.3 Δo ⁣−22.6%\Delta_{o}\!-22.6\%27.1 Δo ⁣−51.6%\Delta_{o}\!-51.6\%41.1 Δo ⁣−22.6%\Delta_{o}\!-22.6\%0.627
++ JensUn25.526.724.00.123.2 Δo ⁣−62.0%\Delta_{o}\!-62.0\%24.1 Δo ⁣−56.9%\Delta_{o}\!-56.9\%31.5 Δo ⁣−40.6%\Delta_{o}\!-40.6\%0.093
++ RMU28.626.656.534.552.6 Δo ⁣−13.9%\Delta_{o}\!-13.9\%29.5 Δo ⁣−47.2%\Delta_{o}\!-47.2\%50.2 Δo ⁣−5.5%\Delta_{o}\!-5.5\%1.440
Table 3: Aligned WMDP comparison across direct forgetting, general utility, and forget-adjacent utility. In forget-adjacent columns, badges such as Δo ⁣−15.7%\Delta_{o}\!-15.7\% report Δo=100×(Mmethod−Morig)/Morig\Delta_{o}=100\times(M_{\mathrm{method}}-M_{\mathrm{orig}})/M_{\mathrm{orig}} , where MM is the corresponding same-backbone metric. Values are means over ten seed runs. Full error bars are in Table 14 .
 MMLU-PWMDP →\rightarrow MMLU
MethodLLROUGE-LLLROUGE-L
Original−3.52-3.5215.00−10.99-10.9927.91
GA−435.88-435.880.00−472.27-472.270.00
GradDiff−673.02-673.020.00−757.97-757.970.00
RMU−8.30-8.3015.25−14.72-14.7218.13
RMU-LAT−39.81-39.819.87−95.42-95.425.48
Table 4: Complete-answer likelihood and free generation on 500 examples per task. LL is sentence log-likelihood; ROUGE-L is measured against the gold answer and reported as a percentage. Higher is better for both metrics.
Figure 3: Directional sweeps of the forget-side update. Repulsive directions can achieve even stronger direct forgetting, but only attraction preserves local retain continuity over a useful range of strengths.
Changed componentBio ↓\downarrowCyber ↓\downarrowMMLU ↑\uparrowWMDP →\rightarrow MMLU ↑\uparrow
Forget-Side Force
Default RA 2 (attract, m=5000m=5000 , k=3k=3 , λret=260\lambda_{\mathrm{ret}}=260 )29.327.356.643.0
No attraction ( λattr=0\lambda_{\mathrm{attr}}=0 )64.743.058.955.8
Weak attraction ( λattr=0.003\lambda_{\mathrm{attr}}=0.003 )62.743.958.354.4
Repel direction (sign-flipped, strength 0.010.01 )29.526.357.124.0
Retain Consistency
Table 5: RA 2 ablations under aligned WMDP evaluation. Rows vary forget-side direction, retain consistency, and anchor-field quality.
SplitVariantVerbatim Mem. ↓\downarrowKnowledge Mem. ↓\downarrowPrivacy LeakRetain Know. ↑\uparrow
BooksTarget ftargetf_{\mathrm{target}}99.8059.40-57.5066.90
Retrain fretrainf_{\mathrm{retrain}}14.2129.150.1569.10
GAKLR\mathrm{GA}_{\mathrm{KLR}}16.0021.90-40.2037.20
NPOKLR\mathrm{NPO}_{\mathrm{KLR}}17.0025.00-43.5044.60
Task Vector99.7052.40-57.5064.70
WHP18.0055.7056.5063.60
Table 6: MUSE-Books knowledge-removal results.
Figure 4: t-SNE of answer probabilities.
ModelMethodTrainingEvalTotal
Zephyr-7B- β\betaRMU∼\sim 1m36s∼\sim 8 min∼\sim 9m36s
Zephyr-7B- β\betaRA 2∼\sim 5 min∼\sim 8 min∼\sim 13 min
Zephyr-7B- β\betaRMU-LAT∼\sim 16m28s∼\sim 8 min∼\sim 24m28s
Qwen3-8BRMU∼\sim 1 min∼\sim 11 min∼\sim 12 min
Qwen3-8BRA 2∼\sim 2 min∼\sim 11 min∼\sim 13 min
Qwen3-8BRMU-LAT∼\sim 11m02s∼\sim 11 min∼\sim 22m02s
Table 7: Running time comparison.
QuantizationWMDP Bio ↓\downarrowWMDP Cyber ↓\downarrowMMLU ↑\uparrowMMLU Perturbed ↑\uparrow
unquantized0.29300.27300.56600.5570
int8_bnb0.29850.28280.56150.5702
nf4_bnb0.28990.29290.55750.5692
Table 8: RA 2 robustness under post-training quantization.
Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

NotationMeaning
Models, Data, and Evaluation
RA 2Retain-Anchored Attraction, the proposed latent-space unlearning method.
fθf_{\theta}Parameterized model under evaluation or optimization.
fθ0f_{\theta_{0}}Frozen original model used as the reference for retain consistency and logit-drift comparisons.
θS\theta_{\mathcal{S}}Editable parameter subset optimized during RA 2 training.
Dret\mathcal{D}_{\mathrm{ret}}Retain-side data distribution containing benign examples whose behavior should be preserved.
Appendix
Table 9: Core notation.
MethodPlain outputAfter adding a bio-adjacent prefix
GAThe ;. ( ; @ @ ; ...The\n\n\n...
RMUChlorophyll, gives plants their green color...This is a saaaa...
RMU-LATChlorophyll, specifically chlorophyll a and chlorophyll b, gives plants their green color...Are / In / In / In / T / T / T / R / T / R / T
RA 2 (Ours)Chlorophyll is the pigment that gives plants their green color. It ...answer: Chlorophyll, a green pigment found in the chloroplasts of plant cells...
Appendix
Table 10: Output excerpts for the qualitative bio-prefix example.
Target evaluationOriginalRMURMU-LATRA 2
WMDP-Bio63.727.331.729.3
WMDP-Cyber44.026.729.627.3
MMLU →\to WMDP-Bio62.327.731.329.6
MMLU →\to WMDP-Cyber40.727.731.225.3
Appendix
Table 11: Direct and reverse mixed-query suppression. Zephyr accuracy (%). The direct rows are controls for the reverse-context experiment. The scored target is always WMDP, so lower is preferred.
SettingZephyr-7B- β\betaQwen3-8B
ModelHuggingFaceH4/zephyr-7b-beta [ 55 ]Qwen/Qwen3-8B [ 64 ]
Reshape layer ℓr\ell_{r}710
Retain-consistency layer ℓt\ell_{t}811
Trainable target layers5–88–11
Edited parameter group6 (MLP down-proj)8 (MLP down-proj)
Anchor initializationFPSFPS
Appendix
Table 12: Main RA 2 hyperparameters for the WMDP experiments.
ModelMethodBatchTrainingEvalTotal
Zephyr-7B- β\betaRMU4∼\sim 1m36s∼\sim 8 min∼\sim 9m36s
Zephyr-7B- β\betaRA 24∼\sim 5 min∼\sim 8 min∼\sim 13 min
Zephyr-7B- β\betaRMU-LAT4∼\sim 16m28s∼\sim 8 min∼\sim 24m28s
Qwen3-8BRMU1∼\sim 1 min∼\sim 11 min∼\sim 12 min
Qwen3-8BRA 21∼\sim 2 min∼\sim 11 min∼\sim 13 min
Qwen3-8BRMU-LAT1∼\sim 11m02s∼\sim 11 min∼\sim 22m02s
Appendix
Table 13: Approximate runtime for one training run and standard WMDP/MMLU evaluation.
MethodWMDP Bio ↓\downarrowWMDP Cyber ↓\downarrowWMDP Avg. ↓\downarrowMMLU ↑\uparrowMMLU Perturbed ↑\uparrowWMDP →\boldsymbol{\rightarrow} MMLU ↑\uparrowA−\mathbf{A^{-}} ↑\uparrowGcue\mathbf{G_{\mathrm{cue}}} ↓\downarrow
GA0.2725\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0145}0.2489\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0061}0.2607\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0099}0.2702\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0269}0.2624\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0134}0.2555\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0009}0.2078\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0166}0.0720\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0090}
GradDiff0.2760\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0106}0.2893\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0127}0.2826\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0112}0.5076\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0731}0.4731\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0670}0.2705\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0077}0.4109\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0364}0.6269\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.1502}
JensUn0.2548\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0105}0.2673\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0082}0.2610\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0068}0.2401\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0148}0.2321\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0138}0.2410\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0068}0.3152\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0196}0.0931\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0092}
RMU0.2862\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0103}0.2663\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0103}0.2762\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0038}0.5650\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0026}0.5262\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0142}0.2954\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0199}0.5016\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0226}1.4401\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.1057}
RMU-LAT0.3063\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0105}0.2871\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0082}0.2967\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0068}0.5751\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0148}0.4672\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0138}0.2454\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0068}0.4233\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0196}2.5362\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.2512}
RA 20.2983\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0042}0.2924\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0033}0.2953\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0030}0.5662\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0006}0.5686\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0026}0.4221\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0059}0.5672\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0075}0.6000\,{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.0330}
Appendix
Table 14: Zephyr-7B- β\beta WMDP results over ten seeds; smaller gray terms denote standard deviations.
QuantizationWMDP Bio ↓\downarrowWMDP Cyber ↓\downarrowMMLU ↑\uparrowMMLU Perturbed ↑\uparrow
unquantized reference0.29300.27300.56600.5570
int8_bnb0.29850.28280.56150.5702
nf4_bnb0.28990.29290.55750.5692
Appendix
Table 15: RA 2 robustness under post-training quantization.
StageWMDP-Bio ↓\downarrowMMLU ↑\uparrow
154.6057.43
231.5855.55
329.6954.28
427.4253.34
Appendix
Table 16: Continuous WMDP-Bio unlearning over four sequential forget-data partitions, with 150 training steps per stage.
IdxDirectionStrengthBio ↓\downarrowCyber ↓\downarrowWMDP Avg. ↓\downarrowMMLU ↑\uparrowMMLU-P ↑\uparrowW →\rightarrow M ↑\uparrow
1repel-0.013331.0327.3829.2057.3824.7224.80
2repel-0.010031.2627.2329.2557.1624.8925.91
3repel-0.006730.4026.1728.2957.1824.8425.45
4repel-0.003329.6928.8429.2757.2330.4726.67
5off0.000064.7343.0353.8858.8761.1255.81
6attract+0.003362.5343.9453.2358.3660.6854.64
Appendix
Table 17: Signed forget-strength sweep. Metric values are percentages; strength is the signed forget-side coefficient.
AngleSourceWMDP Avg. ↓\downarrowMMLU ↑\uparrowMMLU-P ↑\uparrowW →\rightarrow M ↑\uparrow
-150planar sweep27.5556.8723.8925.20
-120planar sweep27.0256.8124.0924.91
-90planar sweep28.7756.9530.3223.50
-60planar sweep30.8556.7453.9034.62
-30planar sweep29.7456.9055.1234.55
0canonical attract +0.01+0.0130.5956.7557.2144.64
Appendix
Table 18: Planar angle sweep between the attract and repel endpoints. Metric columns are percentages; angle is in degrees.
SplitVariantVerbatim Mem. ↓\downarrowKnowledge Mem. ↓\downarrowPrivacy LeakRetain Know. ↑\uparrow
NewsTarget ftargetf_{\mathrm{target}}58.4063.90-99.8055.20
Retrain fretrainf_{\mathrm{retrain}}20.2132.34-0.1554.42
GAGDR\mathrm{GA}_{\mathrm{GDR}}4.9031.00108.1027.30
NPOKLR\mathrm{NPO}_{\mathrm{KLR}}26.9049.00-95.8045.40
Task Vector57.2066.20-99.8055.80
WHP19.7021.20109.6028.30
Appendix
Table 19: MUSE-News knowledge-removal results. This split complements MUSE-Books under a different corpus structure.
MethodWMDP avg. ↓\downarrowMMLU ↑\uparrowMMLU-P ↑\uparrowW →\rightarrow M ↑\uparrowBio-MNCP ↑\uparrow
GA29.934.230.725.6—
GradDiff28.551.247.427.6—
JensUn25.622.922.922.9—
GA+KLR26.546.138.523.0—
NPO+KLR52.858.560.655.6—
Task Vector24.524.724.724.7—
Appendix
Table 20: Zephyr-7B- β\beta comparison using the common baseline set. All metrics are percentages; dashes indicate unreported values.
MethodVerbMem-F ↓\downarrowKnowMem-F ↓\downarrowKnowMem-R ↑\uparrowPrivacyLeak
Books
Target99.8059.4066.90-57.50
Retrain14.2129.1569.100.15
GA0.000.000.00-25.00
GradDiff / GA+GDR0.000.0010.70-26.50
JensUn80.6744.3561.34-58.17
Appendix
Table 21: MUSE results for a common baseline set. VerbMem-F and KnowMem-F measure forget-side memorization; KnowMem-R measures retained knowledge. PrivacyLeak is signed, with a target of zero.
MethodMMLU-P LL ↑\uparrowMMLU-P PPL ↓\downarrowW →\rightarrow M LL ↑\uparrowW →\rightarrow M PPL ↓\downarrow
Original-3.521.58-10.994.16
GA-435.883.57×10243.57{\times}10^{24}-472.274.00×10264.00{\times}10^{26}
GradDiff-673.028.14×10378.14{\times}10^{37}-757.974.96×10424.96{\times}10^{42}
RMU-8.302.94-14.726.75
RMU-LAT-39.81174.71-95.422.37×1052.37{\times}10^{5}
RA 2-8.172.89-11.004.17
Appendix
Table 22: Complete-answer sentence log-likelihood (LL) and word-normalized perplexity (PPL) on 500 examples per task.
MethodGold ↑\uparrowOriginal ↑\uparrowDegen. ↓\downarrow
MMLU-P
Original15.00100.000.00
GA0.000.00100.00
GradDiff0.000.00100.00
RMU15.2549.160.00
RMU-LAT9.8742.430.20
Appendix
Table 23: Free-generation scores (percentages). Gold and Original denote ROUGE-L against the gold option text and the original model’s generation. Degen. is the degenerate-output rate.
MethodForget vs. Forget ↓\downarrowRetain vs. Retain ↑\uparrowRetain vs. Combined ↑\uparrow
Original1.00001.00000.2379
GA0.12170.23390.0561
GA+KL0.15910.28690.0799
NPO0.07990.18800.0571
NPO+KL0.08340.19610.0322
SCRUB0.21220.33350.1042
Appendix
Table 24: BLUR ROUGE-L recall against original-model responses. Forget vs. Forget measures residual forget-response overlap; Retain vs. Retain and Retain vs. Combined measure retain-response overlap under retain-only and combined queries.
MethodWMDP ↓\downarrowMMLU-P ↑\uparrowW →\rightarrow M ↑\uparrowBio-MNCP ↑\uparrowMMLU ↑\uparrow
GA31.5733.1625.6823.4436.85
GA + RA 228.3857.2644.5251.5756.96
NPO29.9454.5245.3148.4454.17
NPO + RA 229.8057.8149.1051.5957.26
SimNPO31.3750.1133.5939.0652.49
SimNPO + RA 230.2457.5843.9951.5657.06
Appendix
Table 25: Logit-space objectives combined with retain-anchor routing and retain consistency. All scores are percentages.
Retain fractionExamplesMMLU-P ↑\uparrowW →\rightarrow M ↑\uparrowBio-MNCP ↑\uparrowMMLU ↑\uparrow
100%50056.6741.7756.2556.56
50%25053.2144.1948.4452.85
20%10058.2852.0453.1255.90
10%5057.4150.1150.0054.90
5%2558.1151.8248.4456.03
Appendix
Table 26: Sensitivity to retain-data volume with a fixed number of example presentations.
SettingWMDP ↓\downarrowMMLU ↑\uparrowMMLU-P ↑\uparrowW →\rightarrow M ↑\uparrow
Default: m=5000m=5000 , k=3k=3 , cosine29.3856.5656.6741.77
m=512m=51228.8856.7255.6136.95
m=2000m=200028.5956.7756.3239.42
k=1k=131.0956.6357.7042.13
k=16k=1628.9156.6756.4438.00
Euclidean distance29.6356.7156.7937.26
Appendix
Table 27: Sensitivity to anchor budget mm , neighborhood size kk , and assignment distance. All metric values are percentages.
MethodMain timeMain memoryMatched timeMatched memory
GA194.40 s (1.89 ×\times )17.03 GiB101.86 s (3.03 ×\times )17.00 GiB
GradDiff194.44 s (1.89 ×\times )17.03 GiB101.83 s (3.03 ×\times )17.00 GiB
RMU103.10 s (1.00 ×\times )40.80 GiB33.65 s (1.00 ×\times )36.54 GiB
RA 2308.87 s (3.00 ×\times )63.81 GiB41.35 s (1.23 ×\times )38.26 GiB
RMU-LAT2261.33 s (21.93 ×\times )47.06 GiB212.89 s (6.33 ×\times )41.05 GiB
Appendix
Table 28: Training runtime and peak GPU memory. Parentheses give runtime relative to RMU within the same configuration.
Line of workRepresentative examplesRelevance to this paper
Benchmarks and suitesTOFU, WMDP, MUSE, OpenUnlearning, DUSK [ 37 , 28 , 47 , 7 , 22 ]Define the forgetting, retention, privacy, leakage, and robustness contracts that make LLM unlearning measurable.
Objective-level methodsGradient ascent/difference, NPO, ULD, SPUL, GRU, belief-based and embedding-corrupted objectives [ 8 , 70 , 24 , 1 , 58 , 27 , 57 , 35 , 30 ]Provide practical baselines that alter output probabilities, prompts, or training losses without explicitly building a retain-supported latent target.
Latent and mechanistic methodsRMU, RMU-LAT, CLReg, KUDA, SRMU, EC, LUNAR, mechanistic unlearning [ 28 , 46 , 51 , 11 , 4 , 26 , 45 , 15 ]Closest methodological family; RA 2 keeps the latent-intervention view but changes the target geometry from unsupported displacement to local retain attraction.
Mixed-prompt and overlap robustnessRNA, SEPS, BLUR, form-independent unlearning, retain-set, context-aware, and retain–forget entanglement evaluation [ 20 , 23 , 19 , 68 , 3 , 42 , 6 ]Motivates forget-adjacent utility: benign targets can fail when prompts move near forgotten content even if broad retain metrics remain strong.
Post-unlearning evaluationsRelearning, quantization recovery, adversarial leakage, trace-based evaluations [ 18 , 9 , 50 , 71 , 72 , 5 ]Study how unlearned behavior changes under post-hoc training, compression, prompting, or diagnostic testing.
Appendix
Table 29: A compact map of the related-work landscape. The categories are not mutually exclusive; they separate the main evaluation or method question each line of work makes salient.
More in this topic

Machine Unlearning

Machine unlearning removes the influence of selected data or concepts from trained models while preserving useful behavior and avoiding full retraining.

See all 343 papers

Explore similar work

CardsList

  1. BLADE: Bilevel Low-rank Augmented-Lagrangian Erasure for LLM Unlearning

    Aug 23, 2026Md Toufikuzzaman, Ahmad Mousavi, Dongwon LeeMachine UnlearningLLM Unlearning

    The Pennsylvania State University, USA · American University

  2. RepSelect: Robust LLM Unlearning via Representation Selectivity

    Jun 15, 2026Filip Sondej, Yushi Yang, Adam MahdiLLM SafetyRepresentation Geometry in Language Models

    Independent · University of Oxford

  3. Contrastive Representation Shaping for LLM Unlearning

    Jan 29, 2026Haoran Tang, Rajiv KhannaDisentangled Representation LearningMachine Unlearning

    Department of Computer Science, Purdue University.