Forgotten in Weights, Recovered by Tools: Agentic Tool Unlearning for LLM Agents

Baicheng Chen; Zheyuan Liu; Jingyu Zhang; Kaize Ding; Ningshan Ma; Yue Huang; Meng Jiang

cs.CLAug 21, 2026

Forgotten in Weights, Recovered by Tools: Agentic Tool Unlearning for LLM Agents

Authors: Baicheng Chen, Zheyuan Liu, Jingyu Zhang, Kaize Ding, Ningshan Ma, Yue Huang, Meng Jiang

Organizations: The Chinese University of Hong Kong, Shenzhen · University of Notre Dame · Johns Hopkins University · Northwestern University ·

Abstract

Large language models (LLMs) are increasingly deployed as tool-augmented agents, where responses can depend on tool calls and external observations rather than model parameters alone. This creates an evaluation mismatch for LLM unlearning: previous unlearning methods may suppress direct parametric recall, but an agent can still recover the same forget target through tools such as web search, retrieval, or database lookup. We identify this failure mode as tool-mediated recovery and study agentic tool unlearning, which aims to reduce both parametric recall and tool-mediated recovery while preserving normal tool use for retained knowledge. To address this challenge, we propose Agentic Tool Unlearning (ATU), a two-stage framework. The first stage applies parametric knowledge unlearning to suppress direct recall, while the second stage performs trajectory-level reinforcement learning in simulated tool-augmented environments to penalize target-seeking tool behavior and final-answer leakage. Experiments on RWKU and MUSE across different LLM architectures show that ATU achieves a better balance between target forgetting and retained utility, making unlearning more robust under tool-augmented agent deployment.

Figures & tables

Figure 1: Illustration of tool-mediated recovery. A parametrically unlearned LLM fails to recall the forgotten answer, but the same model can recover it after being deployed as a tool-augmented agent.

Figure 2: Motivation of agentic tool unlearning. Standard parametric unlearning (NPO) reduces forget-set scores in standalone inference, but the same unlearned model recovers much of the forgotten knowledge when deployed as a tool-augmented agent.

Figure 3: Overview of ATU. Stage 1 applies parametric knowledge unlearning on forget and retain data to suppress direct recall while preserving utility. Stage 2 trains the unlearned model in simulated tool-augmented environments, where trajectory-level rewards penalize knowledge recovery and final-answer leakage while encouraging appropriate tool use. The model is updated with GRPO to reduce both parametric recall and tool-mediated recovery.

Reward	Trajectory-level Criterion
$+1.0$	Safe refusal or non-informative response without revealing the target answer.
$+0.3$	Ambiguous response that does not clearly leak the target answer.
$-0.5$	Risky tool use or target-seeking behavior, but no exposed answer.
$-1.0$	Target-related information appears in the trajectory.
$-2.0$	Final response reveals or paraphrases the reference answer.

Table 1: Reward rubric for forget queries. Rewards are assigned at the trajectory level, enabling penalties for tool-mediated recovery attempts even before the target answer appears in the final response.

Reward	Trajectory-level criterion
$+1.0$	Correct and helpful answer, with appropriate tool use when needed.
$+0.5$	Partially correct answer or minor formatting issue.
$0.0$	Unclear answer without severe hallucination or refusal.
$-0.5$	Incorrect answer, unsupported answer, or irrelevant tool use.
$-1.0$	Unnecessary refusal or severe hallucination.

Table 2: Reward rubric for retain queries. Rewards are assigned at the trajectory level to preserve general response quality and discourage over-refusal after Stage 2 training.

Method	Deployment	Forget Set $\downarrow$					Retain Utility $\uparrow$
Method	Deployment	FB	QA	AA	All	$\Delta$ All	Neighbor (All)	Factuality	Fluency
Base model
Base Model	LLM	45.7	55.2	60.9	57.7	–	52.7	34.2	692.7
	LLM w/ Agent	62.5	79.8	74.1	74.6	$+\,16.9$	63.8	–	–
Stage 1: Parametric knowledge unlearning only
GA	LLM	20.7	38.1	54.9	45.8	–	46.6	31.9	677.8

Table 3: Main results on RWKU with Qwen-3-4B. We compare Stage-1 parametric unlearning under standalone and tool-augmented deployment, and evaluate Stage-2 agentic tool unlearning under agent deployment. Forget-set scores are ROUGE-L recall on FB, QA, AA, and weighted average All; lower is better.

\Delta

All measures the change in the All score relative to the corresponding baseline row. Retain utility measures neighboring knowledge, factuality, and fluency; higher is better. Bold numbers denote the best results among agent-deployment settings.

Figure 4: RWKU utility scores with Qwen-3-4B across Base, Stage 1, and Stage 2 models.

Method and Deployment	Qwen-3-4B		Qwen-2.5-7B
Method and Deployment	knowmem_f $\downarrow$	knowmem_r $\uparrow$	knowmem_f $\downarrow$	knowmem_r $\uparrow$
Base Model	37.9	53.4	17.7	25.4
Base Model w/ Agent	43.3	67.8	34.9	43.9
Stage 1 (NPO)	9.7	47.0	10.0	25.4
Stage 1 (NPO) w/ Agent	29.8	56.7	32.9	44.1
Stage 1 (NPO) + Stage 2	15.8	55.1	22.4	40.5

Table 4: Results on MUSE (Books) across backbone models. We report knowledge memorization on the forget split ( knowmem_f ) and retain split ( knowmem_r ) following the MUSE evaluation protocol. Lower knowmem_f indicates better forgetting, while higher knowmem_r indicates better retain-side utility.

Figure 5: General tool-use utility on target-independent retain tasks. Stage 2 preserves high contains-match rates across different Stage-1 initializations, indicating that ATU does not globally suppress tool use.

Figure 6: Tool-use behavior on forget queries. Stage 1 increases tool use as an alternative recovery path, while Stage 2 reduces target-seeking tool calls across different unlearning methods.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Method	Training data	LR / Epochs	Key settings
RWKU	GA	Forget data	$1\times 10^{-5}$ / 10	LoRA; target modules q_proj,v_proj
RWKU	RT	Rejection data	$3\times 10^{-5}$ / 10	Refusal-style SFT; LoRA
RWKU	DPO	Preference pairs	$6\times 10^{-5}$ / 10	Reference model = original base model; dpo_ftx =0.0
RWKU	NPO	Forget data	$5\times 10^{-5}$ / 10	Reference model = original base model; $\beta=0.2$
MUSE	Target FT	Forget + retain data	$1\times 10^{-5}$ / 25	LoRA rank 8; target modules all ; cutoff 2048
MUSE	NPO	Forget data	$1\times 10^{-5}$ / 10	Reference model = pre-unlearning model; $\beta=0.1$

Appendix

Table 5: Stage-1 parametric unlearning hyperparameters. RWKU uses LoRA training with LLaMA-Factory and merges adapters before evaluation. MUSE follows the official iterative unlearning setup.

Method	Deployment	Forget Set $\downarrow$					Retain Utility $\uparrow$
Method	Deployment	FB	QA	AA	All	$\Delta$ All	Neighbor (All)	Factuality	Fluency
Base model
Base Model	LLM	26.8	43.8	56.6	49.3	–	59.1	43.0	707.2
	LLM w/ Agent	54.0	58.3	72.8	67.6	$+\,18.3$	64.7	–	–
Stage 1: Parametric knowledge unlearning only
NPO	LLM	6.3	21.7	47.1	35.1	–	64.1	43.4	669.9

Appendix

Table 6: Additional results on RWKU with Qwen-2.5-7B. We evaluate NPO as the Stage 1 parametric unlearning method and apply Stage 2 agentic tool unlearning on top of the NPO checkpoint. Forget-set metrics are ROUGE-L recall scores on FB, QA, AA, and weighted average All; lower is better.

\Delta

All measures the change in the average forget score: for LLM w/ Agent rows, it is computed against the corresponding standalone LLM row; for the Stage 2 row, it is computed against the Stage 1 NPO LLM w/ Agent row. Retain utility measures neighboring knowledge, factuality, and fluency; higher is better. Bold numbers denote the best result among agent-deployment settings.

Setting	FB $\downarrow$	QA $\downarrow$	AA $\downarrow$	All $\downarrow$	Neigh. $\uparrow$
Base Model	39.3	65.0	60.1	58.2	69.5
Base w/ Agent	69.6	67.9	95.7	86.7	79.3
Stage 1 (NPO)	13.6	23.1	33.7	28.3	45.1
NPO w/ Agent	87.5	92.9	81.9	86.0	80.7
NPO + Stage 2	18.6	37.6	36.2	33.8	85.7

Appendix

Table 7: RWKU results on Qwen-3-8B backbone.

Method	Original Tools			Swapped Tools
Method	Forget (All) $\downarrow$	$\Delta$ All	ToolUse F (%) $\downarrow$	Forget (All) $\downarrow$	$\Delta$ All	ToolUse F (%) $\downarrow$
Stage 1 (NPO) w/ Agent	68.6	$+\,24.0$	76.6	64.0	$+\,19.4$	69.8
Stage 1 (NPO) + Stage 2 w/ Agent	47.5	$-\,21.1$	63.6	46.7	$-\,17.3$	58.4

Appendix

Table 8: Tool-set generalization on RWKU with Qwen-3-4B. Original Tools denotes the tool set used in the main agentic evaluation, while Swapped Tools replaces the retrieval tools with different tools of the same type, such as alternative web-search APIs. The Stage 2 checkpoint is not retrained on the swapped tools. Forget (All) denotes the weighted average forget-set ROUGE-L recall, and ToolUse F denotes the forget-set tool-use rate.

Method	All Forget Score $\downarrow$			$\Delta_{\mathrm{fw}}$	$\Delta_{\mathrm{tool}}$
Method	LLM	Agent w/o Tools	Agent w/ Tools	$\Delta_{\mathrm{fw}}$	$\Delta_{\mathrm{tool}}$
Base Model	57.7	62.2	74.6	$+\,4.5$	$+\,12.4$
Stage 1 (NPO)	44.6	45.8	68.6	$+\,1.2$	$+\,22.8$
Stage 1 (NPO) + Stage 2	–	45.1	47.5	–	$+\,2.4$

Appendix

Table 9: Evaluation without tools on RWKU with Qwen-3-4B.

\Delta_{\mathrm{fw}}=\mathrm{All}_{\mathrm{Agent\ w/o\ Tools}}-\mathrm{All}_{\mathrm{LLM}}

measures the effect of the agent framework itself, while

\Delta_{\mathrm{tool}}=\mathrm{All}_{\mathrm{Agent\ w/\ Tools}}-\mathrm{All}_{\mathrm{Agent\ w/o\ Tools}}

measures the additional recovery caused by tool access.

Model	Held-out accuracy $\uparrow$
Base	80.0%
Stage 1 (NPO)	92.5%
Stage 2 (NPO)	87.5%

Appendix

Table 10: Contains-match accuracy on four unseen tool families.

Method (w/ Agent)	FB	QA	AA	All $\downarrow$	Neigh. $\uparrow$
Stage 1 NPO	48.2	78.3	69.9	68.6	60.6
+ Refusal SFT	25.0	76.7	62.9	59.9	58.8
+ Trajectory DPO	25.0	83.3	57.8	58.1	60.5
+ ATU	12.5	58.3	52.7	47.5	55.8

Appendix

Table 11: Controlled comparison between ATU and simpler Stage 2 alternatives initialized from the same Stage 1 NPO checkpoint. Refusal SFT uses fixed safe-refusal targets, while trajectory-level DPO constructs preference pairs from judged agent responses. All methods use the same prompts and tool-augmented evaluation protocol. We report FB, QA, AA, aggregated All, and Neighbor scores; lower forget-set scores and higher Neighbor scores are better.

Method	Cost (h) $\downarrow$	All $\downarrow$	$\Delta$ All
Stage 1 NPO	0.82	68.6	–
+ Refusal SFT	1.10	59.9	$-\,8.7$
+ Trajectory DPO	1.35	58.1	$-\,10.5$
+ ATU	1.97	47.5	$-\,21.1$

Appendix

Table 12: Controlled cost-benefit comparison. Cost denotes the training time of the stage reported in each row under the same hardware. The common evaluation cost is excluded.

Adaptive directive	Stage 1 NPO All $\downarrow$	Stage 2 ATU All $\downarrow$
No attack	68.6	47.5
Retrieve before answering	89.9 ( $+\,21.3$ )	59.6 ( $+\,12.1$ )
Authority-based retrieval	78.7 ( $+\,10.1$ )	44.3 ( $-\,3.2$ )
Retrieve with explicit target	80.6 ( $+\,12.0$ )	50.6 ( $+\,3.1$ )
Copy exact tool output	71.1 ( $+\,2.5$ )	57.7 ( $+\,10.2$ )
Reproduce search output verbatim	62.2 ( $-\,6.4$ )	40.6 ( $-\,6.9$ )

Appendix

Table 13: Robustness under adaptive prompt directives that explicitly require retrieval, target-focused tool queries, or reproduction of tool outputs. Each directive raises forget-set tool use to 100%, creating a stronger adversarial deployment condition than the default evaluation. We report the aggregated RWKU All score for the Stage 1 NPO agent and the Stage 2 ATU agent; parentheses show the change relative to each checkpoint’s own no-attack condition. Lower scores indicate less target recovery.

Method	FB	QA	AA	All $\downarrow$	Neigh. $\uparrow$
Stage 1 NPO + Agent	33.3	75.0	50.0	52.6	49.7
Stage 2 + Agent	33.3	50.0	33.3	36.8	51.1

Appendix

Table 14: Held-out-query evaluation on a disjoint 60/40 split.

Reward judge	Sign agr. $\uparrow$	Exact match $\uparrow$	Forget All $\downarrow$	$\Delta$ vs. 7B	Neighbor $\uparrow$
Qwen3-4B	58.3%	26.4%	53.1	+5.6	46.9
Qwen2.5-7B	83.3%	63.9%	47.5	–	55.8
Qwen3-8B	61.1%	41.7%	60.0	+12.5	62.6

Appendix

Table 15: Human validation and reward-judge sensitivity analysis. Two annotators independently score a stratified set of 72 forget and retain trajectories, yielding adjudicated human reference labels. Sign agreement measures reward-polarity agreement, while exact match requires the full discrete reward score to coincide. Separate Stage 2 models are trained with Qwen3-4B, Qwen2.5-7B, and Qwen3-8B judges, and are compared using downstream Forget All and Neighbor scores.

Diagnostic	Value
Training steps	128
Mean reward	$-0.44$
First/last 10-step reward	$-0.65/-0.43$
Steps with mean reward $+1.0$	14
$+1.0$ with $<15$ tokens	0
Mean KL	$\approx 0.50$

Appendix

Table 16: Summary statistics for the 128-step GRPO training audit in the main Qwen-3-4B policy and Qwen-2.5-7B judge setting. The diagnostics track reward evolution, the frequency of maximum-reward steps, the occurrence of ultra-short maximum-reward responses, KL divergence from the Stage 1 initialization, and the corresponding change in downstream Forget All. These measurements are used to check for unstable optimization and the specific short-response reward-hacking behavior considered in our analysis.

LLM Unlearning

Large language model unlearning studies how to remove specified knowledge or behaviors while preserving a model’s unrelated capabilities.

See all 191 papers

Can Agents Truly Forget? Explore Machine Unlearning for LLM Agents
Oct 13, 2026Junkai Lin, Ge Xu, Zheng Xu +2Machine Unlearning LLM Unlearning
Institute of Automation, Chinese Academy of Sciences · The Chinese University of Hong Kong · Nanyang Technological University +2
Secure Forgetting: A Framework for Privacy-Driven Unlearning in Large Language Model (LLM)-Based Agents
Apr 1, 2026Dayong Ye, Tainqing Zhu, Congcong Zhu +5LLM Unlearning Privacy-Preserving Language Models
K-Bench: A Benchmark for LLM Unlearning in Agentic Deployments
Sep 11, 2026Guangsheng Yu, Yanna Jiang, Qin Wang +2LLM Agent Evaluation Privacy Leakage in Language Models
University of Technology Sydney, Australia · CSIRO, Australia

Oct 13, 2026J/K move · Enter open · S save

Can Agents Truly Forget? Explore Machine Unlearning for LLM Agents

Junkai Lin, Ge Xu, Zheng Xu, Junkai Chen+1

Institute of Automation, Chinese Academy of Sciences · The Chinese University of Hong Kong · Nanyang Technological University+2

Machine Unlearning LLM Unlearning Machine Unlearning Evaluation LLM Agent Memory

Open paper AskPDF arXiv

Large language model (LLM) agents have shown strong capabilities in interactive problem solving, tool use, and long-horizon task execution. However, they may also acquire sensitive or private knowledge during training and expose it during deployment. Retraining a model from scratch to remove such knowledge is prohibitively costly, making machine unlearning a more practical approach. Machine unlearning aims to remove targeted undesirable knowledge from trained models without retraining from scratch while preserving general utility. When language models are deployed as agents, parameter-level unlearning becomes insufficient: forgotten information may still persist in memory artifacts or be reacquired via tool calls. In this paper, we formulate the problem of agent unlearning as removing private information from an LLM-based agent across three information channels: parametric knowledge, persistent memory, and tool-call acquisition. To study this problem, we introduce we introduce Agent Unlearning Benchmark (AUBench), a controlled benchmark that extends the fictitious-author setting of TOFU into an agent environment with persistent memory, an offline web snapshot, tool-accessible documents, and provenance-tracked interaction trajectories. By evaluating existing parameter-level and memory-level unlearning methods in this setting, AUBench reveals the gap between model-level forgetting and system-level agent unlearning, and provides a foundation for developing unlearning methods that operate across the full lifecycle of LLM agents.

Abstract

LLM Unlearning

Explore similar work

Can Agents Truly Forget? Explore Machine Unlearning for LLM Agents

Secure Forgetting: A Framework for Privacy-Driven Unlearning in Large Language Model (LLM)-Based Agents

K-Bench: A Benchmark for LLM Unlearning in Agentic Deployments