Don't Forget What You Just Learned: Self-Directed Repair Turns Agent Mistakes Into Lasting Lessons

Yuval Kansal; Karen Ullrich; Polina Kirichenko; Ilia Kulikov; Niraj K. Jha; Mohammad Pezeshki; Mark Ibrahim

cs.AIOct 22, 2026

Don't Forget What You Just Learned: Self-Directed Repair Turns Agent Mistakes Into Lasting Lessons

Authors: Yuval Kansal, Karen Ullrich, Polina Kirichenko, Ilia Kulikov, Niraj K. Jha, Mohammad Pezeshki, Mark Ibrahim

Organizations: Princeton University · Meta FAIR

Abstract

Intelligent agents should learn from their mistakes so that experience leads to continuous improvement. Yet today's agents recover from a mistake only temporarily: in-context reflections are easily forgotten across sessions and compete for limited context, while full-parameter fine-tuning is expensive and can interfere with existing capabilities. We introduce Self-Direction, a mechanism for agents to learn from mistakes that combines the benefits of temporary in-context retries and policy updates. When an agent fails a task, it reflects on its own trajectory in order to extract a lesson and immediately distills it into a lightweight low-rank adapter. By keeping the original weights frozen, Self-Direction prevents adapter training from overwriting base-model parameters while training additional parameters equivalent to ~0.02% of the base model's parameter count for each adapter. The agent then saves these adapters in a persistent library of lessons, automatically routing them to solve related problems later. Experiments with agents based on open-weight models from the Qwen family across open-ended research (GAIA), tool-use workflows (Claw-Eval), and terminal tasks (Terminal-Bench 2.1) show that our method improves overall success rates by 8.9 to 17.0 percentage points and performs competitively with frontier systems. We find that Self-Direction reduces repeated mistakes across future encounters of a task, nearly doubling the reliability of an agent's corrections. Learned lessons also transfer: a handful of adapters generalize to dozens of new tasks without additional training. Ultimately, this self-directed repair transforms everyday mistakes into lasting lessons.

Figures & tables

Figure 1: Self-Direction turns an agent’s own failures into reusable LoRA lessons. Left: Current agents fail to reliably reuse lessons learned by spending extra resources correcting a mistake. Self-Direction enables storage and reuse of lessons to avoid repeated mistakes. Right: On Terminal-Bench 2.1, Self-Direction lifts performance from 51.7% to 65.5%, outperforming existing baselines and frontier models 2 2 2 Frontier model numbers were obtained from https://www.tbench.ai/?version=2.1 . .

Figure 2: Self-Direction outperforms alternative mistake-repair strategies on GAIA. SVD-initialized LoRA achieves success on $92/165$ tasks ( $55.8\%$ ), outperforming Retrying (pass@3), Reflections in Context, and Accumulating Reflections in Skills.md. Self-Direction recovers 28 mistakes, yielding a $17.0$ -point gain over the initial base execution. We also compare our results with Failed Trajectory in Context and doubling the tool call budget in Appendix C.2 .

Figure 3: Self-Direction pipeline overview. On detecting a mistake, the agent either self-reflects on the failed trajectory and trains a new LoRA adapter or retrieves an adapter already stored in the library of lessons; the selected adapter is then loaded and the task retried.

Figure 4: Self-Direction improves task success across distinct agentic environments. Left: Across three runs, Self-Direction improves mean success on Claw-Eval from $64.4\%$ to $73.3\%$ and outperforms other baselines. Error bars denote one sample standard deviation; cross-run pass@ $3$ and pass 3 results are reported in Appendix C.3 . Terminal-Bench 2.1 and GAIA results are reported in Figs. 2 and 2 , respectively. Right: Self-Direction enables a 27B model to outperform several larger frontier models across pass@ $3$ and pass 3 metrics for Claw-Eval tasks. 5 5 footnotemark: 5

Figure 5: Self-Direction reduces the performance variability of the base agent. Left: Base agent performance varies within the same benchmark in the exact same setting, ranging from 48.3% to 56.2%. Self-Direction helps maintain reliable performance across all runs. Right: Self-Direction increases pass@ $3$ from $64.0\%$ to $74.2\%$ and pass 3 from $34.8\%$ to $57.3\%$ . Reference frontier scores are provided for context to highlight differences between different harnesses and agentic settings.

Benchmark	Encounters	Mistakes avoided ( $\mathrm{pass}^{k}$ )			Corrections preserved
	$k$	Base retry	Self-Direction	Gain	Base retry	Self-Direction
GAIA	2	$9.9\%$	$21.8\%$	+11.9 pp	$45.5\%$	$78.6\%$
	3	$6.9\%$	$19.8\%$	+12.9 pp	$31.8\%$	$71.4\%$
Terminal-Bench 2.1	2	$13.6\%$	$20.5\%$	+6.8 pp	$50.0\%$	$69.2\%$
	3	$13.6\%$	$15.9\%$	+2.3 pp	$50.0\%$	$53.8\%$
Claw-Eval	2	$15.0\%$	$22.5\%$	+7.5 pp	$75.0\%$	$81.8\%$

Table 1: Self-Direction prevents repaired mistakes from recurring. Learned lessons increase success across repeated encounters and preserve repairs more reliably than base-model retrying. Results cover 101 GAIA, 44 Terminal-Bench 2.1, and 40 Claw-Eval prior mistakes.

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Task group	Tasks	Initial base	Base pass@ $3$	Full loop
			(Qwen3.6-27B)	(no lesson retention)	(lessons retained)
MiniWoB++	social-media-all	25	$6/25$ ( $24\%$ )	$11/25$ ( $44\%$ )	$10/25$ ( $40\%$ )
MiniWoB++	click-collapsible-2	25	$8/25$ ( $32\%$ )	$11/25$ ( $44\%$ )	$11/25$ ( $44\%$ )
MiniWoB++	email-inbox	25	$15/25$ ( $60\%$ )	$18/25$ ( $72\%$ )	$18/25$ ( $72\%$ )
MiniWoB++	email-inbox-nl-turk	25	$13/25$ ( $52\%$ )	$15/25$ ( $60\%$ )	$15/25$ ( $60\%$ )
MiniWoB++	Overall	100	$42/100$ ( $42\%$ )	$55/100$ ( $55\%$ )	$54/100$ ( $54\%$ )

Appendix

Table 2: Additional results on browser use environments. Self-Direction enables meaningful gains in tasks involving browser use.

Generalization	Lessons learned from	Library	Evaluation tasks	Improvement
Tasks in a continual stream GAIA	3 GAIA tasks	3 adapters	GAIA deployment stream ( $n=165$ )	+16.3 pp [-1pt] $36.4\%\rightarrow 52.7\%$
Harder tasks GAIA	Easy Level 1 tasks	28 adapters	Harder Level 2 tasks ( $n=86$ )	+10.47 pp [-1pt] $40.7\%\rightarrow 51.2\%$
Tasks across categories Claw-Eval	Operations tasks	3 adapters	Unseen workflow tasks ( $n=24$ )	+12.5 pp [-1pt] $75.0\%\rightarrow 87.5\%$

Appendix

Table 3: Self-Direction transfers learned lessons to unseen tasks. Stored lessons generalize to new tasks in a continual deployment stream, harder tasks, and related task categories.

Figure 6: Expanded GAIA recovery results by difficulty level. Each panel compares base resampling, Failed Trajectory in Context, Reflections in Context, reflections in Skills.md, and SVD-LoRA. Stacked bars distinguish initial base successes from tasks recovered by each strategy. SVD-LoRA improves performance on all three levels and reaches $92/165$ tasks overall.

Figure 7: Expanded three-run results on Claw-Eval. The left panel decomposes each run into initial base successes, additional full-loop recoveries, and unresolved tasks. The right panel reports cross-run coverage (pass@ $3$ ) and consistency (pass 3 ). The full loop improves both metrics and produces a positive accuracy gain in every run. Reference-model scores are included for context and may use different agent configurations.

Figure 8: Terminal-Bench 2.1 outcomes across eight complete runs. Each bar decomposes success from the initial base execution and additional recovery by the full loop. Self-Direction improves all eight runs and raises mean accuracy from $51.8\%$ to $64.0\%$ .

Panel of Figure 9 — Figure 9: Self-Direction improves repeated-run reliability on Terminal-Bench 2.1. Left: pass k , the fraction of tasks solved in each of $k$ runs, remains consistently higher for the full loop and increases from $25.8\%$ to $48.3\%$ at $k=8$ . Lines report the mean over all subsets of $k$ runs, with shaded bands showing the minimum and maximum. Right: The loop preserves $100\%$ pass 8 on easy tasks while increasing reliability from $29.1\%$ to $58.2\%$ on medium tasks and from $10.0\%$ to $23.3\%$ on hard tasks.

Stage	Attempts	New wins	Stage success	Cumulative wins	Overall accuracy
Phase 1	89	23	25.8%	23	25.8%
Phase 4: adapter	65	7	10.8%	30	33.7%
Phase 4: base fallback	58	1	1.7%	31	34.8%
Final	89	31	–	31	34.8%

Appendix

Table 4: Terminal-Bench 2.1 results for Qwen3.5-9B

Stratification	Group	N	Mean Accuracy		Pass 3
			Initial base	Full loop	Initial base	Full loop
Overall	All tasks	161	64.4%	73.3%	51.6%	64.6%
Category	Workflow	47	78.7%	87.9%	61.7%	78.7%
	Ops	31	89.2%	98.9%	80.6%	96.8%
	Finance	14	28.6%	40.5%	14.3%	28.6%
	Office QA	10	33.3%	43.3%	20.0%	20.0%

Appendix

Table 5: Claw-Eval results stratified by task category and structure. Mean accuracy is computed across three independent runs, while pass 3 requires success in all three runs. Task categories partition the 161 tasks. Task-structure rows describe three separate binary attributes: multi-service, state-changing, and safety-primary.

Stratification	Group	N	Mean Accuracy		Pass 3
			Initial base	Full loop	Initial base	Full loop
Overall	All tasks	89	51.7%	65.5%	34.8%	57.3%
Difficulty	Easy	4	100.0%	100.0%	100.0%	100.0%
	Medium	55	62.4%	78.2%	43.6%	70.9%
	Hard	30	25.6%	37.8%	10.0%	26.7%
Category	Software Engineering	26	38.5%	47.4%	26.9%	42.3%

Appendix

Table 6: Terminal-Bench 2.1 results stratified by difficulty and task category. Mean accuracy is computed across three independent runs, while pass 3 requires success in all three runs. Self-Direction improves both medium and hard tasks and produces gains across a broad range of technical categories.

Figure 10: Self-Direction reaches higher accuracy with comparable task-environment rollouts. Relative to base pass@

k

, Self-Direction outperforms the pass@

k

baseline across all three benchmarks while using a comparable number of full agent rollouts. Reflection, synthesis, and adapter optimization details are presented in Appendix F.3 .

Figure 11: A compact library of lessons amortizes learning across a deployment-like GAIA stream. Existing lessons avoid new adapter training for $88.7$ - $97.1\%$ of initial failures. Bars show adapters trained; annotations report storage and successful reuses; the green line shows training avoided through reuse.

Benchmark Pair	Max. Similarity	Safe $\bm{\tau}$	Cross-Domain Retrievals	Capabilities Preserved
GAIA–TB 2.1	0.546	0.55	0 / 145	$\checkmark$
GAIA–Claw	0.415	0.42	0 / 141	$\checkmark$
TB 2.1–Claw	0.306	0.31	0 / 84	$\checkmark$
All three, one global $\tau$	0.546	0.55	0 / 370	$\checkmark$

Appendix

Table 7: Self-Direction learns new lessons without disturbing existing capabilities. The safe threshold is the smallest two-decimal cutoff above the largest similarity observed across the pair. At this threshold, no evaluated task retrieves an adapter from the other domain in either direction, preventing cross-domain adapter application (

\checkmark

Figure 12: Accuracy across router thresholds

\tau

with a fixed library of cross-benchmark lessons. Left: GAIA tasks failed by the base agent are retried with the corresponding top-matched adapter from Terminal-Bench and Claw-Eval libraries. Right: Terminal-Bench 2.1 tasks failed by the base agent are retried with the corresponding top-matched adapter from GAIA and Claw-Eval libraries. We see existing capabilities are preserved except for low thresholds dipping below 0.30, where adapters from other benchmarks in some cases can help or hurt accuracy on the new benchmark.

Figure 13: Self-Direction continues to improve a stronger base model. We apply the same failure-conditioned loop to Qwen3.6-27B and Qwen3.8-27B on identical benchmark tasks. Blue denotes tasks solved by initial base execution, green denotes additional tasks recovered by the complete loop. Self-Direction produces substantial gains for both models across GAIA, Claw-Eval, and Terminal-Bench 2.1.

Figure 14: Recovery after an initial failure on GAIA. The complete Self-Direction loop recovers $27.7\%$ of the failures. Text memory stores lessons in Skills.md ; the single-LoRA baselines pool experience across failures and the mismatched condition applies an unrelated adapter.

Stored content	Approximate tokens	Native context windows
Full trajectories for 101 failures	$2.72$ M	$10.4\times$
Truncated failure representations	188K	$0.72\times$
2,428 lesson instruction-response pairs	362K	$1.38\times$
Lessons with formatting and rationales	410K	$1.56\times$
Full trajectories and lessons	$3.13$ M	$11.9\times$

Appendix

Table 8: Context required to retain the complete GAIA learning history. Counts use the Qwen3.6 tokenizer. Totals are rounded because this section is intended as a capacity analysis.

Hyperparameter	GAIA / Claw-Eval	Terminal-Bench 2.1
Temperature	$0.6$	$0.6$
Top- $p$	$0.95$	$0.95$
Maximum new tokens per turn	16,384	16,384
Maximum ReAct steps	50	100
Maximum tool calls	100	100
Stop generation after tool call	Yes	Yes

Appendix

Table 9: Agent generation and interaction hyperparameters. The token limit applies independently to each model turn within the ReAct loop, rather than to the complete task trajectory.

Component	Hyperparameter	Value
Corrective data	Maximum examples per failure	24
Corrective data	Minimum examples required	8
Corrective data	Maximum assistant-response length	16,384 tokens
Corrective data	Trajectory steps retained for reflection	24
LoRA	Rank $r$	16
LoRA	Scaling parameter $\alpha$	16

Appendix

Table 10: Adapter synthesis and training hyperparameters. These settings are shared across GAIA, Claw-Eval, and Terminal-Bench 2.1. Unlisted TrainingArguments retain their library defaults.

Reason	Operational definition	Target modules
step_budget	The agent reaches its maximum number of interaction steps before verified completion.	q_proj , v_proj
tool_call_budget	The agent exhausts its permitted tool calls before completing the task.	q_proj , v_proj
no_final_response	The execution ends without an extractable final response and was not terminated by a budget or exception.	q_proj , v_proj
wrong_answer	The agent produces a response but the verifier rejects the resulting answer or environment state.	down_proj
exception	An execution error prevents the agent from completing the task.	down_proj

Appendix

Table 11: Normalized failure reasons used by Self-Direction. Each categorical reason determines which projections are updated during adaptation.

Benchmark	Adapters	Training tokens	Synthesis	Optimization	End-to-end	GPU energy
		per adapter	(min)	(s)	(min)	(Wh)
GAIA	46	4,059	8.51	80.9	9.89	37.0
Claw-Eval	39	3,709	5.82	80.3	7.19	26.5
Terminal-Bench 2.1	45	3,100	5.79	81.4	7.22	27.1
Overall	130	3,622	6.76	80.9	8.16	30.4

Appendix

Table 12: Cost of constructing a modular lesson adapter. Values are averages over 130 failures across the three benchmarks. End-to-end latency includes reflection, data preparation, optimization, and adapter serialization on one NVIDIA H200. Energy denotes measured GPU energy over the same interval.

Benchmark	Available tools
GAIA	web_search , web_fetch , python_exec , shell , file_read , file_open , and list_dir . These tools support web research, numerical computation, shell commands, directory inspection, and extraction from text, PDF, spreadsheet, HTML, and image files.
Claw-Eval	Email and contacts: gmail_list_messages , gmail_get_message , gmail_save_draft , gmail_send_message , contacts_search , contacts_get , and contacts_send_message . Calendar, tasks, and scheduling: calendar_list_events , calendar_get_event , calendar_create_event , calendar_get_user_events , calendar_delete_event , todo_list_tasks , todo_get_task , todo_update_task , todo_create_task , todo_delete_task , scheduler_list_jobs , scheduler_get_job , scheduler_job_history , scheduler_create_job , scheduler_update_job , and scheduler_delete_job . Business services: finance, CRM, inventory, reporting, helpdesk, and integration-management tools for listing, inspecting, updating, and creating benchmark entities. Knowledge and content: kb_search , kb_get_article , kb_update_article , notes_list , notes_get , notes_share , rss_list_feeds , rss_list_articles , rss_get_feed , rss_get_article , rss_publish , web_search , and web_fetch . Documents and media: documents_extract_text , ocr_extract_text , and caption_describe_image . The nine sandbox tasks additionally expose Bash , Read , Write , Edit , Glob , and Grep inside an isolated workspace.
Terminal-Bench 2.1	terminal_shell , which executes Bash commands inside a fresh, task-specific Podman container. Commands share a persistent filesystem for the duration of an attempt and return their exit code, standard output, and standard error. The shell supports inspecting files and processes, editing or creating artifacts, installing dependencies, compiling software, configuring services, and running agent-authored validation checks.

Appendix

Table 13: Tool interfaces used by each benchmark. The Claw-Eval row reports the union of tools across the evaluated tasks; each individual task exposes only its task-specific subset. Tool names are shown as presented to the agent.

Figure 15: Self-Direction recovers failing tasks. A qualitative example showing how Self-Direction reflects on the failure, correctly identifies overuse of invalid tools, and generates corrective examples that ultimately help the agent succeed on a GAIA Level-1 task.

Source adapter	Later target	Shared procedure	Observed recovery
ORCID publication aggregation (Level 3)	Wikipedia revision counting (Level 2)	Traverse a complete remote history, handle pagination, apply a date constraint, and verify completeness before counting.	Initial answer: 2610. Adapter answer: 2732 (correct).
BBC Earth video lookup (Level 1)	Exact quotation from another video (Level 1)	Identify the correct video and extract a localized fact using descriptions, timestamps, comments, or other supporting metadata.	The initial execution exhausted its step budget; the adapter returned the exact quotation.
Spreadsheet aggregation (Level 1)	Statistics over values extracted from an image (Level 2)	Extract structured values, inspect intermediate data, compute with a tool, verify the calculation, and format only after validation.	Initial answer: 2.404. Adapter answer: 17.056 (correct).
Caesar-cipher decoding (Level 2)	Text-grid reconstruction (Level 1)	Apply a text transformation, check linguistic coherence, and preserve exact capitalization, spacing, and output constraints.	The initial execution recovered the meaning but violated formatting; the adapter returned the exact required sentence.

Appendix

Table 14: Representative transfers in the shuffled GAIA stream. Every source task occurs before its target in the stream. The examples illustrate procedural reuse across topics, modalities, and difficulty levels.

Source operations lesson	Target workflow	Shared procedure	Grader score change
Scheduled-task management	Weekly meeting tracking	Collect status across multiple records, identify missing or incomplete actions, and produce a structured progress report.	$0.413\rightarrow 0.946$
Month-end reconciliation	Task backtracking	Reconcile evidence across emails, task lists, and meeting notes; identify discrepancies and report status, blockers, and next actions.	$0.674\rightarrow 0.894$
Scheduled-task management	Multi-project progress reporting	Aggregate task status across several projects, identify risks, and organize the results into separate actionable reports.	$0.586\rightarrow 0.890$

Appendix

Table 15: Qualitative cross-category transfer from Claw-Eval operations tasks to unseen workflows. Scores are assigned by the Claw-Eval grader. No workflow adapter is trained before evaluation.

Self-Improving Agents

Self-improving agents use experience and feedback to update their models, memories, skills, or workflows for better future task performance.

See all 169 papers

Jun 30, 2026

Learning from Failure: Inference-Time Self-Improvement for Computer-Use Agents

Computer-use agents, which leverage multimodal large language models (MLLMs) to operate computers and complete tasks, have attracted significant attention for their utility and versatility. A major challenge in developing these agents is collecting large-scale, high-quality trajectories. The standard approach generates synthetic data through a self-improving loop: an agent is placed in a verifiable environment and iteratively fine-tuned on its successful trajectories. Despite its effectiveness, this paradigm exploits only successful trajectories and discards the failed ones, even though failures carry rich information about a model's weaknesses. In this work, we explore a complementary failure-driven self-improvement loop, a data-centric paradigm that turns failed trajectories into agent improvements. Specifically, we employ an LLM to diagnose failure modes, propose inference-time solutions, and generate code patches -- lightly verified by humans -- that upgrade the agent. We validate this approach with the state-of-the-art OpenCUA-72B model on the OSWorld benchmark, improving the success rate from 42.3% to 48.9%, a gain of 6.6 percentage points, without any additional training cost and with only modest inference overhead. Our results demonstrate that failure-driven self-improvement is a viable complement to success-based pipelines, enabling more efficient agent improvement.

Xueqiao Sun, Xiaohan Wang, Ludwig Schmidt +2

1Stanford University, Stanford, CA 94305, USA

Computer-Use Agents Self-Improving Agents

Ask PDF

Jul 15, 2026

Experience Memory Graph: Extracting Reusable Correction Knowledge from Agent Trajectories

Large Language Model (LLM) agents have shown remarkable capabilities in autonomous decision-making by generating sequential trajectories of states, actions, and observations. However, in complex, long-horizon tasks, these agents frequently suffer from compounding errors and struggle to recover from failures. Despite advances in experience memory, extracting reliable and transferable correction knowledge from historical trajectories remains challenging. Existing methods, whether based on test-time reflection or offline memory construction, largely rely on LLMs to diagnose failures and formulate corrective strategies, which can produce brittle or task-specific insights that are difficult to generalize. To address this challenge, we propose Experience Memory Graph (EMG), a framework that reformulates agent failure recovery as a graph matching problem. During offline construction on the training set, we convert both failed exploration trajectories and successful expert trajectories into directed action decision graphs. By matching these graphs, we extract common subgraphs and graph edit paths that explicitly indicate how to correct failures, and store them in a memory graph with intra-task nodes and cross-task edges. At test time, EMG retrieves relevant insights and guides the agent in a single test-time attempt without additional reflection-and-replay rounds. Experiments on ALFWorld and ScienceWorld demonstrate consistent improvements over the evaluated memory construction and competitive baselines across seen and unseen splits.

Wenjun Wang, Yuchen Fang, Fengrui Liu +2

University of Electronic Science and Technology of China, Chengdu, China · Yangtze Delta Region Institute (Quzhou), School of Computer Science and Engineering, UESTC · Shenzhen Institute for Advanced Study, UESTC

Agent Memory LLM Self-Correction Graph-Based Agent Memory Long-Horizon LLM Agents Memory-Augmented Language Model Agents

Ask PDF

Mar 25, 2026

Experiential Reflective Learning for Self-Improving LLM Agents

Recent advances in large language models (LLMs) have enabled the development of autonomous agents capable of complex reasoning and multi-step problem solving. However, these agents struggle to adapt to specialized environments and do not leverage past interactions, approaching each new task from scratch regardless of their accumulated experience. We introduce Experiential Reflective Learning (ERL), a simple self-improvement framework that enables rapid environment adaptation through experiential learning. ERL reflects on task trajectories and outcomes to generate heuristics, capturing actionable lessons that transfer across tasks. At test time, relevant heuristics are retrieved based on the current task and injected into the agent's context to guide execution. On the Gaia2 benchmark, ERL improves success rate by 7.8% over a ReAct baseline, with large gains in task completion reliability, and outperforms prior experiential learning methods. Through systematic ablations, we find that selective retrieval is essential and that heuristics provide more transferable abstractions than few-shot trajectory prompting. These results demonstrate that reflecting on single-attempt experiences to extract transferable heuristics enables effective agent self-improvement.

Marc-Antoine Allard, Arnaud Teinturier, Victor Xing +1

Illuin Technology

Self-Improving Agents LLM Agent Training Memory-Augmented Language Model Agents

Ask PDF

Abstract

Self-Improving Agents

Explore similar work

Learning from Failure: Inference-Time Self-Improvement for Computer-Use Agents

Experience Memory Graph: Extracting Reusable Correction Knowledge from Agent Trajectories

Experiential Reflective Learning for Self-Improving LLM Agents