Trajectory-Informed Memory Generation for Self-Improving Agent Systems

Gaodan Fang; Vatche Isahagian; K. R. Jayaram; Ritesh Kumar; Vinod Muthusamy; Punleuk Oum; Gegi Thomas

cs.AIMar 11, 2026

Trajectory-Informed Memory Generation for Self-Improving Agent Systems

Authors: Gaodan Fang, Vatche Isahagian, K. R. Jayaram, Ritesh Kumar, Vinod Muthusamy, Punleuk Oum, Gegi Thomas

Organizations: Agents and Automation Lab, IBM Research USA

Abstract

LLM-powered agents face a persistent challenge: learning from their execution experiences to improve future performance. While agents can successfully complete many tasks, they often repeat inefficient patterns, fail to recover from similar errors, and miss opportunities to apply successful strategies from past executions. We present a novel framework for automatically extracting actionable learnings from agent execution trajectories and utilizing them to improve future performance through contextual memory retrieval. Our approach comprises four components: (1) a Trajectory Intelligence Extractor that performs semantic analysis of agent reasoning patterns, (2) a Decision Attribution Analyzer that identifies which decisions and reasoning steps led to failures, recoveries, or inefficiencies, (3) a Contextual Learning Generator that produces three types of guidance -- strategy tips from successful patterns, recovery tips from failure handling, and optimization tips from inefficient but successful executions, and (4) an Adaptive Memory Retrieval System that injects relevant learnings into agent prompts based on multi-dimensional similarity. Unlike existing memory systems that store generic conversational facts, our framework understands execution patterns, extracts structured learnings with provenance, and retrieves guidance tailored to specific task contexts. Evaluation on the AppWorld benchmark demonstrates consistent improvements, with up to 14.3 percentage point gains in scenario goal completion on held-out tasks and particularly strong benefits on complex tasks (28.5~pp scenario goal improvement, a 149% relative increase).

Figures & tables

Figure 1. Overview of our approach

Type	Task Goal	Scenario Goal
Aggregate	73.2	64.3
Difficulty 1	91.2	89.5
Difficulty 2	70.8	56.2
Difficulty 3	58.7	47.6

Table 1. Subtask Tips + LLM Selection: Test-Normal

Type	Task Goal	Scenario Goal
Aggregate	69.6	50.0
Difficulty 1	89.5	79.0
Difficulty 2	66.7	56.2
Difficulty 3	54.0	19.1

Table 2. Baseline Agent (No Memory): Test-Normal

Type	Task Goal	Scenario Goal
Aggregate	66.7	48.2
Difficulty 1	86.0	68.4
Difficulty 2	70.8	56.2
Difficulty 3	46.0	23.8

Table 3. Task-Level Tips + Cosine (

\tau\geq 0.5

, top-3): Test-Normal

Type	Task Goal	Scenario Goal
Aggregate	72.0	62.5
Difficulty 1	91.2	84.2
Difficulty 2	72.9	68.8
Difficulty 3	54.0	38.1

Table 4. Task-Level Tips + Cosine (

\tau\geq 0.6

): Test-Normal

Type	Task Goal	Scenario Goal
Aggregate	70.2	57.1
Difficulty 1	91.2	84.2
Difficulty 2	64.6	43.8
Difficulty 3	55.6	42.9

Table 5. Task-Level Tips + Cosine (

\tau\geq 0.5

): Test-Normal

Type	Task Goal	Scenario Goal
Aggregate	73.8	57.1
Difficulty 1	91.2	73.7
Difficulty 2	72.9	56.2
Difficulty 3	58.7	42.9

Table 6. Subtask Tips + Cosine (

\tau\geq 0.6

): Test-Normal

Tip Granularity	Retrieval Strategy	TGC	$\Delta$ TGC	SGC	$\Delta$ SGC
Baseline (no memory)		69.6	—	50.0	—
Subtask-level	LLM-guided selection	73.2	+3.6	64.3	+14.3
Subtask-level	Cosine sim. ( $\tau\geq 0.6$ )	73.8	+4.2	57.1	+7.1
Task-level	Cosine sim. ( $\tau\geq 0.6$ )	72.0	+2.4	62.5	+12.5

Table 7. Configuration Comparison on Test-Normal (Aggregate)

Type	Task Goal	Scenario Goal
Aggregate	91.1	83.3
Difficulty 1	94.4	83.3
Difficulty 2	88.9	83.3
Difficulty 3	88.9	83.3

Table 8. Subtask Tips + LLM Selection: Train

Type	Task Goal	Scenario Goal
Aggregate	86.7	73.3
Difficulty 1	100.0	100.0
Difficulty 2	77.8	58.3
Difficulty 3	77.8	50.0

Table 9. Baseline Agent (No Memory): Train

Type	Task Goal	Scenario Goal
Aggregate	89.5	73.7
Difficulty 1	90.0	80.0
Difficulty 2	87.5	62.5
Difficulty 3	100.0	100.0

Table 10. Subtask Tips + LLM Selection: Dev

Type	Task Goal	Scenario Goal
Aggregate	77.2	47.4
Difficulty 1	80.0	60.0
Difficulty 2	70.8	25.0
Difficulty 3	100.0	100.0

Table 11. Baseline Agent (No Memory): Dev

Partition	Task Goal	Task Goal	Scenario Goal	Scenario Goal
	(Baseline)	(+Memory)	(Baseline)	(+Memory)
Test-Normal	69.6	73.2 (+3.6)	50.0	64.3 (+14.3)
Train	86.7	91.1 (+4.4)	73.3	83.3 (+10.0)
Dev	77.2	89.5 (+12.3)	47.4	73.7 (+26.3)

Table 12. Summary of Aggregate Improvements: Subtask Tips + LLM Selection

LLM Agent Self-Improvement

Large language model agent self-improvement studies how agents use experience and feedback to improve their behavior, tools, and workflows.

See all 257 papers

Retrieval-Augmented LLM Agents: Learning to Learn from Experience
Mar 18, 2026Thomas Palmeira Ferraz, Romain Deffayet, Vassilina Nikoulina +2Fine-Tuning LLM Agent Training
NAVER LABS Europe, Meylan, France
Experiential Reflective Learning for Self-Improving LLM Agents
Mar 25, 2026Marc-Antoine Allard, Arnaud Teinturier, Victor Xing +1Self-Improving Agents LLM Agent Training
Illuin Technology
Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents
Jul 15, 2026Eric Hanchen Jiang, Zhi Zhang, Yuchen Wu +11Persistent Memory for Language Models Memory Management for LLM Agents
University of California Los Angeles · University of Washington · Northwestern University

Mar 18, 2026J/K move · Enter open · S save

Retrieval-Augmented LLM Agents: Learning to Learn from Experience

Thomas Palmeira Ferraz, Romain Deffayet, Vassilina Nikoulina, Hervé Déjean+1

NAVER LABS Europe, Meylan, France

Fine-Tuning LLM Agent Training Memory-Augmented Language Model Agents

Open paper AskPDF arXiv

While large language models (LLMs) have advanced the development of general-purpose agents, robust generalization to unseen tasks remains challenging. Two common approaches are supervised fine-tuning and training-free memory-augmented generation using retrieved experience; yet both have limitations: fine-tuning often fails to extrapolate to new tasks, while experience retrieval often underperforms compared to supervised baselines. In this work, we combine these approaches and study how retrieval-augmented LLM agents can learn to use retrieved trajectories in-context. First, we establish a strong LoRA fine-tuning baseline that outperforms several state-of-the-art agent training pipelines. Second, we analyze key design choices for experience retrieval, including storage, querying, and trajectory selection. We then integrate experience retrieval directly into the fine-tuning process, finding that this substantially improves generalization to unseen tasks. Finally, we show that these gains often persist with imperfect experience and, even when agents reuse their own failed attempts without test-time parameter updates. Overall, our results establish simple episodic retrieval as a strong foundation for agent memory and retrieval-aware fine-tuning as a practical and effective framework for building agents that learn to learn from experience.

Abstract

LLM Agent Self-Improvement

Explore similar work

Retrieval-Augmented LLM Agents: Learning to Learn from Experience

Experiential Reflective Learning for Self-Improving LLM Agents

Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents