CoMem: Learning to Anticipate Failures from Retrieved Experience

Yun Shen; Ming Shen; Tianjun Yao

cs.CLOct 19, 2026

CoMem: Learning to Anticipate Failures from Retrieved Experience

Authors: Yun Shen, Ming Shen, Tianjun Yao

Organizations: The Ohio State University · Shenzhen Loop Area Institute · Shanghai Jiaotong University

Abstract

Reflection lets LLM agents improve from their own attempts, but the memory it builds is local to a single task, so an agent reflecting on its own tends to revisit the same explanation for failure. Cross-task reflective memory addresses this in two forms: retrieved memory keeps past experience explicit and looks up related cases, which is specific but bounded by the bank, and parametric memory internalizes experience into parameters, which generalizes but is grounded in nothing beyond the task description. When both are used, existing agents combine them additively, feeding their outputs to the solver side by side, so retrieved experience never changes what the learned memory produces. We introduce CoMem (Compounding Memory), which makes retrieved experience an input to the learned memory itself, at training and at inference. The agent builds a bank of its own successful and failed reflections, learns from retrieved cases to write failure hypotheses, candidate explanations of how an attempt is likely to go wrong, and at test time reasons from newly retrieved cases to guide each attempt. Varying the retrieved cases gives the memory different perspectives, yielding reflective diversity that is not merely sampled but also grounded in retrieved experience. Across two backbones and five benchmarks in math, code, and multi-hop QA, CoMem achieves strong overall performance. Additional evaluations show that it generalizes to out-of-distribution tasks and enables smaller models to provide effective guidance to stronger solvers.

Figures & tables

Figure 1: Reflective memory designs and CoMem. Retrieved memory recalls past experience, parametric memory generates guidance from the task alone, and additive hybrids combine both. CoMem uses retrieved experience to construct supervision and train an experience-conditioned generator. At inference, different retrieved pairs elicit diverse failure hypotheses.

Model	Method	MATH	HumanEval+	LiveCodeBench	HotpotQA	MuSiQue
Qwen3-4B	Base	$72.90^{\pm 0.74}$	$73.42^{\pm 0.77}$	$32.56^{\pm 0.31}$	$74.6^{\pm 1.3}$	$36.3^{\pm 0.3}$
	Reflexion	$85.73^{\pm 0.61}$	$80.37^{\pm 1.00}$	$43.67^{\pm 0.27}$	$87.4^{\pm 0.5}$	$63.5^{\pm 2.6}$
	DoT-Bank	$87.77^{\pm 0.29}$	$85.28^{\pm 0.87}$	$50.44^{\pm 0.96}$	$91.3^{\pm 0.9}$	$68.0^{\pm 2.8}$
	ParamMem	$92.33^{\pm 1.22}$	$92.02^{\pm 1.62}$	$53.53^{\pm 1.37}$	$\mathbf{92.4^{\pm 0.2}}$	$\mathbf{80.6^{\pm 1.0}}$
	CoMem	$\mathbf{93.05^{\pm 0.61}}$	$\mathbf{95.30^{\pm 0.29}}$	$\mathbf{54.67^{\pm 0.47}}$	$\mathbf{92.4^{\pm 0.6}}$	$78.6^{\pm 0.8}$
Qwen3-8B	Base	$79.38^{\pm 0.45}$	$81.19^{\pm 0.58}$	$41.00^{\pm 0.27}$	$82.4^{\pm 0.2}$	$43.1^{\pm 0.9}$

Table 1: Performance comparison of CoMem with baselines across three task domains using Qwen3-4B and Qwen3-8B as base models. We report accuracy for MATH and QA, and Pass@1 for code. Results are averaged over three random seeds, with standard deviations in superscript. All values are percentages. Bold marks the best result within each dataset and model.

Figure 2: Hypothesis diversity on HumanEval+ with Qwen3-8B. Curves show

D_{\mathrm{mean}}

distributions, with dashed lines marking means.

Method	Self-BLEU-4 $\downarrow$	Distinct-1 $\uparrow$	Distinct-2 $\uparrow$
Frozen	0.598	0.149	0.414
Vanilla SFT	0.508	0.172	0.412
CoMem	0.419	0.195	0.483

Table 2: Hypothesis diversity on MuSiQue with Qwen3-8B. Lower Self-BLEU-4 and higher Distinct-1/2 indicate greater textual diversity. Bold marks the best result in each column.

Figure 3: Weak-to-strong evaluation. We use Qwen3-Next-80B-A3B-Instruct as the task solver, guided by hypotheses generated by CoMem with Qwen3-8B. Bars show mean accuracy or Pass@1 over three random seeds. Error bars indicate one standard deviation. Compared with the baselines, the hypothesis-guided solver achieves the best or near-best performance across the three tasks.

Method	AIME24	AIME25	OlympiadBench	BigCodeBench- Hard	2Wiki MultiHopQA
Base	$16.67^{\pm 0.00}$	$15.56^{\pm 3.14}$	$46.11^{\pm 2.68}$	$26.35^{\pm 3.76}$	$56.8^{\pm 0.3}$
Reflexion	$33.33^{\pm 0.00}$	$26.67^{\pm 2.72}$	$62.50^{\pm 2.50}$	$39.86^{\pm 2.95}$	$94.5^{\pm 0.5}$
DoT-Bank	$33.33^{\pm 2.72}$	$30.00^{\pm 4.71}$	$65.28^{\pm 2.55}$	$45.50^{\pm 1.03}$	$92.3^{\pm 1.2}$
ParamMem	$42.22^{\pm 1.92}$	$31.11^{\pm 5.09}$	$71.11^{\pm 1.27}$	$45.50^{\pm 0.78}$	$\mathbf{96.7^{\pm 0.8}}$
CoMem	$\mathbf{44.44^{\pm 3.85}}$	$\mathbf{36.67^{\pm 0.00}}$	$\mathbf{73.06^{\pm 0.96}}$	$\mathbf{50.23^{\pm 0.39}}$	$96.1^{\pm 0.5}$

Table 3: Out-of-distribution evaluation with Qwen3-8B. Memory modules trained on MATH, APPS, and MuSiQue generate problem-specific hypotheses that guide the solver on the corresponding out-of-distribution benchmarks. Results are averaged over three random seeds. CoMem performs best on math and code and remains competitive on multi-hop QA.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 4 — Figure 4: Additional hypothesis diversity results on MATH and code generation. The top row shows MATH with Qwen3-4B and Qwen3-8B, the middle row shows LiveCodeBench with the same backbones, and the bottom row shows HumanEval+ with Qwen3-4B. Dashed lines mark mean $D_{\mathrm{mean}}$ scores. All configurations generate 10 hypothesis sets per question at $T=1.0$ .

Figure 5: Clustering-based diversity of generated hypothesis sets on HumanEval+ and LiveCodeBench with Qwen3-4B and Qwen3-8B. Curves show the mean entropy-based effective cluster count across problems. For each problem, k-means is applied jointly to 40 hypothesis sets, with 10 sets from each of Frozen, Vanilla SFT, Frozen + Retrieval, and CoMem, using

K

shared clusters.

Memory-Augmented Language Model Agents

Memory-augmented language model agents use persistent facts, interaction histories, and skills to support personalization and long-horizon tasks.

See all 428 papers

Oct 15, 2026

MemComposer: Learning to Assemble Memories for LLM Agents via Reinforcement Learning

Agent memory enables large language model (LLM) agents to reuse past experiences without updating their parameters. However, semantic similarity alone does not guarantee memory usefulness: downstream performance depends on how memories are selected and presented to the agent. We propose MemComposer, a memory composer that jointly examines the current query and retrieved candidates to decide which experiences to include, how many to select, and how to order them. We train the composer using Experience Policy Optimization (ExPO), a GRPO-based framework that evaluates assembled memory contexts through complete agent interactions and uses task-success feedback to optimize the assembly policy. Across three

τ^2

-bench domains with two frozen executors, MemComposer outperforms the strongest baseline for each executor while reducing average memory-token usage by 16.9% and 4.9%, respectively. The learned policy also transfers to unseen domains, executors, and memory banks, and benefits from memories generated by stronger models without further training. These results demonstrate the value of learning query-specific memory assembly using downstream task feedback.

Yixuan Xie, Xueqiang Xu, Xinyi Fan +8

University of Illinois Urbana-Champaign · Amazon

Memory-Augmented Language Model Agents RL for LLM Agents

Ask PDF

Jul 15, 2026

Experience Memory Graph: One-Shot Error Correction for Agents

Large Language Model (LLM) agents have shown remarkable capabilities in autonomous decision-making by generating sequential trajectories of states, actions, and observations. However, in complex, long-horizon tasks, these agents frequently suffer from compounding errors and struggle to recover from failures. Existing self-correction mechanisms rely on prompt-based reflection, which is inherently brittle, incurs heavy time and API costs due to iterative trial-and-error loops, and produces task-specific memory that may be hard to generalize to new scenarios. To address this, we propose Experience Memory Graph (EMG), a framework that reformulates agent failure recovery as a graph matching problem. At training time, we convert both failed exploration trajectories and successful expert trajectories into directed action decision graphs. By matching these graphs, we extract common subgraphs (successful workflows) and graph edit paths that explicitly indicate how to correct failures (e.g., which actions to add, delete, or relabel under a given observation), and store them in a memory graph with intra-task nodes and cross-task edges. At test time, EMG retrieves relevant insights and guides the agent in a single, loop-free execution. Experiments on ALFWorld and ScienceWorld show that EMG consistently outperforms state-of-the-art reflection baselines in success rate and average reward, while requiring no test-time trial-and-error.

Wenjun Wang, Yuchen Fang, Fengrui Liu +2

University of Electronic Science and Technology of China

Agent Memory LLM Self-Correction Graph-Based Agent Memory Long-Horizon LLM Agents Memory-Augmented Language Model Agents

Ask PDF

Mar 18, 2026

Retrieval-Augmented LLM Agents: Learning to Learn from Experience

While large language models (LLMs) have advanced the development of general-purpose agents, robust generalization to unseen tasks remains challenging. Two common approaches are supervised fine-tuning and training-free memory-augmented generation using retrieved experience; yet both have limitations: fine-tuning often fails to extrapolate to new tasks, while experience retrieval often underperforms compared to supervised baselines. In this work, we combine these approaches and study how retrieval-augmented LLM agents can learn to use retrieved trajectories in-context. First, we establish a strong LoRA fine-tuning baseline that outperforms several state-of-the-art agent training pipelines. Second, we analyze key design choices for experience retrieval, including storage, querying, and trajectory selection. We then integrate experience retrieval directly into the fine-tuning process, finding that this substantially improves generalization to unseen tasks. Finally, we show that these gains often persist with imperfect experience and, even when agents reuse their own failed attempts without test-time parameter updates. Overall, our results establish simple episodic retrieval as a strong foundation for agent memory and retrieval-aware fine-tuning as a practical and effective framework for building agents that learn to learn from experience.

Thomas Palmeira Ferraz, Romain Deffayet, Vassilina Nikoulina +2

NAVER LABS Europe, Meylan, France

Fine-Tuning LLM Agent Training Memory-Augmented Language Model Agents

Ask PDF

Abstract

Memory-Augmented Language Model Agents

Explore similar work

MemComposer: Learning to Assemble Memories for LLM Agents via Reinforcement Learning

Experience Memory Graph: One-Shot Error Correction for Agents

Retrieval-Augmented LLM Agents: Learning to Learn from Experience