CoMem: Learning to Anticipate Failures from Retrieved Experience

Yun Shen; Ming Shen; Tianjun Yao

cs.CLOct 19, 2026

CoMem: Learning to Anticipate Failures from Retrieved Experience

Authors: Yun Shen, Ming Shen, Tianjun Yao

Organizations: The Ohio State University · Shenzhen Loop Area Institute · Shanghai Jiaotong University

Abstract

Reflection lets LLM agents improve from their own attempts, but the memory it builds is local to a single task, so an agent reflecting on its own tends to revisit the same explanation for failure. Cross-task reflective memory addresses this in two forms: retrieved memory keeps past experience explicit and looks up related cases, which is specific but bounded by the bank, and parametric memory internalizes experience into parameters, which generalizes but is grounded in nothing beyond the task description. When both are used, existing agents combine them additively, feeding their outputs to the solver side by side, so retrieved experience never changes what the learned memory produces. We introduce CoMem (Compounding Memory), which makes retrieved experience an input to the learned memory itself, at training and at inference. The agent builds a bank of its own successful and failed reflections, learns from retrieved cases to write failure hypotheses, candidate explanations of how an attempt is likely to go wrong, and at test time reasons from newly retrieved cases to guide each attempt. Varying the retrieved cases gives the memory different perspectives, yielding reflective diversity that is not merely sampled but also grounded in retrieved experience. Across two backbones and five benchmarks in math, code, and multi-hop QA, CoMem achieves strong overall performance. Additional evaluations show that it generalizes to out-of-distribution tasks and enables smaller models to provide effective guidance to stronger solvers.

Figures & tables

Figure 1: Reflective memory designs and CoMem. Retrieved memory recalls past experience, parametric memory generates guidance from the task alone, and additive hybrids combine both. CoMem uses retrieved experience to construct supervision and train an experience-conditioned generator. At inference, different retrieved pairs elicit diverse failure hypotheses.

Model	Method	MATH	HumanEval+	LiveCodeBench	HotpotQA	MuSiQue
Qwen3-4B	Base	$72.90^{\pm 0.74}$	$73.42^{\pm 0.77}$	$32.56^{\pm 0.31}$	$74.6^{\pm 1.3}$	$36.3^{\pm 0.3}$
	Reflexion	$85.73^{\pm 0.61}$	$80.37^{\pm 1.00}$	$43.67^{\pm 0.27}$	$87.4^{\pm 0.5}$	$63.5^{\pm 2.6}$
	DoT-Bank	$87.77^{\pm 0.29}$	$85.28^{\pm 0.87}$	$50.44^{\pm 0.96}$	$91.3^{\pm 0.9}$	$68.0^{\pm 2.8}$
	ParamMem	$92.33^{\pm 1.22}$	$92.02^{\pm 1.62}$	$53.53^{\pm 1.37}$	$\mathbf{92.4^{\pm 0.2}}$	$\mathbf{80.6^{\pm 1.0}}$
	CoMem	$\mathbf{93.05^{\pm 0.61}}$	$\mathbf{95.30^{\pm 0.29}}$	$\mathbf{54.67^{\pm 0.47}}$	$\mathbf{92.4^{\pm 0.6}}$	$78.6^{\pm 0.8}$
Qwen3-8B	Base	$79.38^{\pm 0.45}$	$81.19^{\pm 0.58}$	$41.00^{\pm 0.27}$	$82.4^{\pm 0.2}$	$43.1^{\pm 0.9}$

Table 1: Performance comparison of CoMem with baselines across three task domains using Qwen3-4B and Qwen3-8B as base models. We report accuracy for MATH and QA, and Pass@1 for code. Results are averaged over three random seeds, with standard deviations in superscript. All values are percentages. Bold marks the best result within each dataset and model.

Figure 2: Hypothesis diversity on HumanEval+ with Qwen3-8B. Curves show

D_{\mathrm{mean}}

distributions, with dashed lines marking means.

Method	Self-BLEU-4 $\downarrow$	Distinct-1 $\uparrow$	Distinct-2 $\uparrow$
Frozen	0.598	0.149	0.414
Vanilla SFT	0.508	0.172	0.412
CoMem	0.419	0.195	0.483

Table 2: Hypothesis diversity on MuSiQue with Qwen3-8B. Lower Self-BLEU-4 and higher Distinct-1/2 indicate greater textual diversity. Bold marks the best result in each column.

Figure 3: Weak-to-strong evaluation. We use Qwen3-Next-80B-A3B-Instruct as the task solver, guided by hypotheses generated by CoMem with Qwen3-8B. Bars show mean accuracy or Pass@1 over three random seeds. Error bars indicate one standard deviation. Compared with the baselines, the hypothesis-guided solver achieves the best or near-best performance across the three tasks.

Method	AIME24	AIME25	OlympiadBench	BigCodeBench- Hard	2Wiki MultiHopQA
Base	$16.67^{\pm 0.00}$	$15.56^{\pm 3.14}$	$46.11^{\pm 2.68}$	$26.35^{\pm 3.76}$	$56.8^{\pm 0.3}$
Reflexion	$33.33^{\pm 0.00}$	$26.67^{\pm 2.72}$	$62.50^{\pm 2.50}$	$39.86^{\pm 2.95}$	$94.5^{\pm 0.5}$
DoT-Bank	$33.33^{\pm 2.72}$	$30.00^{\pm 4.71}$	$65.28^{\pm 2.55}$	$45.50^{\pm 1.03}$	$92.3^{\pm 1.2}$
ParamMem	$42.22^{\pm 1.92}$	$31.11^{\pm 5.09}$	$71.11^{\pm 1.27}$	$45.50^{\pm 0.78}$	$\mathbf{96.7^{\pm 0.8}}$
CoMem	$\mathbf{44.44^{\pm 3.85}}$	$\mathbf{36.67^{\pm 0.00}}$	$\mathbf{73.06^{\pm 0.96}}$	$\mathbf{50.23^{\pm 0.39}}$	$96.1^{\pm 0.5}$

Table 3: Out-of-distribution evaluation with Qwen3-8B. Memory modules trained on MATH, APPS, and MuSiQue generate problem-specific hypotheses that guide the solver on the corresponding out-of-distribution benchmarks. Results are averaged over three random seeds. CoMem performs best on math and code and remains competitive on multi-hop QA.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 4 — Figure 4: Additional hypothesis diversity results on MATH and code generation. The top row shows MATH with Qwen3-4B and Qwen3-8B, the middle row shows LiveCodeBench with the same backbones, and the bottom row shows HumanEval+ with Qwen3-4B. Dashed lines mark mean $D_{\mathrm{mean}}$ scores. All configurations generate 10 hypothesis sets per question at $T=1.0$ .

Figure 5: Clustering-based diversity of generated hypothesis sets on HumanEval+ and LiveCodeBench with Qwen3-4B and Qwen3-8B. Curves show the mean entropy-based effective cluster count across problems. For each problem, k-means is applied jointly to 40 hypothesis sets, with 10 sets from each of Frozen, Vanilla SFT, Frozen + Retrieval, and CoMem, using

K

shared clusters.

Memory-Augmented Language Model Agents

Memory-augmented language model agents use persistent facts, interaction histories, and skills to support personalization and long-horizon tasks.

See all 428 papers

MemComposer: Learning to Assemble Memories for LLM Agents via Reinforcement Learning
Oct 15, 2026Yixuan Xie, Xueqiang Xu, Xinyi Fan +8Memory-Augmented Language Model Agents RL for LLM Agents
University of Illinois Urbana-Champaign · Amazon
Experience Memory Graph: One-Shot Error Correction for Agents
Jul 15, 2026Wenjun Wang, Yuchen Fang, Fengrui Liu +2Agent Memory LLM Self-Correction
University of Electronic Science and Technology of China
Retrieval-Augmented LLM Agents: Learning to Learn from Experience
Mar 18, 2026Thomas Palmeira Ferraz, Romain Deffayet, Vassilina Nikoulina +2Fine-Tuning LLM Agent Training
NAVER LABS Europe, Meylan, France

Oct 15, 2026J/K move · Enter open · S save

MemComposer: Learning to Assemble Memories for LLM Agents via Reinforcement Learning

Yixuan Xie, Xueqiang Xu, Xinyi Fan, Xinxi Lyu+7

University of Illinois Urbana-Champaign · Amazon

Memory-Augmented Language Model Agents RL for LLM Agents

Open paper AskPDF arXiv

Agent memory enables large language model (LLM) agents to reuse past experiences without updating their parameters. However, semantic similarity alone does not guarantee memory usefulness: downstream performance depends on how memories are selected and presented to the agent. We propose MemComposer, a memory composer that jointly examines the current query and retrieved candidates to decide which experiences to include, how many to select, and how to order them. We train the composer using Experience Policy Optimization (ExPO), a GRPO-based framework that evaluates assembled memory contexts through complete agent interactions and uses task-success feedback to optimize the assembly policy. Across three

τ^2

-bench domains with two frozen executors, MemComposer outperforms the strongest baseline for each executor while reducing average memory-token usage by 16.9% and 4.9%, respectively. The learned policy also transfers to unseen domains, executors, and memory banks, and benefits from memories generated by stronger models without further training. These results demonstrate the value of learning query-specific memory assembly using downstream task feedback.

Abstract

Memory-Augmented Language Model Agents

Explore similar work

MemComposer: Learning to Assemble Memories for LLM Agents via Reinforcement Learning

Experience Memory Graph: One-Shot Error Correction for Agents

Retrieval-Augmented LLM Agents: Learning to Learn from Experience