REM: What to Store, When to Recall, What to Forget in Experience Memory for Coding Agents

Xinyi Lisa Chen; Pedro Herrero-Vidal; Xiaohan Chen; Xiaojing Yu; Cris Liu; Bingqing Song; Bin Xu; Suleiman A. Khan

cs.AIOct 19, 2026

REM: What to Store, When to Recall, What to Forget in Experience Memory for Coding Agents

Authors: Xinyi Lisa Chen, Pedro Herrero-Vidal, Xiaohan Chen, Xiaojing Yu, Cris Liu, Bingqing Song, Bin Xu, Suleiman A. Khan,

Organizations: Yale University · Amazon

Abstract

Large language model (LLM) agents now autonomously complete long-horizon tasks, such as repository-level coding, through hundreds of think-act steps, yet the experience each run produces is rarely reused on the next. Prior memory systems concentrate on retrieval, finding the most relevant past trajectory. Retrieval cannot observe the running agent, so it cannot decide whether the agent needs an experience, at which point, or for how long. We introduce REM (Reusable Experience Memory), a framework that manages the memory lifecycle from inside the agent's think-act loop through three explicit decisions. What to store: each completed trajectory is consolidated into a phase-structured document, two orders of magnitude smaller than the trajectory. When to recall: the document is injected at the start of the task; the costlier implementation record is withheld until an in-loop gate detects the agent struggling. What to forget: the record is evicted after a single exposure. On repository-level bug fixing (SWE-ContextBench Lite), REM retrieves the related trajectory more accurately than every state-of-the-art memory system (68.7% vs. 55.6% match@1) and improves, in every replicate, end-task resolution (27.6% vs. 24.9%) at 17% lower cost. Injecting the same experience as a summary or raw trajectory, even when oracle-retrieved, does not. Ablating any of the three decisions reduces resolution. Altogether, these results demonstrate that the bottleneck in experience reuse is not retrieval fidelity but how experience is stored and governed across its lifecycle, opening a control-theoretic perspective on agent memory.

Figures & tables

Figure 1: REM manages the memory lifecycle in three stages around the agent’s think-act loop. Left: the write path consolidates each completed trajectory into a phase-structured document ( localize , understand , fix pattern , verify ); the extraction fields come from a domain module discovered automatically from example trajectories. Top right: the read path retrieves candidates by cosine similarity, reranks with one LLM call, and injects the top document at the start of the run. Bottom right: the in-loop recall node recalls the implementation record on a struggle signal and evicts it after one exposure.

Figure 2: Every configuration that injects the phased document matches or exceeds no-memory resolution, and the retrieved ones do so at lower cost; summaries and raw trajectories, even oracle ones, fall below it. Resolution against cost per task (Sonnet 4.5); up and to the left is preferable. Cost is weighted by token type (fresh

\times

1, cache write

\times

1.25, cache read

\times

0.1, output

\times

5, the agent model’s relative list prices ( Anthropic, 2026 ) ). Blue: phased document (diamond: document alone; triangle: document and implementation record). Gray: summary or raw trajectory. Hollow: oracle. Black, dashed: no memory. Filled blue points and no memory are three-replicate means; hollow and gray points are single runs.

Method	Stored unit	Candidates	Match@1
Free trajectory †	raw trajectory ( $\sim$ 105K tok)	self-selected	22.2%
Supermemory	managed chunks ( $\sim$ 30 tok)	15	23.2%
Mem0	extracted facts ( $\sim$ 12 tok)	3	25.3%
Free summary †	trajectory summary ( $\sim$ 373 tok)	self-selected	35.4%
OpenViking	indexed file chunks ( $\sim$ 246 tok)	3	38.4%
LangMem	extracted memory items ( $\sim$ 48 tok)	10	55.6%

Table 1: The phase-structured document is the most accurate retrieval unit, ahead of every existing memory system. Match@1 over the 99 tasks, after reranking for REM. Numbers for existing systems from Zhu et al. (2026) , under each system’s own retrieval configuration; our token counts are approximated as one token per four characters. † From our runs, agent searches the pool.

Injected form	Tokens	Injected	Referenced	Coverage
Wrong summary	$\sim$ 380	8.9	0.3	3.4%
Free trajectory	$\sim$ 105K	67.7	17.9	26.7%
Oracle trajectory	$\sim$ 111K	67.7	18.1	27.2%
Oracle summary	$\sim$ 390	9.3	4.3	45.7%
Free summary	$\sim$ 373	9.3	4.8	52.5%
Oracle phased document	$\sim$ 540	12.4	7.8	64.7%

Table 2: The phased document is read more completely than any other stored form, and coverage tracks resolution. Tokens are the stored unit’s size, approximated as one token per four characters; the agent’s file reader returned about half of each oracle trajectory (

\sim

54K). Identifiers injected/referenced are per-run means; coverage is their ratio (Sonnet 4.5). The final row is the standardized mean difference between resolved and unresolved runs (two-sided Mann-Whitney

p

), pooled over the four start-of-run conditions (

n=91

resolved, 284 unresolved);

{}^{*}p<0.05

. REM row is a mean over three replicates; others are one.

Recall policy for the record	Injected in	Resolution (sd)	$\Delta$ ( $p$ )	Cost tok/task
Never, document only	0%	26.9% (1.5)	$-0.7$ (0.82)	374K
Eager, at first source edit	100%	25.3% (1.7)	$-2.3$ (0.17)	423K
On struggle, then pinned	56%	24.9% (1.5)	$-2.7$ (0.08)	353K
On struggle, then evicted (REM)	58%	27.6% (1.5)	—	374K

Table 3: Both gating on struggle and evicting after use improve resolution over the other record policies. Every row injects the same retrieved document and varies only the record policy (Sonnet 4.5). Resolution is the mean over three replicates of 99 tasks, sd across replicates in parentheses.

\Delta

is the difference from REM in points, exact McNemar

p

over 297 paired task-runs; none reaches

p<0.05

at this sample size. Per-replicate counts in Table 11 .

Figure 3: Memory helps agents in a capability band and costs the strongest ones. Resolution by memory condition for five agent models, one replicate each (the first for Sonnet 4.5). Phased doc injects the retrieved document; REM adds the struggle-gated, evicted record; Pinned keeps that record; Oracle doc substitutes the labeled trajectory’s document. Shading marks no-memory rates at or below which the phased document does not reduce resolution.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Condition	Resolved / 99	ITT	Cost tok/task	Turns	Min	PatchNA
No memory	23, 26, 25	24.9%	452K	70.4	11.9	7.0
Wrong summary †	23	23.2%	431K	72.0	14.4	7
Free trajectory †	22	22.2%	637K	66.3	13.1	11
Free summary †	24	24.2%	484K	68.6	13.8	10
Oracle summary †	23	23.2%	378K	67.4	13.7	6
Oracle trajectory †	24	24.2%	768K	49.9	9.5	11

Appendix

Table 4: End-task outcomes (Sonnet 4.5). Resolved counts per replicate; ITT is their mean over all 99 tasks. Cost, turns, minutes, and PatchNA (patch does not apply) are means over replicates. † One replicate; other rows three.

Extraction schema	Cosine top-1	Gold in top-5	Match@1	Conversion	Parse fail.
Hand-written	65.7%	83.8%	66.7%	79.6%	5
Discovered	62.6%	78.8%	68.7%	87.2%	0

Appendix

Table 5: Retrieval by stage on the 99 tasks. Cosine top-1 is first-stage accuracy, gold in top-5 is first-stage recall at the rerank budget, match@1 is post-rerank accuracy, conversion is match@1 over gold-in-top-5, and parse failures fell back to the cosine top-1.

condition	res/patch/N	ITT	prec.	min	turns	tok/inst	failures
No memory	20/88/99	20.2%	22.7%	9.5	71.1	444,070	error 8, max_turns 3
Oracle phased document	23/91/99	23.2%	25.3%	10.6	72.1	442,141	error 6
REM, document only	24/93/99	24.2%	25.8%	10.2	68.3	381,828	error 4, max_turns 2
REM, gate without eviction	24/96/99	24.2%	25.0%	10.0	66.6	385,833	error 2, max_turns 1
REM	21/92/99	21.2%	22.8%	9.7	68.7	419,678	error 3, max_turns 4

Appendix

Table 6: Full run summary, Haiku 4.5 ( claude-haiku-4-5 ).

condition	res/patch/N	ITT	prec.	min	turns	tok/inst	failures
No memory	23/97/99	23.2%	23.7%	14.5	71.5	452,536	max_turns 1, no_patch 1
Wrong summary	23/95/99	23.2%	24.2%	14.4	72.0	430,526	max_turns 2, no_patch 2
Free trajectory	22/94/99	22.2%	23.4%	13.1	66.3	636,529	context_overflow 3, max_turns 2
Free summary	24/98/99	24.2%	24.5%	13.8	68.6	484,142	max_turns 1
Oracle summary	23/97/99	23.2%	23.7%	13.7	67.4	378,265	none
Oracle trajectory	24/90/99	24.2%	26.7%	9.5	49.9	767,810	context_overflow 3, error 1, max_turns 3

Appendix

Table 7: Full run summary, Sonnet 4.5 ( claude-sonnet-4-5 ). First replicate; three-replicate means for REM and its recall ablations are in Tables 4 and 3 .

condition	res/patch/N	ITT	prec.	min	turns	tok/inst	failures
No memory	31/99/99	31.3%	31.3%	14.6	56.7	279,947	none
Oracle phased document	32/94/99	32.3%	34.0%	15.2	56.6	296,278	error 1, max_turns 2
REM, document only	32/97/99	32.3%	33.0%	13.2	50.5	230,611	max_turns 2
REM, gate without eviction	29/98/99	29.3%	29.6%	14.5	51.9	248,034	max_turns 1
REM	31/98/99	31.3%	31.6%	13.8	49.9	242,595	max_turns 1

Appendix

Table 8: Full run summary, Opus 4.6 ( claude-opus-4-6 ).

condition	res/patch/N	ITT	prec.	min	turns	tok/inst	failures
No memory	47/95/99	47.5%	49.5%	17.0	24.5	129,623	error 4
Oracle phased document	42/93/99	42.4%	45.2%	16.9	32.5	191,712	error 1, timeout 3
REM, document only	39/98/99	39.4%	39.8%	17.0	25.6	140,603	error 1
REM, gate without eviction	40/98/99	40.4%	40.8%	17.0	25.9	137,940	max_turns 1
REM	40/97/99	40.4%	41.2%	16.7	24.4	136,395	error 2

Appendix

Table 9: Full run summary, Opus 4.8 ( claude-opus-4-8 ).

condition	res/patch/N	ITT	prec.	min	turns	tok/inst	failures
No memory	52/98/99	52.5%	53.1%	25.3	49.0	330,975	no_patch 1
Oracle phased document	50/94/99	50.5%	53.2%	28.0	54.1	364,212	error 1, max_turns 1, timeout 1
REM, document only	50/98/99	50.5%	51.0%	26.3	50.6	326,477	max_turns 1
REM, gate without eviction	43/98/99	43.4%	43.9%	24.2	46.7	301,082	error 1
REM	47/98/99	47.5%	48.0%	25.3	47.3	303,257	no_patch 1

Appendix

Table 10: Full run summary, Sonnet 5 ( claude-sonnet-5 ).

condition	res/patch/N	ITT	PatchNA	min	turns	tok/inst
No memory, replicate 1	23/97/99	23.2%	8	14.5	71.5	452,536
replicate 2	26/95/99	26.3%	6	10.5	69.0	442,914
replicate 3	25/98/99	25.3%	7	10.8	70.7	459,158
mean	24.7/96.7/99	24.9%	7.0	11.9	70.4	451,536
REM, document only, replicate 1	27/99/99	27.3%	7	9.1	65.7	375,712
replicate 2	28/98/99	28.3%	3	10.0	65.6	373,133

Appendix

Table 11: Per-replicate accounting for the three-replicate conditions (Sonnet 4.5). Every replicate runs all 99 tasks. Columns: resolved/patches/instances, ITT rate, count of runs whose patch does not apply, and mean minutes, turns, and cost-weighted tokens per instance. Mean rows are the values the body reports.

Condition	$n$	Coverage	Window	Dead weight	First infl.
REM, phased document only	$3\times$ 99	69.1%	38.8	35.3%	4.1
REM, gate and evict	$3\times$ 99	50.3%	42.9	27.2%	4.8
REM, gate without eviction	$3\times$ 99	50.5%	42.1	28.1%	4.3
REM, eager record	$3\times$ 99	38.8%	44.9	19.1%	3.8
Oracle phased document	97	64.7%	45.6	35.7%	2.1
Free summary	97	52.5%	28.2	52.8%	3.6

Appendix

Table 12: Identifier utilization per condition (Sonnet 4.5). Window is the span in turns from first to last reference, dead weight the fraction of the run after the last reference, and first influence the turn of the first reference. REM rows are means over three replicates; start-of-run rows are one.

Grouping	Quantity	Resolved	Unresolved	$d$	MW $p$
Start-of-run forms	coverage	0.423	0.378	$+0.20$	0.03
	identifiers referenced	12.9	11.0	$+0.18$	0.10
	identifiers injected	39.7	36.2	$+0.10$	0.84
REM recall policies	coverage	0.511	0.527	$-0.06$	0.34
	identifiers referenced	12.4	12.7	$-0.02$	0.42
	identifiers injected	28.8	28.3	$+0.02$	0.76

Appendix

Table 13: Coverage against resolution, with its two component quantities tested separately. Coverage is identifiers referenced over injected. Start-of-run forms pool the oracle and free summary and trajectory conditions (

n=91

resolved, 284 unresolved); REM recall policies pool the four phased conditions over three replicates (

n=311

resolved, 868 unresolved).

d

is the standardized mean difference; MW

p

is two-sided Mann-Whitney. Runs without an evaluation verdict are excluded (13 start-of-run, 9 REM); start-of-run forms also exclude the two tasks without a labeled trajectory in the pool.

Hand-written	Discovered
files_modified	files_modified
root_cause	root_cause
fix_approach	fix_approach
functions_changed , classes_involved	related_functions
test_names , test_commands	test_cases
files_read , key_identifiers	—

Appendix

Table 14: Extraction fields, hand-written against discovered, for repository-level coding. Rows pair fields serving the same purpose; the last two list fields present in only one schema.

Memory Management for LLM Agents

See all 79 papers

DELTAMEM: Incremental Experience Memory for LLM Agents via Residual Trees
Jun 2, 2026Haoran Tan, Zeyu Zhang, Zhicheng Cao +2Continual Learning for LLM Agents LLM Agent Memory
Gaoling School of Artificial, Renmin University of China, Beijing, China · Beijing Key Laboratory of Research on Large Models and Intelligent Governance · Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE +1
Memex(RL): Scaling Long-Horizon LLM Agents via Indexed Experience Memory
Mar 4, 2026Zhenting Wang, Huancheng Chen, Jiayun Wang +1LLM Agent Memory Memory Management for LLM Agents
Center for Advanced AI, Accenture
CoMem: Learning to Anticipate Failures from Retrieved Experience
Oct 19, 2026Yun Shen, Ming Shen, Tianjun YaoMemory-Augmented Language Model Agents Self-Reflection in Language Model Agents
The Ohio State University · Shenzhen Loop Area Institute · Shanghai Jiaotong University

Jun 2, 2026J/K move · Enter open · S save

DELTAMEM: Incremental Experience Memory for LLM Agents via Residual Trees

Haoran Tan, Zeyu Zhang, Zhicheng Cao, Rui Li+1

Gaoling School of Artificial, Renmin University of China, Beijing, China · Beijing Key Laboratory of Research on Large Models and Intelligent Governance · Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE+1

Continual Learning for LLM Agents LLM Agent Memory Agent Skill Learning Memory Consolidation

Open paper AskPDF arXiv

Large Language Model (LLM)-based agents increasingly rely on memory to learn from experiences over continual interactions. However, storing experiences as independent, flat units leads to substantial redundancy and retrieval conflicts, as similar episodes repeat overlapping content and subtle scene variations cause retrieved memories to offer contradictory guidance. To address this, we introduce residual experience, positing that newly acquired experience is often an incremental variation of existing knowledge. We propose DeltaMem, a framework that organizes experience memory into two independent residual trees, one storing goal-conditioned task experience as reusable skills and another for scene-level environment knowledge. Each tree uses a root node for generalized base experiences and incremental delta nodes for subsequent variations, allowing related experiences to share a common foundation without duplication. For retrieval, a failure-penalized similarity scan locates the best match, reconstructing the full experience via root-to-match chain composition. An autonomous consolidation mechanism distills high-frequency paths into new root nodes, enabling the trees to self-organize from general heuristics to specialized variants. Experiments across diverse interactive environments show that DeltaMem consistently outperforms existing baselines. To facilitate future research, we release the code at https://github.com/import-myself/DeltaMem.

Abstract

Memory Management for LLM Agents

Explore similar work

DELTAMEM: Incremental Experience Memory for LLM Agents via Residual Trees

Memex(RL): Scaling Long-Horizon LLM Agents via Indexed Experience Memory

CoMem: Learning to Anticipate Failures from Retrieved Experience