Recursive Experiential-Working Memory Evolution for Long-Horizon Agent Harnesses

Zhaochen Yu; Yingcheng Wu; Zhenfei Yin; Kaiyuan Chen; Zhe Zhao; Mengdi Wang; Shuicheng Yan; Ling Yang

cs.AIAug 25, 2026

Recursive Experiential-Working Memory Evolution for Long-Horizon Agent Harnesses

Authors: Zhaochen Yu, Yingcheng Wu, Zhenfei Yin, Kaiyuan Chen, Zhe Zhao, Mengdi Wang, Shuicheng Yan, Ling Yang

Organizations: NUS · Princeton University · Stanford University · University of Oxford

Abstract

Recursive self-improvement (RSI) remains hard in long-horizon tasks, where growing histories obscure the task state and misalign skill invocation. We introduce Recuris, a recursive Experiential-Working Memory architecture for long-horizon agent harnesses, in which Working Memory tracks task progress and guides skill selection from Experiential Memory, grounding skill use in current needs rather than the full history. This coupling also turns execution into structured evidence that localizes failures to specific memory components. Across tasks, a fixed Meta-Agent turns that evidence into localized, validation-gated updates to Skill Memory that reshape execution and yield new evidence, forming a bounded recursive memory-evolution loop. Across four long-horizon benchmarks and ten models, Recuris improves task success in 35 of the 37 completed model-benchmark pairs, carrying frontier models to SOTA-level task success: on tau-bench it adds +17.8 points to GPT-5.6 Sol and +15.6 to Claude Opus 5, taking Opus 5 to 87.9%, and +16.6/+13.5 points on Qwen3.6-27B/35B on SkillFlow. The advantage widens as the interaction horizon grows, to +32.2 points on the longest tasks, and common long-horizon failures fall by up to 80%. These results position recursively evolving memory as a scalable foundation for RSI, enabling agents to continuously transform accumulated experience into increasingly effective long-horizon behavior. Code: https://github.com/Gen-Verse/Recuris

Figures & tables

Figure 2: Two shifts separate Recuris from prior memory-based agent harnesses. How memory is used (left): prior harnesses retrieve against a growing chat history and lose track of unresolved goals. Recuris retrieves at execution events, and a checker turns the environment’s response into a verified state update rather than trusting the dialogue. How memory evolves (right): prior harnesses rewrite the whole memory from one task outcome. Recuris reads the structured trajectory, attributes each failure to a component, patches exactly the components implicated, and admits the result only through a validation gate on held-out tasks.

Figure 3: Overview of Recuris. (a) Within a task: the working memory $w_{t}$ tracks each goal with its verified state. At a defined execution event the invocation policy $\rho$ retrieves the matching skill from experiential memory $\mathcal{E}$ ; after the agent acts, the checker set $\mathcal{C}$ commits only the state changes the observation $o_{t}$ supports. (b) Across tasks: the structured trace records $(w_{t},\mathcal{E}_{t},a_{t},o_{t})$ at every step, so a fixed Meta-Agent attributes each diagnosed failure to a component and patches exactly the components implicated. A fixed gate admits the candidate only if it repairs the source tasks without regressing a held-out development set; otherwise the memory is kept unchanged. The base LLM and the outer procedure remain fixed.

	Cross-task evolution			Within-task adaptation
Model	$\tau^{2}$ -Retail	$\tau^{2}$ -Airline	SkillFlow	Terminal-Bench 2.1
Open-weight models
Granite-4.1-3B	9.7	34.3	0.3	0.6
$+$ Recuris	23.0 ( $+$ 13.4 † )	39.8 ( $+$ 5.5)	0.0 ( $-$ 0.3)	3.1 ( $+$ 2.5)
Qwen3.5-4B	68.0	75.3	6.0	10.1
$+$ Recuris	68.3 ( $+$ 0.3)	79.0 ( $+$ 3.8)	7.1 ( $+$ 1.1)	13.0 ( $+$ 2.9)

Table 1: Recuris improves task success where the task family has shared structure, and its value does not follow model scale. Cells are avg@4 task success (%); higher is better. Each model contributes a pair of rows, the benchmark’s own reference agent alone and that same agent with Recuris (grey). Bold marks the better of each pair; the subscript is

\Delta

, shaded by the size of the gain; † marks a paired task-clustered bootstrap 95% CI excluding zero. Agents, models and protocol are given in section 3.1 . New A not evaluated on that benchmark.

Figure 4: Long-horizon failure is an execution problem, not a retrieval problem. Task success (left) and required-write recall (right) for the four ablation variants on

\tau^{2}

-Retail, stratified into quartiles by the number of turns a task takes to complete; quartile sizes appear under each tick and the grey band spans read-action recall for every variant in every quartile. Shaded regions are paired task-clustered bootstrap

95\%

CIs (

10{,}000

resamples of tasks). Quartiles are defined by task-intrinsic completion length, never by the length a variant realises, which is an outcome of the variant.

	$\tau^{2}$ -Retail (114 tasks, 456 episodes)		$\tau^{2}$ -Airline (50 tasks, 200 episodes)
Variant	Success ( $\uparrow$ )	$\Delta$ [95% CI]	Success ( $\uparrow$ )	$\Delta$ [95% CI]
Base (no EM, no WM)	58.1 (265/456)	New A	75.5 (151/200)	New A
EM only	60.1 (274/456)	$+2.0$ [ $-4.0$ , $+7.9$ ]	77.0 (154/200)	$+1.5$ [ $-4.5$ , $+7.5$ ]
WM only	82.0 (374/456)	$+23.9^{\dagger}$ [ $+17.5$ , $+30.3$ ]	79.5 (159/200)	$+4.0$ [ $-5.0$ , $+13.5$ ]
Model-controlled invocation	65.6 (299/456)	$+7.5^{\dagger}$ [ $+1.5$ , $+13.4$ ]	New A	New A
EM + WM (Recuris)	83.6 (381/456)	$\mathbf{+25.4}^{\dagger}$ [ $+18.4$ , $+32.5$ ]	84.0 (168/200)	$\mathbf{+8.5}$ [ $-1.0$ , $+18.5$ ]

Table 2: Coupling both memories gives the strongest variant in each domain. Each variant removes one component from one fixed Skill Memory; variants within a domain are matched on model, user simulator, skill library, tools, tasks, seeds and budget. Cells give success rate (successful/total episodes);

\Delta

is against that domain’s base agent ( appendix A ) and † marks an interval excluding zero. The two domains were evaluated separately, so values compare within a column; the model-controlled airline cells are omitted because that batch cannot be paired.

Figure 5: Coupling of EM and WM. Bars give each variant’s gain over the base agent of the same domain, whose absolute success rate is stated above each panel, so the base agent is the zero of this chart and the axis is not truncated. Confidence intervals are reported in table 2 rather than drawn here; on

\tau^{2}

-Airline every interval includes zero, so that panel shows a direction and not an established effect. Panels use separate scales because the two domains were evaluated in separate runs.

Figure 6: The critical mechanism differs between domains. Each bar removes one harness mechanism from the full system and reports the change in task success against that domain’s full system. The two mechanisms that matter are disjoint: write review carries

\tau^{2}

-Airline and is inert on

\tau^{2}

-Retail, while the status board does the reverse. † marks a paired task-clustered bootstrap 95% CI that excludes zero:

[-22.6,-12.3]

for the status board on

\tau^{2}

-Retail and

[-21.0,-6.5]

for write review on

\tau^{2}

-Airline. Every other interval includes zero. Gate termination was not ablated on

\tau^{2}

-Retail.

	Base	WM only	Model-controlled	Recuris
Task success (%)	58.1	82.0	65.6	83.6
Required-write recall (%)	55.7	80.9	61.1	82.4
Omitted required writes / episode	0.596	0.145	0.417	0.121
Agent tokens per success (k)	116	102	147	101

Table 3: Control over skill invocation matters more than skill content.

\tau^{2}

-Retail, 114 tasks,

k=4

, 456 episodes per variant at matched model, decoding and harness. Model-controlled and Recuris carry the same ten skills, byte-identical. Required-write recall is the share of the reference plan’s database-mutating actions the agent performed; omitted counts those never issued at all. Cost charges every episode, including failures, to the variant that spent it.

Figure 7: State-grounded invocation recovers required writes the agent would otherwise omit, and enters the error-compounding regime least often. (a) share of all 456 episodes that have issued a correct required write by turn

T

; the dashed rule marks the median turn of the first correct write, which is identical in all four variants. (b) probability that a required write mismatches, split by whether an earlier write in the same episode already mismatched.

Figure 8: Case study of verified EM–WM coupling on $\tau^{2}$ -Retail Task 91.

	Injected fault
Evidence given to the judge	$\mathcal{E}$	$\mathcal{W}$	$\rho$	Macro	Macro-F1
Outcome only	0.0	38.9	0.0	13.0	10.4
Raw trajectory	61.1	50.0	0.0	37.0	31.2
Structured trace $\Gamma$	72.2	83.3	38.9	64.8	63.4

Table 4: Structured traces make component faults observable, and the gain is concentrated on the faults the transcript cannot show. A known fault is injected into one component of a working package, a skill (

\mathcal{E}

), the working-memory specification (

\mathcal{W}

), the invocation policy (

\rho

), or the checkers (

\mathcal{C}

) of eq. 2 ; a fixed judge names the component at fault from one of three evidence conditions. Nine admitted cases per class, two repeats, 54 verdicts per condition. Cells are per-class recall (%); Macro averages the three classes against a

33.3\%

constant-answer floor.

Evolution run	Package	Success	$\Delta$ vs. $\mathcal{M}_{0}$	Reach
$\mathcal{M}_{0}$ (shared starting memory)		54.07	New A	0/86
Run A	round 1	63.08	$+9.01^{\dagger}$ [ $+1.45$ , $+16.57$ ]	83/86
	round 2	65.99	$+11.92^{\dagger}$ [ $+4.65$ , $+19.19$ ]	78/86
	round 3	65.99	$+11.92^{\dagger}$ [ $+4.65$ , $+18.90$ ]	–
Run B	round 1	64.53	$+10.47^{\dagger}$ [ $+3.78$ , $+17.15$ ]	–
	round 2	64.83	$+10.76^{\dagger}$ [ $+3.20$ , $+18.31$ ]	86/86

Table 5: Recursive evolution yields consistent held-out gains, and a second round compounds them. Every package is evaluated on the same 86 tasks held out from the Meta-Agent at

k=4

. Run A’s Meta-Agent runs on Claude Code, Runs B and C on DeepSeek Harness under the identical protocol; only Run C admits candidates inside the loop.

\Delta

is against

\mathcal{M}_{0}

( appendix A ); † marks an interval excluding zero. Reach is the share of held-out tasks on which the memory was invoked at least once, and is blank where no instrumented run exists.

Figure 9: Held-out gains are consistent across evolution runs, and finer than the in-round gate can resolve. (a) Task success of each final package relative to

\mathcal{M}_{0}

on the 86 held-out tasks: Run A plateaus, Run B’s round-4 package is never invoked (open marker), and Run C rises for two rounds and then reverses. The band is the difference between two runs of the identical

\mathcal{M}_{0}

package. (b) The dev-split reading of each rejected candidate at the moment of rejection. Every interval contains zero, and the three marked candidates later cleared zero on the held-out split.

Comparison	$\Delta$ (points)	95% CI	$p$
Settled gain over $\mathcal{M}_{0}$
Claude Code (Run A)	$+11.92^{\dagger}$	$[+4.65,\,+19.19]$	New A
DeepSeek Harness (Run B)	$+10.47^{\dagger}$	$[+3.78,\,+17.15]$	0.0022
DeepSeek Harness, progressive (Run C)	$+9.30^{\dagger}$	$[+1.45,\,+17.44]$	0.0068
Direct paired contrasts
DeepSeek Harness $-$ Claude Code	$-1.45$	$[-7.85,\,+4.65]$	0.72

Table 6: Swapping the Meta-Agent from Claude Code to DeepSeek Harness reproduces the gain, the endpoint and the learned components. All packages are evaluated on the frozen 86-task split at

k=4

( table 5 );

\Delta

carries a paired task-clustered bootstrap 95% CI,

p

is McNemar’s exact test, and † marks an interval excluding zero. The implementation contrast pairs the DeepSeek Harness round-1 package (Run B) against the Claude Code round-2 package (Run A); the threshold contrast pairs Run C’s final package against the same Run B package. The A/A row re-runs the byte-identical

\mathcal{M}_{0}

package and sets the instrument’s resolution.

Model	Domain	agent alone	$+$ Recuris	$\Delta$ (95% CI)
GPT-5.6 Sol	$\tau^{2}$ -Retail	58.33	76.10	$\mathbf{+17.76}^{\dagger}$ $[+11.84,+23.90]$
Claude Opus 5	$\tau^{2}$ -Retail	72.37	87.94	$\mathbf{+15.57}^{\dagger}$ $[+10.96,+20.39]$
Gemini 3.7 Flash	$\tau^{2}$ -Retail	73.46	78.29	$+4.82$ $[-0.22,+10.09]$
GPT-5.6 Sol	$\tau^{2}$ -Airline	79.00	86.00	$\mathbf{+7.00}^{\dagger}$ $[+1.50,+13.00]$
Claude Opus 5	$\tau^{2}$ -Airline	89.50	90.50	$+1.00$ $[-3.00,+4.00]$
Gemini 3.7 Flash	$\tau^{2}$ -Airline	86.50	85.00	$-1.50$ $[-5.50,+2.00]$

Table 7: One package, evolved on a mid-sized deployment model, lifts frontier models it never saw. Each configuration runs the full task set at

k=4

with zero null episodes;

\Delta

carries a paired task-clustered bootstrap 95% CI and † marks an interval excluding zero. Rows are ordered by the level the agent reaches alone.

Figure 10: Illustration of component-specific Skill Memory evolution.

Configuration	Budget	Solved	$\Delta$	$p$	$\Delta$ isolates
Terminus-2 (baseline)	1	30/87 (34.5)	New A	New A	New A
$+$ seed memory	1	28/87 (32.2)	$-2.3$	0.824	the layer alone
$+$ seed memory, retry	4	51/87 (58.6)	$+26.4$	$<10^{-4}$	the attempt budget
$+$ test-time adaptation	4	53/87 (60.9)	$+2.3$	0.774	learning, matched budget

Table 8: Terminal-Bench 2.1, decomposed: the attempt budget carries the headline, and the memory terms are read at a matched budget. Every configuration stops at its first success. Solved counts tasks solved within budget at one rollout per attempt.

\Delta

and

p

compare each row with the row above it on the same tasks,

p

from McNemar’s exact test; the last column names what that comparison isolates.

Metric	Tasks	seed	adapted	$\Delta$	95% CI
avg@4, learned	56	17.4	21.9	$+4.5$	$[-0.9,\,+9.8]$
pass@4, learned	56	39.3	42.9	$+3.6$	$[-8.9,\,+16.1]$
avg@4, all	87	46.1	48.9	$+2.9$	$[-0.6,\,+6.3]$
pass@4, all	87	60.9	63.2	$+2.3$	$[-5.7,\,+10.3]$

Table 9: Per-attempt success, the metric extra attempts cannot move. Four untruncated rollouts per configuration, seed memory against the memory test-time adaptation produced. Learned is the 56 tasks whose adapted memory carries a learned skill; all splices those with the remaining tasks, which adaptation leaves unchanged. CIs are paired task-clustered bootstrap

95\%

intervals.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Target model	bare	$\mathcal{M}_{0}$	$\Delta$	95% CI	up/down
GPT-OSS-20B	45.35	50.58	$+5.23$	$[-1.16,\,+11.63]$	31 / 20
Qwen3.5-9B	77.84	77.56	$-0.28$	$[-5.11,\,+4.83]$	23 / 29
Qwen3.6-35B	78.78	78.20	$-0.58$	$[-5.81,\,+4.65]$	21 / 25
GLM-4.7-Flash	61.05	61.92	$+0.87$	$[-4.94,\,+6.98]$	27 / 26

Appendix

Table 10: The harness on its own contributes nothing measurable. Bare agent against

\mathcal{M}_{0}

\tau^{2}

-Retail, 86–88 tasks per model at

k=4

, zero null episodes, one harness commit and byte-identical decoding settings. Every interval contains zero and the point estimates split two positive, two negative.

Target model	harness	evolution	total
GPT-OSS-20B	$+5.23$	$\mathbf{+10.17}^{\dagger}$ $[+4.07,+16.28]$	$\mathbf{+15.41}^{\dagger}$ $[+8.14,+22.38]$
Qwen3.5-9B	$-0.28$	$+1.99$	$+1.70$
Qwen3.6-35B	$-0.58$	$+0.29$	$-0.29$
GLM-4.7-Flash	$+0.87$	$+0.00$	$+0.87$

Appendix

Table 11: Where the effect lives. The harness term is

\mathcal{M}_{0}

minus bare, evolution is the final package minus

\mathcal{M}_{0}

, total is the final package minus bare. † marks an interval excluding zero.

Regime	Success (%)	First-call prompt	Tokens/episode	Tokens/success
Bare agent	58.11	5163	67,315	115,833
Working memory only	82.02	5636	83,938	102,341
Model-controlled invocation	65.57	8274	96,289	146,849
Recuris	83.55	5632	84,275	100,865

Appendix

Table 12: More context buys a worse result. The model-controlled configuration keeps the whole skill library standing in context,

3{,}111

more prompt tokens at the first call than Recuris, and is 18 points worse while costing

46\%

more per success.

Family	canonical	selected in family
weighted-risk-assessment	0/8	7/8
embedded-data-repair	4/8	8/8
healthcare-cost-benefit-analysis	1/9	$\sim$ 6/9

Appendix

Table 13: The same model, the same tasks, a different procedural description. Tasks solved in a family under the canonical template and under the template selected inside that family. Selection is in-sample, so these figures measure what procedure is worth, not generalization.

Long-Horizon LLM Agents

Long-horizon large language model agents plan, use tools, and preserve task state across extended, multi-step interactions.

See all 429 papers

Let Them Sleep: Recursive Self-Pruning Memory for Long-Horizon LLM Agents
Oct 18, 2026Kargi Chauhan, Sadiba Nusrat Nur, Pratibha Revankar +2LLM Agent Memory Memory Management for LLM Agents
University of California, Santa Cruz
RRSI: Regularized Recursive Self-Improvement of Agent Harnesses
Sep 21, 2026Peng Xia, Rujun Han, Zifeng Wang +11OOD Generalization Agent Harness Optimization
Google Cloud AI Research · UNC-Chapel Hill · Stanford University +1
Memex(RL): Scaling Long-Horizon LLM Agents via Indexed Experience Memory
Mar 4, 2026Zhenting Wang, Huancheng Chen, Jiayun Wang +1LLM Agent Memory Memory Management for LLM Agents
Center for Advanced AI, Accenture

Oct 18, 2026J/K move · Enter open · S save

Let Them Sleep: Recursive Self-Pruning Memory for Long-Horizon LLM Agents

Kargi Chauhan, Sadiba Nusrat Nur, Pratibha Revankar, Kyle Montgomery+1

University of California, Santa Cruz

LLM Agent Memory Memory Management for LLM Agents Agent Memory Management Long-Horizon LLM Agents Long-Term Conversational Memory

Open paper AskPDF arXiv

Agents that run for a long time have a memory problem, and it is not the one most systems optimize for. Storing every trace and retrieving the nearest neighbors works early and quietly stops working later: the store fills up with failures the agent then repeats, and with facts that were true fifty turns ago. RSPM (Recursive Self-Pruning Memory) borrows the fix biology already found. At intervals the agent sleeps. It clusters its recent failures, distills each cluster into a short rule, and deletes the raw episodes it just learned from. Rules decay unless something reinforces them, and rules that recur across many contexts get promoted into general ones. We tested this on three settings with little in common: a difficulty-calibrated SWE-bench simulation, temporal consistency across 12 conversational benchmarks, and long-horizon QA on LoCoMo-10. All three improve, by 9.1, 11.9, and 7.6 points respectively. More interestingly, the two studies that instrument the store itself converge on nearly the same operating point without being tuned toward it: roughly 93% fewer retry loops and roughly 30% fewer stored entries. That hints that knowing what to forget is a general property of agent memory rather than a benchmark-specific trick.

Abstract

Long-Horizon LLM Agents

Explore similar work

Let Them Sleep: Recursive Self-Pruning Memory for Long-Horizon LLM Agents

RRSI: Regularized Recursive Self-Improvement of Agent Harnesses

Memex(RL): Scaling Long-Horizon LLM Agents via Indexed Experience Memory