Lifelong Context Adaptation through Bidirectional Memory-Skill Learning

Tianyi Xu; Huazheng Wang

cs.AIOct 23, 2026

Lifelong Context Adaptation through Bidirectional Memory-Skill Learning

Authors: Tianyi Xu, Huazheng Wang

Organizations: Oregon State University

Abstract

Lifelong LLM agents must learn from an expanding history of interactions while making each decision within a bounded context window. Memory makes past experience available, but the agent may still need to infer how to solve each new task from the retrieved records. Skills provide reusable procedures, yet may omit the task-specific facts needed to apply them. Because retrieved memory and skills shape the agent's behavior, they also influence the experience available for future learning. We introduce MemCryst, a framework for lifelong context adaptation through bidirectional learning of memory and skills with a frozen language model. Recurring episodes stored in memory support skill creation and revision, and the interactions guided by those skills supply new memory for subsequent learning. Facts retrieved from memory and learned skills jointly form the agent's context, while task outcomes adjust the future retrieval influence of the records actually served without rewriting their content. Across ten settings spanning embodied control, long-document reading, conversation, mathematics, and code, MemCryst leads seven reported comparisons. On ALFWorld, joint serving achieves 63.17% success, compared with 59.83% for memory-only and 50.50% for skill-only serving. Code is available at https://github.com/tianyi0216/Memcryst.git

Figures & tables

Figure 1: Bidirectional memory–skill learning through context and behavior. Recurring episodes support skill creation and revision. Jointly served memory and skills guide the frozen actor, whose completed interactions supply new memory for later skill consolidation. Task outcomes separately update the retrieval influence of exposed facts and skills without rewriting their text. Solid arrows show content and behavior flow, and dashed arrows show statistical updates. Return applies before new records are written.

Method	ALFWorld	RULER	BABILong	Overall
ReAct	$51.50$	$41.30$	$34.50$	$42.43$
ExpeL	$44.80$ $\downarrow 6.70$	$\mathbf{71.27}$ $\uparrow 29.97$	$44.50$ $\uparrow 10.00$	$53.52$ $\uparrow 11.09$
MemP	$48.80$ $\downarrow 2.70$	$62.55$ $\uparrow 21.25$	$43.00$ $\uparrow 8.50$	$51.45$ $\uparrow 9.02$
SimpleMem	$44.30$ $\downarrow 7.20$	$35.10$ $\downarrow 6.20$	$33.30$ $\downarrow 1.20$	$37.57$ $\downarrow 4.87$
LightMem	$52.30$ $\uparrow 0.80$	$12.20$ $\downarrow 29.10$	$14.50$ $\downarrow 20.00$	$26.33$ $\downarrow 16.10$
MemCryst	$\mathbf{63.17}$ $\uparrow 11.67$	$64.70$ $\uparrow 23.40$	$\mathbf{59.00}$ $\uparrow 24.50$	$\mathbf{62.29}$ $\uparrow 19.86$

Table 1: Prequential performance across ten task streams. Scores are percentages, with changes from ReAct in pp. Overall equally weights benchmarks within panels and the three panel means in panel d.

System	RULER	BABILong	MATH-hard	MBPP	LoCoMo	Overall
ReAct	$36.90$	$35.00$	$61.60$	$76.00$	$0.50$	$28.30$
Specialist	$64.70$ $\uparrow 27.80$	$50.00$ $\uparrow 15.00$	$60.90$ $\downarrow 0.70$	$77.40$ $\uparrow 1.40$	$31.80$ $\uparrow 31.30$	$\mathbf{48.50}$ $\uparrow 20.20$
MemCryst	$55.90$ $\uparrow 19.00$	$52.00$ $\uparrow 17.00$	$58.30$ $\downarrow 3.30$	$80.00$ $\uparrow 4.00$	$25.30$ $\uparrow 24.80$	$44.50$ $\uparrow 16.20$

Table 2: Performance on a stream that interleaves five domains. Scores are percentages. Specialist uses separate domain runs. MemCryst uses one shared external state over 1,088 events. Overall averages 1,018 scored queries.

Backbone	ReAct	MemCryst	$\Delta$ pp
Qwen3-1.7B	$13.50$	$27.67$	$\uparrow 14.17$
Qwen3-8B	$51.50$	$63.17$	$\uparrow 11.67$
GPT-4o-mini	$28.33$	$40.00$	$\uparrow 11.67$

Table 3: ALFWorld performance across backbones. Rates are percentages. Each task allows at most 30 actions.

Configuration	SR%	$\Delta$ vs. Full
ReAct	$56.17$	$-7.00$
Memory-only	$59.83$	$-3.33$
Skill-only serving	$50.50$	$-12.67$
No skill revision	$62.00$	$-1.17$
Full MemCryst	$\mathbf{63.17}$	$0.00$

Table 4: ALFWorld component ablations over 600 tasks. All configurations use the same revisit guard and navigation hint. Panel a removes skills, memory retrieval, or skill revision and reports differences from Full in percentage points. Skill-only serving retains internal memory for skill learning. Panel b switches fact and skill return while retaining construction and retrieval of memory and skills.

Panel of Figure 2 — Figure 2: Evolution and reuse of the learned state. The ALFWorld run with both return channels enabled processes $T=600$ tasks and achieves $S_{\mathrm{preq}}=63.17\%$ . Successful deployments accumulate throughout the stream. After cycle 2, all accepted updates revise existing skills instead of creating new ones.

Figure 3: Frozen-state context controls on 100 held-out ALFWorld tasks. All conditions use the same learned state and a 1,024-token injected-context limit. Episode replacement keeps MemCryst ’s facts and replaces skills with retrieved episodic records. Random selection matches MemCryst ’s record types and approximate context length. Panel a shows overall and task-family success, with task counts below the axis. Panels b and c show mean injected context and actor input per task, respectively. Actor input includes retries.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Meaning	Symbol	Meaning
$t,T$	event index and stream horizon	$q_{t},m_{t},\iota_{t},\mathcal{I}_{T}$	event, mode, scoring flag, and scored set
$\pi_{\theta_{0}},\tau_{t}$	frozen policy and produced trajectory	$o_{t}=(r_{t},y_{t})$	reward and evaluator-defined success
$\mathcal{E}_{t},\mathcal{F}_{t},\mathcal{S}_{t}$	episode store, fact store, and skill catalog	$\Theta_{t}$	disjoint union of the three content stores
$\rho_{t}$	mutable retrieval and outcome statistics	$\Xi_{t}$	behaviorally active external state
$\mathcal{A}_{s}$	supporting episode IDs	$e,f,s$	one episode, fact, and skill record
$\delta,\ell,\varphi,z,$ $\chi,\psi,\varepsilon$	diagnosis, lesson, claim, type, applicability, procedure, and scope	$P_{f},a_{e}$	fact provenance and episode lifecycle state

Appendix

Table 5: Notation used throughout the paper. The index

X\in\{E,F,S\}

identifies one of the three stores.

	Check	Passing condition
1	Instance literals	No names, identifiers, or varying numeric values from a single episode are baked into the procedure.
2	Borrowed framing	The skill is a reusable procedure rather than a paraphrase of one episode.
3	Runtime probes	Relevant preconditions are checked when the action space supplies a probe, or are stated explicitly otherwise.
4	Parametricity	Values that vary across uses appear as parameter slots.
5	Necessary setup	The procedure contains no setup step that cannot contribute to the task.
6	Workflow integrity	No declared tool call or required stage is silently bypassed.

Appendix

Table 6: Skill audit checks. Each skill proposal is evaluated in a fresh context.

Order	Successes	SR%	Clean	Cool	Examine	Heat	Move
Seed 101	359/600	$59.83$	$60.2$	$54.3$	$66.1$	$62.2$	$59.6$
Seed 202	383/600	$63.83$	$62.2$	$60.9$	$86.4$	$62.2$	$61.0$
Seed 303	379/600	$63.17$	$61.2$	$62.0$	$83.1$	$62.2$	$60.3$
Seed 404	345/600	$57.50$	$57.1$	$56.5$	$64.4$	$52.7$	$57.8$
Mean		$61.08$	$60.2$	$58.4$	$75.0$	$59.8$	$59.7$
Sample SD		$2.96$	$2.2$	$3.6$	$11.4$	$4.7$	$1.4$

Appendix

Table 7: ALFWorld performance across four shuffled-order runs. Scores are percentages, and standard deviations are in percentage points computed over the four new runs only. Each run contains 98 clean, 92 cool, 59 examine, 74 heat, and 277 move tasks.

Benchmark	Metric	Terminal score	Prequential score	$\Delta$
ALFWorld	Success rate	$70.00$	$63.17$	$\uparrow 6.83$
RULER	Recall	$65.70$	$64.70$	$\uparrow 1.00$
BABILong	Mean reward	$57.00$	$59.00$	$\downarrow 2.00$
LoCoMo	Token F1	$23.70$	$31.75$	$\downarrow 8.05$
MBPP	Unit test reward	$81.59$	$80.42$	$\uparrow 1.17$

Appendix

Table 8: Terminal and prequential performance of MemCryst . Scores are percentages, and

\Delta=100(S_{\mathrm{term}}-S_{\mathrm{preq}})

is in percentage points.

Method	SR%	$\Delta$ vs. ReAct
ReAct	$51.5$	n/a
Reflexion	$52.7$	$\uparrow 1.2$
SAGE	$49.5$	$\downarrow 2.0$
Live-Evo	$49.0$	$\downarrow 2.5$
Dynamic Cheatsheet	$47.8$	$\downarrow 3.7$
A-Mem	$43.2$	$\downarrow 8.3$

Appendix

Table 9: Additional ALFWorld success rates under the benchmark interface.

System	Successes / $N$	SR%	30 actions reached	Invalid actions
ReAct	337/600	$56.17$	268	905
MSCE	296/600	$49.33$	308	1,082
MemCryst	379/600	$\mathbf{63.17}$	226	661

Appendix

Table 10: Comparison with MSCE on ALFWorld and DialSim. Scores are percentages. The “30 actions reached” count includes tasks solved on the final allowed action. Panel b uses the highest-scoring tested MSCE configuration, with 957 questions and 535 timed questions. ReAct and MemCryst answer 955 questions, including 533 timed questions. All three runs include 295 untimed and 127 unanswerable questions.

Panel of Figure 4 — Figure 4: ALFWorld state dynamics with both return channels enabled. Panel 4(a) reports deployments, successes, and success rates by family. Panels 4(b) and 4(c) show cumulative successful deployments and terminal co-serving links by fact type. Panel 4(d) shows candidates as a dashed line, accepted new skills and revisions as stacked bars, and task success in each cycle.

Figure 5: Lifecycle of the most frequently deployed final skill. From top to bottom, the traces show admission and recorded name changes, all 201 deployments with 155 successes, and the 97 episodes in the final synthesis ancestry. The 176 distinct terminal fact co-serving links have no timestamps and may involve earlier skill texts. Same-name revisions overwrite prior versions, so the records do not recover every revision or ancestry size over time.

Panel of Figure 6 — Figure 6: ALFWorld actor-call efficiency. All methods use Qwen3-8B and 600 tasks. ReAct uses its original unguarded setting, while the other configurations retain the revisit guard. Calls include action retries but exclude memory maintenance. Panel 6(b) shows the 250 shared successes, with marker size indicating task count.

Panel of Figure 7 — Figure 7: MBPP with and without episodic context. Curves show cumulative mean reward and actor steps over the same 252 tasks with Qwen3-8B and retrieval limits of four facts and two skills.

Block	Hyperparameter	Default
LLM	Primary backbone	Frozen Qwen3-8B
	Role temperatures	actor $0$ , writer $.2$ , synthesizer $.3$ , and auditor $0$
State	Initial external state	empty episode store, fact store, and skill catalog
Memory	Episode weight	$1.3$ after success and $.7$ after failure
	Prior for a new fact	$n_{f}=h_{f}=0$ , hence $C_{f}=.5$
	Fact merge	same entity and scope with cosine threshold $\eta=.86$

Appendix

Table 11: Default MemCryst framework settings.

Benchmark	State scope	Evaluation mode	Native reward $r_{t}$	Return signal $y_{t}$
Embodied control and reading long documents
ALFWorld	persistent	legal text actions with a limit of 30 steps	binary success	$r_{t}$
RULER	persistent for MemCryst	one document and one question per item with an answer context of 8,192 tokens	mean recall reward	$\mathbf{1}[r_{t}\geq.99]$
BABILong	persistent for MemCryst	one document and one question per item with an answer context of 8,192 tokens	mean reward	$\mathbf{1}[r_{t}\geq.99]$
Conversation and memory
LoCoMo	per dialogue	session ingestion followed by short answers	token F1	$\mathbf{1}[\mathrm{F1}\geq.5]$

Appendix

Table 12: Benchmark evaluation settings. Persistent streams begin empty and score each item before writeback.

Benchmark	Domain	$N$	Metric
ALFWorld	Embodied control	600	Success rate
RULER	Long document reading	400	Recall
BABILong	Long document reading	200	Mean reward
LoCoMo	Conversation memory	1,977	Token F1
LongMemEval	Conversation memory	60	Token F1
DialSim-subst.	Conversation memory	955	Success rate

Appendix

Table 13: Benchmarks, sample counts, and metrics.

N

counts the items scored for MemCryst .

Memory-Augmented Language Model Agents

Memory-augmented language model agents use persistent facts, interaction histories, and skills to support personalization and long-horizon tasks.

See all 444 papers

LifeMem: Enabling Lifelong Experience Reuse for LLM Agents
Sep 11, 2026Yuli Qiu, Yutong Li, Wei Su +5Continual Learning for LLM Agents LLM Agent Skill Learning
School of Computer Science and Technology, Beijing Institute of Technology · School of Computer Science and Engineering, Beihang University · Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology +1
Mem- $π$ : Adaptive Memory through Learning When and What to Generate
May 20, 2026Xiaoqiang Wang, Chao Wang, Hadi Nekoei +5LLM Agent Memory RL for LLM Agents
1ServiceNow AI Research · 2Mila – Quebec AI Institute · 3Université de Montréal +3
ERSkill: Evolving for Skill-Guided Adaptive Memory Retrieval
Aug 13, 2026Haolong Chen, Liang Zhang, Zhuo Li +2LLM Agent Memory LLM Agent Skill Learning
Shenzhen International Center for Industrial and Applied Mathematics · Shenzhen Research Institute of Big Data · The Chinese University of Hong Kong, Shenzhen +2

Sep 11, 2026J/K move · Enter open · S save

LifeMem: Enabling Lifelong Experience Reuse for LLM Agents

Yuli Qiu, Yutong Li, Wei Su, Zeming Liu+4

School of Computer Science and Technology, Beijing Institute of Technology · School of Computer Science and Engineering, Beihang University · Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology+1

Continual Learning for LLM Agents LLM Agent Skill Learning Transfer Learning Memory-Augmented Language Model Agents

Open paper AskPDF arXiv

Large language model agents are expected to continuously adapt to new tasks and environments over their lifetime by reusing past experience. However, existing memory-based agents struggle to transfer reusable experience across environments and suffer from catastrophic forgetting as experience accumulated. To address these challenges, we propose LifeMem, a lifelong learning framework that enables agents to transfer knowledge across multiple environments. During learning, LifeMem clusters accumulated interaction trajectories based on underlying workflows to extract reusable skills. When solving a new task at inference time, the agent recalls relevant skills and trajectories to guide actions. To validate our method, we conduct experiments across 10 environments and over 13k tasks with 2k newly annotated interaction trajectories. Results show that LifeMem enables effective experience reuse in lifelong learning, achieving both reduced forgetting on learned tasks and superior cross-task transfer. Further analysis reveals that task streaming impacts learning, while consolidating structurally similar trajectories within memory boosts performance.

Abstract

Memory-Augmented Language Model Agents

Explore similar work

LifeMem: Enabling Lifelong Experience Reuse for LLM Agents

Mem-πππ: Adaptive Memory through Learning When and What to Generate

ERSkill: Evolving for Skill-Guided Adaptive Memory Retrieval

Mem- $π$ : Adaptive Memory through Learning When and What to Generate