Self-Evolving LLM Memory Extraction Across Heterogeneous Tasks

Yuqing Yang; Tengxiao Liu; Wang Bill Zhu; Taiwei Shi; Linxin Song; Robin Jia

cs.CLApr 13, 2026

Self-Evolving LLM Memory Extraction Across Heterogeneous Tasks

Authors: Yuqing Yang, Tengxiao Liu, Wang Bill Zhu, Taiwei Shi, Linxin Song, Robin Jia

Organizations: University of Southern California · University of California, Santa Barbara

Abstract

As LLM-based assistants become persistent and personalized, they must extract and retain useful information from past conversations as memory. However, the types of information worth remembering vary considerably across tasks. We formalize the heterogeneous memory extraction task and introduce BEHEMOTH, a benchmark that repurposes 18 existing datasets spanning personalization, problem-solving, and agentic tasks, using a downstream utility-driven metric for systematic evaluation. Our empirical analysis confirms that no single static extraction prompt dominates across all task categories, and that existing self-evolving prompt optimization frameworks, originally designed for homogeneous distributions, degrade when training tasks are heterogeneous. To address this, we propose CluE, a cluster-based self-evolving strategy that groups training examples into clusters by extraction scenarios, analyzes each cluster independently, and synthesizes cross-cluster insights to update the extraction prompt. Experiments on BEHEMOTH show that CluE generalizes effectively across heterogeneous tasks ( $+$ 9.04% relative gain), consistently outperforming prior self-evolving frameworks.

Figures & tables

Figure 1: Illustration of heterogeneous memory extraction. A general-purpose assistant $\mathrm{LLM}_{g}$ encounters diverse previous conversations spanning technical debugging, math problem-solving, and personal preferences, from which an extraction model $\mathrm{LLM}_{e}$ must produce different types of memory (e.g., reusable insights, solution steps, personal facts). $\mathrm{LLM}_{g}$ then leverages these memories to improve responses in new conversations. How well these responses address user needs can then serve as a utility-driven signal for evaluating extraction quality.

Figure 2: Dataset Composition of BEHEMOTH.

	Personalization		Problem-Solving		Agentic		Overall
	MA	RG	MA	RG	MA	RG	MA	RG
No Memory	34.15	–	46.52	–	30.36	–	37.84	–
Simple	58.76	0	48.76	0	32.46	0	46.00	0
Mem0	73.31	$+$ 29.96	45.97	$-$ 3.70	31.62	$+$ 0.44	48.48	$+$ 5.79
ReasoningBank	50.67	$-$ 10.78	50.72	$+$ 6.90	32.12	$+$ 2.49	44.51	$+$ 0.45
OpenMemory	57.39	$-$ 1.04	50.47	$+$ 5.59	34.93	$+$ 8.57	47.14	$+$ 4.74

Table 1: Static prompt evaluation on in-distribution test sets. MA : Macro Accuracy (%); RG : Relative Gain (%) over Simple . Best and second best MA are highlighted; best RG in bold . Overall aggregates across all datasets.

Figure 3: Overview of our method CluE in one evolution round.

	Personalization		Problem-Solving		Agentic		Overall
	MA	RG	MA	RG	MA	RG	MA	RG
No Memory	34.15	–	46.52	–	30.36	–	37.84	–
Simple	58.76	0	48.76	0	32.46	0	46.00	0
GEPA	56.59	$-$ 2.16	50.05	$+$ 2.86	37.23	$+$ 14.08	47.52	$+$ 5.06
ACE	53.73	$-$ 4.38	51.04	$+$ 8.04	33.49	$+$ 4.93	45.91	$+$ 3.56
MemEvolve	63.67	$+$ 10.76	49.49	$+$ 1.17	30.92	$-$ 3.25	47.08	$+$ 2.11

Table 2: Evaluation of self-evolving frameworks starting from the Simple prompt on in-distribution test sets. RG denotes the Relative Gain (%) over Simple .

	LME	GPQA-D	ToolBench
No Memory	20.45	45.62 ±3.12	23.33
Simple	46.02	47.14 ±4.15	25.30
GEPA	35.06	50.00 ±0.82	24.09
ACE	29.71	46.13 ±0.86	26.82
MemEvolve	56.82	47.98 ±1.09	26.67
CluE	63.07	48.48 ±1.80	26.67

Table 3: Evaluation on the out-of-distribution test sets of BEHEMOTH.

	Personalization		Problem-Solving		Agentic		Overall
	MA	RG	MA	RG	MA	RG	MA	RG
No Memory	34.15	–	46.52	–	30.36	–	37.84	–
Survey	62.05	0	49.86	0	33.58	0	47.69	0
GEPA †	62.05	0	49.86	0	33.58	0	47.69	0
ACE	54.80	$-$ 10.48	50.06	$+$ 0.75	34.41	$+$ 3.69	46.11	$-$ 1.44
MemEvolve	62.88	$-$ 0.31	51.22	$+$ 2.51	31.34	$-$ 4.85	47.70	$-$ 0.74

Table 4: Evaluation of self-evolving frameworks starting from the Survey prompt on in-distribution test sets. RG denotes the Relative Gain (%) over Survey .

\dagger

GEPA did not find a better prompt.

	Game of 24		AlfWorld
	Single-Step	Continual	Single-Step	Continual
No Memory	$26.67_{\pm 3.86}$	$35.42_{\pm 5.03}$	$30.41_{\pm 2.71}$	$63.74_{\pm 0.41}$
Simple	${\hbox{\pagecolor[HTML]{DFE9E2}27.92}}_{\pm 3.86}$	$29.58_{\pm 3.12}$	${\hbox{\pagecolor[HTML]{DFE9E2}47.66}}_{\pm 2.19}$	${\hbox{\pagecolor[HTML]{DFE9E2}64.91}}_{\pm 5.01}$
MemEvolve	$27.50_{\pm 2.70}$	${\hbox{\pagecolor[HTML]{DFE9E2}43.33}}_{\pm 3.58}$	$40.64_{\pm 0.41}$	$62.57_{\pm 2.30}$
CluE	${\hbox{\pagecolor[HTML]{AEC8B5}37.08}}_{\pm 2.12}$	${\hbox{\pagecolor[HTML]{AEC8B5}50.83}}_{\pm 7.52}$	${\hbox{\pagecolor[HTML]{AEC8B5}55.85}}_{\pm 3.23}$	${\hbox{\pagecolor[HTML]{AEC8B5}67.25}}_{\pm 2.98}$

Table 5: Single-step vs. continual memory extraction on Game of 24 and AlfWorld. The No Memory baselines differ between settings because the two settings evaluate on different example pools; see Appendix A.3 for details.

Figure 4: Structural comparison of the best prompts evolved by each framework from the Simple seed. Common sections (task description, input/output format) are omitted; only the distinctive components are shown. CluE produces a structured memory taxonomy with per-category definitions and guidelines alongside domain-agnostic general guidelines. GEPA embeds a large amount of domain-specific content (e.g., AlfWorld examples) directly into its rules. MemEvolve lacks a memory taxonomy and instead introduces a penalty clause that prohibits specific extraction patterns.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	# Train	# Test	$R$	Memory Work
Personalization
MemBench-High ( Tan et al., 2025 )	20	200	Acc.	MemBench ( Tan et al., 2025 )
MemBench-Low ( Tan et al., 2025 )	20	80	Acc.	MemBench ( Tan et al., 2025 )
PersonaMem-v2 ( Jiang et al., 2025 )	20	272	Acc.	PersonaMem-v2 ( Jiang et al., 2025 )
PrefEval ( Zhao et al., 2025 )	20	200	Acc.	PrefEval ( Zhao et al., 2025 )
LongMemEval ( Wu et al., 2025 )	–	176	LLM Judge	LongMemEval ( Wu et al., 2025 )

Appendix

Table 6: Datasets used in BEHEMOTH, grouped by task category.

R

denotes the reward function: Acc. = accuracy, EM = exact match, Rule = rule-based verification, SR = success rate, Pass@1 = code execution pass rate, SoPR = solvable pass rate (LLM judge), LLM Judge = LLM-as-judge evaluation. Memory Work indicates the prior work that uses each dataset for memory evaluation.

	Personalization		Problem-Solving		Agentic		Overall
	MA	RG	MA	RG	MA	RG	MA	RG
No Memory	34.15	–	46.52	–	30.36	–	37.84	–
Simple	65.86	0	50.19	0	38.46	0	50.46	0
GEPA	68.59	$+$ 5.64	52.91	$+$ 7.71	33.77	$-$ 9.99	50.71	$+$ 0.93
ACE	66.65	$-$ 0.43	48.43	$-$ 1.98	34.45	$-$ 8.88	48.63	$-$ 3.93
MemEvolve	65.46	$-$ 0.93	51.05	$+$ 2.37	36.84	$-$ 2.89	50.15	$-$ 0.29

Appendix

Table 7: Evaluation of Gemini-3-Flash as the extraction backend from the Simple prompt. RG denotes the Relative Gain (%) over Simple .

	Wall Time	Optimization LLM Calls	Eval Calls
GEPA	$\sim$ 7.4h	46	1,645
ACE	$\sim$ 12.4h	610	1,120
MemEvolve	$\sim$ 5.0h	30	1,150
CluE	$\sim$ 5.5h	221	1,150

Appendix

Table 8: Efficiency comparison of self-evolving frameworks.

Dataset	No Memory	Simple	GEPA	ACE	MemEvolve	CluE
Personalization
MemBench-High	20.50	87.50	83.00	77.00	87.00	87.00
MemBench-Low	55.00 ±1.77	60.83 ±1.18	59.58 ±1.56	58.33 ±3.12	64.58 ±2.12	63.75 ±1.02
PersonaMem	19.12	21.69	22.79	26.10	26.10	24.63
PrefEval	42.00	65.00	61.00	53.50	77.00	87.50
Problem-Solving

Appendix

Table 9: Per-dataset accuracy (%) on in-distribution test sets for Qwen3-32B evolving from Simple . Subscripts denote standard deviations over three runs.

Dataset	No Memory	Survey	GEPA*	ACE	MemEvolve	CluE
Personalization
MemBench-High	20.50	87.00	87.00	82.00	85.50	86.00
MemBench-Low	55.00 ±1.77	64.58 ±6.24	64.58 ±6.24	57.08 ±1.18	67.08 ±2.36	80.83 ±3.12
PersonaMem	19.12	24.63	24.63	24.63	22.43	26.84
PrefEval	42.00	72.00	72.00	55.50	76.50	89.00
Problem-Solving

Appendix

Table 10: Per-dataset accuracy (%) on in-distribution test sets for Qwen3-32B evolving from Survey .

Dataset	No Memory	Simple	GEPA	ACE	MemEvolve	CluE
Personalization
MemBench-High	20.50	84.50	84.00	86.50	84.50	87.00
MemBench-Low	55.00 ±1.77	52.50 ±4.45	60.00 ±1.02	57.08 ±1.56	52.50 ±2.70	60.42 ±2.57
PersonaMem	19.12	34.93	37.87	30.51	33.82	37.13
PrefEval	42.00	91.50	92.50	92.50	91.00	94.00
Problem-Solving

Appendix

Table 11: Per-dataset accuracy (%) on in-distribution test sets for Gemini-3-Flash evolving from Simple .

LLM Input Optimization

See all 24 papers

Beyond Static Summarization: Proactive Memory Extraction for LLM Agents
Jan 8, 2026Chengyuan Yang, Zequn Sun, Wei Wei +1LLM Agent Memory Persistent Memory for Language Models
State Key Laboratory for Novel Software Technology, Nanjing University, China · National Institute of Healthcare Data Science, Nanjing University, China
EvolveMem:Self-Evolving Memory Architecture via AutoResearch for LLM Agents
May 13, 2026Jiaqi Liu, Xinyu Ye, Peng Xia +4LLM Agent Memory LLM Memory
UNC-Chapel Hill · UC Berkeley · UCSC
Generalizable Self-Evolving Memory for Automatic Prompt Optimization
Mar 23, 2026Guanbao Liang, Yuanchen Bei, Sheng Zhou +5Automatic Prompt Optimization Language Model Self-Improvement
Zhejiang University · University of Illinois Urbana-Champaign · Alibaba Group

Jan 8, 2026J/K move · Enter open · S save

Beyond Static Summarization: Proactive Memory Extraction for LLM Agents

Chengyuan Yang, Zequn Sun, Wei Wei, Wei Hu

State Key Laboratory for Novel Software Technology, Nanjing University, China · National Institute of Healthcare Data Science, Nanjing University, China

LLM Agent Memory Persistent Memory for Language Models LLM Memory Long-Term Conversational Memory Agent Memory Management Memory Management for LLM Agents

Open paper AskPDF arXiv

Memory management is vital for LLM agents in long-term and personalized interactions. Most previous work studies how to retrieve and use memory, but pays less attention to how memory is extracted. We find two main limitations in existing methods. First, extraction is "ahead-of-time": the agent saves information before it knows future tasks. A single summary prompt often mixes details, events, and relations, so useful information is lost. Second, extraction is usually one-off. Without verification, errors and hallucinations may stay in memory for a long time. To address these limitations, we propose ProMem, a proactive memory extraction framework. It separates details, events, and relations, and uses different extraction strategies for each type. It also checks completeness to recover missed events and verifies facts at the atomic level to reduce hallucinations. Experiments show that ProMem improves memory completeness and QA accuracy, while keeping a good balance between quality and token cost.

Abstract

LLM Input Optimization

Explore similar work

Beyond Static Summarization: Proactive Memory Extraction for LLM Agents

EvolveMem:Self-Evolving Memory Architecture via AutoResearch for LLM Agents

Generalizable Self-Evolving Memory for Automatic Prompt Optimization