Mem$^2$Evolve: Towards Self-Evolving Agents via Co-Evolutionary Capability Expansion and Experience Distillation

Zihao Cheng; Zeming Liu; Yingyu Shan; Xinyi Wang; Xiangrong Zhu; Yunpu Ma; Hongru Wang; Yuhang Guo

cs.CLApr 13, 2026

Mem $^2$ Evolve: Towards Self-Evolving Agents via Co-Evolutionary Capability Expansion and Experience Distillation

Authors: Zihao Cheng, Zeming Liu, Yingyu Shan, Xinyi Wang, Xiangrong Zhu, Yunpu Ma, Hongru Wang, Yuhang Guo,

Organizations: Beihang University · Beijing Institute of Technology · Independent Researcher · Munich Center for Machine Learning ·

Abstract

While large language model--powered agents can self-evolve by accumulating experience or by dynamically creating new assets (i.e., tools or expert agents), existing frameworks typically treat these two evolutionary processes in isolation. This separation overlooks their intrinsic interdependence: the former is inherently bounded by a manually predefined static toolset, while the latter generates new assets from scratch without experiential guidance, leading to limited capability growth and unstable evolution. To address this limitation, we introduce a novel paradigm of co-evolutionary Capability Expansion and Experience Distillation. Guided by this paradigm, we propose the \textbf{Mem $^{\textbf{2}}$ Evolve}, which integrates two core components: Experience Memory and Asset Memory. Specifically, Mem $^{2}$ Evolve leverages accumulated experience to guide the dynamic creation of assets, thereby expanding the agent's capability space while simultaneously acquiring new experience to achieve co-evolution. Extensive experiments across 6 task categories and 8 benchmarks demonstrate that Mem $^{2}$ Evolve achieves improvement of 18.53% over standard LLMs, 11.80% over agents evolving solely through experience, and 6.46% over those evolving solely through asset creation, establishing it as a substantially more effective and stable self-evolving agent framework. Code is available at: https://buaa-irip-llm.github.io/Mem2Evolve.

Figures & tables

Figure 1: Paradigms of Self-Evolving Agents : (a) Experience-centric evolution, (b) Capability-centric evolution, and (c) Our co-evolutionary framework that jointly expands capabilities and distills experience.

Framework	Experience Distillation			Capability Expansion				Exp.-Guided Creation
Framework	Optimization	Persistence	Source	Tool Crea.	Agent Crea.	Tool/Agent	Crea. Grounding	Exp.-Guided Creation
DSPy Khattab et al. (2023)	✓	✗		✗	✗	Static	–	✗
DyLAN Liu et al. (2023)	✓	✗		✗	✗	Static	–	✗
ReasoningBank Ouyang et al. (2025)	✗	✓		✗	✗	Static	–	✗
AFlow Zhang et al. (2025a)	✓	✗		✗	✗	Static	–	✗
AgentSquare Shang et al. (2025)	✓	✗		✗	✗	Static	–	✗

Table 1: Comparison of self-evolving agent frameworks. Optimization indicates whether experience is used to optimize the agent (e.g., prompts). Persistence denotes whether experiences are persistently stored for future reuse. Source : agent task execution trajectory, tool creation process. Tool Crea. and Agent Crea. indicate whether the framework supports creation of tools and expert agents, respectively. Tool/Agent denotes whether the toolset and expert agents are static or dynamic. Crea. Grounding indicates the knowledge sources used for asset creation, parametric knowledge, web search information, experience. Exp.-Guided Creation indicates whether new assets are created under the guidance of past experience. Details in the Appendix A.1 and A.2 .

Figure 2: Overview of Mem ${}^{\textbf{2}}$ Evolve , a self-evolving agent framework built on a Dual-Memory mechanism. The evolution proceeds in two phases. During Forward Inference , the agent recruits tools and expert agents from Asset Memory to execute the current task. When the task exceeds its current capability boundary, Experience Memory is leveraged to guide the stable creation of new assets on demand. During Backward Evolution , newly validated assets are preserved in Asset Memory to achieve persistent capability expansion, while strategic insights distilled from execution trajectories are accumulated into Experience Memory. This forward–backward loop enables the co-evolution of capabilities and experience, forming a stable self-evolving cycle.

	GAIA				Embodied	Multi-Hop QA		Math		Planning	Web Interaction
Method	L1	L2	L3	Total	ALFWorld	HotpotQA	2Wiki	AIME24	AIME25	TravelPlanner	WebShop	Avg.
Naive-Large Language Model
GPT-5-Chat (Direct)	16.98	12.79	7.69	12.49	83.58	50.40	81.80	60.00	46.67	38.68	22.31	49.49
GPT-5-Chat (CoT)	24.53	17.44	11.54	17.84	83.58	47.40	74.40	66.67	56.67	39.51	27.49	51.71
GPT-5-Chat (ReAct)	26.42	17.44	11.54	18.47	86.87	41.40	48.40	66.67	60.00	39.13	25.10	48.27
OpenAI-DeepResearch †	74.29	69.06	47.60	67.36	—	—	—	—	—	—	—	—

Table 2: Main results across 6 tasks and 8 benchmarks , reported as Pass@1 for each benchmark. The best results are highlighted in bold , and the second-best results are underlined . † Results are from the original paper.

Framework	Avg. Pass@1	$\Delta_{\textbf{rel}}^{\textbf{\%}}$
Mem ${}^{\textbf{2}}$ Evolve	70.24	–
w/o Asset Creation
w/o Tool Creation	59.96	$\downarrow 10.28$
w/o Expert Agent Creation	68.52	$\downarrow 1.72$
w/o Experience Distillation
w/o Tool Memory	67.11	$\downarrow 3.13$

Table 3: Ablation study of Mem

{}^{\textbf{2}}

Evolve. Full results are provided in Appendix 6 .

Benchmark	w/o Exp.-Guide	w/ Exp.-Guide	$\Delta_{\text{{rel}}}^{\textbf{\%}}$
First-Pass Validity ( $\uparrow$ )
GAIA	32.7%	51.0% (+18.3%)	$\uparrow$ 56.0
AIME24	64.9%	83.8% (+18.9%)	$\uparrow$ 29.1
AIME25	61.8%	82.4% (+20.6%)	$\uparrow$ 33.3
Avg.	53.1%	72.4% (+19.3%)	$\uparrow$ 36.3
Avg. Improve Iter. ( $\downarrow$ )

Table 4: Analysis of Experience-Guided Asset Creation. We report first-pass validity and average improvement iterations on benchmarks requiring extensive tool generation. Experience guidance is associated with higher first-pass validity, with a relative improvement of up to 56.0% on GAIA, and fewer fix iterations, with reductions of nearly 68% on AIME benchmarks.

Figure 3: Single-task self-evolving performance. The results show that initializing the agent with prior memory consistently improves performance compared to the setting without initial memory, indicating that Mem 2 Evolve can effectively leverage accumulated experience to enhance the task execution performance.

Figure 4: Cross-task self-evolving performance. When initialized with heterogeneous memory from GAIA, Mem 2 Evolve consistently outperforms the setting without initial memory and achieves performance comparable to single-task initialization.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Domain	Test Size	Metric
GAIA ( Mialon et al., 2024 )	General Assistant	166	Pass@1
ALFWorld ( Shridhar et al., 2020 )	Embodied Task	134	Success Rate
HotpotQA ( Yang et al., 2018 )	Multi-hop QA	500 †	Exact Match (EM)
2WikiMultihopQA ( Ho et al., 2020 )	Multi-hop QA	500 †	Exact Match (EM)
AIME 24	Math Reasoning	30	Pass@1
AIME 25	Math Reasoning	30	Pass@1

Appendix

Table 5: Overview of the benchmarks, domains, test set sizes, and evaluation metrics used in experiments. † indicates that the test set was randomly sampled. ‡ Average Score is the mean of Delivery Rate, Micro/Macro Commonsense Constraint Pass Rate, and Micro/Macro Hard Constraint Pass Rate.

Method	GAIA	Embodied	Multi-Hop QA		Math		Planning	Web	Avg.
Method	Total	ALFWorld	HotpotQA	2Wiki	AIME24	AIME25	TravelPlanner	WebShop	Avg.
Mem ${}^{\textbf{2}}$ Evolve (Ours)	76.31	94.31	60.80	82.00	76.70	73.33	59.25	39.20	70.24
w/o Asset Creation
w/o Tool Creation	21.69 ( $\downarrow$ 54.62)	94.10 ( $\downarrow$ 0.21)	59.50 ( $\downarrow$ 1.30)	81.50 ( $\downarrow$ 0.50)	66.67 ( $\downarrow$ 10.03)	60.00 ( $\downarrow$ 13.33)	57.15 ( $\downarrow$ 2.10)	39.05 ( $\downarrow$ 0.15)	59.96 ( $\downarrow$ 10.28)
w/o Expert Agent Creation	75.30 ( $\downarrow$ 1.01)	93.65 ( $\downarrow$ 0.66)	59.00 ( $\downarrow$ 1.80)	81.00 ( $\downarrow$ 1.00)	73.33 ( $\downarrow$ 3.37)	70.00 ( $\downarrow$ 3.33)	57.08 ( $\downarrow$ 2.17)	38.80 ( $\downarrow$ 0.40)	68.52 ( $\downarrow$ 1.72)
w/o Experience Distillation

Appendix

Table 6: Ablation study of Mem

{}^{\textbf{2}}

Evolve . Pass@1 scores are reported. Performance drops (

\downarrow

) relative to the full model are shown in parentheses.

Figure 7: Case Study 2 on YouTube Video Subtitle Extraction. When initialized with only a web search tool, (a) experience-centric frameworks fail to handle tasks situated beyond their capability boundary, such as retrieving internal video content, leading to incorrect answers based on general common sense. In contrast, (b) Mem 2 Evolve leverages the guidance of accumulated experience to dynamically generate high-quality tools (e.g., a custom subtitle transcriber), effectively breaking through capability boundaries to access the necessary context and derive the correct answer.

Figure 9: Case Study 4: Experience-Guided Tool Generation for Attribute-Preserving Excel Parsing. This case illustrates how Experience Memory guides Mem 2 Evolve to generate task-appropriate tools that preserve critical non-textual attributes. When required to extract color-coded cells from an Excel file, Mem 2 Evolve leverages past experience to synthesize a tool capable of accurately retrieving both cell values and their original color information in a standardized output format (full implementation in Code 4 ). In contrast, without experiential guidance, the generated tool relies solely on pandas, which fails to retain color attributes (full implementation in Code 5 ), leading to unsuccessful task execution.

LLM Agent Self-Improvement

Large language model agent self-improvement studies how agents use experience and feedback to improve their behavior, tools, and workflows.

See all 273 papers

Learning What to Remember and What to Internalize in LLM Self-Evolution via Adaptive Memory-Parameter Coordination
Aug 2, 2026Tianyun Ji, Zhenya Huang, Jiayu Liu +3Continual Learning for LLM Agents Agent Harness Optimization
University of Science and Technology of China Hefei, Anhui, China · City University of Hong Kong Hong Kong, China · Hefei Normal University Hefei, Anhui, China +1
Evolving-RL: End-to-End Optimization of Experience-Driven Self-Evolving Capability within Agents
May 11, 2026Zhiyuan Fan, Wenwei Jin, Feng Zhang +4Reinforcement Learning Language Model Self-Improvement
1Xiaohongshu Inc. · School of Computer Science, Peking University
EXG: Self-Evolving Agents with Experience Graphs
May 18, 2026Yuxin Jin, Siyuan Zhang, Hanchen Wang +3Graph-Based Agent Memory Memory-Augmented Agents
University of Technology Sydney Sydney, Australia · The University of New South Wales Sydney, Australia

Aug 2, 2026J/K move · Enter open · S save

Learning What to Remember and What to Internalize in LLM Self-Evolution via Adaptive Memory-Parameter Coordination

Tianyun Ji, Zhenya Huang, Jiayu Liu, Zirui Liu+2

University of Science and Technology of China Hefei, Anhui, China · City University of Hong Kong Hong Kong, China · Hefei Normal University Hefei, Anhui, China+1

Continual Learning for LLM Agents Agent Harness Optimization LLM Agent Self-Improvement Self-Evolving Agents

Open paper AskPDF arXiv

Large language model agents increasingly operate in dynamic environments where tool interfaces, APIs, and user requirements change after deployment. Existing self-evolution methods mainly follow two paradigms: harness-based approaches, which externalize feedback into editable memories or skills for rapid adaptation, and parameter-based approaches, which internalize experience into model parameters for deeper capability improvement. However, using either mechanism alone creates a trade-off between flexibility and performance. This paper asks how an agent can coordinate both channels to achieve robust self-evolution. We present COVE, a unified agent self-evolution framework that combines harness-based and parameter-based learning through task-aware routing, stage-aware scheduling, and knowledge optimization. Through this design, COVE treats self-evolution not as indiscriminate accumulation of experience, but as a coordinated process that matches tasks and knowledge types to appropriate learning mechanisms. Experiments across multiple task categories show that COVE outperforms single-channel evolution strategies, demonstrating more robust and efficient improvement under changing environments.

Abstract

LLM Agent Self-Improvement

Explore similar work

Learning What to Remember and What to Internalize in LLM Self-Evolution via Adaptive Memory-Parameter Coordination

Evolving-RL: End-to-End Optimization of Experience-Driven Self-Evolving Capability within Agents

EXG: Self-Evolving Agents with Experience Graphs