CoEvolve: Counterfactual-Guided Co-Evolution of Memory and Collaboration in Multi-Agent Systems

Xufei Shi; Derong Xu; Wei Chen; Yanyu Yao; Zhi Zheng; Tong Xu

cs.AIOct 24, 2026

CoEvolve: Counterfactual-Guided Co-Evolution of Memory and Collaboration in Multi-Agent Systems

Authors: Xufei Shi, Derong Xu, Wei Chen, Yanyu Yao, Zhi Zheng, Tong Xu

Organizations: State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China, Hefei, China

Abstract

Large language model (LLM)-based multi-agent systems have demonstrated remarkable capabilities across diverse tasks. Yet their memory and collaboration architectures often remain static, constraining continued adaptation and improvement over time. Recent work addresses this limitation by evolving memory or collaboration based on interaction experience. However, updates to these interdependent components are not necessarily aligned: an evolved workflow may require experience that memory does not provide, while evolved memory may encode guidance that the workflow cannot use. To address this problem, we propose CoEvolve, a skill-centered framework for aligned memory--collaboration evolution. CoEvolve represents reusable local collaboration patterns as skills that link evolving collaboration decisions to their supporting memories. It further compares original and counterfactual trajectories to identify local evolution signals, which coordinate updates to the skill library, skill memory, and executor memories across rounds. Experiments on the ALFWorld and WebShop benchmarks confirm that CoEvolve achieves the best overall performance among the evaluated baselines, while ablation studies further validate the effectiveness of aligned co-evolution and counterfactual-guided evolution.

Figures & tables

Figure 1: Cross-component mismatch under independent evolution: (a) memory-only and (b) collaboration-only updates leave workflow requirements and memory support misaligned, whereas (c) CoEvolve aligns both via shared skills.

Figure 2: Overview of CoEvolve . Skills couple collaboration with supporting memories; skill-conditioned execution produces trajectories, and counterfactual comparisons yield local signals for aligned updates across rounds.

Dataset	ALFWorld							WebShop
Method	Overall	put	clean	heat	cool	examine	puttwo	Score
DyLAN	$0.6940$	$0.7500$	$0.6129$	$0.6522$	$0.6667$	$0.8333$	$0.7059$	$0.5935$
MemEvolve	$0.7239$	$0.8333$	$0.6452$	$0.7826$	$0.6190$	$0.8333$	$0.6471$	$0.5249$
EvolveR	$0.7015$	$0.7917$	$0.6774$	$0.8261$	$0.7619$	$0.6111$	$0.4706$	$0.5715$
AWM	$0.6716$	$\mathbf{0.8750}$	$0.7097$	$0.5652$	$0.6190$	$0.6667$	$0.5294$	$0.5364$
AutoGen-GMemory	$0.7537$	$0.8333$	$0.7097$	$\mathbf{0.8696}$	$\mathbf{0.8095}$	$0.7778$	$0.4706$	$0.6681$

Table 1: Performance on ALFWorld and WebShop. ALFWorld columns report overall and task-wise success rates; WebShop reports purchased-product score.

(a) Evolved components
	ALFWorld		WebShop
Setting	SR $\uparrow$	Stages $\downarrow$	Score $\uparrow$	Stages $\downarrow$
None	$0.6119$	$21.4328$	$0.5238$	$16.7540$
Skill	$0.6567$	$20.1496$	$0.5718$	$15.2360$
SMem	$0.6418$	$21.0149$	$0.5521$	$15.6730$
EMem	$0.6194$	$20.7090$	$0.5763$	$16.1980$

Table 2: Ablation studies on ALFWorld and WebShop. Blocks compare (a) evolved components and (b) counterfactual sampling and delayed promotion; stages are average executed skill stages.

Panel of Figure 3 — Figure 3: Skill calls and task coverage: (a) ALFWorld; (b) WebShop.

Figure 4: Performance gains over no evolution under different orchestrator capacities.

Figure 5: Average execution length on ALFWorld and WebShop.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Sem. $\lambda_{1}$	Util. $\lambda_{2}$	Explor. $\lambda_{3}$	ALF. SR	WS Score
Default	1.0	1.0	1.0	0.7985	0.7072
Semantic $\times 0.5$	0.5	1.0	1.0	0.7463	0.6732
Semantic $\times 1.5$	1.5	1.0	1.0	0.7687	0.6863
Utility $\times 0.5$	1.0	0.5	1.0	0.7388	0.6629
Utility $\times 1.5$	1.0	1.5	1.0	0.7612	0.6781
Exploration $\times 0.5$	1.0	1.0	0.5	0.7164	0.6748

Appendix

Table 3: Retrieval-weight sensitivity on the evolved CoEvolve system.

Round	Tokens	Time (s)
R1	4.286M	6,090
R2	3.704M	5,238
R3	3.746M	5,295
Avg.	3.912M	5,541

Appendix

Table 4: Counterfactual sampling overhead on ALFWorld across three evolution rounds.

Executor	Persona
Executor 1	A calm and methodical thinker who brings structure to messy situations, surfaces implicit assumptions, and works carefully within defined boundaries.
Executor 2	A steady and results-oriented contributor who turns direction into momentum, makes practical decisions, and keeps work moving without unnecessary overhead.
Executor 3	A rigorous and independent-minded evaluator who pressure-tests ideas, notices fragile assumptions, and improves outcomes by identifying what may fail.

Appendix

Table 5: Personas assigned to the three executors.

Multi-Agent LLM Systems

Multi-agent large language model systems coordinate multiple AI agents to solve tasks through communication, specialization, shared tools, and memory.

See all 1,050 papers

Mem $^2$ Evolve: Towards Self-Evolving Agents via Co-Evolutionary Capability Expansion and Experience Distillation
Apr 13, 2026Zihao Cheng, Zeming Liu, Yingyu Shan +7LLM Agent Self-Improvement Self-Evolving Agents
Beihang University · Beijing Institute of Technology · Independent Researcher +2
EvoTeam: Multi-Agent Collaboration via Evolving Teams
Oct 29, 2026Qinuo Cao, Xin He, Dianxuan Fu +7Multi-Agent Collaboration Multi-Agent Systems
Jilin University · Nanjing University of Science and Technology · Shanghai AI Lab
CoMem: Collective-Individual Memory Synergy for Evolutionary Multi-Agent Systems
Sep 14, 2026Chengxin Yu, Zhaoxin Fan, Faguo Wu +3Multi-Agent LLM Systems Multi-Agent Collaboration
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, School of Artificial Intelligence, Beihang University · Beijing academy of blockchain and edge computing · National University of Defense Technology +1

Apr 13, 2026J/K move · Enter open · S save

Mem $^2$ Evolve: Towards Self-Evolving Agents via Co-Evolutionary Capability Expansion and Experience Distillation

Zihao Cheng, Zeming Liu, Yingyu Shan, Xinyi Wang+6

Beihang University · Beijing Institute of Technology · Independent Researcher+2

LLM Agent Self-Improvement Self-Evolving Agents Tool-Augmented Language Model Agents Memory-Augmented Language Model Agents

Open paper AskPDF arXiv

While large language model--powered agents can self-evolve by accumulating experience or by dynamically creating new assets (i.e., tools or expert agents), existing frameworks typically treat these two evolutionary processes in isolation. This separation overlooks their intrinsic interdependence: the former is inherently bounded by a manually predefined static toolset, while the latter generates new assets from scratch without experiential guidance, leading to limited capability growth and unstable evolution. To address this limitation, we introduce a novel paradigm of co-evolutionary Capability Expansion and Experience Distillation. Guided by this paradigm, we propose the \textbf{Mem

^{\textbf{2}}

Evolve}, which integrates two core components: Experience Memory and Asset Memory. Specifically, Mem

^{2}

Evolve leverages accumulated experience to guide the dynamic creation of assets, thereby expanding the agent's capability space while simultaneously acquiring new experience to achieve co-evolution. Extensive experiments across 6 task categories and 8 benchmarks demonstrate that Mem

^{2}

Evolve achieves improvement of 18.53% over standard LLMs, 11.80% over agents evolving solely through experience, and 6.46% over those evolving solely through asset creation, establishing it as a substantially more effective and stable self-evolving agent framework. Code is available at: https://buaa-irip-llm.github.io/Mem2Evolve.

Abstract

Multi-Agent LLM Systems

Explore similar work

Mem2^22Evolve: Towards Self-Evolving Agents via Co-Evolutionary Capability Expansion and Experience Distillation

EvoTeam: Multi-Agent Collaboration via Evolving Teams

CoMem: Collective-Individual Memory Synergy for Evolutionary Multi-Agent Systems

Mem $^2$ Evolve: Towards Self-Evolving Agents via Co-Evolutionary Capability Expansion and Experience Distillation