SEPM: Self-Evolving Procedural Memory Grounded in Task Graphs for Multi-Agent Collaboration

Yi Liu; Junwei Zhao; Li Sun; Jiansong Zhang; Sen Su

cs.MAOct 10, 2026

SEPM: Self-Evolving Procedural Memory Grounded in Task Graphs for Multi-Agent Collaboration

Authors: Yi Liu, Junwei Zhao, Li Sun, Jiansong Zhang, Sen Su

Organizations: Beijing University of Posts and Telecommunications · China Telecom

Abstract

Long-horizon multi-agent collaboration requires memory that preserves both successful actions and the coordination structure underlying them. Existing memory systems support information sharing or procedural reuse, but often leave the evolution of shared procedures disconnected from the task dependencies and execution states of collaborating agents. This coupling also limits the direct transfer of procedural memory designed for individual agents to team settings. We propose Self-Evolving Procedural Memory (SEPM), a framework that uses task dependency graphs as a common representation for online coordination and procedural memory evolution. During execution, a lightweight shared blackboard records structured evidence alongside task dependencies, supporting the detection of goal, plan, and world-state divergence and evidence-based realignment. After execution, verified actions and dependencies are distilled into reusable procedures, while recurring failures inform warnings and constraints. The shared procedure pool evolves through creation, refinement, specialization, and retirement, guided by success and cost utility and constrained by evidence and safety requirements. Together, these mechanisms preserve the collaborative structure and applicability conditions needed to reuse experience across tasks. Evaluations on ALFWorld, WebArena, and OfficeBench under AutoGen and DyLAN show that SEPM achieves the highest task scores among the compared methods in all six benchmark settings. Under AutoGen, it improves task scores over the no-added-memory baseline by 29.85, 25.86, and 13.00 percentage points, respectively. The codebase is available at https://github.com/cookieApril/SEPM.

Figures & tables

Figure 1: Task Score (TS) and Communication Score (CS) across the coding, research, and database environments of MultiAgentBench under AutoGen.

Figure 2: Overview of SEPM . Retrieved procedures instantiate collaborative task graphs. Execution records linked to task nodes support goal, plan, and state alignment. Verified outcomes and prerequisite relations support procedure creation, structural revision, and updates to conditional warnings.

Figure 3: Illustrative cleaning-and-storage case. Agent B introduces an unrelated action, while Agent C requests a cup before cleaning has been verified. Realignment restores the assigned plan and the cleaning-before-handoff prerequisite. The revised procedure retains this dependency and adds guidance on state verification and resumption from verified checkpoints.

Panel of Figure 4 — Figure 4: Task performance and token cost. (a) Mean task scores across ALFWorld, WebArena, and OfficeBench; each segment connects the no-added-memory baseline to a memory strategy within the same host. (b) ALFWorld task scores versus total input and output tokens over the complete evaluation under AutoGen, measured in millions.

MAS	Memory	ALFWorld	WebArena	OfficeBench
AutoGen	No added memory	59.70 $\uparrow$ 0.00	12.44 $\uparrow$ 0.00	41.67 $\uparrow$ 0.00
	Generative	85.82 $\uparrow$ 26.12	15.27 $\uparrow$ 2.83	38.33 $\downarrow$ 3.34
	MetaGPT	78.36 $\uparrow$ 18.66	25.99 $\uparrow$ 13.55	45.33 $\uparrow$ 3.66
	G-Memory	83.58 $\uparrow$ 23.88	30.30 $\uparrow$ 17.86	50.00 $\uparrow$ 8.33
	SEPM	89.55 $\uparrow$ 29.85	38.30 $\uparrow$ 25.86	54.67 $\uparrow$ 13.00
DyLAN	No added memory	53.73 $\uparrow$ 0.00	14.90 $\uparrow$ 0.00	33.33 $\uparrow$ 0.00

Table 1: Task performance across host multi-agent systems. Boldface marks the highest score within each benchmark setting. Arrows indicate percentage-point changes relative to the same host without added memory.

Model	AW	WA	OB
qwen3.5-0.8b	58.21	14.53	45.67
qwen3.5-2b	57.46	14.78	45.67
qwen3.5-9b	67.16	16.01	49.67
qwen3.5-27b	81.34	30.42	55.67
gemma-4-12b-it	72.39	22.54	53.00
gemma-4-31b-it	77.61	28.69	54.33

Table 2: Sensitivity to execution and memory models. The main planning agent and goal-consistency judge are fixed. AW, WA, and OB denote ALFWorld, WebArena, and OfficeBench, respectively.

Panel of Figure 5 — Figure 5: Ablation results under AutoGen. (a) Full SEPM versus the textual procedural memory control. (b) Component configurations. AW, WA, and OB denote ALFWorld, WebArena, and OfficeBench. Memory denotes the complete cross-task procedure extraction, revision, and reuse mechanism built on the shared blackboard.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Query cues	Modified logit	Increment
code , debug , implement , coding	Vector	$1.0$
latest , current , real-time	BM25	$0.8$
routine , repeat , standard	BM25	$0.6$

Appendix

Table 3: Query-dependent logit adjustments for adaptive procedure retrieval.

	Coding		Research		Database
Method	TS	CS	TS	CS	TS	CS
No added memory	54.90	39.81	62.40	55.16	64.93	62.43
Generative	55.74	46.09	61.16	64.64	65.21	77.16
MetaGPT	57.62	47.23	70.11	62.27	70.14	80.29
G-Memory	64.20	51.17	74.29	77.77	69.73	88.20
SEPM	69.09	60.64	78.37	88.91	77.03	91.94

Appendix

Table 4: Task and communication scores on MultiAgentBench. All methods use AutoGen, gpt5.5-mini for execution, and claude-opus-5 for evaluation. TS and CS retain the benchmark’s reporting scale.

Comparison	ALFWorld	WebArena	OfficeBench
Add memory to blackboard	$17.17$	$6.28$	$0.33$
Add memory to blackboard + alignment	$7.46$	$3.82$	$3.67$
Add alignment to blackboard	$12.69$	$6.77$	$6.00$
Add alignment to blackboard + memory	$2.98$	$4.31$	$9.34$
Interaction $I$	$-9.71$	$-2.46$	$3.34$

Appendix

Table 5: Conditional gains from the component experiment, in percentage points. Differences are computed from the displayed task scores.

Agent Memory

Agent memory lets AI systems retain, update, and use facts, experiences, and skills across sessions and long-horizon tasks.

See all 190 papers

CONTRAMEM: Learning Self-Evolving Procedural Memory from Contrasting Multi-Model Trajectories
Aug 23, 2026Zheyuan Deng, Binghang Lu, Hanqi Feng +11Computer-Use Agents LLM Agent Memory
Brown University · Purdue University · Carnegie Mellon University +6
MACE: Memory-Agent Co-Evolution with Adaptive Memory Graphs for Multi-Agent Systems
Sep 18, 2026Kairui Yang, Minghao An, Xunkai Li +4LLM Agent Memory Multi-Agent LLM Systems
StackPlanner: A Centralized Hierarchical Multi-Agent System with Task-Experience Memory Management
Jan 9, 2026Ruizhe Zhang, Xinke Jiang, Zhibang Yang +12Multi-Agent LLM Systems Multi-Agent Collaboration
School of Computer Science, Peking University, Beijing, China · 2National Engineering Research Center for Software Engineering, Peking University, Beijing, China · 3Key Laboratory of High Confidence Software Technologies, Ministry of Education, Beijing, China +5

Aug 23, 2026J/K move · Enter open · S save

CONTRAMEM: Learning Self-Evolving Procedural Memory from Contrasting Multi-Model Trajectories

Zheyuan Deng, Binghang Lu, Hanqi Feng, Shirley Huang+10

Brown University · Purdue University · Carnegie Mellon University+6

Computer-Use Agents LLM Agent Memory Procedural Memory for LLM Agents Self-Evolving Agents

Open paper AskPDF arXiv

Autonomous computer-use agents are increasingly applied to long-horizon tasks requiring coordinated application calls, persistent state tracking, and verifier-sensitive writes, yet they remain prone to procedural failures: misreading application state, tool semantics, or task progress. Procedural memory promises more consistent decisions and less redundant exploration, but constructing high-quality memory without model training remains challenging. We introduce CONTRAMEM, a source-flexible, training-free framework for self-evolving procedural memory that treats same-task outcome variation as supervision: differences in correctness, efficiency, recovery, and failure modes expose outcome-relevant procedural distinctions, distilled into a compact bank of app-level Function Cards and task-level Skill Cards that evolves through localized curation rather than append-only accumulation or whole-bank rewriting. On held-out GAIA2/ARE computer-use tasks, CONTRAMEM more than doubles the success rate across the three source-model targets (26.2% to 55.3%), with consistent per-model gains (GPT-5.5: 27.5 to 61.0; Claude Sonnet 4.6: 28.0 to 52.5; DeepSeek V4 Pro: 23.0 to 52.5). The same bank transfers unchanged to the unseen Qwen3.7 Plus (18.5 to 35.5), indicating transferable procedural knowledge rather than model-specific behavior. The same construction carries over unchanged to AppWorld, beating both no memory and its own single-source self-memory variant for all three mid-tier agents on both public test splits. Under a matched trajectory budget, heterogeneous multi-model trajectories yield stronger memory than self- or same-model multi-rollout memory: the margin comes from contrastive behavioral diversity, not stronger source agents or more sampling.

Abstract

Agent Memory

Explore similar work

CONTRAMEM: Learning Self-Evolving Procedural Memory from Contrasting Multi-Model Trajectories

MACE: Memory-Agent Co-Evolution with Adaptive Memory Graphs for Multi-Agent Systems

StackPlanner: A Centralized Hierarchical Multi-Agent System with Task-Experience Memory Management