LLM-based multi-agent systems generate collaboration traces that record how agents plan tasks, verify intermediate results, and repair failures. Reusing these procedures requires preserving an action's prerequisites and the outputs needed by subsequent agents. Our empirical studies show that grouping these dependencies into functional memory units improves their retention, while connecting units increases retrieval of the units and links jointly required by a task. The preferred combination of units also changes between instructions and checklists, even when each combination's content is fixed across formats. Updating choices from the outcomes of each combination and format pairing outperforms scoring combinations and formats separately. These findings motivate MACE, a memory-agent co-evolution framework that adapts memory organization and agent memory use through execution feedback. Its MemGoG structure represents functional units as subgraphs of related conditions, actions, and outputs, connecting them through support, conflict, and repair relations. MACE Loop selects task-relevant units and relations within a memory budget and provides each agent with instructions or checklists for its current operation. It records the selected units, presentation formats, agent outputs, and task outcomes to update unit scores and relations for retrieval and inform subsequent presentation choices. Across eight benchmarks, MACE outperforms ten baselines with an average score of 81.11%, compared with 78.97% for the strongest baseline, SAGE.
Figures & tables
Figure 1: Diagnostics of memory reuse. (a) Task success across memory budgets. (b) Dependency retention and complete support across experiences. (c) Composition and use across four task contexts. (d) Probe success as calibration feedback accumulates.
Figure 2: Overview of MACE. MemGoG constructs candidate memory units from execution needs and long-term memory. The Composer selects a working graph, the Coupler generates agent-specific patches, and the Co-Evolver updates memory and its use through execution feedback.
Figure 3: Measured cost-performance visualization for EQ3. Left: average accuracy versus wall-clock latency per query. Right: runtime comparison; each bar label reports seconds per query and total wall-clock minutes. All markers and bars correspond to measured methods or measured baselines.
Figure 4: Measured robustness results for EQ4. The four panels report stress-test performance under serving-backbone replacement, network fluctuation, memory-cell budget reduction, and noisy memory retrieval. Dashed or dotted references denote Direct and SAGE where applicable. Higher is better.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Task family
Required output
Scoring field
Multiple choice
Final option plus optional short rationale
Parsed option label
Short-answer QA
Compact final answer plus optional rationale
Normalized answer string
Mathematical reasoning
Final numerical or option answer plus optional derivation
Parsed final value or option
Fact verification
Entailed / refuted style label as required by the benchmark
Parsed label
Table reasoning
Final cell, value, span, or expression answer
Exact-match normalized answer
Code generation
Executable completion under the official benchmark interface
Official pass@1 result
Appendix
Table 5: Task-family output contracts.
Condition
Serving change
MACE
Direct ref.
SAGE ref.
Primary API
Clean serving endpoint
81.11
71.53
78.97
Low-cost API
Cost-constrained serving endpoint
80.04
71.53
78.97
Local model
On-premise fallback endpoint
79.28
71.53
78.97
Appendix
Table 12: Backbone replacement results for EQ4.
Method
25% budget
50% budget
75% budget
100% budget
Direct
74.63
74.63
74.63
74.63
MAGMA
69.64
72.36
74.10
75.77
SAGE
80.44
83.56
85.30
86.64
MACE
81.76
84.40
85.74
86.63
Appendix
Table 14: Memory-cell budget results averaged over five benchmarks.
Method
0% noise
10% noise
20% noise
30% noise
Direct
68.40
65.03
60.62
55.65
MAGMA
69.67
66.38
62.08
56.45
SAGE
80.10
76.82
72.35
66.95
MACE
78.97
77.00
73.80
70.30
Appendix
Table 15: Noisy memory retrieval results averaged over four task groups.
Task group
0% noise
10% noise
20% noise
30% noise
Knowledge QA
84.10
82.30
79.40
76.30
Math Reasoning
92.30
90.10
86.70
83.40
Code Generation
62.10
59.80
56.20
52.40
Table Reasoning
77.40
75.80
72.90
69.10
Appendix
Table 16: Per-task-group noisy retrieval values for MACE.
Step
Stage
Output
1
Task adaptation
A normalized task specification with public context, question, benchmark tag, and answer schema.
2
Need-aware composition
A compact working MemGoG subgraph and a memory-need decision.
3
Agent-memory coupling
Role-specific memory patches with provenance and confidence notes.
4
Answer generation
A raw completion plus parsed prediction under the benchmark answer contract.
5
Scoring
Task metric, success flag, and error type when available.
6
Feedback co-evolution
Utility updates, reusable traces, and revised memory state for future tasks.
Appendix
Table 17: High-level MACE execution stages.
Object
Stored content
Role in the loop
Task node
Benchmark id, task family, public metadata
Anchors retrieval and feedback to the evaluated instance.
Memory cell
Reusable strategy, warning, fact pattern, or solution trace
Provides prior experience to the composer.
Role patch
Cell subset assigned to an answer role
Controls which memory is exposed to generation.
Feedback trace
Score, parsed answer, error type, useful-cell signal
Supervises utility and memory-state updates.
Utility state
Freshness, confidence, and usefulness estimates
Ranks cells and resolves memory-budget pressure.
Typed edge
Selection, assignment, support, conflict, or update link
Preserves provenance and relation structure.
Appendix
Table 18: Conceptual node and edge types in the MACE memory graph.
Invariant
Purpose
Fixed task file per benchmark
Ensures that local methods answer the same instances in the same evaluation set.
Read-before-write memory order
Prevents current-task feedback from influencing the current prediction.
Shared answer parser
Avoids method-specific parsing advantages.
Scorer-only gold access
Keeps gold answers outside model-facing prompts.
Logged raw completions
Makes parsing and scoring decisions auditable.
Stable perturbation axis in EQ4
Changes one deployment factor at a time for interpretability.
Appendix
Table 19: Execution invariants for controlled MACE evaluation.
Category
Setting
Value / rule
Task pool
Total local queries
4,728 queries across eight benchmarks.
MMLU sampling
Local split rule
Validation first153 in original CSV order.
MMLU-Pro sampling
Local split rule
Seed-42 sample of 500 test examples without replacement.
TabFact sampling
Local split rule
Seed-42 NumPy permutation, first 500 test indices.
Other local datasets
Local split rule
Complete available test split in original order.
Evaluation statistic
Main tables
Single-run percentage score for each reported row.
Appendix
Table 20: Shared evaluation settings used by the controlled local rows.
Designing effective memory mechanisms is crucial for advancing LLM-driven Multi-Agent Systems (MAS), helping agents learn together and perform better over time. While recent work has led to strong cooperation skills, most methods still use flat, unstructured memories, which easily get filled with noise and erase differences between agents. To address this, we introduce the concept of collective-individual memory synergy and propose CoMem, an architecture that unifies both private experience and shared knowledge for multi-agent learning. CoMem features:(i) Private Experience Sedimentation, which lets each agent keep and update its own useful memories over time;(ii) Collective Wisdom Curation, which carefully selects only widely proven ideas to be shared among agents;(iii)Parallel Dual-Stream Retrieval, which allows agents to draw both from their own memory and the group's wisdom, using clustering to ensure diversity.Experiments on ALFWorld and PDDL benchmarks show that CoMem achieves strong overall performance and robustly avoids memory pollution.
Chengxin Yu, Zhaoxin Fan, Faguo Wu +3
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, School of Artificial Intelligence, Beihang University · Beijing academy of blockchain and edge computing · National University of Defense Technology +1
Large language model (LLM)-based multi-agent systems increasingly rely on memory to transform execution trajectories into reusable procedural knowledge. Yet repeated retrieval also makes memory errors persistent: memory pollution arises when outdated, weakly supported, or spuriously successful procedures become recurring components of future reasoning. Multi-agent execution introduces an additional structural risk. Scope collapse occurs when procedural knowledge escapes the coordination scope in which it was shown effective and is repeatedly reused at incompatible decision levels, allowing local errors to influence cascades of downstream decisions. Meanwhile, task-level failures provide ambiguous supervision because they rarely reveal which recalled knowledge was responsible. We introduce AECG, a framework for asymmetric experience consolidation and governance for multi-agent systems. AECG turns memory from static experience storage into a dynamic reliability-governance loop, preserving coordination scope and using multi-scale, confidence-aware reliability to detect degradation. It then combines degradation with downstream impact to prioritize high-risk knowledge under a bounded review budget, applies targeted interventions, and reactivates revised skills only after paired replay. Across three multi-agent frameworks and four benchmarks, AECG achieves the best score in 11 of 12 framework--benchmark settings and improves over the strongest competing memory method by as much as 10.23 percentage points; removing scope preservation reduces accuracy by up to 16.89 points. AECG thereby reframes multi-agent memory from passive accumulation into auditable reliability governance. Code is available at https://github.com/fenhg297/AECG
Ao Tian, Jialong Liu, Daqi Zheng +7
Beihang University · Wuhan University · ModelBest +1
LLM-based agents increasingly operate in persistent environments where they must store, update, and reason over information across many sessions. While prior benchmarks evaluate only single-entity updates, MEME defines six tasks spanning the full space defined by the multi-entity and evolving axes, including three not scored by prior work: Cascade and Absence (dependency reasoning) and Deletion (post-removal state). Evaluating six memory systems spanning three memory paradigms on 100 controlled episodes, we find that all systems collapse on dependency reasoning under the default configuration (Cascade: 3%, Absence: 1% in average accuracy) despite adequate static retrieval performance. Prompt optimization, deeper retrieval, reduced filler noise, and most stronger LLMs fail to close this gap. Only a file-based agent paired with Claude Opus 4.7 as its internal LLM partially closes the gap, but at ~70x the baseline cost, indicating closure currently depends on configurations that are not practical at scale. Code and data are available on the project page: https://seokwonjung-jay.github.io/meme-eval/.
Seokwon Jung, Alexander Rubinstein, Arnas Uselis +2
1KAIST AI · 2Tübingen AI Center, University of Tübingen · 3NAVER AI Lab