With the rise of Group Agent such as Claude Tag, agents increasingly serve as digital workers for long-term team collaboration. These agents operate over shared contexts that continuously accumulate through multi-user interactions. However, existing agent memory systems are mainly designed for individual users and struggle to capture group-level knowledge in shared contexts, including group relationships and collaboratively formed knowledge. Manual memory strategies further limit adaptation to diverse and evolving shared contexts. To this end, we propose EvoGroup (EvoG), a self-evolving harness for group agents that starts from a minimal six-tool cold start and adapts through interaction with shared context. EvoG uses confidence and task outcomes to select informative trajectories, while also eliciting self-reflections on uncertainty in low-confidence cases. The analysis agent then examines selected trajectories through adaptive progressive disclosure, followed by bucketed analysis to synthesize high-level findings that guide the evolve agent in revising the Harness. Empirically, pass@1 on held-out split of EverMemBench improves from 71.07% to 89.58%, surpassing both self-evolving memory and human-designed harnesses, including Claude Code (85.83%). To evaluate generalization, we freeze the evolved harness and evaluate it on the unseen GroupMemBench, where, with GPT-5.5, it demonstrates effective generalization and outperforms both memory baselines and human-designed harnesses.
Figures & tables
Figure 1: Performance of EvoGroup with DeepSeek-V4-Flash on EverMemBench . (a) Evolution over 5 iterations on the held-in split, improving pass@1 by 18.2 pp from H0 to H∗ . (b) Comparison on the held-out split, where H∗ outperforms all the baselines including Codex and Claude Code.
Figure 2: Overview of the EvoGroup loop , which operates in four phases: I) Interaction, which Group Agent collaborates with shared contexts; II) Trajectory Selection , which thresholds trajectories by confidence; III) Analysis, which performs per-trajectory and bucketed analysis; and IV) Revision, which updates harness through four interfaces.
Table 1: Performance on EverMemBench. All category scores are aligned to the 1,680-question held-out counts (Single=149, Multi=174, Temp=210, Const=282, Proact=299, Update=188, Style=123, Skill=118, Role=137). Baseline entries are count-aligned projections from the reported full-benchmark category accuracies, not rerun results on the held-out question IDs. Average is recomputed from the aligned integer correct counts. Bold marks the best aligned result within each answer-model block, excluding FullContext.
Figure 3: Component ablation on the held-out EverMemBench split using DeepSeek-V4-Flash. Each variant uses one evolved component, while H∗ uses all five. Δ indicates the gains over H0 .
Figure 4: Process ablation on the held-in EverMemBench split using DeepSeek-V4-Flash. Pass@1 under three ablation settings: w/o Trajectory Selection, w/o APD Analysis, and w/o Bucketed Analysis.
Transition
Fail → Pass
Pass → Fail
Always → Pass
Always → Fail
Retention ↑
Regression ↓
H0→H1
124
34
469
93
93.24%
6.76%
H1→H2
73
32
561
54
94.60%
5.40%
H2→H3
38
53
581
48
91.64%
8.36%
H3→H4
45
49
570
56
92.08%
7.92%
H4→H5
53
44
571
52
92.85%
7.15%
Table 2: Capability-transition analysis of self-evolution on the held-in EverMemBench under (a) DeepSeek-V4-Flash and (b) GPT-5.5. Fail → Pass and Pass → Fail denote questions from incorrect to correct and from correct to incorrect, respectively. Always → Pass and Always → Fail denote questions that remain consistently correct or incorrect across consecutive harnesses. Retention is the fraction of previously correct questions that remain correct, while Regression is the fraction that become incorrect in the next harness.
Iteration
Time (min)
Tokens (M)
APD
Full
APD
Full
Iteration 1
64.6
110.8
54.58
82.09
Iteration 2
26.1
43.0
33.34
66.55
Iteration 3
21.5
37.9
22.08
34.18
Iteration 4
17.7
30.1
28.46
56.12
Iteration 5
24.9
47.5
23.88
38.64
Table 3: Effectiveness of Adaptive Progressive Disclosure (APD) for trajectory analysis. We compare APD Analysis with Full Trace Analysis across self-evolution iterations in terms of analysis time and token consumption. APD consistently reduces both analysis time and token consumption compared with Full Trace Analysis, demonstrating its effectiveness in improving trajectory-analysis efficiency. Time is reported in minutes (min), and token usage in millions of tokens (M).
Figure 5: Agreement between self-reflection and trajectory-based diagnosis . Localization Agreement indicates whether self-reflection correctly locates the problematic region, Cause-Attribution Agreement whether it identifies the underlying cause, and Diagnostic-Evidence Consistency whether its diagnosis is supported by trajectory evidence.
Figure 6: Revision-interface edits across evolution rounds for DeepSeek-V4-Flash and GPT-5.5. The figure illustrates which revision interfaces EvoGroup relies on most during self-evolution.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Table 4: Performance on GroupMemBench. All metrics are computed using the official scripts and reported as Pass@1 (%). Average is the reported aggregate score. Bold marks the best result within each answer-model block, excluding FullContext. “-” indicates that the input exceeds the model’s context limit.
Iter.
Correct.
Adoption
Repair
Adopted
Denied
Unused
Iter. 1
87.50
62.50
0.00
37.50
59.40
Iter. 2
88.90
66.70
0.00
33.30
57.10
Iter. 3
100.00
57.10
14.30
28.60
41.40
Iter. 4
100.00
66.70
0.00
33.30
54.50
Iter. 5
71.40
57.10
14.30
28.60
57.60
Appendix
Table 5: Effectiveness of bucketed analysis in producing useful evolution signals. Correct. measures the consistency between the analyzed findings and the actual task issues; Adopted indicates that the Evolve Agent accepts a finding and uses it to update the harness; Denied indicates that the Evolve Agent considers the finding incorrect and explicitly rejects it; and Unused indicates that the finding is not selected for a harness update. Repair measures the proportion of affected tasks successfully repaired by adopted findings. All the metrics are reported on a 0–100 scale.
Figure 7: Distribution of trajectory types across self-evolution for DeepSeek-V4-Flash and GPT-5.5. CC denotes confident-and-correct trajectories, representing routine successful behavior. CW denotes confident-but-wrong trajectories, revealing potentially undetected reasoning failures or systematic answer bias. UC denotes unconfident-but-correct trajectories, capturing fragile successes that the current harness may not sufficiently support. UW denotes unconfident-and-wrong trajectories, capturing failures that the agent can partially recognize.
Table 6: Efficiency on EverMemBench . Tok. (K) denotes the average number of tokens consumed per question (in thousands), Calls denotes the average number of tool calls per question, QA denotes the average time required to complete each question, and Cold denotes the one-time preparation cost before answering, i.e., the time required for the initial Harness execution. QA and Cold are measured in seconds.
Long-horizon autonomous agents require memory systems to retain historical information, track evolving states, and reuse relevant knowledge beyond finite context windows. Existing agentic memory systems typically follow a memory construction-retrieval (MCR) pipeline, but often adapt mainly the memory bank while keeping the surrounding pipeline fixed after deployment. This fixed-pipeline design struggles to handle heterogeneous task-specific failure modes and can become misaligned with memory banks that evolve in scale and structure over time. To address these limitations, we propose MemPro, a system-level evolution framework that treats the entire MCR pipeline as an evolvable program rather than adapting only the memory bank or prompt text. MemPro maintains a version tree of runnable memory-system implementations, where an Evolving Agent iteratively selects promising versions, diagnoses recurring failures, and creates improved child versions through failure-mode-guided edit-debug refinement. Experiments on LongMemEval, LoCoMo, HotpotQA, and NarrativeQA show that MemPro consistently outperforms strong static and prompt-level evolving baselines within a few iterations, continues to improve with evolution, and achieves a favorable performance-cost trade-off. Code is available at https://github.com/wanghai673/MemPro.
An LLM agent can acquire persistent improvements by evolving its harness---the executable program governing model calls, tool use, verification, and recovery---without updating model weights. When the same harness serves users with different workloads, isolated evolution limits the sharing of useful discoveries, while centralized adaptation can introduce conflicting task-specific changes. We introduce EvolveNet, a framework for collaborative harness evolution that connects local specialization with shared capability accumulation. Clients evolve the same broadcast harness on their local workloads in parallel, and a server composes their adaptations into an updated shared program. Redistribution makes accepted discoveries available to all clients, allowing subsequent local evolution to build on improvements accumulated across the collaboration. EvolveNet supports this process through evidence-guided, scope-typed program aggregation and behavioral validation. Our theoretical analysis provides sufficient conditions for retaining and combining complementary improvements despite program interactions. Experiments across five settings show improved shared-harness performance, with ablation and behavioral analyses supporting the contribution of composition beyond specialist selection alone.
Jun Nie, Yonggang Zhang, Qianshu Cai +3
Hong Kong Baptist University · University of Science and Technology of China · The Hong Kong University of Science and Technology
Recent benchmarks for Large Language Model (LLM) agents mainly evaluate reasoning, planning, and execution. However, memory is also essential for agents, as it enables them to store, update, and retrieve information over time. This ability remains under-evaluated, largely because existing benchmarks do not provide a systematic way to assess memory mechanisms. In this paper, we study agent memory from a self-evolving perspective and introduce EvoMemBench, a unified benchmark organized along two axes: memory scope (in-episode vs. cross-episode) and memory content (knowledge-oriented vs. execution-oriented). We compare 15 representative memory methods with strong long-context baselines under a standardized protocol. Results show that current memory systems are still far from a general solution: long-context baselines remain highly competitive, memory helps most when the current context is insufficient or tasks are difficult, and no single memory form works consistently across all settings. Retrieval-based methods remain strong for knowledge-intensive settings, whereas procedural and long-term memory methods are more effective for execution-oriented tasks when their stored experience matches the task structure. We hope EvoMemBench facilitates future research on more effective memory systems for LLM-based agents. Our dataset and code are available at https://github.com/DSAIL-Memory/EvoMemBench.
Yuyao Wang, Zhongjian Zhang, Mo Chi +7
Hong Kong University of Science and Technology (Guangzhou) · Beijing University of Posts and Telecommunications · Beijing Institute of Technology +1