Cooperative language-model agents can improve without weight updates by revising reusable skills, but only if a shared team reward is traced to the responsible agent and skill. Existing methods credit the skills present in each trajectory, yet presence is not responsibility: a harmful skill active in successful runs still earns positive credit. We present GRSE, Group-Relative Skill Evolution, which assigns behavioral credit by comparison: inspired by GRPO, it executes each question several times under the same skill libraries and contrasts the executions to credit agent--skill pairs, with structured diagnosis of failures. On HotpotQA and LoCoMo with two open-weight actors, GRSE outperforms MASkills, the closest prior method, on HotpotQA and every LoCoMo category except open-domain, with the largest gains on multi-hop questions (+6.7 and +3.2 F1). Its gains rest on better-grounded retrieval rather than answers given without evidence, and its learned skills transfer between the multi-hop tasks. GRSE is robust to corrupted libraries: starting from planted harmful skills, it credits every planted skill with the correct sign and recovers the score of a clean library (+6.3 points), whereas MASkills misattributes two and falls below its starting point.
Figures & tables
Figure 1: Each iteration (1) samples a group of team executions per question (§ 3.1 ), (2) credits agent–skill pairs by contrasting these executions and diagnoses failures in a structured form (§ 3.2 ), and (3) aggregates the credit across questions into skill edits, each kept only if it does not lower the validation score (§ 3.3 – 3.4 ). The example is from one training run on LoCoMo single-hop questions, in which Agent 1 retrieves evidence and Agent 2 answers. In iteration 1, a failure that surfaces in Agent 2’s answer is traced to Agent 1’s retrieval, and a new retrieval skill is induced. In iteration 3, this skill is credited as harmful in the shown group but helpful across the iteration’s 16 groups, so it is refined rather than pruned. In Agent 1’s search queries, | denotes OR.
Actor Model
Qwen3.8
Gemma4
Single model, no agents
IO
81.6
81.3
CoT
80.7
82.5
CoT-SC
82.6
83.3
Self-Refine
84.1
81.1
Two agents, 5 iterations
Table 1: HotpotQA test F1 ( × 100) with a greedy actor on 500 test questions. Columns give the actor model (Qwen3.8-27B, Gemma-4-31B-it). Qwen3.8-27B is critic and optimizer throughout. Empty library is this system before any skill is learned (iteration 0). Mean over three runs, with SD as a subscript. CoT-SC uses 5 samples, Self-Refine at most 3 feedback rounds. Best per column in bold . Best two-agent setting is underlined .
Task ( N )
Multi-hop (156)
Temporal (179)
Open-domain (52)
Single-hop (453)
Method
F1
BLEU
F1
BLEU
F1
BLEU
F1
BLEU
Actor: Qwen3.8-27B
Full-context
49.9
40.0
48.3
25.9
36.4
31.6
70.8
57.6
Empty library (iter. 0)
49.7
42.8
70.6
48.5
24.7
16.4
59.7
51.0
MASkills
46.7 ± 1.4
39.4 ± 0.8
72.8 ± 1.3
51.8 ± 2.1
31.8 ± 4.4
22.8 ± 5.1
59.0 ± 0.6
49.5 ± 1.2
GRSE (ours)
53.4 ± 0.8
44.7 ± 0.7
73.4 ± 2.1
52.9 ± 2.2
24.7 ± 1.1
15.9 ± 1.3
59.1 ± 4.5
50.0 ± 4.1
Table 2: LoCoMo test results per question category: token F1 and smoothed BLEU ( × 100) with a greedy actor. Full-context is a single model that receives the entire conversation, following the LoCoMo protocol (no agents or skills). The remaining rows use the two-agent system trained for 5 iterations. Empty library is this system before any skill is learned (iteration 0). Mean over three runs, with SD as a subscript. Qwen3.8-27B serves as critic and optimizer for both skill methods. Best per column in bold ; best two-agent result underlined .
Contribution to Δ
% of questions
Method
Δ
D → C
W → C
C → W
Decl.
No ev.
#D → C
HotpotQA, Qwen3.8
MASkills
+0.7
0.0
+2.3
−1.8
0→0
–
–
GRSE
+2.3
0.0
+2.6
−0.6
0→0
–
–
HotpotQA, Gemma4
MASkills
+0.7
0.0
+1.2
−0.8
1→1
–
–
Table 3: Answer transitions from the empty to the final library. Each test question is declined (empty or “not mentioned”), correct (F1 ≥0.5 ), or wrong under each library. Δ : change in mean token F1 ( × 100). D → C, W → C, C → W: contribution to Δ of questions moving from declined to correct, wrong to correct, and correct to wrong; the remaining transitions make up the rest. Decl.: share of declined answers, before → after. No ev.: share of questions whose retrieved evidence cites no dialogue turn (e.g., [D3:5]), before → after. #D → C: number of declined-to-correct questions, with those answered without cited evidence in (). No ev. and #D → C are reported for LoCoMo only, as HotpotQA has no retrieval step. All rows use the first run of each method; on LoCoMo, each category has its own library, and empty-library answers come from iteration 0 of the same GRSE run.
Library
F1
BLEU
Δ F1
Abst. (%)
Perturbed (start)
60.6
52.1
–
9.7
Helpful skills only
66.7
57.4
+6.1
4.2
MASkills
58.4
49.3
−2.2
7.9
GRSE
66.9
57.4
+6.3
4.6
Table 4: Repairing a perturbed skill library on LoCoMo single-hop questions (Qwen3.8 actor, 453 test questions, greedy decoding, one run per method). Token F1 and smoothed BLEU ( × 100) are measured with the final library. Δ F1 is the change in F1 from the perturbed starting library. Abst. is the percentage of refusal answers such as “not mentioned”. Bold marks the best learned library. GRSE beats MASkills by +8.5 F1.
GRSE
MASkills
Planted skill
Kind
u1
Fate
u1
Fate
Agent 1 (retriever)
context expansion
help
+0.18
merged 2
+0.03
merged 2
latest session
harm
−0.12
merged 2
+0.07
merged 2
equation solving
irr.
0.00
kept
0.00
merged 2
Agent 2 (reasoner)
Table 5: Credit assigned to the planted skills of the robustness study (Figure 4 ). Kind: helpful (help), harmful (harm), or irrelevant (irr.). u1 : aggregated utility after the first update; it has the correct sign if positive for helpful and negative for harmful skills ( red : wrong sign). Fate: the skill’s status by iteration 5, with the iteration of the change as subscript. Same runs as Table 4 .
Target
LoCoMo MH
HotpotQA
Empty
50.2
80.7
MASkills
In-domain
46.7 ± 1.4
80.6 ± 0.6
Transfer
50.2 ± 1.5
82.7 ± 0.7
GRSE
In-domain
53.4 ± 0.8
83.7 ± 0.5
Table 6: Transfer between the two multi-hop tasks (Qwen3.8 actor, test F1 × 100, mean ± SD over three runs). Transfer : final libraries from the other task (HotpotQA libraries on LoCoMo multi-hop questions and vice versa); each agent keeps the target task’s role. In-domain : libraries learned on the target (Tables 1 and 2 ).
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Datasets
HotpotQA : distractor setting, development set (all questions labeled hard ). Split: 100 training, 50 validation, and 500 test questions (indices 150–649). Source: hotpot_dev_distractor_v1.json from https://hotpotqa.github.io . License: CC BY-SA 4.0.
LoCoMo : 10 conversations, question categories 1–4. Split: 2 training, 2 validation, and 6 test conversations (seed 42, fixed for all runs); open-domain uses 2/2/5, since conv-30 has no open-domain question. Test questions: 156 multi-hop, 179 temporal, 52 open-domain, and 453 single-hop; 842 in Mix. Source: locomo10.json from https://github.com/snap-research/locomo . License: CC BY-NC 4.0.
Table 7: Datasets, data splits, and model checkpoints.
LoCoMo
HotpotQA
Per Cat.
Mix
Iterations
5
5
5
Batch size ∣Bt∣
100
24
32
Rollouts per question G
4
4
4
Validation set size ∣V∣
50
21–159
80
Validation executions H
1
1
1
Appendix
Table 8: Training settings, shared by GRSE and MASkills except that MASkills uses one rollout per question. Per-category validation uses every question of that category in the two validation conversations (multi-hop 68, temporal 60, open-domain 21, single-hop 159).
Share of skills (%)
Method
n
Fmt
Retr
Verif
Reas
Abst
Words
HotpotQA, Qwen3.8
MASkills
6
17
0
33
50
0
504
GRSE w/o struct.
8
50
0
25
25
0
606
GRSE w/ struct.
7
57
0
29
14
0
502
HotpotQA, Gemma4
Appendix
Table 9: Final skill libraries by primary skill type. Each skill is assigned one primary type by Qwen3.8-27B: answer formatting (Fmt), retrieval (Retr), verification (Verif), reasoning (Reas), or abstention (Abst). n : number of skills. Words: mean length of description and body. On LoCoMo, the four per-category libraries are pooled. w/ and w/o struct.: with and without structured failure analysis. First runs.
Figure 2: Learned Skill Case A: a declined question becomes correct with cited evidence (Gemma4 actor, LoCoMo single-hop, GRSE w/o structure). The answer turn does not contain the keyword of the question; the learned skill fetches the turn that follows a matched question. Evidence abridged.
Figure 3: Learned Skill Case B: a wrong answer becomes correct (Qwen3.8 actor, HotpotQA, GRSE w/o structure). The final Agent 1 follows the steps of its learned skills, whose headings it reproduces.
Figure 4: The perturbed seed library of the robustness experiment (§ 5.3 ). Each agent starts with three skills: one helpful skill and two harmful or irrelevant ones. Skill bodies are reproduced verbatim; labels are for the reader only.
Figure 5: Robustness Case 1: the harmful skill focused-latest-session-retrieval . The same single-hop test question is answered with the perturbed library and with the final libraries of MASkills and GRSE (greedy decoding; evidence abridged to the cited turns). MASkills keeps the single-keyword behavior of the harmful skill. GRSE keeps the question–answer expansion of the helpful one.
Figure 6: Robustness Case 2: the harmful skill verbatim-evidence-guard during training. Utility is the library’s running estimate of the skill’s value, aggregated from critic credits.
Multi-hop
Temporal
Open-domain
Single-hop
All
Avg.
Setting
F1
BLEU
F1
BLEU
F1
BLEU
F1
BLEU
F1
BLEU
F1
BLEU
Empty library (iter. 0)
50.9
43.5
70.8
48.2
26.8
16.7
59.6
50.8
58.3
46.7
52.0
39.8
GRSE
52.3
44.0
73.1
51.5
33.9
22.3
60.2
50.9
59.8
47.9
54.9
42.2
w/o structure
51.5
42.1
70.6
48.5
31.6
20.9
62.0
52.4
59.9
47.7
53.9
41.0
G=1
53.7
44.6
72.0
51.3
30.5
19.4
60.4
51.4
59.7
48.1
54.1
41.7
DAPO filter
53.4
44.0
71.6
51.0
29.8
18.1
62.6
53.4
60.7
48.9
54.4
41.6
Appendix
Table 10: Ablation on LoCoMo with one shared library learned on all four categories together (Qwen3.8 actor, greedy decoding, one run per setting, 842 test questions). Token F1 and smoothed BLEU ( × 100) per question category, over all questions (All), and averaged over the four categories with equal weight (Avg.). Single-hop questions make up 54% of All. Best per column in bold , second best underlined . w/o structure removes the failure-diagnosis schema and its checks. G=1 samples one team trajectory per question instead of four, so no group-relative comparison is possible. DAPO filter keeps four rollouts but, following the dynamic sampling of DAPO, sends only groups whose rewards differ to the critic. w/ label conditions the critic on the question category and groups update evidence by category.
HotpotQA
LoCoMo
Qwen3.8
Gemma4
Qwen3.8
Gemma4
Wall-clock training time (h)
MASkills
2.8 ± 0.8
4.6 ± 1.1
15.9 ± 0.7
19.0 ± 1.3
GRSE
4.3 ± 0.6
5.9 ± 0.6
20.8 ± 1.6
23.4 ± 2.5
Actor tokens, training rollouts (M)
MASkills
2.7 ± 0.3
2.6 ± 0.2
6.1 ± 0.2
4.2 ± 0.3
Appendix
Table 11: Training cost of the main-result runs (mean ± SD over three runs of five iterations; LoCoMo sums the four per-category runs). Actor tokens are the input and output tokens of both agents; critic and optimizer tokens are not logged and are excluded.
Figure 7: Agent roles on HotpotQA.
Figure 8: Agent 1 role on LoCoMo.
Figure 9: Agent 2 role on LoCoMo.
Figure 10: Prompts of the single-model baselines.
Figure 11: Critic prompt for group-relative credit assignment.
A persistent skill library allows language model agents to reuse successful strategies across tasks. Maintaining such a library requires three coupled capabilities. The agent selects a relevant skill, utilizes it during execution, and distills new skills from experience. Existing methods optimize these capabilities in isolation or with separate reward sources, resulting in partial and conflicting evolution. We propose Skill1, a framework that trains a single policy to co-evolve skill selection, utilization, and distillation toward a shared task-outcome objective. The policy generates a query to search the skill library, re-ranks candidates to select one, solves the task conditioned on it, and distills a new skill from the trajectory. All learning derives from a single task-outcome signal. Its low-frequency trend credits selection and its high-frequency variation credits distillation. Experiments on ALFWorld and WebShop show that Skill1 outperforms prior skill-based and reinforcement learning baselines. Training dynamics confirm the co-evolution of the three capabilities, and ablations show that removing any credit signal degrades the evolution.
Yaorui Shi, Yuxin Chen, Zhengxi Lu +6
University of Science and Technology of China · 2Meituan · 3National University of Singapore +2
Agent skills, structured artifacts distilled from interaction trajectories and dynamically reused from skill banks, have become a central mechanism for enabling large language model (LLM)-based self-evolving agents to learn from past experience. Yet existing work has largely focused on the operational aspects of skills, such as acquisition, evolution, and retrieval, while leaving a more fundamental reliability question unresolved: Do accumulated skills in an agent's skill bank actually make positive mechanistic contributions? We investigate this question through systematic skill-bank audits across alternative bank compositions and deployment domains, measuring the resulting changes in agent behavior. These audits reveal two recurring reliability failures: coalition pollution, where bank-level gains conceal negative coalition-level skill contributions, and cross-domain utility reversal, where source-beneficial skills reverse their effects after transfer. These findings motivate two reliability interventions: coalition-aware skill selection during skill accumulation and label-free skill masking after transfer. Coalition-Aware Skill Selection (CASS) selects more reliable candidate skills for the current bank using sampled Shapley marginals. Unsupervised Skill-Masked Coalition Optimizer (u-SMCO) masks transferred skills whose exclusion improves retrieval quality on unlabeled target-domain data. Agentic experiments on LoCoMo, LongMemEval, HotpotQA, and ALFWorld show that CASS and u-SMCO consistently improve task performance and cross-domain generalization over strong skill-based self-evolving agent baselines. Beyond accuracy, coalition-conditioned reliability modeling reduces sensitivity to noisy outcome-reward fluctuations during reinforcement learning and exposes the limits of isolation-based skill evaluation.
Agent skills are crucial external instructions that enable language agents to execute long procedural tasks such as coding or document processing. Existing agent skills are primarily created through human manual crafting or agent execution traces, with limited understanding of how each step contributes to overall skill performance on specific tasks; i.e., there remains an open problem in quantifying the contribution of individual steps within an agent skill. To address this issue, we first model skill-step attribution as a Shapley value-based contribution estimation problem, and then propose SkillShapley, a step-level attribution framework for agent skills. Notably, SkillShapley operates in two phases, motivated by key empirical insights, i.e., discretized benchmark rewards that create sharp performance cliffs, and step interactions that are largely additive rather than synergistic. Specifically, it first identifies informative coalitional regions, and then adaptively samples new coalitions that can yield reusable marginal evidence. Experiments on skills from the widely adopted SkillsBench demonstrate that our SkillShapley can effectively and efficiently identify high- or low-value skill steps, providing several key takeaways for agent skill creation.
Chang Liu, Yuqi Zhang, Yiman Zhong +3
1Beihang University, Beijing, China · 2Shandong University, Jinan, China