EvoSkill: Automated Skill Discovery for Multi-Agent Systems

Salaheddin Alzubi; Noah Provenzano; Jaydon Bingham; Weiyuan Chen; Tu Vu

cs.AIMar 3, 2026

EvoSkill: Automated Skill Discovery for Multi-Agent Systems

Authors: Salaheddin Alzubi, Noah Provenzano, Jaydon Bingham, Weiyuan Chen, Tu Vu

Organizations: Sentient · Virginia Tech

Abstract

Coding agents are increasingly used as general-purpose problem solvers, but their flexibility does not by itself confer the domain expertise needed for specialized tasks. Recent work addresses this through agent skills: reusable workflows, and code, that augment agents with domain-specific capabilities. Most skills today are hand-crafted, and existing evolutionary approaches optimize low-level artifacts (e.g. prompts & code) that are tightly coupled to specific models and tasks. We introduce EvoSkill, a self-evolving framework that automatically discovers and refines agent skills through iterative failure analysis. EvoSkill analyzes execution failures, proposes new skills or edits to existing ones, and materializes them into structured, reusable skill folders. A Pareto frontier of agent programs governs selection, retaining only skills that improve held-out validation performance while the underlying model remains frozen. We evaluate EvoSkill on two benchmarks: OfficeQA, a grounded reasoning benchmark over U.S.\ Treasury data, where it improves exact-match accuracy by 7.3% (60.6% $\to$ 67.9%); and SealQA, a search-augmented QA benchmark with noisy retrieval, where it yields a 12.1% gain (26.6% $\to$ 38.7%). We also investigate the zero-shot transfer capabilties of skills evolved on one task to the other; in particular: skills evolved from SealQA transfers zero-shot to BrowseComp, improving accuracy by 5.3% without modification demonstrating that skill-level optimization produces transferable capabilities beyond the training task.

Figures & tables

Figure 1: Overview of the EvoSkill loop.

Figure 2: EvoSkill performance OfficeQA benchmark across training splits and tolerance levels. The skill-merge configuration, which combines unique skills from independent runs, achieves the highest exact-match accuracy (67.9%), a 7.3 percentage point improvement over the baseline (60.6%).

Train Split/Tol.	0.00%	0.10%	1.00%	5.00%	10.00%
base	60.6	66.3	72.8	77.2	79.7
5%	63.4	67.4	74.3	77.6	80.1
10%	65.8	69.9	76.4	80.5	82.5
15%	64.5	69.9	75.6	79.3	81.3
merge-unique	68.1	70.8	77.1	80.5	82.4

Table 1: Accuracy across tolerance thresholds for different training data splits.

Agent Skill Learning

Agent skill learning develops reusable procedures and behaviors that help AI agents solve new tasks and adapt from experience.

See all 166 papers

Oct 19, 2026

EvoSkill v2: Fully Agentic Skill Evolution from Benchmark Adaptation to a Stateful Evolver

EvoSkill v2 is a three-agent system that edits the instructions (skills) an agent is given, leaving its model and harness intact. The user starts a run by describing a benchmark and supplying a grader, which a benchmark adaptor turns into a benchmark-specific plugin. A solver attempts the tasks, and an evolver reads the solver's training rollouts and rewrites the skills at each epoch. We implement the evolver as one stateful coding agent with a bash tool for the whole run, unlike the two most common designs among the skill optimizers we surveyed: a sequence of single model calls, as in SkillOpt, and an agent that starts afresh each round, as in SkillBoost. We also measure how much the solver's rollouts vary from one attempt to the next, give the evolver several attempts per training task, and accept every epoch's skills instead of selecting among them on noisy scores. With fine-grained ground truth, EvoSkill v2's skills raised the held-out pass rate above the stronger of SkillOpt and SkillBoost by 7.7 percentage points on SpreadsheetBench v2 and by 5.4 on

τ^3

-bench banking. In both benchmarks and both feedback settings, EvoSkill v2's evolver cost less per percentage point of held-out pass rate gained than either baseline's optimizer. EvoSkill v2 also enables a less capable model to improve its own skills, demonstrated by a gain of up to 21.5 percentage points in the

τ^3

-bench banking experiment where Claude Haiku 5.5 powers both solver and evolver.

Dastin Huang, Baran Nama, Salaheddin Alzubi

Sentient Labs

LLM Agent Skill Learning Self-Evolving Agents

Ask PDF

May 11, 2026

SkillEvolver: Skill Learning as a Meta-Skill

Agent skills today are static artifact: authored once -- by human curation or one-shot generation from parametric knowledge -- and then consumed unchanged, with no mechanism to improve from real use. We propose SkillEvolver, a lightweight, plug-and-play solution for online skill learning, in which a single meta-skill iteratively authors, deploys, and refines domain-specific skills. The learning target of SkillEvolver is the skill's prose and code, not model weights, so that the resulting artifact drops into any agent without retraining; and the meta-skill itself is just another skill, loaded through the same interface by any protocol-compliant CLI-agent. Unlike trace-distillation, the meta-skill refines only after deploying the learnt skill, such that the learning signal comes from failures another agent encounters while using it -- not from exploratory traces alone. Refinement iterations are governed by a fresh-agent overfit audit that catches possible leakage as well as deployed-skill-specific failures, including the silent-bypass mode in which a skill appears valid in content but is never invoked at runtime. On

83

SkillsBench tasks spanning

15^{+}

domains, SkillEvolver reaches

56.8\%

accuracy versus

43.6\%

for curated human skills and

29.9\%

for the no-skill baseline; on three GPU kernel optimization tasks from KernelBench, it also raises mean speedup from

1.16

1.51

on average.

Genrui Zhang, Erle Zhu, Jinfeng Zhou +2

2Beijing Jiaotong University · 1Tsinghua University

LLM Agent Self-Improvement LLM Agent Skill Learning

Ask PDF

Aug 6, 2026

Learning Globally Reusable Skills for Coding Agents

Automated skill evolution enables Large Language Model (LLM) agents to continuously improve without expensive retraining. However, existing approaches typically treat skill evolution as a sequence of local updates, overlooking relationships among skills and often producing overfitted skill updates that fail to generalize across tasks. We propose GSE, a globalized skill evolution framework that jointly optimizes skill compatibility and skill generalization. To preserve consistency across the skill bank, GSE maintains a Skill Relation Graph (SRG) that explicitly models and co-evolves inter-skill relationships. To improve generalization, GSE performs cluster-based skill consolidation to abstract reusable capabilities from local updates and employs replay-driven verification to prevent overfitting and behavioral regressions. We evaluate GSE on two representative software engineering tasks: bug-revealing test generation and false-positive bug report filtering. Across two state-of-the-art coding agents, OpenHands and mini-SWE-agent, GSE consistently achieves the best precision, recall, and F1-score. Compared with existing evolution techniques, GSE improves precision and recall by 6.1%~34.1% and 31.8%~180.0% for test generation, and by 15.4%~96.4% and 13.1%~19.8% for false-positive filtering. Deployment on an internal industrial agent further yields a 61.4% improvement in F1-score, demonstrating the effectiveness and generalizability of GSE for evolving effective skills.

Chen Yang, Jiashuo Tian, Ziqi Wang +3

Tianjin University · Tsinghua University

Continual Learning for LLM Agents Software Engineering Agents LLM Agent Skill Learning Self-Evolving Agents

Ask PDF

Abstract

Agent Skill Learning

Explore similar work

EvoSkill v2: Fully Agentic Skill Evolution from Benchmark Adaptation to a Stateful Evolver

SkillEvolver: Skill Learning as a Meta-Skill

Learning Globally Reusable Skills for Coding Agents