Skills let language agents reuse experience across tasks as natural-language documents. Recent work evolves skills from interaction, but existing methods have limitations when skills evolve online. Prior methods update a skill offline from a batch of trajectories collected over many tasks. Some then adopt the new skill without any check, which can degrade the skill over time. Others keep it only if it improves performance on a held-out validation set, which is too slow and costly to run after every update. We introduce EvoGuard, a pairwise skill selector that mitigates skill degradation at low latency and cost. We first run the agent iteratively to build pools of skills and score each skill on a private test set to obtain utility labels. We pair the skills and use rejection sampling to keep only pairs with a clear utility gap. The selector is trained on the logits of two label tokens to pick the better skill. At deployment it decides in a single forward pass, which is significantly faster than evaluation on the private test set. Compared with adopting every new skill, EvoGuard improves success by 11.0% on ALFWorld, 5.9% on ScienceWorld, and 2.7% on LiveCodeBench.
Figures & tables
Figure 1: (a) Success rate of the base-written and expert-written skills on the private test set for each task family. (b) Change in success rate after a single-trajectory rewrite (child minus parent). The shaded region marks degradation.
Figure 2: Overview of EvoGuard . (a) A rewrite loop on training tasks fills a skill pool from two sources, a base generator and an expert generator. (b) The utility of every skill is estimated on a private test set. Rejection sampling keeps same-family pairs with a clear utility gap to train a pairwise selector. (c) At deployment the frozen selector accepts or rejects each rewrite in one forward pass.
Seen
Unseen
Method
Pick
Look
Clean
Heat
Cool
Pick2
All
Pick
Look
Clean
Heat
Cool
Pick2
All
(35)
(13)
(27)
(16)
(25)
(24)
(140)
(24)
(18)
(31)
(23)
(21)
(17)
(134)
Executor: Qwen3-4B
No-Skill
82.9
10.3
23.5
10.4
1.3
48.6
36.0
81.9
29.6
29.0
13.0
11.1
23.5
32.3
Accept-All
90.5
20.5
27.2
29.2
14.7
30.6
41.0
86.1
11.1
78.5
30.4
54.0
19.6
51.2
Random
87.6
12.8
37.0
33.3
16.0
30.6
42.1
75.0
16.7
58.1
30.4
44.4
25.5
44.5
Table 1: Online success rate (%, ↑ ) on ALFWorld by task family for three executors. Each value is the mean of three runs. Bold marks the best value in each column for each executor.
Qwen3-4B
Qwen3-8B
Method
Valid
Accepted
Tokens
Valid
Accepted
Tokens
Accept-All
100.0
100.0
553
100.0
100.0
567
Random
100.0
12.8
303
100.0
13.0
307
LLM-Judge
99.8
68.9
541
99.9
70.9
536
LLM-Judge (Think)
98.4
68.4
565
99.1
68.8
548
LLM-Judge (Logit)
100.0
69.0
543
100.0
72.0
528
Table 2: Acceptance statistics of the runs in Table 1 . Valid is the share of rewrites that pass the format check (%). Accepted is the share of valid rewrites that are adopted (%). Tokens is the mean length of the skill in use.
ScienceWorld
LiveCodeBench
Method
Success
Score
Pass Rate
No-Skill
10.2
20.6
26.1
Accept-All
9.4
21.2
25.7
Random
10.0
20.7
25.7
LLM-Judge
10.0
21.6
26.7
LLM-Judge (Think)
10.8
21.3
25.3
Table 3: Online results on ScienceWorld and LiveCodeBench (%). ScienceWorld reports the success rate and the mean score. LiveCodeBench reports the pass rate. Bold marks the best value in each column.
Figure 3: Selector accuracy during training on the training batches (Train), on validation pairs (Validation), and on parent-child validation pairs (Parent-child).
Figure 4: Share of validation pairs for which the selector keeps the same skill after the two skills swap positions. The dashed line marks a random choice.
Qwen3-4B
Qwen3-8B
Variant
Seen
Unseen
Seen
Unseen
Avg.
EvoGuard
55.7
62.4
53.3
54.7
56.5
w/o expert-written pool
47.1
58.2
48.3
54.5
52.0
w/o rejection sampling
49.5
57.0
48.3
54.7
52.4
Table 4: Ablation study of EvoGuard , reported as overall online success (%).
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Number of LiveCodeBench problems per category in the training and the test set.
Qwen3-4B
Qwen3-8B
Method
Success
Score
Success
Score
No-Skill
14.7
53.7
16.1
54.9
Accept-All
15.8
55.6
18.1
55.5
Random
16.0
55.0
16.9
54.3
LLM-Judge
15.6
55.3
18.1
55.3
LLM-Judge (Think)
15.3
55.1
16.9
54.8
Appendix
Table 5: Online results on WebShop with two executors. Success is the success rate (%) and Score is the mean purchase score. Bold marks the best value in each column.
Method
Seen
Unseen
(140)
(134)
SkillRise (our reproduction)
86.0
77.9
+ EvoGuard
86.4
80.1
Appendix
Table 6: EvoGuard attached to our reproduction of SkillRise on ALFWorld, reported as overall success rate (%).
Method
Tokens
Latency (ms)
LLM-Judge
15.7
71
LLM-Judge (Think)
1,144
4,184
LLM-Judge (Logit)
0
26
EvoGuard
0
24
Appendix
Table 7: Cost of one selection during the online runs: generated tokens and latency.
Language-model agents increasingly improve by converting execution experience into reusable external skills. Yet repeated skill updates form a learning process of their own: locally useful edits can accumulate into redundant or task-specific instructions, while new updates can disrupt behavior that previously worked. We study this problem as skill-evolution overfitting and introduce SkillEvoReg, a general regularization framework for skill evolution inspired by anti-overfitting techniques in neural-network training. SkillEvoReg combines training-time skill dropout, which perturbs update generation, and complexity-aware local regularization, which controls unnecessary structural growth, with causal counterexample validation (CCV), which provides targeted behavioral validation of candidate-specific regressions. We instantiate the framework across heterogeneous skill-evolution systems while retaining each system's native skill evolver and task evaluator. Across SkillOpt, SkillEvolBench, and ContinualSkillBench, SkillEvoReg consistently controls skill-state growth while preserving competitive downstream capability, improves several transfer and later-stage evolution outcomes, and identifies update-level regressions that structural metrics alone cannot reveal. These results suggest that explicit regularization is a useful complement to increasingly capable skill updaters.
Agent skills today are static artifact: authored once -- by human curation or one-shot generation from parametric knowledge -- and then consumed unchanged, with no mechanism to improve from real use. We propose SkillEvolver, a lightweight, plug-and-play solution for online skill learning, in which a single meta-skill iteratively authors, deploys, and refines domain-specific skills. The learning target of SkillEvolver is the skill's prose and code, not model weights, so that the resulting artifact drops into any agent without retraining; and the meta-skill itself is just another skill, loaded through the same interface by any protocol-compliant CLI-agent. Unlike trace-distillation, the meta-skill refines only after deploying the learnt skill, such that the learning signal comes from failures another agent encounters while using it -- not from exploratory traces alone. Refinement iterations are governed by a fresh-agent overfit audit that catches possible leakage as well as deployed-skill-specific failures, including the silent-bypass mode in which a skill appears valid in content but is never invoked at runtime. On 83 SkillsBench tasks spanning 15+ domains, SkillEvolver reaches 56.8% accuracy versus 43.6% for curated human skills and 29.9% for the no-skill baseline; on three GPU kernel optimization tasks from KernelBench, it also raises mean speedup from 1.16 to 1.51 on average.
Genrui Zhang, Erle Zhu, Jinfeng Zhou +2
2Beijing Jiaotong University · 1Tsinghua University
A persistent skill library allows language model agents to reuse successful strategies across tasks. Maintaining such a library requires three coupled capabilities. The agent selects a relevant skill, utilizes it during execution, and distills new skills from experience. Existing methods optimize these capabilities in isolation or with separate reward sources, resulting in partial and conflicting evolution. We propose Skill1, a framework that trains a single policy to co-evolve skill selection, utilization, and distillation toward a shared task-outcome objective. The policy generates a query to search the skill library, re-ranks candidates to select one, solves the task conditioned on it, and distills a new skill from the trajectory. All learning derives from a single task-outcome signal. Its low-frequency trend credits selection and its high-frequency variation credits distillation. Experiments on ALFWorld and WebShop show that Skill1 outperforms prior skill-based and reinforcement learning baselines. Training dynamics confirm the co-evolution of the three capabilities, and ablations show that removing any credit signal degrades the evolution.
Yaorui Shi, Yuxin Chen, Zhengxi Lu +6
University of Science and Technology of China · 2Meituan · 3National University of Singapore +2