EvoSkill-S1: Evolving Reusable Skills with Genetic-Inspired Operators
Organizations: LMU Munich · Munich Center for Machine Learning (MCML) · Huawei Heisenberg Research Center · Zhejiang University ·
Abstract
Reusable skills allow language-model agents to accumulate and reuse problem-solving experience. However, most existing methods improve a skill bank only indirectly through rollout-driven accumulation and heuristic reflection, often resulting in shallow and weakly controlled updates. We present EvoSkill-S1, an RL-driven framework that instead treats reusable skills themselves as the direct objects of evolution. Given hard-negative failures, EvoSkill-S1 selects parent skills and transforms them into candidate child skills through five genetic-inspired operators: revision, cross-revision, mutation, dual-reflection, and crossover. Downstream task rewards train the skill evolver to generate more effective child skills, while evaluator-guided write-back selectively retains beneficial updates. Complementary Skill Bank Fusion consolidates operator-specific banks to exploit their distinct recovery patterns, while a Procedural Skill Interface supports transfer to unseen problems. Across all reported math and code benchmarks, EvoSkill-S1 improves the corresponding initialized skill banks under both Qwen and GPT-family solvers, with relative gains of up to 12.39%. Compared with the strongest external baseline without direct skill evolution, the largest relative improvement reaches 20.39%.
Figures & tables
| Math | Code | |||||||
| Model | Method | AIME 24 | AIME 25 | AMC 23 | Minerva Math | Olymp. Bench | APPS- test | LiveCode Bench |
| Qwen3-4B-Instruct | No skill | 53.75 | 42.08 | 87.50 | 16.18 | 40.65 | 36.90 | 41.98 |
| Traj RAG | 57.08 | 40.83 | 91.56 | 29.04 | 54.15 | 28.58 | 31.17 | |
| Reflexion | 57.92 | 42.92 | 93.12 | 32.35 | 66.77 | 41.82 | 42.18 | |
| ExpeL-SI | 59.17 | 41.67 | 92.50 | 30.88 | 66.32 | 41.64 | 39.63 | |
| Tiny Skill Bank | ||||||||
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Skill bank | Method | AIME 24 | AIME 25 | AMC 23 | Minerva Math | OlympiadBench |
|---|---|---|---|---|---|---|
| None | No skill | |||||
| Traj RAG | ||||||
| Reflexion | ||||||
| ExpeL-SI | ||||||
| Tiny | Init | |||||
| Single-op |
| Skill bank | Method | APPS-test | LiveCodeBench |
|---|---|---|---|
| None | No skill | ||
| Traj RAG | |||
| Reflexion | |||
| ExpeL-SI | |||
| Tiny | Init | ||
| Single-op |
| Skill bank | Method | AIME 24 | AIME 25 | AMC 23 | Minerva Math | OlympiadBench |
|---|---|---|---|---|---|---|
| None | No skill | |||||
| Traj RAG | ||||||
| Reflexion | ||||||
| ExpeL-SI | ||||||
| Tiny | Init | |||||
| Fusion |
| Skill bank | Method | APPS-test | LiveCodeBench |
|---|---|---|---|
| None | No skill | ||
| Traj RAG | |||
| Reflexion | |||
| ExpeL-SI | |||
| Tiny | Init | ||
| Fusion |
| Benchmark | Domain | Split / source | Size | Use and metric |
|---|---|---|---|---|
| DAPO-hard | Math | Hard subset mined from the DAPO initialization pool after base-solver rollouts. | 4k | Evolution target pool; answer-level correctness. |
| AIME24 | Math | Official 2024 AIME I and II problems ( Mathematical Association of America, 2024 ) . | 30 | Held-out math evaluation; answer-level correctness. |
| AIME25 | Math | Official 2025 AIME I and II problems ( Mathematical Association of America, 2025 ) . | 30 | Held-out math evaluation; answer-level correctness. |
| AMC23 | Math | 2023 AMC 12 A/B aggregate ( Mathematical Association of America, 2023 ) . | 40 | Held-out math evaluation; answer-level correctness. |
| Minerva Math | Math | Minerva quantitative-reasoning test set ( Lewkowycz et al., 2022 ) . | 272 | Held-out math evaluation; answer-level correctness. |
| OlympiadBench | Math | Text-only open-ended math subset ( He et al., 2024 ) . | 675 | Out-of-distribution math evaluation; answer-level correctness. |
| Math | Code | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Method | DAPO -hard | AIME 24 | AIME 25 | AMC 23 | Minerva Math | Olympiad Bench | APPS- train-hard | APPS- test | LiveCode Bench |
| Revision-500 | 12.77 | 61.25 | 46.25 | 92.19 | 34.56 | 67.95 | 20.61 | 44.98 | 44.15 |
| Cross-revision-500 | 13.50 | 62.92 | 45.42 | 91.56 | 34.93 | 69.29 | 21.12 | 44.39 | 43.12 |
| Mutation-500 | 13.00 | 60.83 | 46.25 | 92.19 | 35.66 | 68.69 | 21.49 | 45.12 | 42.75 |
| Crossover-500 | 13.09 | 61.25 | 45.00 | 92.81 | 34.93 | 69.14 | 21.42 | 44.64 | 43.23 |
| Dual-reflection-500 | 13.27 | 61.25 | 45.83 | 92.81 | 34.56 | 68.10 | 20.66 | 44.83 | 43.96 |
| Setting | Value |
|---|---|
| Learning rate | |
| Rollout batch | 32 |
| Global update batch | 32 |
| Update micro-batch / GPU | 2 |
| Experience micro-batch / GPU | 8 |
| Candidates per target | 4 |
LLM Agent Skill Learning
Large language model agent skill learning studies how agents acquire, represent, reuse, and refine procedures for multi-step tasks.
See all 428 papers