EvoSkill-S1: Evolving Reusable Skills with Genetic-Inspired Operators
Authors: Yujun Wang, Jiqiang Li, Yinuo Lu, Jinhe Bi, Aniri, Wenxuan Ye, Boliang Liu, Sikuan Yan, Shuning Wang, Xuebing Zhou, S{ö}ren Pirk, Hinrich Sch{ü}tze, Yunpu Ma
Organizations: LMU Munich · Munich Center for Machine Learning (MCML) · Huawei Heisenberg Research Center · Zhejiang University · University of Hong Kong · Technical University of Munich (TUM) · TU Berlin · Kiel University
Reusable skills allow language-model agents to accumulate and reuse problem-solving experience. However, most existing methods improve a skill bank only indirectly through rollout-driven accumulation and heuristic reflection, often resulting in shallow and weakly controlled updates. We present EvoSkill-S1, an RL-driven framework that instead treats reusable skills themselves as the direct objects of evolution. Given hard-negative failures, EvoSkill-S1 selects parent skills and transforms them into candidate child skills through five genetic-inspired operators: revision, cross-revision, mutation, dual-reflection, and crossover. Downstream task rewards train the skill evolver to generate more effective child skills, while evaluator-guided write-back selectively retains beneficial updates. Complementary Skill Bank Fusion consolidates operator-specific banks to exploit their distinct recovery patterns, while a Procedural Skill Interface supports transfer to unseen problems. Across all reported math and code benchmarks, EvoSkill-S1 improves the corresponding initialized skill banks under both Qwen and GPT-family solvers, with relative gains of up to 12.39%. Compared with the strongest external baseline without direct skill evolution, the largest relative improvement reaches 20.39%.
Figures & tables
Figure 1: Heuristic skill-bank updates versus direct skill-level evolution in EvoSkill-S1 , where an RL-trained evolver transforms existing skills through genetic-inspired operators.
Figure 2: Overview of the EvoSkill-S1 pipeline. An initial skill bank is evolved through fixed-operator runs, where an RL-trained evolver generates candidate child skills and evaluator-guided write-back updates the bank. Operator-specific banks are then fused, and the resulting evolved skills are accessed through the Procedural Skill Interface at inference time.
Figure 3: Diagnostics motivating multi-operator skill evolution. The left panel separates shared and operator-unique recoveries among failures of the initialized bank. The right panel reports judge-scored changes in retrieved skill guidance after evolution.
Figure 4: Effect of RL training on skill-evolution efficiency. Solved once counts hard negatives solved at least once during evolution, while written-back children counts child skills accepted into the bank.
Figure 5: Operator-level comparison for the five skill evolution operators. The operators emphasize different support metrics, explaining why Complementary Skill Bank Fusion can outperform reliance on a single operator.
Math
Code
Model
Method
AIME 24
AIME 25
AMC 23
Minerva Math
Olymp. Bench
APPS- test
LiveCode Bench
Qwen3-4B-Instruct
No skill
53.75
42.08
87.50
16.18
40.65
36.90
41.98
Traj RAG
57.08
40.83
91.56
29.04
54.15
28.58
31.17
Reflexion
57.92
42.92
93.12
32.35
66.77
41.82
42.18
ExpeL-SI
59.17
41.67
92.50
30.88
66.32
41.64
39.63
Tiny Skill Bank
Table 1: Main results on math and code benchmarks. Math columns report accuracy (%), while code columns report mean unit-test pass rate (%). Single-op denotes the arithmetic mean across the five independently evolved operator-specific banks, while Fusion denotes Complementary Skill Bank Fusion under the same overall bank-size budget. Shaded rows indicate our evolved skill-bank variants, and bold values mark the best result within each backbone and skill-bank-size block. For the GPT Family rows, math uses GPT-4.1 mini and code uses GPT-4o mini.
Figure 6: Effect of proceduralization on downstream performance and skill-interface quality. Top: performance before and after proceduralization for the initialized and Crossover skill banks. Bottom: paired-judge score differences between proceduralized skills and source skills across six interface-quality dimensions; positive values indicate improvement.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Skill bank
Method
AIME 24
AIME 25
AMC 23
Minerva Math
OlympiadBench
None
No skill
53.75±6.53
42.08±6.41
87.50±3.27
16.18±1.35
40.65±1.07
Traj RAG
57.08±6.53
40.83±6.36
91.56±2.97
29.04±1.66
54.15±1.08
Reflexion
57.92±6.41
42.92±6.53
93.12±2.59
32.35±1.71
66.77±1.03
ExpeL-SI
59.17±6.36
41.67±6.42
92.50±2.67
30.88±1.67
66.32±1.02
Tiny
Init
60.42±6.28
45.00±6.42
91.56±2.97
33.82±1.72
66.32±1.03
Single-op
61.50±6.42
45.75±6.61
92.31±2.81
34.93±1.75
68.63±1.00
Appendix
Table 2: Repeated-evaluation results for the Qwen3-4B-Instruct math benchmarks. Values are mean ± standard deviation over eight inference seeds; the corresponding means appear in Table 1 .
Skill bank
Method
APPS-test
LiveCodeBench
None
No skill
36.90±0.43
41.98±3.17
Traj RAG
28.58±0.41
31.17±2.97
Reflexion
41.82±0.45
42.18±3.22
ExpeL-SI
41.64±0.45
39.63±3.14
Tiny
Init
44.31±0.45
41.92±3.17
Single-op
44.79±0.45
43.44±3.20
Appendix
Table 3: Repeated-evaluation results for the Qwen3-4B-Instruct code benchmarks. Values are mean ± standard deviation over eight inference seeds; the corresponding means appear in Table 1 .
Skill bank
Method
AIME 24
AIME 25
AMC 23
Minerva Math
OlympiadBench
None
No skill
44.17±6.61
42.50±6.61
79.38±4.17
25.37±1.60
45.99±1.08
Traj RAG
50.42±6.53
44.68±6.41
85.32±3.64
32.35±1.71
61.35±1.06
Reflexion
50.42±6.53
46.54±6.67
86.12±3.54
31.99±1.71
60.83±1.06
ExpeL-SI
47.92±6.41
45.83±6.61
86.57±3.52
33.89±1.73
59.94±1.06
Tiny
Init
48.75±6.65
42.92±6.53
86.25±3.54
33.46±1.74
61.72±1.06
Fusion
50.83±6.61
43.75±6.53
86.88±3.47
34.12±1.75
62.02±1.06
Appendix
Table 4: Repeated-evaluation results for the GPT-family mini math benchmarks. Values are mean ± standard deviation over eight inference seeds; the corresponding means appear in Table 1 .
Skill bank
Method
APPS-test
LiveCodeBench
None
No skill
52.51±0.45
41.54±3.16
Traj RAG
41.85±0.45
50.76±3.21
Reflexion
42.23±0.44
46.44±3.21
ExpeL-SI
52.63±0.45
47.01±3.21
Tiny
Init
53.80±0.45
52.80±3.20
Fusion
53.85±0.45
53.50±3.24
Appendix
Table 5: Repeated-evaluation results for the GPT-family mini code benchmarks. Values are mean ± standard deviation over eight inference seeds; the corresponding means appear in Table 1 .
Benchmark
Domain
Split / source
Size
Use and metric
DAPO-hard
Math
Hard subset mined from the DAPO initialization pool after base-solver rollouts.
∼ 4k
Evolution target pool; answer-level correctness.
AIME24
Math
Official 2024 AIME I and II problems ( Mathematical Association of America, 2024 ) .
30
Held-out math evaluation; answer-level correctness.
AIME25
Math
Official 2025 AIME I and II problems ( Mathematical Association of America, 2025 ) .
30
Held-out math evaluation; answer-level correctness.
AMC23
Math
2023 AMC 12 A/B aggregate ( Mathematical Association of America, 2023 ) .
40
Held-out math evaluation; answer-level correctness.
Minerva Math
Math
Minerva quantitative-reasoning test set ( Lewkowycz et al., 2022 ) .
272
Held-out math evaluation; answer-level correctness.
OlympiadBench
Math
Text-only open-ended math subset ( He et al., 2024 ) .
675
Out-of-distribution math evaluation; answer-level correctness.
Appendix
Table 6: Benchmark splits, sources, and evaluation sizes. Internal hard subsets are mined only from initialization rollouts; held-out and out-of-distribution benchmarks are used for evaluation.
Math
Code
Method
DAPO -hard
AIME 24
AIME 25
AMC 23
Minerva Math
Olympiad Bench
APPS- train-hard
APPS- test
LiveCode Bench
Revision-500
12.77
61.25
46.25
92.19
34.56
67.95
20.61
44.98
44.15
Cross-revision-500
13.50
62.92
45.42
91.56
34.93
69.29
21.12
44.39
43.12
Mutation-500
13.00
60.83
46.25
92.19
35.66
68.69
21.49
45.12
42.75
Crossover-500
13.09
61.25
45.00
92.81
34.93
69.14
21.42
44.64
43.23
Dual-reflection-500
13.27
61.25
45.83
92.81
34.56
68.10
20.66
44.83
43.96
Appendix
Table 7: Detailed tiny-skill-bank single-operator results for Qwen3-4B. The average row is the arithmetic mean over the five independently evolved operator-specific banks. Code columns report mean unit-test pass rate (%).
Setting
Value
Learning rate
1.0×10−6
Rollout batch
32
Global update batch
32
Update micro-batch / GPU
2
Experience micro-batch / GPU
8
Candidates per target
4
Appendix
Table 8: GRPO training and write-back settings for the skill evolver.