Agent skills are documents in which experts write down procedural knowledge, and models increasingly maintain them by rewriting them without feedback. Such rewriting could adapt a skill to the model that uses it or erode the expert knowledge that makes it useful, yet how skills evolve under this feedback-free self-evolution has not been studied. We let models rewrite expert skills round after round without any task, run or judgment and run agents built on the same models with the rewritten skills. Our experiments show that the skills lose value rather than improve, as every model we test performs worse with them on every benchmark, by 15 points on average on SkillsBench after thirty rounds. Sampling each model's prior with skills it writes from the topic alone, we find that rewriting pulls skills toward this prior and gives up first the knowledge outside it. Putting this knowledge back recovers much of the lost pass rate, so an expert's skill is valuable largely for what the model does not write on its own. Building on this insight, SkillDiff marks this knowledge in place and raises the pass rate on SkillsBench by 4.8 points on average without feedback or any change to the skill's text. Project repository: https://github.com/LitaoGuo/How-Self-Evolution-Reshapes-Skills.
Figures & tables
Figure 1: By round 30, rewriting lowers every agent’s SkillsBench pass rate. Original and rewritten skills, with the same model used for rewriting and evaluation.
Figure 2: Distance to self-written skills falls early, then levels off. Mean TF-IDF cosine distance to ten self-written skills, normalized by their mean pairwise distance. At 1, a skill is as close to the self-written skills as they are to one another.
Figure 3: Across models and benchmarks, 10/10 items survive more often than 0/10 items. Retention after rounds 1 and 30, grouped by how many of ten self-written skills state each item. SkillsBench uses one rewriting chain per skill. On LogicBench and MedCalc-Bench, results pool three chains per skill for models with repeated chains; thin lines show individual chains.
SkillsBench
LogicBench
MedCalc-Bench
Model
0/10
10/10
Δ
0/10
10/10
Δ
0/10
10/10
Δ
GPT-5.6-Luna
10
83
+73
19
80
+61
20
85
+65
GPT-6.1-Sol
7
81
+74
11
82
+71
15
85
+70
MiMo-V2.5-Pro
32
90
+58
27
90
+63
21
90
+69
Qwen3.7-Plus
21
86
+65
22
88
+66
38
87
+49
Claude Haiku 5.5
37
82
+45
56
92
+36
52
88
+36
Table 1: After thirty rounds, 10/10 items survive more often than 0/10 items. Share of original skill items retained after thirty rounds, in percent. The labels 0/10 and 10/10 mean that none or all ten self-written skills state an item. Δ is the difference in retention between the 10/10 and 0/10 groups, in percentage points. The final row averages retention across models.
Figure 4: On average across agents and on every benchmark, restoring 0/10 items recovers more of the lost pass rate than restoring ≥ 1/10 items. Pass rates with original, round-30 and restored skills. Lost items stated by none or at least one of ten self-written skills return to the common base built from round-30 skills.
Round 30, lost items restored
Benchmark
Original
Round 30
≥ 1/10
0/10
All
SkillsBench
40.2
25.7
30.1
34.5
35.8
LogicBench
86.2
71.4
75.0
80.4
80.6
MedCalc-Bench
89.7
78.7
80.0
83.1
84.4
Table 2: Restoring 0/10 items recovers more performance than restoring ≥ 1/10 items on every benchmark. Mean pass rates across agents, in percent, for original skills, round-30 skills and the three restored versions. The 0/10 and ≥ 1/10 groups contain lost items stated by none or at least one of ten self-written skills; All restores every lost item.
Figure 5: SkillDiff. Generate ten skills from the topic alone, compare each original item against them, and mark in place the items that none states. A note asks the agent to carry these items into its plan as written.
Model
Δ pass rate, pp
GPT-5.6-Luna
+12.6
GPT-6.1-Sol
+5.7
MiMo-V2.5-Pro
+3.4
Qwen3.7-Plus
+2.3
Claude Haiku 5.5
+3.4
Qwen3.8-Flash
+1.1
Table 3: Marking items outside the prior improves every agent on SkillsBench. Pass-rate gain in percentage points over the original skill, evaluated in the same batch.
Appendix figures & tables22 assets
Supplementary material from the paper’s appendix.
Appendix
SkillsBench
LogicBench
MedCalc-Bench
Model
R1
R30
R1
R30
R1
R30
GPT-5.6-Luna
−11.5
−23.0
−8.7
−19.5
−3.1
−3.6
GPT-6.1-Sol
−8.0
−12.6
−10.4
−16.0
−10.9
−16.5
MiMo-V2.5-Pro
+2.3
−11.5
−0.1
−12.5
−1.7
−6.0
Qwen3.7-Plus
−9.2
−17.2
−4.0
−11.1
−8.1
−10.1
Claude Haiku 5.5
−2.3
−12.6
−1.9
−6.9
−2.3
−17.4
Appendix
Table 4: By round 30, every agent loses pass rate on every benchmark. Change in pass rate or accuracy from the original skill after rounds 1 and 30. R1 and R30 denote these rounds; pp denotes percentage points.
SkillsBench
LogicBench
MedCalc-Bench
Model
Orig.
R30
Δ
Orig.
R30
Δ
Orig.
R30
Δ
GPT-5.6-Luna
1.50
1.18
−0.32
1.38
1.23
−0.15
1.36
1.18
−0.18
GPT-6.1-Sol
1.68
1.34
−0.34
1.43
1.22
−0.21
1.68
1.39
−0.29
MiMo-V2.5-Pro
1.29
1.18
−0.11
1.21
1.07
−0.14
1.22
1.10
−0.12
Qwen3.7-Plus
1.41
1.28
−0.13
1.32
1.18
−0.14
1.31
1.23
−0.08
Claude Haiku 5.5
1.45
1.35
−0.10
1.25
1.22
−0.03
1.41
1.36
−0.05
Appendix
Table 5: On every benchmark, rewriting brings every model’s skills closer to its self-written skills. Distance of the original skill and of the round-30 skill to the model’s ten self-written skills over TF-IDF word vectors, in units of the mean distance between two self-written skills. Orig. and R30 denote the two skill versions; Δ denotes the change in distance between them.
SkillsBench
LogicBench
MedCalc-Bench
Model
0/10
≥ 1/10
Δ
0/10
≥ 1/10
Δ
0/10
≥ 1/10
Δ
After round 1
GPT-5.6-Luna
73.1
31.6
+41.5
39.4
15.6
+23.8
45.5
21.8
+23.7
GPT-6.1-Sol
75.4
33.3
+42.1
40.9
16.6
+24.3
57.5
18.9
+38.6
MiMo-V2.5-Pro
23.1
11.9
+11.2
25.7
4.3
+21.4
26.0
8.7
+17.3
Qwen3.7-Plus
37.5
20.1
+17.4
38.3
8.7
+29.6
30.6
13.2
+17.4
Appendix
Table 6: After one round and after thirty, every model loses the items outside the prior more often than the other items, on every benchmark. Share of items lost, in percent, for the 0/10 items outside the prior, which none of the model’s ten self-written skills states, and for the ≥ 1/10 items that some self-written skill states; Δ is the difference between them in pp.
List entries
Paragraphs
Code blocks
Headings
Model
0/10
≥ 1/10
Δ
0/10
≥ 1/10
Δ
0/10
≥ 1/10
Δ
0/10
≥ 1/10
Δ
GPT-5.6-Luna
90.3
42.6
+47.7
89.0
41.7
+47.3
92.4
59.5
+32.9
85.5
40.3
+45.2
GPT-6.1-Sol
94.5
47.2
+47.3
93.3
49.6
+43.7
90.4
57.4
+33.0
92.4
47.2
+45.2
MiMo-V2.5-Pro
68.5
33.5
+35.0
68.5
26.9
+41.6
73.2
30.2
+43.0
50.6
23.3
+27.3
Qwen3.7-Plus
80.8
45.2
+35.6
79.5
41.3
+38.2
81.5
52.8
+28.7
55.0
30.0
+25.0
Claude Haiku 5.5
59.3
34.2
+25.1
65.1
36.5
+28.6
73.5
51.7
+21.8
49.0
29.2
+19.8
Appendix
Table 7: Within list entries, paragraphs, code blocks and headings alike, every model loses the items outside the prior more often. Share of the items of each kind lost after thirty rounds on SkillsBench, in percent, for the 0/10 items that none of the model’s ten self-written skills states and for the other items in the ≥ 1/10 group; Δ is the difference between them in pp.
Second judge
Main judge, rerun
Third judge
GPT-6.1-Sol
GPT-5.6-Luna
Claude Haiku 5.5
Agreement on self-written skills, %
89.7
92.7
87.2
Cohen’s κ
0.78
0.84
0.72
Agreement on 0/10 class, %
88.9
94.8
86.0
Agreement on rewritten skills, %
87.2
89.8
82.4
Cohen’s κ
0.74
0.79
0.64
Appendix
Table 8: Two further judges and a second run agree with the judgments of the main text. Share of the labels on which each agrees with the main judgments, in percent, and Cohen’s κ , on 20 SkillsBench tasks with 1,800 items. An item is in the 0/10 class when none of the ten self-written skills states it.
Main judge
Second judge
Main judge, rerun
Third judge
GPT-5.6-Luna
GPT-6.1-Sol
GPT-5.6-Luna
Claude Haiku 5.5
0/10
89.2
93.3
89.9
93.4
≥ 1/10
39.8
40.9
38.4
52.6
Δ , pp
+49.5
+52.5
+51.4
+40.7
Appendix
Table 9: Every judgment finds a large gap. Share of items lost after thirty rounds, in percent, for the 0/10 items outside the prior and for the other items in the ≥ 1/10 group, on the same 20 SkillsBench tasks under each judgment; Δ is the difference in pp.
Samples of ten
Agreement on 0/10 class, %
Cohen’s κ
Original vs. further 1
88.1
0.76
Original vs. further 2
88.2
0.76
Original vs. further 3
87.7
0.75
Further 1 vs. 2
92.0
0.84
Further 1 vs. 3
91.7
0.83
Further 2 vs. 3
91.4
0.83
Appendix
Table 10: Every sample of ten self-written skills finds nearly the same items outside the prior. Share of the 6,605 SkillsBench items that two samples of ten self-written skills of GPT-5.6-Luna place in the same class, outside the prior or not, and Cohen’s κ . The original sample is the one of the main text.
Class under the original ten
Items
Stated by the further thirty, %
0/10
2,757
1.1
1–9/10
2,584
39.6
10/10
1,264
95.1
Appendix
Table 11: Items outside the prior are almost never written by further self-written skills. SkillsBench items of GPT-5.6-Luna by how many of the original ten self-written skills state them, and the mean share of the thirty further self-written skills that state an item.
Dropped outright
Weakened
Model
0/10
≥ 1/10
0/10
≥ 1/10
GPT-5.6-Luna
55
37
28
48
GPT-6.1-Sol
60
39
24
43
MiMo-V2.5-Pro
47
40
32
42
Qwen3.7-Plus
52
43
27
39
Claude Haiku 5.5
40
41
38
42
Appendix
Table 12: On average, lost items outside the prior are dropped outright more often, and weakened less often, than the other lost items. Of the items lost after thirty rounds on SkillsBench, the share dropped outright and the share weakened, in percent of the lost items, for the 0/10 items outside the prior and for the other items in the ≥ 1/10 group. The remaining lost items are changed to different content.
Model
0/10
≥ 1/10
GPT-5.6-Luna
3
26
GPT-6.1-Sol
2
21
MiMo-V2.5-Pro
10
42
Qwen3.7-Plus
6
29
Claude Haiku 5.5
12
38
Qwen3.8-Flash
2
26
Appendix
Table 13: Lost items outside the prior rarely return. Of the SkillsBench items lost in the first rewrite, the share kept again after thirty rounds, in percent, for the 0/10 items outside the prior and for the other items in the ≥ 1/10 group.
0/10
Also 0/10 for another model, %
Model
items
Observed
Expected
GPT-5.6-Luna
2,757
87
49
GPT-6.1-Sol
3,019
84
48
MiMo-V2.5-Pro
3,419
78
47
Qwen3.7-Plus
3,410
78
47
Claude Haiku 5.5
2,924
84
48
Appendix
Table 14: The items outside the prior are largely the same for every model. For the SkillsBench items outside the prior of each model, the share that also lies outside the prior of another model, averaged over the other models, against the share expected if the two sets were unrelated, which is the share of all items that lie outside the other model’s prior.
Held-out
Distance
Pairs closer than the original
Measure
self-written skill
Original
Round 5
Round 30
Round 5
Round 30
TF-IDF in main text
1.00
1.37
1.19
1.22
18/18
18/18
Embedding
1.01
2.24
1.78
1.93
17/18
15/18
Word frequency
1.00
2.42
1.62
1.77
18/18
16/18
Appendix
Table 15: Every measure places a held-out self-written skill at 1 and moves the rewritten skills toward the self-written skills in the first rounds. Distance to the model’s ten self-written skills, in units of the mean distance between two self-written skills, averaged over the 18 pairs of model and benchmark, and the number of pairs in which the rewritten skill lies closer than the original. The held-out self-written skill is measured against the other nine on the same scale.
Figure 6: Over Qwen3-Embedding vectors, too, rewriting moves the skills of almost every model and benchmark toward the self-written skills in the first rounds. Distance of the skills to the model’s ten self-written skills over Qwen3-Embedding vectors, in units of the mean distance between two self-written skills, on which a further self-written skill lies at 1.
Figure 7: Over raw word frequencies, too, rewriting moves the skills toward the self-written skills in the first rounds. Distance of the skills to the model’s ten self-written skills over word-frequency vectors without TF-IDF weights, in units of the mean distance between two self-written skills, on which a further self-written skill lies at 1.
Figure 8: On τ2 -bench, the analyses of the main text give the same picture for both models. (a) Pass rate over the 164 tasks of the airline and retail domains with the original policy and after each round of rewriting. (b) TF-IDF distance of the policy to the model’s ten self-written policies, in units of the mean distance between two self-written policies. (c) Share of the items of the original policies kept after rounds 1 and 30, by the number of self-written policies that state the item. (d) Pass rate of the round-30 policy and of the round-30 policy with the lost 0/10 items outside the prior, the other lost items in the ≥ 1/10 group or all lost items restored.
Figure 9: Without web search, rewriting lowers the accuracy of GPT-5.6-Luna on MedCalc-Bench more, and the other analyses keep their shape. (a) Accuracy on the 1,100 questions after each round. (b) TF-IDF distance to the ten self-written skills of the same version, written with or without web search. (c) Share of the items kept after thirty rounds, by the number of self-written skills that state the item. (d) Accuracy of the round-30 skill and of the round-30 skill with the lost 0/10 items outside the prior, the other lost items in the ≥ 1/10 group or all lost items restored. Both versions share the runs without a skill and with the original skill.
Figure 10: With up to sixteen attempts per task, the round-30 skill stays below the original skill. (a) pass@ k of GPT-5.6-Luna on SkillsBench, the chance that at least one of k runs passes a task. (b) Round-30 skill minus original skill, with its 95% interval over tasks.
Figure 11: Over repeated runs, the rewritten skill stays below the original skill, and putting back the items outside the prior recovers the most. (a) Change in pass rate on SkillsBench from the original skill, with every run of a task averaged, and its 95% interval over tasks. (b) Full evaluations of the benchmark, each taking the j -th run of every task; a line joins the original and the round-30 skill of the same evaluation. (c) Pass rate of the round-30 skill and of the round-30 skill with the lost 0/10 items outside the prior, the other lost items in the ≥ 1/10 group or all lost items restored. Bars show averages over runs; dots show each full evaluation.
Figure 12: Three independent rewriting chains move the skills toward the self-written skills along nearly the same path. TF-IDF distance of the skills to the model’s ten self-written skills after each round, in units of the mean distance between two self-written skills, for the rewriting chain of the main text and two further chains of every skill.
Kept after round 30, %
Model
0/10
10/10
Δ , pp
LogicBench, three chains per skill
GPT-5.6-Luna
18–21
77–83
59–62
GPT-6.1-Sol
11–11
80–83
69–72
MiMo-V2.5-Pro
25–28
85–93
60–65
Qwen3.7-Plus
18–26
88–90
62–72
Appendix
Table 16: Every rewriting chain keeps the items that all self-written skills state far more often than the items outside the prior. Range over the independent rewriting chains of each model of the share of items kept after thirty rounds, for the 0/10 items outside the prior and for the 10/10 items that all ten self-written skills or policies state, and of their difference Δ in pp.
Δ pass rate, pp
Both models
GPT-5.6-Luna
GPT-6.1-Sol
Against the original skill
0/10 items deleted
−4.6
−5.7
−3.4
≥ 1/10 items deleted
+0.6
+2.3
−1.1
Between the two deletions
0/10 −≥ 1/10
−5.2
−8.0
−2.3
Appendix
Table 17: Deleting the items outside the prior lowers the pass rate more than deleting as many words of the other items. Change in pass rate on SkillsBench when the same number of words is deleted from the original skill, only from the 0/10 items outside the prior or only from the other items in the ≥ 1/10 group. Each version is run in the same batch as the original skill.
Round-30 skill rewritten by
Δ own −
Agent
Original
GPT-5.6-Luna
GPT-6.1-Sol
other’s, pp
GPT-5.6-Luna
36.8
31.0
27.6
+3.4
GPT-6.1-Sol
44.8
42.5
37.9
−4.6
Appendix
Table 18: Both agents do worse with either round-30 skill, and about as badly with their own rewrite as with the other one. Pass rate on SkillsBench, in percent, with the original skill and with the round-30 skills rewritten by each model, all run in the same batch, and the difference between the own and the other rewrite, in pp. Shaded cells: an agent running its own rewrite.