Large language model (LLM) agents for health can accumulate reusable textual skills from task feedback, such as calculation criteria, measurement time points, and formula units. Deciding which skills to retain is consequential: useful rules improve later answers, while faulty generalizations propagate errors. We propose CausalEvo, a method for causal skill self-evolution that estimates the relative effects of deliberately changed skill sets while holding the model weights, feedback cases, and rule count fixed. At each evolution step, a frozen LLM proposes candidate rules, which are pooled with the incumbent skills so that the current skill set remains a feasible choice. Paired skill replacements against shared background rules are scored on the same feedback cases; least squares converts the score differences into relative skill values, and the top-ranked rules are retained, favoring incumbents on ties. A decision-loss bound separates candidate support, approximation, and estimation errors. On six MedCalc-Bench-Verified tasks with 120 held-out cases, CausalEvo achieves 60.00% accuracy, compared with 55.83% for SkillOpt and 51.67% without skills. Across the recorded training configurations, which use different optimization budgets, it consumes 55.6% fewer training tokens than SkillOpt. Gains concentrate in particular calculations alongside regressions on others, and paired accuracy intervals include zero. CausalEvo turns task feedback into controlled decisions about skill retention and replacement. Project repository: https://github.com/txbrook/CausalEvo
Figures & tables
Figure 1: Overview of CausalEvo. Incumbent skills ( St=Mt ) and model-generated proposals form a candidate pool. On a fixed feedback case, two skill sets share k−1 background rules and exchange one rule. The frozen actor scores both skill sets; paired score differences fit relative skill values for top- k selection, with ties favoring incumbent rules.
Method
Accuracy (%)
Accuracy 95% CI
Valid outputs (%)
Accuracy on valid outputs (%)
No skill
51.67
[43.33, 60.00]
100.00
51.67
SkillOpt
55.83
[47.50, 64.17]
100.00
55.83
CausalEvo
60.00
[52.50, 67.50]
99.17
60.50
Table 1: Main results on 120 held-out cases with thinking-off inference. Accuracy gives equal weight to the six tasks; invalid outputs count as incorrect. Valid-only accuracy is a secondary diagnostic with method-specific denominators. Bold marks the highest observed overall accuracy.
Figure 2: Final task accuracy for the three frozen skill states. Each task has 20 held-out cases, so five percentage points equal one case. Invalid outputs count as incorrect. Exact values appear in Appendix C.4 .
Figure 3: Macro accuracy over learning steps on the fixed 120-case test panel. Step 0 uses an empty skill. Error bars are conditional 95% task-stratified bootstrap intervals. Dashed lines show no skill and the selected SkillOpt endpoint as fixed references.
Setting
Final accuracy (%)
Phase mean (%)
No skill
62.50
61.94
Shuffled credit
62.50
68.68
CausalEvo
70.83
73.33
Table 2: Credit-assignment ablation on a separate dataset with a 24-case test set. Phase mean averages the six phase-end seen-task accuracies. Bold marks the highest observed value in each column.
Figure 4: Training token use in the recorded configurations, whose optimization budgets differ; output includes reasoning. Exact counts are in Appendix C.2 .
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Actor prompt. Selected rules follow the fixed instruction; the no-skill state leaves that field empty. Field headings indicate message structure.
Figure 6: Skill-proposal template, with m=6 at initialization and m=2 for a nonempty incumbent. The lower fields summarize the structured user payload. The program selects rules from the scored comparisons.
Setting
Value
Model identifier
deepseek/deepseek-v4-flash-0731
Temperature
Actor 0, proposer 0.3
Output caps
Actor 4,096; proposer 8,192 tokens
State constraint
Four rules, 5–80 words each
Comparison design
Random six-cycle; six pairs, 12 actor calls per valid evolution step
Appendix
Table 3: Settings for skill evolution.
Method
Requests
Input tokens
Output tokens including reasoning
Total tokens
SkillOpt
839
3,924,153
1,036,499
4,960,652
CausalEvo
336
1,186,029
1,016,551
2,202,580
Appendix
Table 4: Training usage for the main experiment.
Method
SIRS
HEART
PERC
CURB-65
Child-Pugh
Cockcroft– Gault
No skill
65
25
40
65
45
70
SkillOpt
85
40
55
70
40
45
CausalEvo
75
50
35
80
30
90
Appendix
Table 5: Final accuracy (%) on each task, with 20 held-out cases per task. Bold marks the highest observed accuracy in each column.
Method
Correct
Valid but incorrect
Invalid
Length truncations
No skill
62
58
0
0
SkillOpt
67
53
0
0
CausalEvo
72
47
1
0
Appendix
Table 6: Final-state output decomposition on the 120-case panel. All values are counts. Invalid outputs remain errors in the primary accuracy metric; truncation is one possible invalid-output cause.
Figure 7: Complete deployed skill from the main experiment at step 24. The actor receives these four rules after its fixed task instruction. The rule text is preserved from the saved skill artifact.
ID
Task
Teaching
Feedback
Development
51
SIRS
4
32
4
18
HEART
4
18
4
48
PERC
4
10
4
45
CURB-65
4
6
4
15
Child-Pugh
4
4
4
2
Cockcroft–Gault
4
2
4
Appendix
Table 7: Fixed allocation. Each task is introduced for four consecutive rounds. Feedback can come from tasks introduced previously; early tasks consequently appear more often.
Figure 8: Accuracy at each phase end on all tasks introduced so far. The evaluated task mixture expands with each phase.
Agent skill evolution seeks to improve reusable procedural guidance for large language model (LLM) agents through iterative revision. Existing methods base each revision mainly on execution trajectories or feedback, leaving recurring behavioral requirements across tasks implicit and tying revision to the behavior of the current skill. We introduce SkillFocus, which decomposes recurring task requirements into a capability space that remains fixed as the skill evolves, separating what tasks require from how the current skill behaves. SkillFocus maps current task outcomes to this space to identify the capability that leaves the most tasks unresolved, then uses that capability to determine what to revise and which evidence to use. Across four benchmarks spanning heterogeneous tasks, SkillFocus achieves the best held-out accuracy on all four, outperforming the strongest competing result by 5.7 points on average while using 24% fewer evolution tokens on average than the closest iterative baseline. Controlled studies further show that capabilities derived from recurring task requirements outperform task-semantic and execution-derived alternatives, while randomizing task--capability assignments reduces final accuracy by up to 20.2 points. Matching evidence to the selected capability increases candidate gain by 4.4 points under prioritized revision.
Ning Wang, Zhiren Gong, Bingdong Li +2
East China Normal University · Nanyang Technological University · Southern University of Science and Technology +1
Skill documents, structured natural-language instructions that guide Large Language Model (LLM) agents, are critical to modern agent frameworks, yet LLMs struggle to write skills that actually work. On SkillsBench, human-authored skills improve pass rates by 16.2 percentage points, while LLM-authored skills provide no measurable gain. We introduce SkillAxe, a fully unsupervised framework that enables LLMs to iteratively diagnose and refine their own skills. SkillAxe decomposes skill quality into four interpretable dimensions (quality impact, trigger precision, instruction compliance with fault attribution, and solution-path coverage), producing structured improvement briefs that require no ground-truth labels, test suites, or environment rewards. On SkillsBench, SkillAxe improves pass rates by 28% relative over unimproved LLM skills and closes 47--67% of the gap to human-authored skills. We validate the approach as a continuous improvement engine in the wild on SpreadsheetBench, where a SkillAxe-built skill library learns from past agent trajectories and raises pass rate from 16.0% to 52.0% using only 22 skills.
Natural-language skills are textual procedural memories through which large language model (LLM) agents retain reusable task knowledge without updating model weights. Existing methods typically treat skills as either static artifacts or monolithic documents optimized using aggregate validation scores as feedback. However, representing a skill as a monolithic document restricts optimization to its textual content, without explicitly modeling the structure through which procedural knowledge is retrieved and executed. We identify a key distinction between learning what knowledge to retain and determining how to organize it: textual updates should first be validated through execution evidence, after which the retained knowledge should be structured according to its procedural dependencies and retrieval requirements. To this end, we introduce SkillSpec, a two-phase framework comprising consensus-gated evolution and representation specialization. In the consensus-gated phase, complementary editing intents generate complete candidate skills. An update is committed only when paired evaluations reach consensus, requiring sufficient overall improvement and non-negative aggregate paired gain in every repeated evaluation. In the specialization phase, signals of process and redundancy sensitivity derived from the full optimization trajectory, including accepted and rejected candidates, guide the selection of a flat, graph, or hybrid representation.Across six benchmarks and three target language models, SkillSpec improves average success rate over SkillOpt by 6.89%, averaged across the three models. These results demonstrate that reliable skill evolution and representation specialization address complementary objectives: deciding what knowledge to retain and how to structure it for inference.