Self-Evolving Skills via Surrogate-Guided Solve-and-Reproduce

Jiale Liu; Pinze Ren; Yuqi Xia; Huan Wang; Zhenlin Zhao; Siming Dong

cs.AIAug 10, 2026

Self-Evolving Skills via Surrogate-Guided Solve-and-Reproduce

Authors: Jiale Liu, Pinze Ren, Yuqi Xia, Huan Wang, Zhenlin Zhao, Siming Dong

Organizations: Cleer Science · The University of Edinburgh · Tsinghua University

Abstract

Agent skills are portable packages of instructions and resources an agent consults at deployment. Self-evolving them fails in two ways today. First, skills evolved from scratch underperform human-curated ones and, on a weak model, using no skill at all. Second, an evolution-time pass records one lucky trajectory that a fresh stochastic agent often fails to reproduce at deployment. We present reSolve, a per-task, oracle-in-the-loop framework built on three components. It decouples interactive solving from a self-contained deliverable that is independently re-executed in a fresh container, a protocol we call solve-and-reproduce. It enhances the sparse reward signal with a surrogate verifier that cannot access hidden tests or reference answers. It then runs verifier-guided beam search over a solution-construction graph. Within a fixed harness, a cheap model self-evolves skills that reach $74.9\%$ mean-of-3, $+14.8$ points over the $60.1\%$ human-curated baseline, exceeding the strongest official curated-skill result ( $67.3\%$ , GPT-5.5/OpenHands). We also report observed failure cases and domain-level results, including performance on the 14 Natural Science tasks, to clarify when the approach does and does not help.

Figures & tables

Figure 1: reSolve evolves a skill package through a five-stage solve-and-reproduce loop. The top row follows one candidate through stages 1–5, which are task and optional curated anchor, interactive authoring, frozen re-execution by a new agent, dual evaluation (surrogate score and hidden grader), and search control. A full pass returns the frozen skill. Otherwise, the beam keeps the top candidates under $\widetilde{R}$ and refines them. Panel A details how the blind verifier generates and runs executable checks, and Panel B illustrates the search configuration ( $m{=}4$ , $B{=}2$ , $K{=}2$ , $D{=}2$ , at most 12 evaluations).

Rank	Model	Agent	No skills	Curated	$\Delta$
SkillsBench Leaderboard
1	GPT-5.5	OpenHands	51.5	67.3	+15.8
2	GPT-5.5	Codex	46.8	66.5	+19.7
3	Opus 4.7	Claude Code	43.0	61.2	+18.2
4	Gemini 3.1 Pro	Gemini CLI	36.0	60.8	+24.8
5	GLM 5.1	OpenHands	32.7	58.4	+25.7

Table 1: Main result in complete SkillsBench v1.1 context. Scores and

\Delta

are percentages. Background colors mark the first , second , and third best value per official column, and ties share a color.

Condition	Mean-of-3	95% CI	$\Delta$ curated	Pass@3
No skill	30.6	[23.1, 38.4]	-29.6	45.3
Curated skill	60.1	[51.2, 68.8]	+0.0	68.6
reSolve Self-Evolved	74.9	[66.3, 83.0]	+14.8	79.1

Table 2: reSolve aggregate. Entries are percentages. Confidence intervals are task-bootstrap intervals for the mean, and differences are paired against curated.

Figure 2: 25 tasks yield improvements on reSolve. Each marker compares the reSolve Curated and reSolve Self-Evolved conditions. Tasks with identical score pairs are aggregated, with marker area proportional to the task count. Points above the diagonal represent tasks where reSolve found a better skill.

Condition	Mean-of-3	Effect vs. full	95% CI
Full reSolve	74.9	+0.0	[0.0, 0.0]
w/o surrogate ranking	66.0	-8.9	[-14.3, -4.5]
w/o beam search (greedy only)	59.4	-15.5	[-20.3, -11.3]
w/o curated skill package	44.0	-30.9	[-40.7, -21.6]

Table 3: Ablation effects relative to the full reSolve framework. Entries are mean reward percentages and paired task-bootstrap intervals.

Figure 3: Paired ablation effects with 95% task-bootstrap intervals. Zero is the full reSolve framework. These intervals describe heterogeneity across tasks for one evolution realization per arm.

Figure 4: Distribution of full passes across three deployment trials. Each bar partitions the 86 tasks by how many of the three trials end in a full pass. The reSolve Self-Evolved row applies the keep-better rule of Section ˜ 3.4 ; the evolved-package and direct-artifact rows deploy each packaging variant unconditionally.

Condition	Full pass	Graded fail	Exec. error	Mixed tasks
		Non-passing slots
No skill	72	160	26	27
Curated skill	148	93	17	22
Evolved package only	169	76	13	21
Direct artifact (no agent)	177	76	5	3
reSolve Self-Evolved	188	57	13	12

Table 4: Slot-level decomposition of the

86\times 3=258

deployment trials. A slot is a full pass when its reward reaches 1, even if the runner also flagged an error. A graded fail is a trial that completes with no flagged error and scores below 1. An exec. error is a flagged agent, environment, or verifier error. Mixed tasks pass one or two of their three trials.

Figure 5: Task-level movement of full-pass counts along the condition ladder. Cells count tasks by full passes before (rows) and after (columns) each step; dashed diagonal cells are unchanged, and red counts below the diagonal mark tasks that lose passing trials. Curated rescue concentrates above the diagonal of the left panel, and the keep-better rule leaves no cell below the diagonal in the right panel. Row sums appear in the right margin of each panel.

Figure 6: reSolve Self-Evolved mean reward by official SkillsBench category. Values are percentages under the primary non-null convention. Natural Science tasks are highlighted, and category samples are small ( $n=5$ – $16$ ).

Layer	Observable evidence	Representative case	Agent-system implication
Search output	No retained package for 9/86 tasks	ADA bathroom repair has rewards $[0,1,0]$ , but all three are bare-agent runs	Distinguish “no package produced” from “package failed to reproduce”
Package discovery	Semantic path attempted but physical path not recovered	Earthquake plate calculation, where find A && find B short-circuits before /skills ; the two trials that find the solver pass	Expose a canonical path and machine-readable entrypoint
Execution policy	Bundled solver runs, then later commands replace its behavior	FJSP optimization, where direct execution passes; post-execution repair fails	Run the frozen entrypoint first and preserve its output as a rollback checkpoint
Execution noise	Same packaged solver is invoked but numerical outputs differ	HVAC control, where settling times $50.0$ , $58.5$ , and $77.5$ under sensor noise yield rewards $[1,1,0]$	Seed stochastic components and test a distribution, not one replay
Termination	Runner error and grader reward disagree	5 of 13 error-flagged trials still receive full reward; some timed-out agents had already written valid outputs	Grade the latest checkpoint separately from executor termination status

Table 5: Failure modes. “Interpretation” is derived from saved commands and outputs.

LLM Agent Skill Learning

Large language model agent skill learning studies how agents acquire, represent, reuse, and refine procedures for multi-step tasks.

See all 402 papers

Do Self-Evolving Skills Generalize to Held-Out Tasks?
Sep 30, 2026Xihao Piao, Zifeng Wang, Zhen ChenAgent Skill Learning Self-Evolving Agents
SANKEN, Osaka University
EvoSkill: Automated Skill Discovery for Multi-Agent Systems
Mar 3, 2026Salaheddin Alzubi, Noah Provenzano, Jaydon Bingham +2Agent Skill Learning LLM Agent Self-Improvement
Sentient · Virginia Tech
SkillEvolver: Skill Learning as a Meta-Skill
May 11, 2026Genrui Zhang, Erle Zhu, Jinfeng Zhou +2LLM Agent Self-Improvement LLM Agent Skill Learning
2Beijing Jiaotong University · 1Tsinghua University

Sep 30, 2026J/K move · Enter open · S save

Do Self-Evolving Skills Generalize to Held-Out Tasks?

Xihao Piao, Zifeng Wang, Zhen Chen

SANKEN, Osaka University

Agent Skill Learning Self-Evolving Agents

Open paper AskPDF arXiv

AI agents can externalize what they learn from past tasks into reusable skills, such as procedures, checklists, code, or other executable artifacts, that can be retrieved and reused when solving new tasks. Self-evolving skill methods keep rewriting these skills after each round of practice on training tasks, and the skill is then used on new tasks of the same kind. We ask a question: does the improvement a skill shows on its training tasks carry over to new test tasks? We test five self-evolving methods and a one-shot skill on six benchmarks, with the same model, the same agent, and the same train/test split for every method. Of the 21 skills that improve on their training tasks, 5 keep all of that improvement on the test tasks, 13 keep part of it, and 3 keep none of it. No existing method is best everywhere. When we read the skills, the ones that carry over badly often fix details that should depend on the task, such as column names and output files, or turn a fix for one failure into a rule for every task. An LLM judge that reads the skill content can often see this: it ranks finished skills the same way the test results do in 86% of pairs. But it predicts the effect of a single edit poorly, so edits still have to be tested by running them. Based on these findings, we describe Generalizable Skill Optimization (GSO), which keeps only a guide for writing skills and writes a new skill for each task; it scores highest on all six benchmarks.

Abstract

LLM Agent Skill Learning

Explore similar work

Do Self-Evolving Skills Generalize to Held-Out Tasks?

EvoSkill: Automated Skill Discovery for Multi-Agent Systems

SkillEvolver: Skill Learning as a Meta-Skill