StraTune: Adaptive Selection of Revision Operators for Self-Evolving LLM Skills

Zeping Liu; Yan Li; Ni Lao; Gil Wolff; Gengchen Mai

cs.AISep 26, 2026

StraTune: Adaptive Selection of Revision Operators for Self-Evolving LLM Skills

Authors: Zeping Liu, Yan Li, Ni Lao, Gil Wolff, Gengchen Mai

Organizations: The University of Texas at Austin · Amazon

Abstract

Large language models (LLMs) can learn reusable textual skills from execution feedback without updating their parameters, but effectively deciding how to revise these skills remains a key challenge. Existing methods typically rely on a fixed revision operator, a search strategy and the revision forms applied under it. However, we observe that no single revision operator consistently performs best across tasks, and repeatedly applying an unsuitable operator can limit further improvement. We propose StraTune (strategy-guided skill tuning), which lets a frozen optimizer LLM choose the revision operator at every round from the optimization state, which is defined as the current execution feedback together with the recorded outcomes of earlier strategies and forms. Candidate skills from every revision operator pass one candidate evaluation, which screens for gains and regressions on a small sample set and validates them on a larger one, and every outcome is written back to the optimization state for later choices. Across four benchmarks and two LLM settings, StraTune outperforms all five baselines in most settings. Ablations attribute the gains to the adaptive choice of the revision operator, since fixed, random, scheduled, and bandit strategy choices all score lower, and skills learned with a small target LLM also improve a stronger one. Code and learned skills are available at https://github.com/seai-lab/StraTune.

Figures & tables

Figure 1: (a) Existing skill-learning methods by search strategy (columns) and revision form (rows); bold marks the baselines used in this paper. (b) Test scores (Table 1 ) on DocVQA (Doc), LiveMath (Math), Mind2Web (Web), and SpreadsheetBench (SSB) under two model settings; filled circles mark the best method in a column, open circles the second best. The row labelled Best baseline gives the strategy and form of the best baseline in each column, which change from column to column, whereas StraTune ranks first in most.

Figure 2: Adaptive skill optimization with StraTune. (a) From optimization state $\Omega_{t}$ , optimizer $G_{\phi}$ adaptively selects a revision operator $o_{t}$ (a search strategy and the revision forms applied under it) and generates candidate skills, using execution feedback from target $F_{\theta}$ and task scores. (b) Candidate skills pass initial screening and then further validation; both stages record every outcome, including rejections, in the evaluation history $H_{t}$ , and only candidates passing both replace the current skill. (c) After optimization, final skill selection compares the current skill with saved candidate skills to obtain $\hat{s}$ .

Haiku setting Target: Haiku 4.5 Optimizer: Sonnet 4.6
Method	DocVQA	LiveMath	Mind2Web	SpreadsheetBench
Initial skill	47.88	29.38	46.09	40.00
GEPA	91.26	43.60	46.18	52.50
TextGrad	90.69	31.75	46.16	45.00
SkillOpt	91.87	45.02	40.55	50.71
Trace2Skill	90.62	33.18	46.18	46.79

Table 1: Test scores (0–100) with Haiku and Opus as target LLMs under a training budget of

6|\mathcal{D}_{\mathrm{train}}|

target-LLM executions. Bold marks the highest score in each model setting.

Fixed search strategy
Configuration	DocVQA	LiveMath	Mind2Web	SpreadsheetBench
StraTune	92.12	65.88	47.49	56.43
I1 only	91.92	39.34	46.09	54.64
I2 only	91.56	29.38	47.34	52.14
I3 only	90.73	63.03	45.99	55.00
Strategy chosen without the per-round decision

Table 2: Test scores of ablations of candidate generation with Haiku/Sonnet (defined in Section 4.1 ).

Configuration	DocVQA	LiveMath	Mind2Web	SpreadsheetBench
StraTune	92.12	65.88	47.49	56.43
No valid.	91.51	40.28	46.09	47.86
No final sel.	92.12	65.88	47.49	51.43

Table 3: Test scores of ablations of candidate evaluation with Haiku/Sonnet (defined in Section 4.1 ).

Figure 4: The StraTune runs of Table 1 with Haiku/Sonnet. (a) Strategy and revision form (marker) chosen in each round and

\dagger

marks initialization. (b) Test score of the current skill against the number of

F_{\theta}

executions divided by

|\mathcal{D}_{\mathrm{train}}|

; filled circles mark changes of the current skill.

Figure 5: Test scores with Opus 4.8 as the target LLM of skills learned with Haiku (squares) and with Opus itself (triangles). The dashed line is the initial skill, and StraTune is highlighted by blue.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Form	Name	Content introduced or revised
F1	Conditional rules	Instructions for particular situations, such as converting quantities to a common unit before comparison.
F2	Worked examples	A sample paired with a correct answer or solution procedure that illustrates the intended behavior.
F3	Reasoning procedure	Steps for solving a task, including reasoning and answer checking or a workflow of task operations. Examples include comparing two derivations, or reading a table, filtering rows, and writing the result.
F4	Full rewrite	A rewrite of the entire skill to incorporate guidance, group related instructions, and resolve conflicts.

Appendix

Table 5: Revision forms used to generate candidate skills.

Dataset	Training	Test	Budget ( $6\|\mathcal{D}_{\mathrm{train}}\|$ )
DocVQA	500	1,000	3,000
LiveMath	397	211	2,382
Mind2Web	800	252	4,800
SpreadsheetBench	629	280	3,774

Appendix

Table 6: Dataset sizes and training budgets of target-LLM executions shared by all methods. One execution runs one skill on one training sample; a Mind2Web task counts once despite containing multiple action steps.

Dataset	Solved	Random	Further validation
	Screening
DocVQA	18	18	150
LiveMath	24	24	119
Mind2Web	18	18	240
SpreadsheetBench	18	18	189

Appendix

Table 7: Screening quotas of previously solved and randomly sampled samples, and further-validation set sizes (30% of the training set), with Haiku/Sonnet.

Dataset	Target	Optimizer	Ratio (%)	Calls	Tokens	Of optimizer (%)	Of target (%)
	All optimizer calls			Strategy selection
DocVQA	9.75	0.51	5.3	14	49	9.5	0.50
LiveMath	6.47	0.51	7.9	10	49	9.6	0.76
Mind2Web	45.00	0.50	1.1	16	57	11.3	0.13
SpreadsheetBench	7.20	1.63	22.7	25	104	6.4	1.45

Appendix

Table 8: Tokens consumed by one StraTune training run per dataset with Haiku/Sonnet. Target and optimizer tokens are in millions, strategy-selection tokens in thousands. Ratio is optimizer tokens as a percentage of target tokens.

	F1 rules	F2 examples	F3 procedure	F4 rewrite
I1 direct revision	46.92	45.50	43.13	34.12
I2 iterative refinement	28.44	29.38	29.38	29.38
I3 parallel sampling	37.91	36.49	63.51	58.29
Initial skill	29.38
StraTune, adaptive strategy and form	65.88

Appendix

Table 9: LiveMath test scores of StraTune with one fixed revision operator (rows: search strategy, columns: revision form), compared with the initial skill and with adaptive selection of strategy and form (Table 1 ); Haiku/Sonnet,

6|\mathcal{D}_{\mathrm{train}}|

budget.

Strategy selection	DocVQA	LiveMath	Mind2Web	SpreadsheetBench
StraTune, optimizer LLM (Table 1 )	92.12	65.88	47.49	56.43
Rot. , fixed schedule	91.74	65.40	46.06	45.71
Replay , copied sequence	92.00	51.66	46.18	53.21
$\epsilon$ -greedy	91.94	55.45	46.09	49.64
UCB1	90.92	38.39	46.02	53.21
Thompson sampling	91.41	45.02	46.07	53.57

Appendix

Table 10: Test scores with the strategy chosen by the optimizer LLM (StraTune), by a fixed or replayed schedule, and by three bandit rules over the recorded outcomes; Haiku/Sonnet,

6|\mathcal{D}_{\mathrm{train}}|

budget. The schedule and bandit rows use the same training samples.

Setting	DocVQA	LiveMath	Mind2Web	SpreadsheetBench
Haiku	$92.52\pm 0.80$	$65.40\pm 1.16$	$47.52\pm 0.61$	$56.16\pm 0.90$
Opus	$96.88\pm 0.40$	$81.16\pm 0.80$	$51.48\pm 0.56$	$72.41\pm 0.70$

Appendix

Table 11: Mean

\pm

standard deviation of StraTune test scores (0–100) over four optimization runs per dataset–model setting.

Language Model Self-Improvement

Language model self-improvement studies how models use their own outputs, feedback, and experience to improve with less external supervision.

See all 174 papers

Sep 28, 2026

SkillFocus: Evolving Agent Skills via Capability Decomposition

Agent skill evolution seeks to improve reusable procedural guidance for large language model (LLM) agents through iterative revision. Existing methods base each revision mainly on execution trajectories or feedback, leaving recurring behavioral requirements across tasks implicit and tying revision to the behavior of the current skill. We introduce SkillFocus, which decomposes recurring task requirements into a capability space that remains fixed as the skill evolves, separating what tasks require from how the current skill behaves. SkillFocus maps current task outcomes to this space to identify the capability that leaves the most tasks unresolved, then uses that capability to determine what to revise and which evidence to use. Across four benchmarks spanning heterogeneous tasks, SkillFocus achieves the best held-out accuracy on all four, outperforming the strongest competing result by 5.7 points on average while using 24% fewer evolution tokens on average than the closest iterative baseline. Controlled studies further show that capabilities derived from recurring task requirements outperform task-semantic and execution-derived alternatives, while randomizing task--capability assignments reduces final accuracy by up to 20.2 points. Matching evidence to the selected capability increases candidate gain by 4.4 points under prioritized revision.

Ning Wang, Zhiren Gong, Bingdong Li +2

East China Normal University · Nanyang Technological University · Southern University of Science and Technology +1

LLM Agent Skill Learning Task Decomposition

Ask PDF

May 10, 2026

Skill-R1: Agent Skill Evolution via Reinforcement Learning

Agentic large language models often rely on skills, reusable natural language procedures that guide planning, action, and tool use. In practice, skills are typically improved through prompt engineering or by aligning the task LLM itself, which is costly, model-specific, and often infeasible for closed-source models. Skill optimization is not a one-step problem but a recurrent process with two coupled levels of credit assignment: a useful skill must improve rollout quality under current conditioning, while a useful revision must turn observed outcomes into a better skill for the next round. We propose Skill-R1, a reinforcement learning framework for instance-level recurrent skill optimization from verifiable rewards. Rather than updating the task LLM, Skill-R1 trains a lightweight skill generator that conditions on the task context, prior rollouts, and their verified outcomes to produce skills that steer a frozen task LLM. This preserves black-box compatibility with both open- and closed-source models while making adaptation substantially cheaper than model-level updates. Skill-R1 proceeds over multiple generations: at each step, the current skill induces rollouts whose verified outcomes are fed back to produce the next revision. To optimize this recurrent process, we introduce a bi-level group-relative policy optimization objective combining intra-generation and inter-generation advantages. The intra-generation term compares rollouts under shared skill conditioning, while the inter-generation term rewards revisions that improve behavior across successive generations. Together, these provide a principled objective for directional skill evolution rather than one-shot self-refinement. Empirically, Skill-R1 achieves consistent gains over no-skill baselines and standard GRPO across benchmarks with verifiable rewards, with particularly strong improvements on complex, multi-step tasks.

Yash Vishe, Rohan Surana, Xunyi Jiang +8

UC San Diego · Adobe Research

Agent Skill Learning RL for LLM Agents

Ask PDF

Jun 9, 2026

SkillAxe: Sharpening LLM-Authored Agent Skills Through Evaluation-Guided Self-Refinement

Skill documents, structured natural-language instructions that guide Large Language Model (LLM) agents, are critical to modern agent frameworks, yet LLMs struggle to write skills that actually work. On SkillsBench, human-authored skills improve pass rates by 16.2 percentage points, while LLM-authored skills provide no measurable gain. We introduce SkillAxe, a fully unsupervised framework that enables LLMs to iteratively diagnose and refine their own skills. SkillAxe decomposes skill quality into four interpretable dimensions (quality impact, trigger precision, instruction compliance with fault attribution, and solution-path coverage), producing structured improvement briefs that require no ground-truth labels, test suites, or environment rewards. On SkillsBench, SkillAxe improves pass rates by 28% relative over unimproved LLM skills and closes 47--67% of the gap to human-authored skills. We validate the approach as a continuous improvement engine in the wild on SpreadsheetBench, where a SkillAxe-built skill library learns from past agent trajectories and raises pass rate from 16.0% to 52.0% using only 22 skills.

Srishti Gautam, Arjun Radhakrishna, Sumit Gulwani

Microsoft

LLM Self-Refinement LLM Agent Self-Improvement LLM Agent Skill Learning LLM Agent Skill Evaluation

Ask PDF

Abstract

Language Model Self-Improvement

Explore similar work

SkillFocus: Evolving Agent Skills via Capability Decomposition

Skill-R1: Agent Skill Evolution via Reinforcement Learning

SkillAxe: Sharpening LLM-Authored Agent Skills Through Evaluation-Guided Self-Refinement