SkillEvoReg: Regularizing Agent Skill Evolution Against Overfitting

Guanyu Nie; Fangzhou Zhu; Shixiong Kai; Xiongwei Han; Tao Zhong; Mingxuan Yuan

cs.AISep 25, 2026

SkillEvoReg: Regularizing Agent Skill Evolution Against Overfitting

Authors: Guanyu Nie, Fangzhou Zhu, Shixiong Kai, Xiongwei Han, Tao Zhong, Mingxuan Yuan

Organizations: Huawei Noah’s Ark Lab

Abstract

Language-model agents increasingly improve by converting execution experience into reusable external skills. Yet repeated skill updates form a learning process of their own: locally useful edits can accumulate into redundant or task-specific instructions, while new updates can disrupt behavior that previously worked. We study this problem as skill-evolution overfitting and introduce SkillEvoReg, a general regularization framework for skill evolution inspired by anti-overfitting techniques in neural-network training. SkillEvoReg combines training-time skill dropout, which perturbs update generation, and complexity-aware local regularization, which controls unnecessary structural growth, with causal counterexample validation (CCV), which provides targeted behavioral validation of candidate-specific regressions. We instantiate the framework across heterogeneous skill-evolution systems while retaining each system's native skill evolver and task evaluator. Across SkillOpt, SkillEvolBench, and ContinualSkillBench, SkillEvoReg consistently controls skill-state growth while preserving competitive downstream capability, improves several transfer and later-stage evolution outcomes, and identifies update-level regressions that structural metrics alone cannot reveal. These results suggest that explicit regularization is a useful complement to increasingly capable skill updaters.

Figures & tables

Panel of Figure 1 — Figure 1: A motivating longitudinal study of native skill evolution on SpreadsheetBench. (a) Performance across evolution epochs. (b) The IID–transfer gap narrows and then re-expands after the transfer peak. (c) Skill structure continues to accumulate.

Figure 2: Overview of SkillEvoReg . The three regularizers act on update generation, skill complexity, and behavioral validation, respectively. Colors distinguish the native skill evolver , SkillEvoReg regularization modules and skill-states .

Dataset	Method	Delivery Test $\uparrow$	Final Test $\uparrow$	$\Delta$ Test $\uparrow$	Skill tokens $\downarrow$	Complexity $\downarrow$
Spreadsheet	SkillOpt	54.75	47.00	$-7.75$	6,060	116.96
Spreadsheet	SkillEvoReg	58.50	59.25	$\mathbf{+0.75}$	2,341	57.21
SearchQA	SkillOpt	80.46	80.86	$\mathbf{+0.39}$	7,924	124.42
SearchQA	SkillEvoReg	81.14	80.39	$-0.75$	1,833	39.51
LiveMath	SkillOpt	34.79	36.41	$\mathbf{+1.61}$	6,749	110.52
LiveMath	SkillEvoReg	35.02	30.18	$-4.84$	1,606	36.52

Table 1: SkillOpt results. Delivery denotes the validation-selected checkpoint and Final the epoch-8 checkpoint;

\Delta

Test is Final minus Delivery. Skill tokens and complexity describe the final skill state.

Metric	1-pass	2-pass	Ours
Replay (T1–T3) $\uparrow$	40.00	38.89	42.22
T1 Canonical $\uparrow$	43.33	40.00	48.33
T2 Enriched $\uparrow$	41.67	36.67	38.33
T3 Variant $\uparrow$	35.00	40.00	40.00
Deployment (T4–T6) $\uparrow$	31.67	29.44	33.33
T4 Context shift $\uparrow$	36.67	40.00	31.67

Table 2: SkillEvolBench results. T1–T3 are replay tasks and T4–T6 are unseen deployment tasks.

Domain	Method	Held-out capability $\uparrow$			Skill state $\downarrow$
Domain	Method	Overall	Struct.	Rubric	# Skills	Tokens	Complexity
Mathematics	Baseline	67.90	74.00	61.80	33	21,952	612.26
Mathematics	SkillEvoReg	69.50	74.00	65.00	32	16,368	438.30
Law	Baseline	65.42	81.82	60.79	24	25,826	645.78
Law	SkillEvoReg	70.69	86.36	66.27	20	19,941	507.97
Finance	Baseline	68.63	65.58	69.59	28	78,193	1,596.72

Table 3: ContinualSkillBench frozen-library results after learning tasks 1–50. Scores average two evaluations of held-out tasks 51–100; skill-state statistics use the complete step-50 library.

Method	Test $\uparrow$	Chars. $\downarrow$
Native updater	47.00	31.7K
Complexity only	55.50	13.1K
Dropout only	57.50	19.5K
Full SkillEvoReg	59.25	11.7K

Table 4: SpreadsheetBench ablation at the final checkpoint.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Characteristic	IID	Structural transfer
Multi-sheet workbooks	5/40	12/40
Three-or-more sheets	0/40	3/40
Existing cross-sheet formulas	1/40	8/40
Cross-sheet requirements	6/40	13/40
Merged-cell workbooks	2/40	8/40
Mean number of sheets	1.125	1.475

Appendix

Table 5: Characteristics of the two held-out SpreadsheetBench pilot sets. The two sets have closely matched task semantics, while the transfer set contains greater workbook-level structural variation.

Checkpoint	Train	Val.	IID	Transfer	IID–Transfer
Initial	40.00	28.33	50.83	12.50	38.33
E1	61.67	65.83	63.33	35.83	27.50
E2	54.58	55.83	59.17	39.17	20.00
E3	56.25	55.83	60.00	40.00	20.00
E4	60.42	54.17	54.17	46.67	7.50
E5	57.92	53.33	50.00	39.17	10.83

Appendix

Table 6: Checkpoint performance in the motivating SpreadsheetBench pilot. Values are mean task accuracy across three independent evaluation replays of the same learned trajectory.

Checkpoint	Lexical length	Strict lines	Conditional lines	Artifact matches
Initial	402	4	1	13
E1	1,944	12	17	44
E2	3,252	22	34	71
E3	4,416	31	41	84
E4	5,586	40	50	95
E5	6,655	50	60	113

Appendix

Table 7: Deterministic lexical diagnostics along the motivating pilot trajectory. These are descriptive proxies and are distinct from the complexity measure

C(S)

used by SkillEvoReg .

Dataset	Method	Complexity $C$ $\downarrow$
		Value	Relative to baseline
SpreadsheetBench	SkillOpt	116.96	1.00
SpreadsheetBench	SkillEvoReg	57.21	0.49
SearchQA	SkillOpt	124.42	1.00
SearchQA	SkillEvoReg	39.51	0.32
LiveMath	SkillOpt	110.52	1.00

Appendix

Table 8: Final composite complexity

C

on the SkillOpt benchmarks.

Framework	Native proposal	Complexity handling	Final candidate handling
SkillOpt	Ranked edits to one instruction document	Complexity contributes to the regularized validation gate	Regularized acceptance gate; current/best checkpoint bookkeeping retained
SkillEvol	SkillAuthor patch over a multi-skill family	Growth budget can trigger bounded local regularization; budget alone does not reject	Native library commit API reused with staging, optional repair, and finalization
Continual	Native Create-Skill / Modify-Skill over a skill directory	CREATE uses an absolute budget and library-level comparison; MODIFY uses relative growth	No validation reject of the final regularized candidate; final snapshot is written back

Appendix

Table 9: Framework-specific integration of SkillEvoReg .

Framework	CCV case construction	Regression signal	Repair scope
SkillOpt	Dataset-specific candidate-conditioned attacks over workbook, context, or reasoning inputs	Hard incumbent pass and candidate failure, with clean-candidate and dataset-specific attribution controls	Candidate delta and authorized related atoms
SkillEvol	Original instruction plus a bounded robustness appendix	Qualified clean pair, old attacked score $\geq 0.8$ , and attacked score drop $\geq 0.05$	Changed skill and implicated local regions
Continual	Answer-preserving transformation or validated boundary case	Incumbent passes and candidate either fails or drops by more than $0.1$	Responsible skill paths

Appendix

Table 10: Framework-specific CCV instantiations used in our experiments.

Task block	1-pass	2-pass	SkillEvoReg
T1–T3, Replay	0.8557	0.8588	0.8558
T1, Canonical replay	0.8604	0.8625	0.8807
T2, Enriched replay	0.8484	0.8439	0.8381
T3, Variant replay	0.8584	0.8699	0.8487
T4–T6, Deployment	0.8299	0.8263	0.8269
T4, Context shift	0.8334	0.8447	0.8227

Appendix

Table 11: Mean normalized SkillEvolBench verifier scores, averaged over two frozen evaluations. Scores range from 0 to 1.

Domain	Method	EM	F1	Num.	Prog.	Rubric	Raw	Norm.
Mathematics	Baseline	0.641	–	0.909	–	–	0.700	0.700
Mathematics	SkillEvoReg	0.615	–	0.818	–	–	0.660	0.660
Law	Baseline	0.853	–	–	0.800	0.615	0.795	0.795
Law	SkillEvoReg	0.824	–	–	0.800	0.774	0.810	0.810
Finance	Baseline	0.364	0.488	0.583	1.000	0.513	0.541	0.541
Finance	SkillEvoReg	0.545	0.392	0.417	0.600	0.492	0.478	0.478

Appendix

Table 12: ContinualSkillBench online learning performance on tasks 1–50, reported in the source benchmark’s format. Entries are rewards in

[0,1]

from a single sequential learning trajectory.

Domain	Method	EM	F1	Num.	Prog.	Rubric
Mathematics	Baseline	–	–	73.08	75.00	61.80
Mathematics	SkillEvoReg	–	–	73.08	75.00	65.00
Law	Baseline	100.00	–	–	60.00	60.79
Law	SkillEvoReg	100.00	–	–	70.00	66.27
Finance	Baseline	–	21.73	–	87.50	69.59
Finance	SkillEvoReg	–	22.86	–	87.50	70.09

Appendix

Table 13: Mean frozen reward by the source benchmark’s evaluator categories. Every score averages two repeats.

Benchmark	Method	Mean	Median	Trimmed mean
SpreadsheetBench	SkillOpt	12,572	10,099	11,266
SpreadsheetBench	SkillEvoReg	8,865	7,328	7,880
SearchQA	SkillOpt	6,280	6,245	6,257
SearchQA	SkillEvoReg	2,759	2,713	2,726
LiveMath	SkillOpt	8,186	7,426	7,789
LiveMath	SkillEvoReg	5,028	4,437	4,637

Appendix

Table 14: Frozen-evaluation model-token usage. Values are total input-plus-output tokens per task, pooled over two repeats; trimmed means remove the top and bottom

10\%

of tasks.

Interface	SkillOpt	SkillEvolBench	ContinualSkillBench
Skill state	Single evolving instruction document.	Multi-skill library containing localized skill files.	Continually evolving skill library.
Native update	Ranked textual edits generated from the execution trajectory.	Localized skill patch proposed by the native skill author.	Explicit Create or Modify operation.
Candidate representation	Updated candidate document and corresponding textual changes.	Proposed patch over one or more localized skill regions.	Raw post-update skill or library snapshot produced by the native operation.
Delta extraction	Compare the candidate document with the temporary skill view used during update generation.	Recover the changed skill and source patch from the native proposal.	Compare pre-update and raw post-update snapshots and identify affected skill paths.
Dropout reconciliation	Restore masked atoms from the incumbent document and apply only the updater-generated change.	Restore masked library content and reconcile the native patch with the complete incumbent library.	Restore the pre-update library and reconcile the raw Create / Modify result with the restored snapshot.
Local regularization scope	Candidate atoms plus related existing atoms.	Changed skill/source delta plus related local content.	Affected skill paths, with whole-skill rewriting permitted within an implicated path.

Appendix

Table 15: Integration of SkillEvoReg with the three native skill-update interfaces. The regularization mechanisms remain largely unchanged; the main differences concern how the native framework represents, exposes, and writes back a candidate update.

Quantity	Observed change
Original candidate scope	$4/14$ major sections affected
Whole-skill repair scope	$14/14$ major sections changed
Candidate $\rightarrow$ repaired text	$18{,}313\rightarrow 9{,}598$ characters ( $-47.6\%$ )
Persistent skill length	$2{,}679\rightarrow 1{,}450$ words ( $-45.9\%$ )
Inherited bullets preserved verbatim	$6/47$
Inherited bullets not preserved verbatim	$41/47$ ( $87.2\%$ )

Appendix

Table 16: Diagnostic failure case illustrating the scope and behavioral consequences of unrestricted whole-skill rewriting. Textual rewrite statistics characterize rewrite scope rather than semantic rule deletion.

Branch	Checkpoint	Train	Validation	Test
Shared	Initial	32.50	37.00	38.75
	E1	44.50	52.00	52.50
	E2	48.00	55.50	56.50
	E3	48.00	55.50	56.50
	E4	48.50	60.00	58.50
	E5	48.50	60.00	58.50

Appendix

Table 17: SpreadsheetBench shared-prefix continuation. E1–E6 are common to both branches; E7–E8 differ after independent update sampling. Test scores average two frozen evaluations.

Framework	Readable context	Writable scope	Enforcement
SkillOpt	Complete incumbent skill and local context	Explicit source-delta regions	Unknown or duplicate region IDs are rejected; linked historical atoms and merge effects are determined by the runtime.
SkillEvolBench	Complete same-family skills as read-only context	Selected regions within implicated skills	Unknown skill/region pairs and out-of-scope edits are rejected before native commit.
ContinualSkillBench MODIFY	Source skill and bounded matched context	Explicit editable windows	Unknown, stale, overlapping, or out-of-scope windows are rejected before application.
ContinualSkillBench CREATE	New skill and retrieved historical skills	Authorized skill paths	Unauthorized paths are rejected; historical paths can change only under an allowed merge.
ContinualSkillBench CCV repair	Probe and responsibility-skill contents	Authorized responsibility paths	Returned paths and the actual post-repair modified-path set are both checked before write-back.

Appendix

Table 18: Writable-scope enforcement across host frameworks. The model may inspect broader context, but the adapter determines which regions, windows, or skill paths are eligible for write-back.

LLM Agent Skill Learning

Large language model agent skill learning studies how agents acquire, represent, reuse, and refine procedures for multi-step tasks.

See all 398 papers

Skill1: Unified Evolution of Skill-Augmented Agents via Reinforcement Learning
May 7, 2026Yaorui Shi, Yuxin Chen, Zhengxi Lu +6LLM Agent Skill Learning LLM Agent Skill Retrieval
University of Science and Technology of China · 2Meituan · 3National University of Singapore +2
SkillEvolver: Skill Learning as a Meta-Skill
May 11, 2026Genrui Zhang, Erle Zhu, Jinfeng Zhou +2LLM Agent Self-Improvement LLM Agent Skill Learning
2Beijing Jiaotong University · 1Tsinghua University
SkillAdam: Stable and Efficient Skill Evolution for Agents
Sep 8, 2026Gaoyuan Li, Meihao Fan, Yizhe Liu +7Large Language Model-Guided Optimization LLM Agent Self-Improvement
Renmin University of China · Tencent

May 7, 2026J/K move · Enter open · S save

Skill1: Unified Evolution of Skill-Augmented Agents via Reinforcement Learning

Yaorui Shi, Yuxin Chen, Zhengxi Lu, Yuchun Miao+5

University of Science and Technology of China · 2Meituan · 3National University of Singapore+2

LLM Agent Skill Learning LLM Agent Skill Retrieval Policy Learning RL for LLM Agents Memory-Augmented Language Model Agents

Open paper AskPDF arXiv

A persistent skill library allows language model agents to reuse successful strategies across tasks. Maintaining such a library requires three coupled capabilities. The agent selects a relevant skill, utilizes it during execution, and distills new skills from experience. Existing methods optimize these capabilities in isolation or with separate reward sources, resulting in partial and conflicting evolution. We propose Skill1, a framework that trains a single policy to co-evolve skill selection, utilization, and distillation toward a shared task-outcome objective. The policy generates a query to search the skill library, re-ranks candidates to select one, solves the task conditioned on it, and distills a new skill from the trajectory. All learning derives from a single task-outcome signal. Its low-frequency trend credits selection and its high-frequency variation credits distillation. Experiments on ALFWorld and WebShop show that Skill1 outperforms prior skill-based and reinforcement learning baselines. Training dynamics confirm the co-evolution of the three capabilities, and ablations show that removing any credit signal degrades the evolution.

Abstract

LLM Agent Skill Learning

Explore similar work

Skill1: Unified Evolution of Skill-Augmented Agents via Reinforcement Learning

SkillEvolver: Skill Learning as a Meta-Skill

SkillAdam: Stable and Efficient Skill Evolution for Agents