WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution

Liyan Tang; Cyrus Rashtchian; Chun-Sung Ferng; Andrew Tomkins; Da-Cheng Juan; Tu Vu

cs.AIAug 27, 2026

WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution

Authors: Liyan Tang, Cyrus Rashtchian, Chun-Sung Ferng, Andrew Tomkins, Da-Cheng Juan, Tu Vu

Organizations: Google Research · Virginia Tech

Abstract

Agent skills package specialized knowledge and workflows into reusable resources that extend AI agent capabilities. Recent work automatically discovers such skills from agent experience, which enables agents to progressively adapt through interaction. However, the insights that guide skill development typically remain scattered across optimization histories, limiting their systematic reuse across iterations. We introduce WikiSkill, a framework that co-evolves agent skills with a persistent knowledge base (wiki). At a high level, WikiSkill separates raw execution experience, accumulated knowledge, and executable skills, while continuously consolidating experience into the wiki, which subsequent skill updates can build on. Across diverse benchmarks and models, WikiSkill consistently outperforms state-of-the-art skill-evolution methods and improves over no-skill baselines in most model-benchmark settings. We find that skill evolution complements model scaling: larger models generally benefit more from evolved skills, while smaller models with skills can outperform substantially larger models without them. We also find that evolved skills transfer effectively across models and model families, and skills evolved by other models can outperform self-evolved skills. Finally, our ablation studies confirm that persistent knowledge accumulation in the wiki is critical for effective skill evolution. These results demonstrate the benefits of systematically accumulating and refining agent experience for developing reusable and transferable skills.

Figures & tables

Figure 1: WikiSkill consistently improves over both the no-skill baseline and existing skill-evolution methods. Interestingly, its advantage becomes more pronounced for stronger models. We report average accuracy across the evaluated benchmarks for each model using no skills or skills evolved by EvoSkill, SkillOpt, and WikiSkill (see Table 1 for details).

Figure 2: Overview of the WikiSkill framework. The agent workspace is structured into three layers: immutable execution traces (Raw Layer), a persistent knowledge base that compounds across iterations (Wiki Layer), and active procedural instructions (Skills Layer). In each evolutionary loop, the Inference Agent runs rollouts (injecting active skills but restricting Wiki access), the Wiki Maintainer consolidates traces into the Wiki, and the Skill Proposer (with the ReAct mechanism) suggests updates while the Wiki is retained across all iterations.

Model	Method	LiveMath	SealQA	SpreadSheet	OfficeQA	ALFWorld	Avg.
Qwen-3.5-4B	No skill	29.1	32.5	14.6	30.2	24.4	26.2
	Trace2Skill	31.5	37.6	17.5	31.0	42.8	32.1
	EvoSkill	41.7	37.3	18.6	29.5	41.5	33.7
	SkillOpt	48.7	33.3	14.0	34.5	45.3	35.2
	WikiSkill	49.7	39.4	21.1	28.5	53.7	38.5
Qwen-3.5-9B	No skill	28.2	26.3	24.3	35.9	34.7	29.9

Table 1: Method comparison across inference models and test sets . Each horizontal block evaluates a specific inference model without skills (No skill) and with skills developed by different skill-evolution methods. To ensure a fair comparison, all skill-evolution methods start with an empty skill set, and evolved skills are injected into the Inference Agent’s prompt at inference time . All reported scores are the average test performance across three independent runs of the full evolution process. Our method ( WikiSkill ) is highlighted. Bold indicates the best performance for each dataset; multiple bold results indicate methods that are not significantly different from the best under a paired bootstrap test with 1,000 iterations (

p<0.05

Model	Skill Source	LiveMath	SealQA	SpreadSheet	OfficeQA	ALFWorld
Qwen-3.5-4B	None	29.1	32.5	14.6	30.2	24.4
	Qwen-3.5-4B	49.7	39.4	21.1	28.5	53.7
	Qwen-3.6-27B	59.7	38.8	33.0	25.4	57.0
	Gemini-3.5-Flash	62.6	37.3	23.0	32.2	-
Qwen-3.5-9B	None	28.2	26.3	24.3	35.9	34.7
	Qwen-3.5-4B	61.0	40.4	25.0	40.3	69.2

Table 2: Cross-model skill transfer results. We evaluate inference models using no skills (None) and skills evolved by WikiSkill with Qwen-3.5-4B, Qwen-3.6-27B, and Gemini-3.5-Flash as source models. Skills are injected into the Inference Agent’s system prompt at inference time. Highlighted rows indicate self-evolved skills, where the inference model and skill source are the same. The highest performance per benchmark within each model block is bolded. ‘

-

’ indicates that the source model reached 100% validation performance before skill evolution, so no skill was evolved.

WikiSkill Components		Benchmarks
Inference Agent	Skill Proposer	LiveMath	SealQA	SpreadSheet	OfficeQA	Avg.
Wiki Access	Wiki Access
No skill		33.0	29.4	50.5	48.6	40.4
		43.8	42.0	44.4	51.0	45.3
		51.3	38.4	49.9	55.2	48.7
		64.8	42.8	80.2	55.6	60.9

Table 3: Ablation study on WikiSkill using Gemini-3.5-Flash. We evaluate performance across benchmarks under four configurations that vary whether the Inference Agent and Skill Proposer have wiki access during skill evolution. When the Skill Proposer has no Wiki access, we also remove the Wiki Maintainer, eliminating persistent knowledge accumulation across iterations. The bottom row represents our default WikiSkill configuration.

Figure 3: Case study of Wiki-guided skill evolution on ALFWorld (Qwen-3.6-27B). The persistent Wiki Layer compiles cross-iteration patterns, an audit trail of past proposal diffs and acceptance decisions, and chronological history. Informed by the rejection of the skill proposal at Iteration 0, the proposer synthesizes the accepted skill update at Iteration 1, and later refines it with new pattern evidence. File contents are simplified for clarity.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Category	Early (Iter 0–1)	Mid (Iter 2–4)	Late (Iter 5–7)
By Model
Qwen-3.5-4B	39%	39%	21%
Qwen-3.5-9B	52%	30%	19%
Qwen-3.6-27B	43%	40%	17%
Gemma-4-31B	52%	37%	11%
Gemini-3.5-Flash	50%	46%	4%

Appendix

Table 5: Distribution of accepted skill updates across evolution iterations, grouped by model (top) and benchmark (bottom). Percentages indicate the proportion of accepted updates that occur during each stage of evolution.

Benchmark	Interaction	Train	Val	Test	Environment Tools
LiveMath	Single-Step	35	18	124	None (Direct Reasoning)
SealQA	Multi-Step	16	10	85	web_search , read_file
SpreadSheet	Multi-Step	80	40	280	bash
OfficeQA	Multi-Step	50	24	172	glob , grep , read
ALFWorld	Multi-Step	39	18	134	Admissible Actions

Appendix

Table 6: Benchmark statistics, data splits, interaction modes, and available environment tools.

Framework	Per-Iteration Formula	Complexity
Trace2Skill	$N_{\text{train}}+\left(1+\frac{1}{c-1}\right)\frac{N_{\text{train}}}{B}+1$	$\mathcal{O}\left(N_{\text{train}}+\frac{N_{\text{train}}}{B}\right)$
EvoSkill	$\frac{2N_{\text{train}}}{B}$	$\mathcal{O}\left(\frac{N_{\text{train}}}{B}\right)$
SkillOpt	$\frac{K_{\text{opt}}\cdot N_{\text{train}}}{B}$	$\mathcal{O}\left(\frac{N_{\text{train}}}{B}\right)$
WikiSkill	$(1+T_{\text{ReAct}})\frac{N_{\text{train}}}{B}$	$\mathcal{O}\left(\frac{N_{\text{train}}}{B}\right)$

Appendix

Table 7: Comparison of optimizer API call complexity per evolution iteration across self-improving agent frameworks.

N_{\text{train}}=|\mathcal{D}_{\text{train}}|

denotes the number of training tasks,

B

denotes the batch size,

T_{\text{ReAct}}

denotes the number of interactive ReAct reasoning turns used by the Skill Proposer agent in WikiSkill,

K_{\text{opt}}

denotes the number of reflection and merging calls per step in SkillOpt, and

c

denotes the reduction-tree branching factor in Trace2Skill. WikiSkill uses

B=N_{\text{train}}

across all datasets. In full-batch mode, WikiSkill’s optimizer call count is independent of training set size

N_{\text{train}}

, requiring

1+T_{\text{ReAct}}

optimizer calls per iteration.

LLM Agent Skill Learning

Large language model agent skill learning studies how agents acquire, represent, reuse, and refine procedures for multi-step tasks.

See all 398 papers

SkillCommit: Evolving Agent Skills through Behaviorally Validated Scope Expansion
Aug 15, 2026Yu He, Weikai YangLLM Agent Self-Improvement LLM Agent Skill Learning
Nanjing University · The Hong Kong University of Science and Technology (Guangzhou)
Skill1: Unified Evolution of Skill-Augmented Agents via Reinforcement Learning
May 7, 2026Yaorui Shi, Yuxin Chen, Zhengxi Lu +6LLM Agent Skill Learning LLM Agent Skill Retrieval
University of Science and Technology of China · 2Meituan · 3National University of Singapore +2
SkillNEO: Non-Elitist Evolutionary Optimization of Agent Skills
Oct 14, 2026Enze Ma, Chunyu Miao, Yankai Chen +14LLM Agent Skill Learning LLM Agent Self-Improvement
University of Illinois at Chicago · Mohamed bin Zayed University of Artiﬁcial Intelligence · Independent Researcher +3

Aug 15, 2026J/K move · Enter open · S save

SkillCommit: Evolving Agent Skills through Behaviorally Validated Scope Expansion

Yu He, Weikai Yang

Nanjing University · The Hong Kong University of Science and Technology (Guangzhou)

LLM Agent Self-Improvement LLM Agent Skill Learning LLM Agents Continual Learning for LLM Agents

Open paper AskPDF arXiv

Large language model (LLM) agents can continually improve without parameter updates by converting historical experience into reusable procedural knowledge. However, existing methods often consolidate experience based on semantic similarity or LLM judgments, which may merge superficially related but behaviorally incompatible strategies and thereby degrade performance. To address the issue, we propose SkillCommit, an online skill evolution framework that continuously transforms experience into a hierarchical library of reusable skills. Rather than generalizing from individual experiences immediately, SkillCommit first preserves each experience as an instance-specific patch, retaining its locally validated behavior. As patches accumulate, SkillCommit abstracts those sharing a common behavioral mechanism into higher-level skills. Specifically, for each incoming skill, embedding-based retrieval first identifies candidate related skills. Cross-instance replay and an LLM-based mechanism check determine whether these skills transfer across cases and share a common underlying mechanism. Candidates that pass both checks are abstracted into a higher-level skill and committed only if it preserves the validated behavior of all constituent skills. Experiments on RuleArena, OpenExempt and KOR-Bench demonstrate that SkillCommit consistently improves agent performance across diverse domains. Moreover, the learned skills transfer across model scales and families, enabling cross-model experience transfer.

Abstract

LLM Agent Skill Learning

Explore similar work

SkillCommit: Evolving Agent Skills through Behaviorally Validated Scope Expansion

Skill1: Unified Evolution of Skill-Augmented Agents via Reinforcement Learning

SkillNEO: Non-Elitist Evolutionary Optimization of Agent Skills