Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills

Jingwei Ni; Yihao Liu; Xinpeng Liu; Yutao Sun; Mengyu Zhou; Pengyu Cheng; Dexin Wang; Erchao Zhao

cs.AIMar 26, 2026

Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills

Authors: Jingwei Ni, Yihao Liu, Xinpeng Liu, Yutao Sun, Mengyu Zhou, Pengyu Cheng, Dexin Wang, Erchao Zhao,

Organizations: ETH Zürich · University of Zurich · Peking University · Zhejiang University ·

Abstract

Large Language Model (LLM) agents increasingly rely on domain-specific skills, yet manually authoring such skills does not scale, and skills generated purely from parametric knowledge often miss critical operational pitfalls. We introduce Trace2Skill, a framework that consolidates broad execution trajectories in parallel into a unified skill directory through inductive reasoning over agent experience. Trace2Skill supports both deepening existing human-written skills and creating useful skills from weak LLM-generated drafts. Experiments demonstrate the effectiveness of Trace2Skill across diverse domains, including office workflows, math reasoning, and vision QA. Importantly, the evolved skills are not merely memorized artifacts of the trajectories used to create them: they often transfer across model scales, across model families, and to out-of-distribution settings. For example, skills evolved from Qwen3.5-35B trajectories improve a Qwen3.5-122B agent by up to $57.65$ percentage points on WikiTableQuestions. Further analyses show that Trace2Skill outperforms sequential skill editing and ReasoningBank-style retrieval memories, compresses recurring failures and workarounds into standard operating procedures (SoPs), and yields portable skills that can be reused without parameter updates or test-time retrieval.

Figures & tables

Figure 1: Left: sequential online skill evolution edits the skill after each incoming trace. Right: Trace2Skill analyzes many traces in parallel and hierarchically consolidates recurring lessons into one portable skill.

Figure 2: Overview of Trace2Skill’s three-stage pipeline: (1) roll out a frozen agent to collect labeled success and failure trajectories, (2) propose trajectory-level skill patches in parallel with separate error and success analysts, and (3) hierarchically merge all patches into one portable skill directory.

	Skill User: Qwen3.5-122B-A10B					Skill User: Qwen3.5-35B-A3B
	SpreadsheetBench			OOD		SpreadsheetBench			OOD
Condition	Vrf $\uparrow$	Soft $\uparrow$	Hard $\uparrow$	WikiTQ $\uparrow$	HiTab $\uparrow$	Vrf $\uparrow$	Soft $\uparrow$	Hard $\uparrow$	WikiTQ $\uparrow$	HiTab $\uparrow$	Avg $\uparrow$
Baseline (absolute scores)
No Skill	27.67	28.90	17.57	21.50	14.42	19.00	18.00	4.60	13.33	19.12	18.19
Human-Written	48.33	36.30	17.03	74.68	41.31	9.67	13.03	3.37	9.02	5.30	26.93
Parametric	26.17	36.60	17.50	23.73	17.36	20.17	13.70	3.87	20.14	13.94	19.23

Table 1: Main spreadsheet results across skill-author (evolves the skill) and skill-user (runs it at inference) models. SpreadsheetBench reports Vrf (verified), Soft (problem pass rate), and Hard (task pass rate); WikiTQ and HiTab are out-of-distribution (OOD) table-QA transfer, and

\mathrm{Avg}{=}(\mathrm{ID}_{\mathrm{avg}}{+}\mathrm{OOD}_{\mathrm{avg}})/2

equally weights in-distribution (ID, SpreadsheetBench) and OOD. Reference rows are absolute scores; evolved rows are signed deltas from the Human-Written (Deepening) or Parametric (Creation) baseline. Bold marks the largest absolute score per column. Per-seed standard deviations are in Table 7 .

Figure 3: Cross-family generalization on SpreadsheetBench. Gemma-4-31B-it and GPT-5.5-high each (i) evolve a skill from their own traces and (ii) run Qwen3.5-authored skills.

Table 5

	Skill User: 122B			Skill User: 35B
Condition	Vrf $\uparrow$	Soft $\uparrow$	Hard $\uparrow$	Vrf $\uparrow$	Soft $\uparrow$	Hard $\uparrow$	Time $\downarrow$
Seq- $B{=}4$	59.00	40.63	20.63	26.17	22.37	7.47	$\sim$ 15 min
Seq- $B{=}1$	61.83	44.40	25.40	26.00	23.83	10.57	$\sim$ 60 min
Parallel (ours)	65.83	46.60	27.43	27.00	22.20	8.20	$\sim$ 3 min

Table 4: Parallel consolidation versus sequential editing on SpreadsheetBench (same trace pool, +Error). Parallel outperforms Seq in effectiveness and efficiency. Table 9 and Table 11 show similar trend in math/VQA.

Setting	User	Vrf $\uparrow$	Soft $\uparrow$	Hard $\uparrow$
ReasoningBank ( Ouyang et al., 2026 )	122B	56.00	40.10	21.30
ReasoningBank ( Ouyang et al., 2026 )	35B	20.50	17.30	4.97
Human-Written +Combined (ours)	122B	69.83	47.17	29.53
Human-Written +Combined (ours)	35B	29.67	18.80	5.73

Table 5: Trace2Skill outperforms ReasoningBank-style retrieval on SpreadsheetBench (same trajectory pool). Table 9 and Table 11 show similar trend in math/VQA.

Skill	Evolve	Vrf	Soft	Hard	WikiTQ	HiTab
No Selection (+Error)	68.00	65.83	46.60	27.43	76.30	43.45
BO Selection (+Error)	72.83	69.83	47.27	28.90	77.51	43.38

Table 6: BO-selected +Error patches versus applying all of them (No Selection), in the 122B Deepening spreadsheet setting. Evolve is the accuracy on evolving-set.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

	Skill User: Qwen3.5-122B-A10B					Skill User: Qwen3.5-35B-A3B
	SpreadsheetBench			OOD		SpreadsheetBench			OOD
Condition	Vrf $\uparrow$	Soft $\uparrow$	Hard $\uparrow$	WikiTQ $\uparrow$	HiTab $\uparrow$	Vrf $\uparrow$	Soft $\uparrow$	Hard $\uparrow$	WikiTQ $\uparrow$	HiTab $\uparrow$	Avg $\uparrow$
Baseline (absolute scores)
No Skill	3.75	0.90	0.70	0.81	0.67	1.80	2.77	1.25	1.27	5.28	0.42
Human-Written	8.10	1.39	0.40	0.65	0.71	2.93	0.57	0.40	0.36	2.68	0.46
Parametric	0.58	1.04	0.75	6.52	8.93	1.44	1.56	0.50	13.32	18.31	3.28

Appendix

Table 7: Standard deviations across seeds 41, 42, and 43 for the main spreadsheet results in Table 1 . Reference rows are standard deviations of absolute scores; evolved rows are standard deviations of the paired deltas defined in Table 1 . The summary metric Avg is stable (standard deviation

\leq 3.4

in every row), and the mean gains in Table 1 stay large relative to the per-cell spread, so the comparisons are robust. The higher variance in individual columns (Vrf and the OOD WikiTQ/HiTab transfer) is expected rather than a sign of instability: each score aggregates long agentic rollouts that often exceed 30 turns of environment interaction, so single-benchmark run-to-run stochasticity accumulates while averaging out in Avg. For conciseness, we omit additional standard-deviation tables for the paper’s other results, whose deviations are similarly small: std ranges for spreadsheet tasks are similar to this table. Std ranges of DAPO-Test, AIME’26, and DocVQA ANLS are 1.1-2.4, 1.8-3.4, and 0.034-0.139 correspondingly.

Table 11

Table 12

Figure 6: Cross-model trace induction on SpreadsheetBench-Vrf (pass rate, %). Gemma-4 and GPT-5.5-high author skills from Qwen traces, which are then run by the Qwen3.5-122B user.

	Skill User: Qwen3.5-122B-A10B					Skill User: Qwen3.5-35B-A3B
	SpreadsheetBench			OOD		SpreadsheetBench			OOD
Condition	Vrf	Soft	Hard	WikiTQ	HiTab	Vrf	Soft	Hard	WikiTQ	HiTab	Avg
Skill Author: Qwen3.5-122B-A10B
Deepening
+Error (ours)	65.83	46.60	27.43	76.30	43.45	36.67	22.47	6.23	18.28	14.85	36.21
+Error LLM	67.00	43.93	25.23	39.81	36.68	25.00	22.43	6.23	11.24	9.74	28.00

Appendix

Table 12: Agentic error analysis (+Error) versus single-call +Error LLM, across Deepening and Creation and all 122B/35B author–user pairs. Vrf/Soft/Hard, the OOD splits (WikiTQ/HiTab), and Avg are as in Table 1 . Bold marks the better of the two methods per column.

Domain	Source traces	Held-out evaluation	Base	Trace2Skill	Gain
PDF	VRDU Registration Forms	VAREX Flat validation	76.9%	85.3%	+8.4 pp
PPTX	TSBench training traces	TSBench deck-disjoint OOD	72.5%	88.8%	+16.3 pp
DOCX	Generated document-operation tasks	OfficeBench DOCX holdout	79.7%	87.5%	+7.8 pp

Appendix

Table 13: Broader document-agent transfer. Each row evolves an official Anthropic document skill from source traces and evaluates on a held-out target domain. Base and Trace2Skill are target pass rates (%); Gain is their difference in percentage points (pp).

Panel of Figure 7 — Figure 7: Head-to-head comparison on SpreadsheetBench-Verified (Vrf, pass rate %) with all systems using the same open base model (Qwen3.5-122B-A10B) and the held-out 200:400 test slice. Bars cover the No Skill floor, the Human-Written starting skill (dashed line), the three reproduced concurrent systems, and Trace2Skill Deepening. EvoSkill’s two bars are the same faithful reproduction differing only in agent harness (Claude Code, following original paper, vs. React, matching our setting).

User/Trace source	Deepening			Creation
User/Trace source	Base	skill-creator	Trace2Skill +Error	Base	skill-creator	Trace2Skill +Error
122B/122B	48.33	27.33	65.83	26.17	26.67	49.00
122B/35B	48.33	19.50	65.00	26.17	18.33	27.17
35B/35B	9.67	18.50	27.00	20.17	17.67	24.00
35B/122B	9.67	28.00	36.67	20.17	27.33	28.83
Average	29.00	23.33	48.63	23.17	22.50	32.25

Appendix

Table 14: SpreadsheetBench-Verified (Vrf) comparison with Anthropic’s skill-creator baseline (pass rate, %). For each skill-user/trace-source pair, skill-creator is compared with the corresponding Base skill and Trace2Skill +Error under Deepening and Creation; the final row averages each column.

LLM Agent Skill Learning

Large language model agent skill learning studies how agents acquire, represent, reuse, and refine procedures for multi-step tasks.

See all 379 papers

SkillGen: Verified Inference-Time Agent Skill Synthesis
May 9, 2026Yuchen Ma, Yue Huang, Han Bao +5LLM Agent Self-Improvement LLM Agent Skill Learning
Munich Center for Machine Learning, LMU Munich · University of Notre Dame · Microsoft Research
Break It Down, Pass It On: Cross-Task Skill Transfer in LLM Agents
Aug 20, 2026Yiyang Feng, Biddut Sarker Bijoy, Niranjan Balasubramanian +1LLM Agent Skill Learning LLM Agent Skill Evaluation
Computer Science Department, Stony Brook University, Stony Brook, USA
SkillAxe: Sharpening LLM-Authored Agent Skills Through Evaluation-Guided Self-Refinement
Jun 9, 2026Srishti Gautam, Arjun Radhakrishna, Sumit GulwaniLLM Self-Refinement LLM Agent Self-Improvement
Microsoft

May 9, 2026J/K move · Enter open · S save

SkillGen: Verified Inference-Time Agent Skill Synthesis

Yuchen Ma, Yue Huang, Han Bao, Haomin Zhuang+4

Munich Center for Machine Learning, LMU Munich · University of Notre Dame · Microsoft Research

LLM Agent Self-Improvement LLM Agent Skill Learning LLM Agents

Open paper AskPDF arXiv

Skills are a promising way to improve LLM agent capabilities without retraining, while keeping the added procedure reusable and controllable. However, high-quality skills are still largely written by hand. We introduce SkillGen, a multi-agent framework that synthesizes a single auditable skill from trajectories generated by a base agent. The output is a human-readable artifact that can be inspected before use. Rather than merely summarizing trajectories, SkillGen leverages contrastive induction over both successful and failed trajectories to identify reusable success patterns, recurring failure modes, and behaviors that appear in nearby successes but are missing from failures. SkillGen then generates candidate skills and iteratively refines the skill. A key novelty in SkillGen is that we model agent skills as interventions to empirically verify the net effect of skills on the overall performance. Specifically, we compare outcomes on the same instances with and without the skill, so that we account for both repairs (cases where the skill fixes a baseline failure) and regressions (cases where the skill breaks a baseline success). Across a broad range of agents and datasets, SkillGen consistently improves held-out performance, outperforms existing skill-generation baselines, and produces skills that transfer across models.

Abstract

LLM Agent Skill Learning

Explore similar work

SkillGen: Verified Inference-Time Agent Skill Synthesis

Break It Down, Pass It On: Cross-Task Skill Transfer in LLM Agents

SkillAxe: Sharpening LLM-Authored Agent Skills Through Evaluation-Guided Self-Refinement