Hierarchical Skill Evolution for Coding Agent

Jun Gao; Yun Peng; Changhai Zhou; Yuhan Zhu; Xiangyu Zeng; Shiyang Zhang Yuhua Zhou; Luozheng Qin; Qian Qiao

cs.CLOct 26, 2026

Hierarchical Skill Evolution for Coding Agent

Authors: Jun Gao, Yun Peng, Changhai Zhou, Yuhan Zhu, Xiangyu Zeng, Shiyang Zhang Yuhua Zhou, Luozheng Qin, Qian Qiao,

Abstract

Skills provide agents with reusable experience to guide future decisions without modifying the underlying model. However, coding trajectories are often entangled with task- and repository-specific contexts, making it challenging to distill procedural skills that generalize across diverse coding environments. To address this challenge, we present Hierarchical Skill Evolution (HSE), which progressively distills procedural knowledge from coding trajectories through hierarchical evidence aggregation. HSE first extracts task-level evidence from individual trajectories, then aggregates recurring evidence within the same repository to form repository-specific skill candidates, and finally consolidates consistent patterns across repositories into transferable skills across repositories. The evolution is driven by an external writer-evaluator loop that generates and assesses skills while keeping the underlying coding model frozen. The loop externally updates the writer and evaluator policies to improve subsequent skill generation and assessment. This hierarchy progressively filters task-specific details while preserving procedures that generalize across diverse coding environments. Successful trajectories yield reusable guidance for future tasks, while failed trajectories reveal recurring failure modes that guide avoidance and recovery. We evaluate HSE on SWE-bench Verified and DeepSWE, using GLM-5.2 across Pi and Claude Code harnesses and GPT-5.5 and Opus 4.8 with Pi. Across both benchmarks, 10 skills evolved solely from SWE-Gym improve coding performance across all evaluated settings, demonstrating transfer across benchmarks, coding models, and agent harnesses. We further evolve the skill library from target-benchmark trajectories without verifier feedback, relying solely on the learned evaluator for candidate assessment and achieving additional gains through benchmark-tailored skills.

Figures & tables

Figure 1: Motivating example: django__django-16950 . Panels 1–4 show the failure and the required primary-key boundary, and Panel 5 compares No Skills, SWESkillsBench, and HSE. Appendix C provides the skill documents, patches, and trace evidence.

Figure 2: HSE training pipeline. Task evidence is aggregated within repositories into repository-conditioned candidates and failure patterns, which are further consolidated across repositories into procedural and recovery skills.

Benchmark	Harness	No Skills	SWESkillsBench	GEPA	ReasoningBank	HSE
SWE-bench Verified	Pi	$73.5_{\scriptscriptstyle\pm\,1.10}$	$74.7_{\scriptscriptstyle\pm\,0.31}$	$74.6_{\scriptscriptstyle\pm\,0.40}$	$72.8_{\scriptscriptstyle\pm\,1.20}$	$\mathbf{75.4_{\scriptscriptstyle\pm\,0.20}}$
SWE-bench Verified	CC	$76.1_{\scriptscriptstyle\pm\,1.30}$	$79.7_{\scriptscriptstyle\pm\,0.42}$	$78.9_{\scriptscriptstyle\pm\,0.50}$	$76.6_{\scriptscriptstyle\pm\,1.40}$	$\mathbf{80.6_{\scriptscriptstyle\pm\,0.20}}$
DeepSWE	Pi	$43.1_{\scriptscriptstyle\pm\,2.23}$	$34.5_{\scriptscriptstyle\pm\,0.88}$	$46.6_{\scriptscriptstyle\pm\,1.35}$	$44.0_{\scriptscriptstyle\pm\,2.23}$	$\mathbf{48.7_{\scriptscriptstyle\pm\,0.88}}$
DeepSWE	CC	$44.8_{\scriptscriptstyle\pm\,3.11}$	$38.6_{\scriptscriptstyle\pm\,1.35}$	$49.0_{\scriptscriptstyle\pm\,1.35}$	$46.0_{\scriptscriptstyle\pm\,2.65}$	$\mathbf{52.5_{\scriptscriptstyle\pm\,0.51}}$

Table 1: Resolved rate (%) of static skill methods on SWE-bench Verified and DeepSWE with GLM-5.2. Pi and CC denote the Pi and Claude Code harnesses. Entries report the mean over three runs (Mean@3), with standard deviations shown as subscripts. HSE uses the frozen skill library evolved on SWE-Gym without target-benchmark adaptation. Bold indicates the best mean in each row.

Benchmark	Harness	HSE	Gate 1	Gate 2	Gate 3	Gate 4	Gate 5	Gate 6	Gate 7	Gate 8
SWE-bench Verified	Pi	$75.4_{\scriptscriptstyle\pm\,0.20}$	$81.1_{\scriptscriptstyle\pm\,0.61}$	$85.1_{\scriptscriptstyle\pm\,0.70}$	$86.1_{\scriptscriptstyle\pm\,0.81}$	$86.3_{\scriptscriptstyle\pm\,0.50}$	$87.6_{\scriptscriptstyle\pm\,0.40}$	$88.7_{\scriptscriptstyle\pm\,0.31}$	$89.4_{\scriptscriptstyle\pm\,0.40}$	$90.1_{\scriptscriptstyle\pm\,0.31}$
SWE-bench Verified	CC	$80.6_{\scriptscriptstyle\pm\,0.20}$	$86.4_{\scriptscriptstyle\pm\,0.72}$	$88.7_{\scriptscriptstyle\pm\,1.01}$	$88.9_{\scriptscriptstyle\pm\,0.81}$	$89.8_{\scriptscriptstyle\pm\,0.92}$	$91.7_{\scriptscriptstyle\pm\,0.70}$	$92.4_{\scriptscriptstyle\pm\,0.40}$	$93.0_{\scriptscriptstyle\pm\,0.40}$	$93.9_{\scriptscriptstyle\pm\,0.46}$
DeepSWE	Pi	$48.7_{\scriptscriptstyle\pm\,0.88}$	$55.8_{\scriptscriptstyle\pm\,1.77}$	$60.5_{\scriptscriptstyle\pm\,1.35}$	$61.7_{\scriptscriptstyle\pm\,1.35}$	$68.4_{\scriptscriptstyle\pm\,2.23}$	$69.6_{\scriptscriptstyle\pm\,2.23}$	$70.8_{\scriptscriptstyle\pm\,0.88}$	$71.1_{\scriptscriptstyle\pm\,0.51}$	$71.7_{\scriptscriptstyle\pm\,0.88}$
DeepSWE	CC	$52.5_{\scriptscriptstyle\pm\,0.51}$	$55.8_{\scriptscriptstyle\pm\,2.65}$	$65.8_{\scriptscriptstyle\pm\,2.23}$	$70.8_{\scriptscriptstyle\pm\,1.77}$	$71.7_{\scriptscriptstyle\pm\,0.88}$	$72.9_{\scriptscriptstyle\pm\,1.35}$	$73.7_{\scriptscriptstyle\pm\,1.02}$	$74.9_{\scriptscriptstyle\pm\,0.51}$	$75.8_{\scriptscriptstyle\pm\,0.51}$

Table 2: Resolved rate (%) of target-benchmark adaptation with GLM-5.2. HSE denotes the transferred skill library before target-benchmark adaptation. Gate 1–Gate 8 report the resolved rates achieved by the updated skill library after each evaluator gate. Entries report Mean@3, with standard deviations shown as subscripts.

Benchmark	Harness	No Skills	HSE	w/o Procedural Skills	w/o Recovery Skills
SWE-bench Verified	Pi	73.5	75.4	74.2 ( $-1.2$ )	74.6 ( $-0.8$ )
SWE-bench Verified	CC	76.1	80.6	77.0 ( $-3.6$ )	79.0 ( $-1.6$ )
DeepSWE	Pi	43.1	48.7	45.1 ( $-3.6$ )	43.4 ( $-5.3$ )
DeepSWE	CC	44.8	52.5	47.8 ( $-4.7$ )	46.9 ( $-5.6$ )

Table 3: Resolved rate (%) after removing HSE components. No Skills and HSE repeat the main-table means without standard deviations. Parentheses show the change from the complete HSE pack.

	SWE-bench Verified		DeepSWE
Method	Pi	CC	Pi	CC
HSE	75.4	80.6	48.7	52.5
w/o Hierarchical Aggregation	74.2	78.6	46.9	48.7
w/o Controller Updates	74.8	79.2	46.9	50.4

Table 4: Ablation of hierarchical aggregation and controller updates. All variants use the same SWE-Gym trajectories.

Benchmark	Harness	HSE	Harness	No Skills	HSE	$\Delta_{\mathrm{NS}}$	$\Delta_{\mathrm{in}}$
	In-harness		Transfer harness
SWE-bench Verified	Pi	75.4	CC	76.1	79.2	$+3.1$	$+3.8$
SWE-bench Verified	CC	80.6	Pi	73.5	80.4	$+6.9$	$-0.2$
DeepSWE	Pi	48.7	CC	44.8	49.6	$+4.8$	$+0.9$
DeepSWE	CC	52.5	Pi	43.1	50.4	$+7.3$	$-2.1$

Table 5: Resolved rate (%) for fixed HSE packs across harnesses. No Skills and in-harness HSE use the corresponding means from Table 1 .

\Delta_{\mathrm{NS}}

and

\Delta_{\mathrm{in}}

subtract the row’s transfer-harness No Skills and in-harness HSE scores from transfer-harness HSE, respectively.

Model	Method	SWE-bench Verified	DeepSWE
GPT-5.5	No Skills	82.6	67.2
GPT-5.5	HSE	85.4	73.5
Opus 4.8	No Skills	88.6	59.3
Opus 4.8	HSE	90.2	61.1

Table 6: Cross-model resolved rate (%) with the Pi harness.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

		SWE-bench Verified				DeepSWE
Harness	Method	Generation	Prefilling	Steps	Total	Generation	Prefilling	Steps	Total
Pi	No Skills	2.1K	312.5K	52.4	314.7K	8.3K	2.4M	141.9	2.4M
	SWESkillsBench	3.4K	578.8K	79.2	582.2K	14.8K	4.6M	223.6	4.6M
	HSE	2.5K	382.6K	58.4	385.1K	9.0K	2.8M	151.6	2.8M
CC	No Skills	3.4K	450.6K	59.2	454.0K	12.3K	4.5M	184.2	4.5M
	SWESkillsBench	6.2K	698.7K	79.8	704.9K	17.3K	6.9M	237.3	6.9M

Appendix

Table 7: Mean word-count cost using one trace per task. The reported average injected skill context is 1.5K words for SWESkillsBench (49 skills) and 0.5K words for HSE (eight procedural skills and two recovery skills).

Step	Recorded action	Observation and relevance
N11	Read add_fields() at the referenced-field/default branch.	The code executes setattr(..., None) on the parent instance.
N28	Create and run a non-PK UUID/inline-form reproducer.	The form and formset are valid, but saving raises IntegrityError: NOT NULL constraint failed: __main___thing.id .
N56	Run model-form and formset suites after an early repair.	The recorded result is FAILED (failures=4, errors=1) ; one visible failure checks that the inline field’s initial is None.
N65	Replace the early change with the original default-field clearing block.	The edit succeeds. This is an actual correction after failure, not an unrevised first guess.
N78	Construct InlineForeignKeyField before the default-field branch and replace instance clearing with form.fields[name].initial = None .	The edit succeeds. No PK/non-PK branch is added; both parent key types remain populated.
N91–N92	Add an alternate-key save regression test and run the UUID suite.	Ran 7 tests in 0.011s ; OK .

Appendix

Table 8: Selected No Skills events, in their original order.

Step	Recorded action	Observation and relevance
S57	Run an admin/inline-form reproducer and print parent UUID state.	UUID is present after parent-form validation but None after formset validation; saving raises a NOT NULL error.
S65	Run the existing UUID suite after an early edit.	Six tests run, four fail. Failures include expectations that the inline field’s initial be None.
S72	Move field construction earlier and clear only its initial .	The successful edit removes setattr(..., None) for both key types. Its comment says the parent instance must not be modified.
S74	Re-run the reproducer and existing UUID tests.	The output reports FORMSET SAVED ; all six existing tests pass.
S86–S87	Add an alternate-key save regression test and run the UUID suite.	Seven tests pass. The output explicitly marks the old default-PK submission test ok .
S89	Substitute the original source, run the new test, then restore the fix.	Equality of parent.uuid and default_uuid fails on the original source. This checks the repair, not the effect of removing a skill.

Appendix

Table 9: Selected SWESkillsBench events, in their original order.

Step	Recorded action	Observation and relevance
H5	Read the exported procedural skill’s SKILL.md .	The tool returns the procedural skill used in the repair.
H98	Run a reproducer that prints parent field values before and after formset validation.	The generated UUID becomes None; saving fails. Both getattr and pre_save output for the non-PK field are None.
H105	Read add_fields() in models.py .	The output exposes the unconditional default-field reset.
H107	Search repository commit history with git log -p -S .	The returned history shows the earlier expansion of clearing from PKs to referenced fields with defaults.
H109–H111	Restrict resetting to default PKs and re-run the reproducer.	The one-test reproducer passes; the non-PK UUID is retained before and after saving. The repair is not yet complete.
H121	Run the existing UUID suite.	Six tests run; the alternate-key test fails because the inline field’s initial is not None. This is a visible repository-test failure.

Appendix

Table 10: Selected HSE Skills events, in their original order.

Condition	New-issue F2P	Existing-behavior P2P	Verifier tests	Reward
No Skills	1/1	2/3	6/7 pass	0
SWESkillsBench	1/1	2/3	6/7 pass	0
HSE Skills	1/1	3/3	7/7 pass	1

Appendix

Table 11: Required checks and final outcomes for the selected attempts. F2P/P2P refer to benchmark-required checks, not the total number of tests executed by the verifier process.

Level	Object	Role
L1	Task evidence	Organizes trajectory evidence into reproduce, locate, edit, validate, and recover stages.
L2	Repository candidate	Summarizes procedures supported by recurring evidence within one repository.
L3	Transferable skill	Retains success or recovery procedures supported across multiple repositories.

Appendix

Table 12: Three levels of evidence aggregation in HSE.

Benchmark	Harness	Method	Run 1	Run 2	Run 3
SWE-bench Verified	Pi	No Skills	362	367	373
		SWESkillsBench	372	373	375
		GEPA	371	373	375
		ReasoningBank	358	364	370
		HSE	376	377	378
	CC	No Skills	374	380	387

Appendix

Table 13: Per-run resolved counts underlying the static-skill results in Table 1 .

Benchmark	Harness	HSE	Gate 1	Gate 2	Gate 3	Gate 4	Gate 5	Gate 6	Gate 7	Gate 8
SWE-bench Verified	Pi 1	376	402	422	426	429	436	442	445	449
	Pi 2	377	406	425	431	431	438	443	447	450
	Pi 3	378	408	429	434	434	440	445	449	452
	CC 1	402	428	439	440	444	455	460	463	468
	CC 2	403	433	443	445	450	458	462	465	468
	CC 3	404	435	449	448	453	462	464	467	472

Appendix

Table 14: Per-run resolved counts underlying the target-benchmark adaptation results in Table 2 .

LLM Agent Skill Learning

Large language model agent skill learning studies how agents acquire, represent, reuse, and refine procedures for multi-step tasks.

See all 396 papers

Aug 6, 2026

Learning Globally Reusable Skills for Coding Agents

Automated skill evolution enables Large Language Model (LLM) agents to continuously improve without expensive retraining. However, existing approaches typically treat skill evolution as a sequence of local updates, overlooking relationships among skills and often producing overfitted skill updates that fail to generalize across tasks. We propose GSE, a globalized skill evolution framework that jointly optimizes skill compatibility and skill generalization. To preserve consistency across the skill bank, GSE maintains a Skill Relation Graph (SRG) that explicitly models and co-evolves inter-skill relationships. To improve generalization, GSE performs cluster-based skill consolidation to abstract reusable capabilities from local updates and employs replay-driven verification to prevent overfitting and behavioral regressions. We evaluate GSE on two representative software engineering tasks: bug-revealing test generation and false-positive bug report filtering. Across two state-of-the-art coding agents, OpenHands and mini-SWE-agent, GSE consistently achieves the best precision, recall, and F1-score. Compared with existing evolution techniques, GSE improves precision and recall by 6.1%~34.1% and 31.8%~180.0% for test generation, and by 15.4%~96.4% and 13.1%~19.8% for false-positive filtering. Deployment on an internal industrial agent further yields a 61.4% improvement in F1-score, demonstrating the effectiveness and generalizability of GSE for evolving effective skills.

Chen Yang, Jiashuo Tian, Ziqi Wang +3

Tianjin University · Tsinghua University

Continual Learning for LLM Agents Software Engineering Agents LLM Agent Skill Learning Self-Evolving Agents

Ask PDF

May 25, 2026

CODESKILL: Learning Self-Evolving Skills for Coding Agents

Coding agents produce rich trajectories while solving software-engineering tasks. To enable agent self-evolution, these trajectories can be distilled into reusable procedural skills that compactly encode experience to guide future behavior. However, existing skill construction and maintenance methods often rely on fixed prompts and heuristic update rules, leaving it unclear how knowledge should be selected, abstracted, and maintained to best serve downstream agents. We propose CODESKILL, an LLM-based framework that reformulates skill extraction and skill-bank maintenance as a learnable management policy. CODESKILL extracts multi-granularity procedural skills from coding-agent trajectories, evolves skills with new experience, and maintains a compact skill bank for future task solving. We train CODESKILL with reinforcement learning, using a hybrid reward that combines dense rubric-based skill-quality feedback with sparse verifiable execution feedback from the frozen downstream agent. Experiments on EnvBench, SWE-Bench Verified, and Terminal-Bench 2 show that CODESKILL improves average pass rate by 9.69 over the no-skill baseline and by 4.01 over the strongest prompt-based or memory baseline, while maintaining the skill bank at a stable size during iterative construction.

Yanzhou Li, Yiran Zhang, Xiaoyu Zhang +2

Nanyang Technological University · 2Zhejiang University

Coding Agents LLM Agent Skill Learning LLM Agents RL for LLM Agents Self-Evolving Agents

Ask PDF

Sep 27, 2026

CodeSkill: Latent Skill Abstraction for Long-Horizon Code Agents

Code agents require long-horizon decision-making over complex interaction trajectories. However, existing reinforcement learning (RL) approaches typically optimize behavior at the token level, creating a mismatch between low-level generation and high-level behavioral reasoning. This limitation leads to inefficient exploration and weak credit assignment under sparse rewards. Moreover, while large-scale agent trajectories often contain recurring multi-step behavioral patterns, their noisy token-level representations hinder effective experience reuse. To address these challenges, we propose CodeSkill, a framework that adapts hierarchical latent skill modeling to the code agent domain. CodeSkill first leverages a teacher model to distill both successful and failed trajectories into multi-level textual abstractions. It then integrates temporal variational inference with reinforcement learning to map these discrete semantics into continuous latent variables, while an adaptive boundary mechanism dynamically gates skill transitions based on execution feedback. The learned skills are injected into a frozen LLM policy as latent semantic prefixes, enabling optimization in a compact semantic space rather than over raw token sequences. By shifting RL from token-level exploration to experience-level reasoning, CodeSkill improves optimization efficiency and long-horizon behavioral coherence. Extensive experiments demonstrate that CodeSkill achieves highly competitive performance against strong open-weight baselines across diverse general and industrial coding benchmarks. Furthermore, the learned skills exhibit strong transferability and robust cross-domain generalization, highlighting the effectiveness of explicit behavioral abstraction for scalable agentic code generation.

Song-Li Wu, Jingyi Wang, Zhaocheng Du +2

Tsinghua University · Huawei Noah’s Ark Lab · Shanghai Jiao Tong University

LLM Agent Skill Learning Long-Horizon LLM Agents RL for Code Generation Hierarchical RL

Ask PDF

Abstract

LLM Agent Skill Learning

Explore similar work

Learning Globally Reusable Skills for Coding Agents

CODESKILL: Learning Self-Evolving Skills for Coding Agents

CodeSkill: Latent Skill Abstraction for Long-Horizon Code Agents