CodeSkill: Latent Skill Abstraction for Long-Horizon Code Agents

Song-Li Wu; Jingyi Wang; Zhaocheng Du; Weinan Gan; Weiwen Liu

cs.AISep 27, 2026

CodeSkill: Latent Skill Abstraction for Long-Horizon Code Agents

Authors: Song-Li Wu, Jingyi Wang, Zhaocheng Du, Weinan Gan, Weiwen Liu

Organizations: Tsinghua University · Huawei Noah’s Ark Lab · Shanghai Jiao Tong University

Abstract

Code agents require long-horizon decision-making over complex interaction trajectories. However, existing reinforcement learning (RL) approaches typically optimize behavior at the token level, creating a mismatch between low-level generation and high-level behavioral reasoning. This limitation leads to inefficient exploration and weak credit assignment under sparse rewards. Moreover, while large-scale agent trajectories often contain recurring multi-step behavioral patterns, their noisy token-level representations hinder effective experience reuse. To address these challenges, we propose CodeSkill, a framework that adapts hierarchical latent skill modeling to the code agent domain. CodeSkill first leverages a teacher model to distill both successful and failed trajectories into multi-level textual abstractions. It then integrates temporal variational inference with reinforcement learning to map these discrete semantics into continuous latent variables, while an adaptive boundary mechanism dynamically gates skill transitions based on execution feedback. The learned skills are injected into a frozen LLM policy as latent semantic prefixes, enabling optimization in a compact semantic space rather than over raw token sequences. By shifting RL from token-level exploration to experience-level reasoning, CodeSkill improves optimization efficiency and long-horizon behavioral coherence. Extensive experiments demonstrate that CodeSkill achieves highly competitive performance against strong open-weight baselines across diverse general and industrial coding benchmarks. Furthermore, the learned skills exhibit strong transferability and robust cross-domain generalization, highlighting the effectiveness of explicit behavioral abstraction for scalable agentic code generation.

Figures & tables

Figure 1: (a) CodeSkill pipeline overview. CodeSkill distills reusable skills from trajectories into a skill library, enabling efficient evolution of code agents, unlike vanilla training that directly learns from raw experience. (b) Training comparison with Qwen2.5-Coder-32B-Instruct backbone. All runs use fixed seeds, repeated three times with std-based confidence intervals; rewards are normalized to identical scales. Vanilla RL refers to skill-free Proximal Policy Optimization (PPO). Curves are averaged over general and industrial code benchmarks. CodeSkill delivers faster reward convergence and lower exploration entropy, validating skill abstraction improves sample efficiency.

Figure 2: Overview of the CodeSkill framework. We present a two-stage framework for code agents: (1) Skill Discovery: A hierarchical VAE distills offline expert trajectories into a multi-level skill bank; (2) Skill Evolution: A planning-execution loop with RL refines skills based on environment feedback.

Model	Size	EvalPlus				BigCodeBench		FullStackBench
Model	Size	HumanEval	HumanEval+	MBPP	MBPP+	Full	Hard	FullStackBench
6B+ Models
DeepSeek-Coder-V2-Lite-Instruct	2.4/16B	81.1	75.6	85.2	70.6	37.8	18.9	49.4
Qwen2.5-Coder-7B-Instruct	7B	87.2	81.7	84.7	72.2	37.8	13.5	42.2
Seed-Coder-8B-Instruct	8B	81.1	75.6	86.2	73.3	44.6	23.6	55.8
Qwen2.5-Coder-14B-Instruct	14B	62.8	59.8	88.6	77.2	47.0	6.1	53.1

Table 1: Performance comparison on general code generation tasks.

Model	Size	VeriScope Score	VeriRepair Fix (%)	RealBench						ArchXBench
Model	Size	VeriScope Score	VeriRepair Fix (%)	System		Module				$n$	$t$
				Syn@1	Syn@5	Syn@1	Syn@5	Func@1	Func@5
6B+ Models
Qwen3.5-9B	9B	32.0	-	-	-	6.3	15.8	4.3	8.5	1.9	44.3
GPT-OSS-20B	3.6/21B	73.9	86.7	3.8	17.4	22.9	47.9	9.8	21.0	3.1	53.5
Qwen3.5-27B	27B	55.7	60.0	6.2	20.1	17.1	33.8	10.6	17.8	2.6	48.3

Table 2: Performance comparison on chip design benchmarks. CodeSkill results are highlighted in blue.

Model	Size	CAD-Coder		EmbedCGen Main (%)	SuperCoder		TritonBench				KernelBench
Model	Size	Comp.	IoU	EmbedCGen Main (%)	Acc. (%)	Spd.	G-call (%)	G-exe (%)	T-call (%)	T-exe (%)	L1	L2	L3
6B+ Models
Qwen3.5-9B	9B	2.0	-	10.0	36.0	1.0×	2.7	100.0	3.6	100.0	-	-	-
GPT-OSS-20B	3.6/21B	2.0	2.0	30.6	16.0	1.0×	2.2	100.0	1.2	100.0	5.1	10.0	2.0
Qwen3.5-27B	27B	4.0	0.3	9.6	6.0	1.9×	5.4	100.0	25.9	97.7	5.1	6.0	4.0
30B+ Models

Table 3: Performance on GPU optimization, code optimization, and 3D modeling benchmarks. InCoder-32B-Thinking results are highlighted in gray.

Model	Origin	SFT	PPO	GRPO	CodeSkill
SWEBench-Verified
Qwen3-4B	1.7 ( $\pm$ 0.1)	2.4 ( $\pm$ 0.3)	4.9 ( $\pm$ 0.6)	7.3 ( $\pm$ 0.8)	14.3 ( $\pm$ 0.2)
Gemma-3-27b	7.1 ( $\pm$ 0.5)	7.0 ( $\pm$ 0.4)	9.3 ( $\pm$ 0.8)	11.6 ( $\pm$ 0.5)	19.7 ( $\pm$ 0.3)
Qwen3-Coder-30B	37.7 ( $\pm$ 0.2)	43.8 ( $\pm$ 0.8)	48.1 ( $\pm$ 0.9)	49.5 ( $\pm$ 0.6)	58.7 ( $\pm$ 0.5)
GLM-4.5-Air	51.4 ( $\pm$ 0.2)	51.5 ( $\pm$ 0.8)	52.9 ( $\pm$ 0.6)	54.2 ( $\pm$ 0.7)	62.8 ( $\pm$ 0.3)
SWEBench-Lite

Table 4: Resolve rate (pass@1) on benchmarks. Results are mean

\pm

std over three runs. For each model, the best result is highlighted in gray.

Model	EvalPlus				BigCodeBench		FullStackBench
Model	HumanEval	HumanEval+	MBPP	MBPP+	Full	Hard	FullStackBench
CodeSkill	99.2	94.8	98.3	83.7	50.3	32.6	71.5
Hierarchy Ablations
w/o hierarchy	95.8	88.7	92.1	78.4	46.9	28.5	61.8
w/o $z_{H}$	96.4	89.6	92.8	79.1	47.8	29.4	62.6
w/o $z_{M}$	95.9	88.9	92.0	78.2	46.5	27.9	61.3

Table 5: Ablation study on general code generation tasks. To ensure fair comparisons, the textual skill baselines are strictly matched with the latent variants in terms of tunable parameter count and prompt context length.

Model	Size	Agentic Coding			General Tool Use
		Terminal-Bench		SWE-bench Verified	Mind2Web	BFCL V3	$\tau^{2}$ -bench
		v1.0	v2.0	SWE-bench Verified	Mind2Web	BFCL V3	Airline	Retail	Telecom
6B+ Models
DeepSeek-Coder-V2-Lite-Instruct	2.4/16B	5.0	-	-	26.7	-	3.5	12.0	-
Qwen2.5-Coder-7B-Instruct	7B	6.3	-	-	38.4	54.2	-	-	-
Seed-Coder-8B-Instruct	8B	7.5	2.5	-	38.2	-	4.3	32.0	-

Table 6: Performance comparison on agentic coding tasks (Terminal-Bench v1.0, Terminal-Bench v2.0, SWE-Verified) and general tool-use tasks (Mind2Web, BFCL V3,

\tau^{2}

-bench).

Category	Hyperparameter	Value
Architecture	Latent Dimension ( $d_{z}$ )	256
	Soft Prompt Length	16
	LoRA Rank ( $r$ )	16
	LoRA Alpha ( $\alpha$ )	32
Variational Inference	High-level KL Weight ( $\beta_{H}$ )	0.1
	Mid-level KL Weight ( $\beta_{M}$ )	0.1

Table 7: Hyperparameter settings for CodeSkill training and inference.

Model	Size	Code Reasoning				Code Efficiency		Text2SQL
		CruxEval		LiveCodeBench		Mercury		Bird	Spider
		Input-COT	Output-COT	V5	V6	Beyond@1	Pass@1	Bird	Spider
6B+ Models
DeepSeek-Coder-V2-Lite-Instruct	2.4/16B	57.1	56.2	13.2	19.4	76.8	91.4	41.6	72.4
Qwen2.5-Coder-7B-Instruct	7B	66.9	66.0	14.4	18.9	69.9	84.8	53.1	79.8
Seed-Coder-8B-Instruct	8B	62.0	66.6	19.2	22.3	78.5	93.8	44.7	72.7

Table 8: Performance comparison on code reasoning (CruxEval, LiveCodeBench), code efficiency (Mercury), and Text2SQL (Bird, Spider) benchmarks.

Model	CruxEval		LiveCodeBench		Mercury		Text2SQL
Model	Input-COT	Output-COT	V5	V6	Beyond@1	Pass@1	Bird	Spider
CodeSkill (32B)	92.8	97.3	83.7	86.1	82.3	97.8	63.2	84.9
Hierarchy Ablations
w/o hierarchy	88.5	93.6	78.4	80.6	76.5	93.9	56.4	78.5
w/o $z_{H}$	89.2	94.4	79.5	81.8	77.8	94.7	57.8	79.8
w/o $z_{M}$	88.7	93.8	78.7	80.9	76.9	94.1	56.7	78.9

Table 9: Ablation study on code reasoning (CruxEval, LiveCodeBench), code efficiency (Mercury), and Text2SQL (Bird, Spider) benchmarks.

Model	Agentic Coding			General Tool Use
	Terminal-Bench		SWE-bench	Mind2Web	BFCL V3	$\tau^{2}$ -bench
	v1.0	v2.0	SWE-bench	Mind2Web	BFCL V3	Airline	Retail	Telecom
CodeSkill (32B)	42.6	29.3	76.2	57.8	68.1	76.9	89.4	97.1
Hierarchy Ablations
w/o hierarchy	38.4	24.8	70.4	52.4	63.2	71.5	84.1	93.5
w/o $z_{H}$	39.1	25.4	71.3	53.1	64.0	72.4	84.9	94.1

Table 10: Ablation study on agentic coding tasks (Terminal-Bench, SWE-bench Verified) and general tool-use tasks (Mind2Web, BFCL V3,

\tau^{2}

-bench).

Model	VeriScope Score	VeriRepair Fix (%)	RealBench						ArchXBench
			System		Module				$n$	$t$
			Syn@1	Syn@5	Syn@1	Syn@5	Func@1	Func@5	$n$	$t$
CodeSkill (32B)	89.6	90.4	43.8	56.4	78.8	85.7	66.2	73.6	4.8	59.5
Hierarchy Ablations
w/o hierarchy	84.5	85.3	39.4	51.2	73.2	80.5	60.8	67.9	4.2	53.4
w/o $z_{H}$	85.2	86.1	40.2	52.1	74.1	81.4	61.5	68.8	4.3	54.2

Table 11: Ablation study on chip design benchmarks.

Model	CAD-Coder		EmbedCGen Main (%)	SuperCoder		TritonBench				KernelBench
	Comp.	IoU		Acc. (%)	Spd.	G-call	G-exe	T-call	T-exe	L1	L2	L3
	Comp.	IoU		Acc. (%)	Spd.	(%)	(%)	(%)	(%)	L1	L2	L3
CodeSkill (32B)	87.0	56.3	93.4	95.0	1.2 $\times$	28.4	100.0	40.7	100.0	25.2	41.0	15.0
Hierarchy Ablations
w/o hierarchy	81.2	51.5	88.6	90.2	1.1 $\times$	24.1	98.5	35.6	98.8	21.5	36.5	12.8
w/o $z_{H}$	82.0	52.3	89.4	91.1	1.1 $\times$	25.0	98.8	36.5	99.1	22.1	37.2	13.2

Table 12: Ablation study on GPU optimization (TritonBench, KernelBench), code optimization (SuperCoder), and 3D modeling (CAD-Coder) benchmarks.

Figure 3: Comparison of trajectory efficiency and interaction steps required to complete coding tasks with and without the skill library.

Figure 4: t-SNE visualization of skill embeddings, illustrating distinct functional clusters.

Figure 5: Invocation frequency of individual code skills across the evaluation dataset, demonstrating a long-tail distribution.

LLM Agent Skill Learning

Large language model agent skill learning studies how agents acquire, represent, reuse, and refine procedures for multi-step tasks.

See all 396 papers

SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning
Feb 9, 2026Peng Xia, Jianwen Chen, Hanyang Wang +10LLM Agent Skill Learning RL for LLM Agents
UNC-Chapel Hill · University of Chicago · University of California San Diego +3
Hierarchical Skill Evolution for Coding Agent
Oct 26, 2026Jun Gao, Yun Peng, Changhai Zhou +10LLM Agent Skill Learning Software Engineering Agents
SkillCommit: Evolving Agent Skills through Behaviorally Validated Scope Expansion
Aug 15, 2026Yu He, Weikai YangLLM Agent Self-Improvement LLM Agent Skill Learning
Nanjing University · The Hong Kong University of Science and Technology (Guangzhou)

Feb 9, 2026J/K move · Enter open · S save

SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning

Peng Xia, Jianwen Chen, Hanyang Wang, Jiaqi Liu+9

UNC-Chapel Hill · University of Chicago · University of California San Diego+3

LLM Agent Skill Learning RL for LLM Agents Memory-Augmented Language Model Agents

Open paper AskPDF arXiv

Large Language Model (LLM) agents have shown stunning results in complex tasks, yet they often operate in isolation, failing to learn from past experiences. Existing memory-based methods primarily store raw trajectories, which are often redundant and noise-heavy. This prevents agents from extracting high-level, reusable behavioral patterns that are essential for generalization. In this paper, we propose SkillRL, a framework that bridges the gap between raw experience and policy improvement through automatic skill discovery and recursive evolution. Our approach introduces an experience-based distillation mechanism to build a hierarchical skill library SkillBank, an adaptive retrieval strategy for general and task-specific heuristics, and a recursive evolution mechanism that allows the skill library to co-evolve with the agent's policy during reinforcement learning. These innovations significantly reduce the token footprint while enhancing reasoning utility. Experimental results on ALFWorld, WebShop and seven search-augmented tasks demonstrate that SkillRL achieves state-of-the-art performance, outperforming strong baselines over 15.3% and maintaining robustness as task complexity increases. Code is available at this https://github.com/aiming-lab/SkillRL.

Abstract

LLM Agent Skill Learning

Explore similar work

SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning

Hierarchical Skill Evolution for Coding Agent

SkillCommit: Evolving Agent Skills through Behaviorally Validated Scope Expansion