Up and Down the Abstraction Ladder: Code-Based Skills for Language Agents

Bartłomiej Cupiał; Jens Tuyls; Maciej Wołczyk; Davide Paglieri; Martin Klissarov; Benjamin Eysenbach; Piotr Miłoś; Karthik R. Narasimhan

cs.AISep 25, 2026

Up and Down the Abstraction Ladder: Code-Based Skills for Language Agents

Authors: Bartłomiej Cupiał, Jens Tuyls, Maciej Wołczyk, Davide Paglieri, Martin Klissarov, Benjamin Eysenbach, Piotr Miłoś, Karthik R. Narasimhan

Organizations: University of Warsaw · AKCES NCBR · Princeton University · IDEAS NCBR ·

Abstract

Language agents struggle to act and learn in environments that require long sequences of low-level actions. Code-based abstractions can make these agents more productive by letting them invoke reusable skills instead of repeatedly selecting individual actions. The code handles recurring local decisions, while the language model decides which skills to use and how to combine them. Yet abstractions are leaky, and situations beyond a skill's capabilities may require a return to primitive actions. Motivated by this tradeoff between productivity and flexibility, we systematically study how code-based action abstraction affects the performance, inference cost, and learning of language agents. We study this in NetHack, a challenging, long-horizon game environment, using CodeHack, our library of code-based skills with natural-language descriptions. We use this library to compare agents restricted to primitives with those using semantic skills alone or in combination with primitives. We evaluate these agents in three settings: zero-shot prompting, supervised fine-tuning, and reinforcement learning. Across a broad zero-shot evaluation on NetHack, we find that compared with primitives, skills nearly triple game progression, while reducing inference cost per episode by 86%. Combining skills with primitives retains much of this benefit while preserving a path back down to low-level actions. Finally, in RL, we find that skill-based agents learn significantly faster than agents acting on primitives, achieving a 7.2x larger average gain in dungeon level over the same training budget. These results show that a supplied skill library can improve performance, efficiency, and learning, while retaining primitives provides flexibility when the library is insufficient. We release CodeHack together with training and evaluation code.

Figures & tables

Figure 1: The value of skills vs. primitives for language agents in NetHack. Higher-level, semantically meaningful skills provided by CodeHack improve performance both in zero-shot use and during RL. Learning curves are for Qwen-3.5-4B .

Figure 2: CodeHack as a shared control layer for primitive-only, skill-only, and mixed agents. 1. The controller can be either a language-model agent or a learned RL policy, since CodeHack supports both natural-language control and a discrete action-space interface. 2. Depending on the configured action interface, the controller selects code skills such as explore or fight_melee , primitive commands such as movement, kick , or search , or both. 3. All three interfaces use the same runtime for symbolic state, inventory tracking, map memory, panic handlers, and no-progress feedback. Primitive commands are registered as one-step strategies. Code skills can expand into multiple primitives. 4. This runtime supports both NetHack and MiniHack, allowing the same skill library to transfer across environments while preserving a path back to low-level control when abstractions are insufficient.

Figure 3: Zero-shot results across the two evaluation domains. (Left) MiniHack success rates for GPT-5, used as a controlled testbed for diagnosing failures of primitive control. (Right) NetHack milestone reach for GPT-5, showing that skill-based control consistently reaches substantially deeper milestones than primitive-only control. Both plots present averages over 32 episodes per task.

Figure 4: Zero-shot NetHack cost-performance frontier. Each point represents a model evaluated with a particular action interface, showing its average dungeon level reached against its average inference cost per episode. For most models, switching from primitives to skills increases dungeon depth while reducing inference cost.

Regime	Interface	Model	Score	Progression	Dungeon Level
Zero-shot	Primitives	Llama-3.1-8B-Instruct	22.0 $\pm$ 2.9	0.12 $\pm$ 0.05	1.08 $\pm$ 0.03
	Primitives	Qwen-3.5-4B	36.3 $\pm$ 6.5	0.29 $\pm$ 0.08	1.17 $\pm$ 0.06
	Mixed	Llama-3.1-8B-Instruct	78.6 $\pm$ 8.0	0.90 $\pm$ 0.11	1.48 $\pm$ 0.08
	Mixed	Qwen-3.5-4B	92.0 $\pm$ 11.1	0.94 $\pm$ 0.11	1.81 $\pm$ 0.12
	Skills	Llama-3.1-8B-Instruct	178.4 $\pm$ 15.5	1.59 $\pm$ 0.10	2.48 $\pm$ 0.15
	Skills	Qwen-3.5-4B	215.9 $\pm$ 22.1	1.49 $\pm$ 0.13	2.45 $\pm$ 0.18

Table 2: Results across learning regimes for Llama-3.1-8B-Instruct and Qwen-3.5-4B . Both models use act-only control. We report final score, progression, and dungeon level reached. Evaluation checkpoints are picked at 264 gradient steps for all RL results. Values are mean ± 1 standard error (SE). For zero-shot and SFT, SE is computed across 64 evaluation episodes, while for RL (Base) and RL (SFT) it is computed across 3 training seeds (each seed averages over its own 64 evaluation episodes).

Appendix figures & tables28 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Example layout from MiniHack-Corridor-R3-v0 .

Figure 6: Example layout from MiniHack-Corridor-R5-v0

Figure 7: Example layout from MiniHack-Corridor-R10-v0

Figure 8: Example layout for MiniHack-Quest-Easy-v0 .

Figure 9: Example layout for MiniHack-Quest-Medium-v0 .

Figure 10: Example layout for MiniHack-Quest-Hard-v0 .

Figure 11: Example layout for MiniHack-WoD-Hard-Full-v0 .

Stage	Hyperparameter	Value
SFT	Teacher model	Gemma-4-31B
SFT	Teacher episodes	1,024 zero-shot episodes
SFT	Trainable parameters	LoRA adapter only
SFT	LoRA rank / alpha / dropout	128 / 64 / 0.0
SFT	Learning rate	$5\times 10^{-5}$
SFT	Local minibatch size	64

Appendix

Table 3: Main SFT and RL hyperparameters used in our experiments.

Model	Action	Progression	Score	Dlvl	L2	L4	L6	L8	Oracle	Mines	Cost/ep. ($)	Tokens/ep.
GPT-5	skills	4.64 $\pm$ 0.38	1279.29 $\pm$ 151.62	5.73 $\pm$ 0.29	0.94 $\pm$ 0.03	0.84 $\pm$ 0.05	0.54 $\pm$ 0.06	0.25 $\pm$ 0.06	0.06 $\pm$ 0.03	0.47 $\pm$ 0.06	6.44	2,007,700
	mixed	3.99 $\pm$ 0.54	922.84 $\pm$ 179.63	5.25 $\pm$ 0.42	0.94 $\pm$ 0.04	0.78 $\pm$ 0.07	0.38 $\pm$ 0.09	0.22 $\pm$ 0.07	0.06 $\pm$ 0.04	0.53 $\pm$ 0.09	14.88	5,196,745
	primitives	1.78 $\pm$ 0.13	232.56 $\pm$ 22.59	1.50 $\pm$ 0.14	0.34 $\pm$ 0.09	0.03 $\pm$ 0.03	0.00 $\pm$ 0.00	0.00 $\pm$ 0.00	0.00 $\pm$ 0.00	0.03 $\pm$ 0.03	50.41	13,866,723
GPT-OSS-120B	skills	2.88 $\pm$ 0.17	447.72 $\pm$ 39.81	4.06 $\pm$ 0.23	0.95 $\pm$ 0.03	0.61 $\pm$ 0.06	0.22 $\pm$ 0.05	0.05 $\pm$ 0.03	0.03 $\pm$ 0.02	0.45 $\pm$ 0.06	0.06	1,176,438
	mixed	3.17 $\pm$ 0.22	495.48 $\pm$ 41.31	4.47 $\pm$ 0.25	0.95 $\pm$ 0.03	0.67 $\pm$ 0.06	0.33 $\pm$ 0.06	0.06 $\pm$ 0.03	0.05 $\pm$ 0.03	0.36 $\pm$ 0.06	0.21	4,151,999
	primitives	1.39 $\pm$ 0.11	152.97 $\pm$ 14.46	1.28 $\pm$ 0.08	0.20 $\pm$ 0.05	0.02 $\pm$ 0.02	0.00 $\pm$ 0.00	0.00 $\pm$ 0.00	0.00 $\pm$ 0.00	0.00 $\pm$ 0.00	0.63	10,399,676

Appendix

Table 4: Aggregate NetHack zero-shot results for the same 14 models and thinking modes as Table 1 . Gray

\pm

terms are standard errors across episodes. Progression is in percent; milestone reach values are fractions. All settings use 64 evaluation episodes except GPT-5 , which uses 32 episodes per action interface due to cost. Mines reach is computed from episode traces.

Metric	Primitives avg	Skills avg	Skills / primitives	% change
Progression (%)	0.690 $\pm$ 0.164	1.978 $\pm$ 0.325	2.866 $\times$	+187%
Score	83.948 $\pm$ 20.157	318.268 $\pm$ 89.226	3.791 $\times$	+279%
Max DLvl	1.188 $\pm$ 0.052	2.880 $\pm$ 0.340	2.425 $\times$	+143%
L2 reach	0.133 $\pm$ 0.032	0.719 $\pm$ 0.062	5.411 $\times$	+441%
L4 reach	0.013 $\pm$ 0.005	0.286 $\pm$ 0.068	21.320 $\times$	+2032%
L6 reach	0.001 $\pm$ 0.001	0.117 $\pm$ 0.043	104.540 $\times$	+10354%

Appendix

Table 5: Average zero-shot NetHack performance and resource use across 14 models with both primitive and skill runs for every metric. Gray

\pm

terms are standard errors across paired model-level means. Ratios and percent changes compare the skills average against the primitives average; lower cost and token values indicate reduced resource use. Ratios with a zero primitive baseline are undefined and shown as –.

Figure 12: Additional zero-shot NetHack tradeoff plot: average maximum dungeon level reached versus tokens per episode. As in the main-text cost-based view, skill-based control defines a stronger performance-efficiency frontier than primitive-only control, while mixed control is typically intermediate.

Figure 13: Additional zero-shot NetHack tradeoff plot: progression versus inference cost per episode. Skill-based control generally achieves higher progression at lower or comparable cost than primitive-only control.

Figure 14: Additional zero-shot NetHack tradeoff plot: progression versus token usage per episode. Skills remain favorable when efficiency is measured in tokens rather than monetary cost.

Figure 15: Additional zero-shot NetHack tradeoff plot: score versus inference cost per episode. Across models, skill-based control tends to produce higher scores at lower or comparable cost than primitive-only control.

Figure 16: Additional zero-shot NetHack tradeoff plot: score versus token usage per episode. The same qualitative advantage of skills persists when efficiency is measured in total input and output tokens.

Figure 17: Sensitivity to incomplete skill coverage for zero-shot Gemma-4-31B on NetHack. The full baseline uses all 78 skills; each other condition removes the named skill family. Mixed control retains primitive actions. Bars show means over 64 episodes per condition. The comparison measures sensitivity to missing skill families and the extent to which primitive alternatives preserve performance.

Example	Skill limitation	Primitive fallback	Return to skills
Ranged combat	732: fight_ranged reports no ranged weapon or ammunition and executes zero environment steps.	733–735: throw , i , east select a javelin and kill the gas spore. 736–739: east moves to the dropped weapons.	740: pickup_weapon recovers the weapons.
Vault guard	480: explore executes zero environment steps. 483: the guard repeats its demand to drop the gold; no drop-gold skill is available.	484–485: drop , $ drop all 650 gold pieces. 486–487: east follows the corridor.	488: explore resumes navigation.

Appendix

Table 6: Primitive fallback in two zero-shot GPT-5 trajectories. Numbers are controller-decision indices shown in the linked videos.

Figure 18: Full training curves for Llama-3.1-8B-Instruct (gradient steps). We plot progression (left), score (middle), and dungeon level (right) vs. gradient steps. All curves are averages across three seeds, and error bands indicate one standard error.

Figure 19: Full training curves for Llama-3.1-8B-Instruct (wall-clock time). We plot progression (left), score (middle), and dungeon level (right) vs. wall-clock time. All curves are averages across three seeds, and error bands indicate one standard error.

Figure 20: Full training curves for Qwen-3.5-4B (gradient steps). We plot progression (left), score (middle), and dungeon level (right) vs. gradient steps. All curves are averages across three seeds (except for mixed, which uses 2 seeds), and error bands indicate one standard error.

Figure 21: Full training curves for Qwen-3.5-4B (wall-clock time). We plot progression (left), score (middle), and dungeon level (right) vs. wall-clock time. All curves are averages across three seeds (except for mixed, which uses 2 seeds), and error bands indicate one standard error.

Figure 22: Aggregate action-type mix before and after RL on Llama-3.1-8B-Instruct. Bars show the share of parsed calls attributed to skills, primitives, and invalid outputs, aggregated across all evaluation episodes in each stage. RL shifts the mixed controller toward more successful skill use and sharply reduces invalid calls, but does not eliminate primitive fallback: the mixed primitive share remains close to one third of all calls.

Figure 23: Per-episode skill and primitive fractions before and after RL. Each histogram is computed over per-episode call fractions derived from parsed action strings. We do not show primitive-only and skill-only interfaces, because the distributions collapse to 0 or 1 by construction. The important change is in the mixed interface: RL leaves the aggregate primitive share nearly unchanged (33.3% to 32.0%), but the typical episode becomes much more skill-heavy because the model outputs fewer invalid actions. The median mixed-episode skill fraction increases from 41.1% to 77.7%, while the median primitive fraction falls from 32.9% to 20.2%.

Figure 24: Top mixed-controller skills and primitives before and after RL. The left panel shows skill calls and the right panel shows primitive calls within the mixed interface. In each panel, we plot the top- $k$ call types with $k=15$ , selected and ordered by the sum of their before-RL and after-RL call counts. After RL, mixed control concentrates its skill usage on a small core of reusable routines, especially explore , descend_stairs , fight_melee , open_doors , and hidden-door search, while still retaining a nontrivial set of movement and staircase primitives.

Figure 25: Top action calls in the mixed interface before and after RL. We plot the top- $k$ actions with $k=15$ , selected and ordered by the sum of their before-RL and after-RL call counts within the mixed interface. When skills and primitives are pooled together, explore remains the single most common action both before and after training. RL elevates high-value skills such as descend_stairs and fight_melee , while several directional primitives decline in relative share, consistent with the controller relying more often on reusable skills rather than long sequences of local movement decisions.

Figure 26: Top action calls in the skill-only interface before and after RL. We plot the top- $k$ actions with $k=15$ , selected and ordered by the sum of their before-RL and after-RL call counts within the skill-only interface. Skill-only RL becomes even more concentrated on a handful of reusable routines, with explore dominating and combat, door handling, staircase navigation, and hidden-door search accounting for much of the remaining mass. This suggests that much of the learned improvement comes from invoking a small core of robust behaviors more effectively, rather than from broad uniform use of the entire library.

Figure 27: Top action calls in the primitive-only interface before and after RL. We plot the top- $k$ actions with $k=15$ , selected and ordered by the sum of their before-RL and after-RL call counts within the primitive-only interface. Primitive-only control remains dominated by local movement and search . Training changes the mix of directions and reduces invalid calls ( Fig. 22 ), but it does not create the same kind of reusable behavioral compression available in the skill-based interfaces.

Figure 28: OOD MiniHack transfer before and after NetHack RL. Bars compare the Llama-3.1-8B-Instruct for primitives and skills before RL and after RL on held-out MiniHack tasks. We report average success rates. Both interfaces transfer from NetHack to MiniHack, but RL changes them differently: primitive gains are concentrated on the Corridor tasks, while skill-based RL pushes the Corridor family to near-perfect success and also improves the Quest tasks. The main exception is WoD-Hard , where the skill controller becomes worse after RL.

Long-Horizon LLM Agents

Long-horizon large language model agents plan, use tools, and preserve task state across extended, multi-step interactions.

See all 443 papers

Jun 4, 2026

SkillComposer: Learning to Evolve Agent Skills for Specification and Generalization

Agent skills, which consist of reusable strategies that guide agent reasoning and action, have shown strong potential for improving model capability at inference time. However, current skill construction methods treat the problem as one-shot extraction, overlooking a fundamental tension: a skill tailored to the specific task fails to transfer, while the abstracted skill often provides insufficient guidance. We attribute this fragility to the absence of explicit mechanisms for skill specification and generalization. To address this gap, we introduce SkillComposer, a framework that decomposes skill construction into three learnable operations: create, improve, and merge. Trained via systematic rejection sampling recipe, SkillComposer enables language models to self-evolve skills at inference time and supports three deployment modes: offline for building generalized libraries, online for task-specific refinement, and hybrid for combining both. Comprehensive experiments on

τ^2

-Bench, LiveCodeBench v6, and AppWorld show that SkillComposer consistently outperforms baselines. Our SkillComposer-4B improves a 27B executor by up to +4.5 on agent tasks and +3.4 on code tasks, while generalizing across domains and task types unseen during training. Analysis reveals that merge and improve address orthogonal quality dimensions and that skill composition is a transferable meta-ability, providing a practical recipe for skill-augmented inference.

Qi Zhang, Zhaopeng Feng, Xiaonan Shi +8

1Zhejiang University · 2Tongyi Lab · 3National University of Singapore

LLM Agent Self-Improvement LLM Agent Skill Learning Agent Skill Learning

Ask PDF

Aug 11, 2026

Better, Faster, Stronger: Programmatic Skill Learning Best Reduces Agent Cost

Recently, the practice of augmenting LLM agent capability with skills has gained prevalence. We explore the cost effective adaptation of agents to novel domains by means of learning skills. Existing works focus on performance gain over cost effectiveness. As a result, little is known about what skill learning strategies save cost. We argue that among all the different skill learning methods, those that view skills as programs can achieve the best cost reduction. By executing sequences of actions deterministically, a program-augmented agent can reliably and cheaply achieve goals that would otherwise require trial and error and risk degenerate behavior over long horizons. An agent can learn at inference time by incrementally discovering these programs and equipping them for future tasks. We hypothesize that past trajectories contain enough signal to guide skill learning, even without replay or validation, provided the agent can learn to analyze them. To test our claims, we propose SpeedRunner, a coding agent that analyzes trajectories and refactors skills for better performance on future tasks. Across three different embodied environments, we show that SpeedRunner consistently achieves the frontier in learning and cost reduction while remaining robust against distribution shifts and environmental randomness.

Zixi Huang, Xiheng Wang, Andrew Wang +4

Johns Hopkins University

LLM Agent Self-Improvement LLM Agent Skill Learning Embodied AI

Ask PDF

Apr 6, 2026

SkillX: Automatically Constructing Skill Knowledge Bases for Agents

Learning from experience is critical for building capable large language model (LLM) agents, yet prevailing self-evolving paradigms remain inefficient: agents learn in isolation, repeatedly rediscover similar behaviors from limited experience, resulting in redundant exploration and poor generalization. To address this problem, we propose SkillX, a fully automated framework for constructing a plug-and-play skill knowledge base that can be reused across agents and environments. SkillX operates through a fully automated pipeline built on three synergistic innovations: (i) Multi-Level Skills Design, which distills raw trajectories into three-tiered hierarchy of strategic plans, functional skills, and atomic skills; (ii) Iterative Skills Refinement, which automatically revises skills based on execution feedback to continuously improve library quality; and (iii) Exploratory Skills Expansion, which proactively generates and validates novel skills to expand coverage beyond seed training data. Using a strong backbone agent (GLM-4.6), we automatically build a reusable skill library and evaluate its transferability on challenging long-horizon, user-interactive benchmarks, including AppWorld, BFCL-v3, and

τ^2

-Bench. Experiments show that SkillKB consistently improves task success and execution efficiency when plugged into weaker base agents, highlighting the importance of structured, hierarchical experience representations for generalizable agent learning. Our code will be publicly available soon at https://github.com/zjunlp/SkillX.

Chenxi Wang, Zhuoyun Yu, Xin Xie +8

1Zhejiang University · 2Ant Digital Technologies, Ant Group.

LLM Agent Skill Learning LLM Agents Long-Horizon LLM Agents

Ask PDF

Abstract

Long-Horizon LLM Agents

Explore similar work

SkillComposer: Learning to Evolve Agent Skills for Specification and Generalization

Better, Faster, Stronger: Programmatic Skill Learning Best Reduces Agent Cost

SkillX: Automatically Constructing Skill Knowledge Bases for Agents