cs.AISep 25, 2026
SaveUp and Down the Abstraction Ladder: Code-Based Skills for Language Agents
Organizations: University of Warsaw · AKCES NCBR · Princeton University · IDEAS NCBR ·
Abstract
Language agents struggle to act and learn in environments that require long sequences of low-level actions. Code-based abstractions can make these agents more productive by letting them invoke reusable skills instead of repeatedly selecting individual actions. The code handles recurring local decisions, while the language model decides which skills to use and how to combine them. Yet abstractions are leaky, and situations beyond a skill's capabilities may require a return to primitive actions. Motivated by this tradeoff between productivity and flexibility, we systematically study how code-based action abstraction affects the performance, inference cost, and learning of language agents. We study this in NetHack, a challenging, long-horizon game environment, using CodeHack, our library of code-based skills with natural-language descriptions. We use this library to compare agents restricted to primitives with those using semantic skills alone or in combination with primitives. We evaluate these agents in three settings: zero-shot prompting, supervised fine-tuning, and reinforcement learning. Across a broad zero-shot evaluation on NetHack, we find that compared with primitives, skills nearly triple game progression, while reducing inference cost per episode by 86%. Combining skills with primitives retains much of this benefit while preserving a path back down to low-level actions. Finally, in RL, we find that skill-based agents learn significantly faster than agents acting on primitives, achieving a 7.2x larger average gain in dungeon level over the same training budget. These results show that a supplied skill library can improve performance, efficiency, and learning, while retaining primitives provides flexibility when the library is insufficient. We release CodeHack together with training and evaluation code.
Figures & tables
Figure 1: The value of skills vs. primitives for language agents in NetHack. Higher-level, semantically meaningful skills provided by CodeHack improve performance both in zero-shot use and during RL. Learning curves are for Qwen-3.5-4B .
Figure 2: CodeHack as a shared control layer for primitive-only, skill-only, and mixed agents. 1. The controller can be either a language-model agent or a learned RL policy, since CodeHack supports both natural-language control and a discrete action-space interface. 2. Depending on the configured action interface, the controller selects code skills such as explore or fight_melee , primitive commands such as movement, kick , or search , or both. 3. All three interfaces use the same runtime for symbolic state, inventory tracking, map memory, panic handlers, and no-progress feedback. Primitive commands are registered as one-step strategies. Code skills can expand into multiple primitives. 4. This runtime supports both NetHack and MiniHack, allowing the same skill library to transfer across environments while preserving a path back to low-level control when abstractions are insufficient.
Figure 3: Zero-shot results across the two evaluation domains. (Left) MiniHack success rates for GPT-5, used as a controlled testbed for diagnosing failures of primitive control. (Right) NetHack milestone reach for GPT-5, showing that skill-based control consistently reaches substantially deeper milestones than primitive-only control. Both plots present averages over 32 episodes per task.
Figure 4: Zero-shot NetHack cost-performance frontier. Each point represents a model evaluated with a particular action interface, showing its average dungeon level reached against its average inference cost per episode. For most models, switching from primitives to skills increases dungeon depth while reducing inference cost.
| Regime | Interface | Model | Score | Progression | Dungeon Level |
| Zero-shot | Primitives | Llama-3.1-8B-Instruct | 22.0 2.9 | 0.12 0.05 | 1.08 0.03 |
| Primitives | Qwen-3.5-4B | 36.3 6.5 | 0.29 0.08 | 1.17 0.06 | |
| Mixed | Llama-3.1-8B-Instruct | 78.6 8.0 | 0.90 0.11 | 1.48 0.08 | |
| Mixed | Qwen-3.5-4B | 92.0 11.1 | 0.94 0.11 | 1.81 0.12 | |
| Skills | Llama-3.1-8B-Instruct | 178.4 15.5 | 1.59 0.10 | 2.48 0.15 | |
| Skills | Qwen-3.5-4B | 215.9 22.1 | 1.49 0.13 | 2.45 0.18 |
Table 2: Results across learning regimes for Llama-3.1-8B-Instruct and Qwen-3.5-4B . Both models use act-only control. We report final score, progression, and dungeon level reached. Evaluation checkpoints are picked at 264 gradient steps for all RL results. Values are mean ± 1 standard error (SE). For zero-shot and SFT, SE is computed across 64 evaluation episodes, while for RL (Base) and RL (SFT) it is computed across 3 training seeds (each seed averages over its own 64 evaluation episodes).
Appendix figures & tables28 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Example layout from MiniHack-Corridor-R3-v0 .
Figure 6: Example layout from MiniHack-Corridor-R5-v0
Figure 7: Example layout from MiniHack-Corridor-R10-v0
Figure 8: Example layout for MiniHack-Quest-Easy-v0 .
Figure 9: Example layout for MiniHack-Quest-Medium-v0 .
Figure 10: Example layout for MiniHack-Quest-Hard-v0 .
Figure 11: Example layout for MiniHack-WoD-Hard-Full-v0 .
| Stage | Hyperparameter | Value |
|---|---|---|
| SFT | Teacher model | Gemma-4-31B |
| SFT | Teacher episodes | 1,024 zero-shot episodes |
| SFT | Trainable parameters | LoRA adapter only |
| SFT | LoRA rank / alpha / dropout | 128 / 64 / 0.0 |
| SFT | Learning rate | |
| SFT | Local minibatch size | 64 |
Table 3: Main SFT and RL hyperparameters used in our experiments.
| Model | Action | Progression | Score | Dlvl | L2 | L4 | L6 | L8 | Oracle | Mines | Cost/ep. ($) | Tokens/ep. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-5 | skills | 4.64 0.38 | 1279.29 151.62 | 5.73 0.29 | 0.94 0.03 | 0.84 0.05 | 0.54 0.06 | 0.25 0.06 | 0.06 0.03 | 0.47 0.06 | 6.44 | 2,007,700 |
| mixed | 3.99 0.54 | 922.84 179.63 | 5.25 0.42 | 0.94 0.04 | 0.78 0.07 | 0.38 0.09 | 0.22 0.07 | 0.06 0.04 | 0.53 0.09 | 14.88 | 5,196,745 | |
| primitives | 1.78 0.13 | 232.56 22.59 | 1.50 0.14 | 0.34 0.09 | 0.03 0.03 | 0.00 0.00 | 0.00 0.00 | 0.00 0.00 | 0.03 0.03 | 50.41 | 13,866,723 | |
| GPT-OSS-120B | skills | 2.88 0.17 | 447.72 39.81 | 4.06 0.23 | 0.95 0.03 | 0.61 0.06 | 0.22 0.05 | 0.05 0.03 | 0.03 0.02 | 0.45 0.06 | 0.06 | 1,176,438 |
| mixed | 3.17 0.22 | 495.48 41.31 | 4.47 0.25 | 0.95 0.03 | 0.67 0.06 | 0.33 0.06 | 0.06 0.03 | 0.05 0.03 | 0.36 0.06 | 0.21 | 4,151,999 | |
| primitives | 1.39 0.11 | 152.97 14.46 | 1.28 0.08 | 0.20 0.05 | 0.02 0.02 | 0.00 0.00 | 0.00 0.00 | 0.00 0.00 | 0.00 0.00 | 0.63 | 10,399,676 |
Table 4: Aggregate NetHack zero-shot results for the same 14 models and thinking modes as Table 1 . Gray terms are standard errors across episodes. Progression is in percent; milestone reach values are fractions. All settings use 64 evaluation episodes except GPT-5 , which uses 32 episodes per action interface due to cost. Mines reach is computed from episode traces.
| Metric | Primitives avg | Skills avg | Skills / primitives | % change |
|---|---|---|---|---|
| Progression (%) | 0.690 0.164 | 1.978 0.325 | 2.866 | +187% |
| Score | 83.948 20.157 | 318.268 89.226 | 3.791 | +279% |
| Max DLvl | 1.188 0.052 | 2.880 0.340 | 2.425 | +143% |
| L2 reach | 0.133 0.032 | 0.719 0.062 | 5.411 | +441% |
| L4 reach | 0.013 0.005 | 0.286 0.068 | 21.320 | +2032% |
| L6 reach | 0.001 0.001 | 0.117 0.043 | 104.540 | +10354% |
Table 5: Average zero-shot NetHack performance and resource use across 14 models with both primitive and skill runs for every metric. Gray terms are standard errors across paired model-level means. Ratios and percent changes compare the skills average against the primitives average; lower cost and token values indicate reduced resource use. Ratios with a zero primitive baseline are undefined and shown as –.
Figure 12: Additional zero-shot NetHack tradeoff plot: average maximum dungeon level reached versus tokens per episode. As in the main-text cost-based view, skill-based control defines a stronger performance-efficiency frontier than primitive-only control, while mixed control is typically intermediate.
Figure 13: Additional zero-shot NetHack tradeoff plot: progression versus inference cost per episode. Skill-based control generally achieves higher progression at lower or comparable cost than primitive-only control.
Figure 14: Additional zero-shot NetHack tradeoff plot: progression versus token usage per episode. Skills remain favorable when efficiency is measured in tokens rather than monetary cost.
Figure 15: Additional zero-shot NetHack tradeoff plot: score versus inference cost per episode. Across models, skill-based control tends to produce higher scores at lower or comparable cost than primitive-only control.
Figure 16: Additional zero-shot NetHack tradeoff plot: score versus token usage per episode. The same qualitative advantage of skills persists when efficiency is measured in total input and output tokens.
Figure 17: Sensitivity to incomplete skill coverage for zero-shot Gemma-4-31B on NetHack. The full baseline uses all 78 skills; each other condition removes the named skill family. Mixed control retains primitive actions. Bars show means over 64 episodes per condition. The comparison measures sensitivity to missing skill families and the extent to which primitive alternatives preserve performance.
| Example | Skill limitation | Primitive fallback | Return to skills |
|---|---|---|---|
| Ranged combat | 732: fight_ranged reports no ranged weapon or ammunition and executes zero environment steps. | 733–735: throw , i , east select a javelin and kill the gas spore. 736–739: east moves to the dropped weapons. | 740: pickup_weapon recovers the weapons. |
| Vault guard | 480: explore executes zero environment steps. 483: the guard repeats its demand to drop the gold; no drop-gold skill is available. | 484–485: drop , $ drop all 650 gold pieces. 486–487: east follows the corridor. | 488: explore resumes navigation. |
Table 6: Primitive fallback in two zero-shot GPT-5 trajectories. Numbers are controller-decision indices shown in the linked videos.
Figure 18: Full training curves for Llama-3.1-8B-Instruct (gradient steps). We plot progression (left), score (middle), and dungeon level (right) vs. gradient steps. All curves are averages across three seeds, and error bands indicate one standard error.
Figure 19: Full training curves for Llama-3.1-8B-Instruct (wall-clock time). We plot progression (left), score (middle), and dungeon level (right) vs. wall-clock time. All curves are averages across three seeds, and error bands indicate one standard error.
Figure 20: Full training curves for Qwen-3.5-4B (gradient steps). We plot progression (left), score (middle), and dungeon level (right) vs. gradient steps. All curves are averages across three seeds (except for mixed, which uses 2 seeds), and error bands indicate one standard error.
Figure 21: Full training curves for Qwen-3.5-4B (wall-clock time). We plot progression (left), score (middle), and dungeon level (right) vs. wall-clock time. All curves are averages across three seeds (except for mixed, which uses 2 seeds), and error bands indicate one standard error.
Figure 22: Aggregate action-type mix before and after RL on Llama-3.1-8B-Instruct. Bars show the share of parsed calls attributed to skills, primitives, and invalid outputs, aggregated across all evaluation episodes in each stage. RL shifts the mixed controller toward more successful skill use and sharply reduces invalid calls, but does not eliminate primitive fallback: the mixed primitive share remains close to one third of all calls.
Figure 23: Per-episode skill and primitive fractions before and after RL. Each histogram is computed over per-episode call fractions derived from parsed action strings. We do not show primitive-only and skill-only interfaces, because the distributions collapse to 0 or 1 by construction. The important change is in the mixed interface: RL leaves the aggregate primitive share nearly unchanged (33.3% to 32.0%), but the typical episode becomes much more skill-heavy because the model outputs fewer invalid actions. The median mixed-episode skill fraction increases from 41.1% to 77.7%, while the median primitive fraction falls from 32.9% to 20.2%.
Figure 24: Top mixed-controller skills and primitives before and after RL. The left panel shows skill calls and the right panel shows primitive calls within the mixed interface. In each panel, we plot the top- call types with , selected and ordered by the sum of their before-RL and after-RL call counts. After RL, mixed control concentrates its skill usage on a small core of reusable routines, especially explore , descend_stairs , fight_melee , open_doors , and hidden-door search, while still retaining a nontrivial set of movement and staircase primitives.
Figure 25: Top action calls in the mixed interface before and after RL. We plot the top- actions with , selected and ordered by the sum of their before-RL and after-RL call counts within the mixed interface. When skills and primitives are pooled together, explore remains the single most common action both before and after training. RL elevates high-value skills such as descend_stairs and fight_melee , while several directional primitives decline in relative share, consistent with the controller relying more often on reusable skills rather than long sequences of local movement decisions.
Figure 26: Top action calls in the skill-only interface before and after RL. We plot the top- actions with , selected and ordered by the sum of their before-RL and after-RL call counts within the skill-only interface. Skill-only RL becomes even more concentrated on a handful of reusable routines, with explore dominating and combat, door handling, staircase navigation, and hidden-door search accounting for much of the remaining mass. This suggests that much of the learned improvement comes from invoking a small core of robust behaviors more effectively, rather than from broad uniform use of the entire library.
Figure 27: Top action calls in the primitive-only interface before and after RL. We plot the top- actions with , selected and ordered by the sum of their before-RL and after-RL call counts within the primitive-only interface. Primitive-only control remains dominated by local movement and search . Training changes the mix of directions and reduces invalid calls ( Fig. 22 ), but it does not create the same kind of reusable behavioral compression available in the skill-based interfaces.
Figure 28: OOD MiniHack transfer before and after NetHack RL. Bars compare the Llama-3.1-8B-Instruct for primitives and skills before RL and after RL on held-out MiniHack tasks. We report average success rates. Both interfaces transfer from NetHack to MiniHack, but RL changes them differently: primitive gains are concentrated on the Corridor tasks, while skill-based RL pushes the Corridor family to near-perfect success and also improves the Quest tasks. The main exception is WoD-Hard , where the skill controller becomes worse after RL.
More in this topic
Long-Horizon LLM Agents
Long-horizon large language model agents plan, use tools, and preserve task state across extended, multi-step interactions.
See all 443 papers