How Much Skill Orchestration Can LLM Agents Compile Away? Execution-Grounded Specialization for Token-Efficient Skill Use
Organizations: NVIDIA.
Abstract
Reusable skills guide large language model (LLM) agents through recurring requests, but each invocation still requires the model to interpret the skill and reconstruct its orchestration. Can agents automatically compile this orchestration into an executable fast path without human intervention? We first study zero-shot skill specialization, which compiles without execution evidence, and find that despite the skill's procedural guidance, incorrect assumptions about runtime bindings can make generated plans brittle and recovery costly. This motivates SkillCache, a one-trace skill specialization method that compiles a completed invocation into a parameterized capsule of programs and natural-language steps, replays it before reuse, and drives it with a plan executor that returns control to the model only when needed, keeping the source skill for recovery. With the Claude Code harness, warm execution of the same request reduces tokens on 87 SkillsBench tasks by 81% with Claude Sonnet 4.6 and by 74% with Claude Opus 4.8 at comparable reward, and capsules stay effective under controlled request changes. Capsules also transfer across models, raising three open-weight models' mean reward by 12-25 percentage points while reducing their tokens by 72-81%, suggesting that execution-grounded compilation is a promising approach to efficient repeated skill use and cross-model transfer.
Figures & tables
| Compile + execution | Execution only | ||||
|---|---|---|---|---|---|
| Tasks | Ratio | Fewer/more | Ratio | Fewer/more | |
| All | 84 | 26/58 ∗ | 54/30 | ||
| No failure | 52 | 17/35 | 37/15 ∗ | ||
| Failure | 32 | 9/23 | 17/15 | ||
| Sonnet 4.6 | Opus 4.8 | |||||
|---|---|---|---|---|---|---|
| Task type | Cold | Warm | Cold | Warm | ||
| Overall | 1313 | 247 | ( 80.9 3.6%) | 1063 | 273 | ( 74.3 6.3%) |
| Cybersecurity | 2020 | 579 | ( 68.9 35.4%) | 1264 | 526 | ( 58.1 17.0%) |
| Finance/econ. | 2382 | 651 | ( 64.4 28.4%) | 1402 | 326 | ( 76.2 14.9%) |
| Industrial/phys. | 1141 | 103 | ( 91.0 0.9%) | 977 | 203 | ( 79.0 4.4%) |
| Math/formal | 1165 | 135 | ( 88.2 2.4%) | 1589 | 186 | ( 88.3 1.5%) |
| Sonnet 4.6 | Opus 4.8 | |||||
| Context | Tokens | Cost | Reward | Tokens | Cost | Reward |
| Totals | ||||||
| Raw trace | 52.6 | 38.48 | 0.541 | 47.4 | 54.75 | 0.700 |
| Procedure | 37.6 | 27.82 | 0.541 | 44.2 | 47.41 | 0.685 |
| Ours | 21.5 | 14.41 | 0.546 | 23.7 | 26.16 | 0.690 |
| Ours better/worse, task by task | ||||||
| Tokens per probe | Mean reward | |||
| Probe | Skill only | With capsule | Skill only | With capsule |
| Sonnet 4.6 | ||||
| Overall | 658k | 117k ( 82 2%) | 0.81 | 0.89 (+0.08 0.04) |
| Instance | 660k | 117k ( 82 2%) | 0.81 | 0.83 (+0.01 0.05) |
| Rename | 649k | 162k ( 74 8%) | 0.81 | 0.88 (+0.07 0.03) |
| Reword | 663k | 83k ( 87 1%) | 0.81 | 0.96 (+0.15 0.13) |
| Sonnet 4.6 | Opus 4.8 | |||||
| Model calls | Tokens | Model calls | Tokens | |||
| Capsule type | Cold | Warm | saved | Cold | Warm | saved |
| All tasks | 23.6 | 5.6 | 81% | 20.9 | 6.7 | 74% |
| Hybrid | ||||||
| Programs only | 18.8 | 3.0 | 89% | 17.2 | 3.4 | 87% |
| Natural-language steps | 31.6 | 8.6 | 78% | 25.1 | 10.1 | 67% |
| Sonnet 4.6 | Opus 4.8 | |||||
| Context | Tokens | Cost | Reward | Tokens | Cost | Reward |
| Totals | ||||||
| Agent-driven | 29.0 | 17.40 | 0.544 | 29.7 | 30.05 | 0.678 |
| Executor-driven | 21.5 | 14.41 | 0.546 | 23.7 | 26.16 | 0.690 |
| Executor-driven better/worse, task by task | ||||||
| vs agent-driven | 78/9 ∗ | 78/9 ∗ | 4/4 | 70/14 ∗ | 67/17 ∗ | 3/0 |
| Sonnet 4.6 ( ) | Opus 4.8 ( ) | |||||
| Context | Tokens | Cost | Reward | Tokens | Cost | Reward |
| Totals | ||||||
| No replay | 80.4 | 48.23 | 0.368 | 48.7 | 52.20 | 0.597 |
| Replay | 28.2 | 19.73 | 0.406 | 31.8 | 35.10 | 0.625 |
| Replay better/worse, per task run | ||||||
| vs no replay | 56/9 ∗ | 53/12 ∗ | 5/3 | 66/13 ∗ | 67/12 ∗ | 3/0 |
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| Compile + execution | Execution only | |||||
|---|---|---|---|---|---|---|
| Tasks | Ratio | Total | Ratio | Total | ||
| All | 1.24 [1.06, 1.46] | % | 0.71 [0.58, 0.85] | % | 0.01 | |
| No failure | 1.19 [0.96, 1.46] | % | 0.02 | 0.60 [0.46, 0.77] | % | 0.003 |
| Failure | 1.34 [1.03, 1.71] | % | 0.02 | 0.92 [0.71, 1.17] | % | 0.86 |
| Executor-driven | 1.14 [0.99, 1.31] | % | 0.01 | 0.60 [0.50, 0.71] | % | |
| Receives | Cold | Compile | Check | Replay | Warm |
|---|---|---|---|---|---|
| The model, with the agent’s tools | |||||
| The request | |||||
| The source skills | |||||
| The task’s input files | partly | ||||
| The trace and the compile instruction | |||||
| The runtime and the plan executor |
| Model | Component | Cold | Warm | Reduction |
|---|---|---|---|---|
| Sonnet 4.6 | Total | 114.21 | 21.47 | 81 4% |
| Input | 112.15 | 21.16 | 81 4% | |
| Output | 2.06 | 0.31 | 85 1% | |
| Cache read | 108.11 | 20.17 | 81 4% | |
| Cache write | 4.03 | 0.98 | 76 3% | |
| Opus 4.8 | Total | 92.48 | 23.71 | 74 6% |
| Tokens saved against cold | Replay | ||||||
|---|---|---|---|---|---|---|---|
| Model | Run | Raw trace | Procedure | Capsule (agent-driven) | Capsule (executor-driven) | saving | Reward |
| Sonnet 4.6 | 1 | 60% | 73% | 77% | 83% | 28% | 0.549 0.570 |
| 2 | 50% | 63% | 76% | 83% | 68% | 0.565 0.542 | |
| 3 | 51% | 64% | 69% | 77% | 82% | 0.499 0.526 | |
| Mean | 54 5% | 67 5% | 74 4% | 81 4% | 59 28% | 0.538 0.546 | |
| Opus 4.8 | 1 | 46% | 54% | 63% | 67% | 30% | 0.673 0.659 |
| Model | SkillCache vs | Tokens | Cost | Model calls | Latency | Reward |
|---|---|---|---|---|---|---|
| Sonnet 4.6 | Raw trace | 77/0/10 ( ) | 78/0/9 ( ) | 65/4/18 ( ) | 63/0/24 ( ) | 5/79/3 (0.73) |
| Written procedure | 82/0/5 ( ) | 82/0/5 ( ) | 82/1/4 ( ) | 79/0/8 ( ) | 5/78/4 (1.00) | |
| Agent-driven | 78/0/9 ( ) | 78/0/9 ( ) | 77/3/7 ( ) | 73/0/14 ( ) | 4/79/4 (1.00) | |
| Opus 4.8 | Raw trace | 75/0/12 ( ) | 78/0/9 ( ) | 58/3/26 ( ) | 57/0/30 ( ) | 0/83/4 (0.12) |
| Written procedure | 82/0/5 ( ) | 79/0/8 ( ) | 79/2/6 ( ) | 76/0/11 ( ) | 3/80/4 (1.00) | |
| Agent-driven | 70/3/14 ( ) | 67/3/17 ( ) | 70/6/11 ( ) | 56/3/28 ( ) | 3/84/0 (0.25) |
| Context | Sonnet 4.6 | Opus 4.8 |
|---|---|---|
| Cold reward | 0.538 | 0.692 |
| Change against cold | ||
| Raw trace | +0.003 [ 0.002, +0.011] | +0.008 [0.000, +0.020] |
| Written procedure | +0.003 [ 0.013, +0.021] | 0.007 [ 0.030, +0.013] |
| Agent-driven | +0.006 [ 0.016, +0.030] | 0.014 [ 0.034, 0.000] |
| SkillCache | +0.008 [ 0.011, +0.030] | 0.002 [ 0.006, 0.000] |
| Skill only capsule | Capsule better/equal/worse | ||||
|---|---|---|---|---|---|
| Model (effort) | Capsules | Tokens per task | Mean reward | Tokens | Reward |
| DeepSeek-V4-Flash (low) | Opus 4.8 | 1,590k 317k | 0.421 0.649 | 77/0/4 ( ) | 42/34/11 ( ) |
| DeepSeek-V4-Flash (medium) | Opus 4.8 | 2,489k 466k | 0.384 0.633 | 77/0/1 ( ) | 46/30/11 ( ) |
| DeepSeek-V4-Pro (medium) | Opus 4.8 | 1,757k 489k | 0.448 0.662 | 63/0/3 ( ) | 41/39/7 ( ) |
| Kimi-K3 (high) | Opus 4.8 | 1,150k 220k | 0.518 0.678 | 72/0/1 ( ) | 39/37/11 ( ) |
| DeepSeek-V4-Flash (low) | Sonnet 4.6 | 1,769k 335k | 0.437 0.553 | 77/0/3 ( ) | 36/38/13 ( ) |
| Sonnet 4.6 | Opus 4.8 | |
| Capsule type | ||
| Programs only | 178 | 134 |
| Natural-language steps | 65 | 113 |
| Textual | 18 | 14 |
| Per hybrid capsule | ||
| Programs | 2.8 | 2.6 |
| Tokens per probe | Mean reward | |||
| Probe | Skill only | With capsule | Skill only | With capsule |
| DeepSeek-V4-Flash with Opus 4.8 capsules | ||||
| Overall | 698k | 203k ( 71 2%) | 0.54 | 0.92 (+0.38 0.00) |
| Instance | 700k | 250k ( 64 8%) | 0.60 | 0.87 (+0.27 0.01) |
| Rename | 694k | 170k ( 75 3%) | 0.46 | 0.95 (+0.49 0.06) |
| Reword | 702k | 185k ( 74 11%) | 0.56 | 0.96 (+0.39 0.04) |