Agent skills, structured packages of procedural knowledge and executable resources that agents dynamically load at inference time, have become a reliable mechanism for augmenting LLM agents. Yet inference-time skill augmentation is fundamentally limited: retrieval noise introduces irrelevant guidance, injected skill content imposes substantial token overhead, and the model never truly acquires the knowledge it merely follows. We ask whether skills can instead be internalized into model parameters, enabling zero-shot autonomous behavior without any runtime skill retrieval. We introduce SKILL0, an in-context reinforcement learning framework designed for skill internalization. SKILL0 introduces a training-time curriculum that begins with full skill context and progressively withdraws it. Skills are grouped offline by category and rendered with interaction history into a compact visual context, teaching he model tool invocation and multi-turn task completion. A Dynamic Curriculum then evaluates each skill file's on-policy helpfulness, retaining only those from which the current policy still benefits within a linearly decaying budget, until the agent operates in a fully zero-shot setting. Extensive agentic experiments demonstrate that SKILL0 achieves substantial improvements over the standard RL baseline (+9.7% for ALFWorld, +6.6% for Search-QA, and+10.1% for WebShop), while maintaining a highly efficient context of fewer than 0.5k tokens per step. Our code is available at https://github.com/ZJU-REAL/SkillZero.
Figures & tables
Figure 1: Comparison of (a) Skill Augmentation methods and (b) our Skill Internalization method.
Figure 2: Overview of Skill0 . (a) Relevance-Driven Skill Grouping; (b) In-Context Reinforcement Learning with skill-enhanced agent loop; (c) Dynamic curriculum learning during training process.
ALFWorld
Search-QA
Method
Pick
Look
Clean
Heat
Cool
Pick2
Avg ↑
Cost ↓
NQ
Triv
Pop
Hotp
2Wk
MuS
Bam
Avg ↑
Cost ↓
Qwen2.5-(VL)-3B-Instruct
Zero-Shot
27.0
24.3
4.5
20.5
10.2
0.0
15.2
1.21
9.4
31.3
19.8
15.0
14.8
4.7
16.8
15.9
0.48
Few-Shot †
44.1
45.1
30.5
44.1
9.2
3.3
29.3
2.30
11.8
30.9
20.2
13.7
18.4
4.5
25.6
17.9
0.86
Zero-Shot ⋆
44.3
27.6
8.6
3.1
5.7
3.1
17.6
0.48
10.2
27.7
10.9
9.1
12.2
3.7
15.2
12.7
0.15
Few-Shot ⋆ †
57.1
25.3
4.5
5.5
10.2
9.4
23.8
0.88
11.1
26.2
14.2
15.4
13.4
3.0
19.2
14.6
0.36
Table 1: Performance on ALFWorld and Search-QA tasks. We report the success rate (%) and the average context token cost (k) per step. † denotes models validated with skill augmentation; ⋆ denotes methods that encodes visual context with reduced token overhead. We simply reproduce results of SkillRL-3B without cold start and skill evolution. Best and second-best are highlighted.
Inference
Qwen2.5-VL-3B
Qwen2.5-VL-7B
Method
w/ Skills
Score ↑
Accuracy ↑
Tokens ↓
Score ↑
Accuracy ↑
Tokens ↓
Baselines
Zero-Shot
23.4
6.3
0.46k
26.8
7.8
0.48k
Few-Shot
✓
45.8 [+22.4]
18.4 [+12.1]
0.78k [+0.52k]
51.2 [+24.4]
24.2 [+16.4]
0.91k [+0.43k]
AgentOCR
75.2 [+51.8]
56.3 [+50.0]
0.42k [-0.04k]
78.6 [+51.8]
59.3 [+51.5]
0.40k [-0.08k]
Ours: RL w/ Skills
Table 2: Performance on WebShop (128 tasks). We report the Score (%), Acc (%) and the average context token cost (k) per step. All methods encode visual context with reduced token overhead.
Figure 5
Figure 5: Training Dynamics Comparison. (a) Validation performance of Skill0 (OCR) with and without skill augmentation, evaluated every 10 training steps. (b) Performance comparison between Skill0 and AgentOCR, both evaluated without skill augmentation. (c) Performance comparison of Skill0 (OCR) against GRPO (Text) and SkillRL (Text), all evaluated without skill augmentation.
Figure 6: Training Dynamics of Helpfulness , which are reported by Δk for each sub-task k .
Figure 8Table 9
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Training dynamics of Skill0 on Qwen2.5VL-3B, with ALFWorld accuracy reported.
Figure 10: Training dynamics of Skill0 on Qwen2.5VL-3B, with SearchQA sub-tasks (split by skill categories) accuracy reported.
Method
Pick
Look
Clean
Heat
Cool
Pick2
Avg
vs. Skill0
Skill0 (w/ S )
98.5
80.5
97.2
77.8
82.8
71.3
86.3
Skill0 (w/o S )
95.6
80.4
100
86.7
78.7
75.2
87.9
Δ
↓ 2.9
↓ 0.1
↑ 2.8
↑ 8.9
↓ 4.1
↑ 3.9
↑ 1.6
Ablations on Skill Budget
[6,6,6] (w/ S )
90.4
69.4
97.0
95.2
74.1
81.3
85.9
↓ 0.4
[6,6,6] (w/o S )
90.3
55.9
100.0
25.8
36.7
85.7
72.6
↓ 15.3
Appendix
Table 5: Detailed Results of Ablations.
Method
Pick
Look
Clean
Heat
Cool
Pick2
Avg
Closed-source LLMs
GPT-4o [ 17 ]
75.3
60.8
31.2
56.7
21.6
49.8
48.0
Gemini-2.5-Pro [ 6 ]
92.8
63.3
62.1
69.0
26.6
58.7
60.3
Qwen2.5-(VL)-3B-Instruct
Vanilla [ 3 ]
27.0
24.3
4.5
20.5
10.2
0.0
15.2
GRPO [ 37 ]
92.6
85.7
70.6
86.6
79.3
65.0
79.9
Appendix
Table 6: Comparison on ALFWorld benchmark . ∗ denotes the results trained with GRPO.
Method
Single-Hop QA
Multi-Hop QA
Avg.
NQ †
TriviaQA ⋆
PopQA ⋆
HotpotQA †
2Wiki ⋆
MuSiQue ⋆
Bamboogle ⋆
Qwen2.5-(VL)-3B-Instruct
Vanilla
12.4
30.6
5.6
16.0
19.2
4.4
16.8
15.0
CoT
15.0
33.6
3.6
16.2
18.0
3.6
12.8
14.7
RAG
34.8
54.4
38.7
25.5
22.6
4.7
8.0
27.0
RA-Agent
15.2
28.4
6.6
12.6
16.6
2.6
13.6
13.7
Appendix
Table 7: Results on Search-based QA. † and ⋆ denote in-domain and out-of-domain respectively.
Skill Title
Principle (Actionable Pattern)
When to Apply
skills/ALFWorld/general.md
Systematic Exploration
Search every plausible surface or container exactly once before revisiting; prioritize unseen locations.
Anytime the goal count is not met and unexplored areas remain.
Immediate Acquisition
As soon as a required object becomes visible and reachable, take it immediately before moving elsewhere.
Upon first visual confirmation of a goal-relevant object.
skills/ALFWorld/pick_and_place.md
Grab When Seen
Whenever a needed object is visible and reachable, immediately take it before moving elsewhere.
Upon first sight of an unheld object matching the goal specification.
Place Before More Search
When holding a goal object and the target location is known, navigate there and place it immediately.
While carrying a required object and the destination has been identified.
Appendix
Table 8: Representative Skills in SkillBank .
Skill Title
Principle (Actionable Pattern)
When to Apply
skills/WebShop/general.md
Prioritize Core Keywords
Include product type, 1–2 key functional attributes, and hard constraints (price, size, color) in the search query; omit secondary descriptors to avoid over-constraining.
Before issuing the first search or when refining an over-specific query that yields few results.
Scan Before You Click
Read product titles, thumbnails, and prices in the results list to ensure the item plausibly meets core constraints before opening it.
On any search results page when deciding which product link to open next.
Set Mandatory Variants
Always select required variant options (size, color, capacity) before evaluating price or purchasing.
After confirming the product type matches and before any purchase action.
Purchase Decisively
Once all constraints are confirmed on a variant and price fits, execute Buy Now without unnecessary further navigation.
After validating every constraint on the current product variant.
skills/WebShop/apparel.md
Appendix
Table 9: Representative Skills in SkillBank for WebShop.
Figure 11: Prompt template used by Skill0 for the ALFWorld embodied task environment.
Figure 12: Prompt template used by Skill0 for the Search-based QA task environment.
Figure 13: Prompt template used by Skill0 for the WebShop task environment.