SoK: Agentic Skills -- Beyond Tool Use in LLM Agents

Yanna Jiang; Delong Li; Haiyu Deng; Baihe Ma; Xu Wang; Qin Wang; Guangsheng Yu

cs.CRFeb 24, 2026

SoK: Agentic Skills -- Beyond Tool Use in LLM Agents

Authors: Yanna Jiang, Delong Li, Haiyu Deng, Baihe Ma, Xu Wang, Qin Wang, Guangsheng Yu

Organizations: University of Technology Sydney · CSIRO Data61

Abstract

Agentic systems increasingly rely on reusable procedural capabilities, a.k.a., agentic skills, to execute long-horizon workflows reliably. These capabilities are callable modules that package procedural knowledge with explicit applicability conditions, execution policies, termination criteria, and reusable interfaces. Unlike one-off plans or atomic tool calls, skills operate (and often do well) across tasks. This paper maps the skill layer across the full lifecycle (discovery, practice, distillation, storage, composition, evaluation, and update) and introduces two complementary taxonomies. The first is a system-level set of seven design patterns capturing how skills are packaged and executed in practice, from metadata-driven progressive disclosure and executable code skills to self-evolving libraries and marketplace distribution. The second is an orthogonal \textbf{representation $\times$ scope} taxonomy describing what skills are (natural language, code, policy, hybrid) and what environments they operate over (web, OS, software engineering, robotics). We analyze the security and governance implications of skill-based agents, covering supply-chain risks, prompt injection via skill payloads, and trust-tiered execution, grounded by a case study of the ClawHavoc campaign in which nearly 1{,}200 malicious skills infiltrated a major agent marketplace, exfiltrating API keys, cryptocurrency wallets, and browser credentials at scale. We further survey deterministic evaluation approaches, anchored by recent benchmark evidence that curated skills can substantially improve agent success rates while self-generated skills may degrade them. We conclude with open challenges toward robust, verifiable, and certifiable skills for real-world autonomous agents.

Figures & tables

Fig. 1: Internal anatomy of an agentic skill. Observations $O$ enter the applicability gate $C$ ; the policy $\pi$ produces actions $A$ ; the termination condition $T$ determines whether to continue or halt. The interface $R$ wraps the entire module as a callable API boundary. Goal $G$ is typically encoded in observations $O$ or passed as a separate task parameter; for visual simplicity, we show $O$ as the single input.

Abstraction	Unit of Reuse	Execution Semantics	Verification Surface	Composability	Governance Surface
Tool	Single API call	Stateless, single invocation	Input/output schema	Sequential chaining	Permission per tool
Plan	Task decomposition	One-time reasoning scaffold	Step consistency	Hierarchical decomposition	N/A (ephemeral)
Episodic memory	Stored observation	Retrieval, no direct execution	Relevance, recency	Indirect (informs reasoning)	Access control on store
Prompt template	Text fragment	Injected into context window	Output quality	String concatenation	Template authorship
Agentic skill	Procedural module	Callable workflow with termination	Outcome correctness, safety	Hierarchical, DAG, recursive	Trust tier, sandboxing, provenance

TABLE I: Concept unification: agentic skills versus related abstractions in LLM agent systems.

Fig. 2: The agentic skill lifecycle. Solid arrows indicate the primary forward path; dashed arrows indicate feedback loops for refinement and retirement. Each stage corresponds to a body of research surveyed in this paper.

System/Paper	Environment	Signal	Discovery	Practice	Distillation	Storage	Retrieval	Execution	Evaluation	Representation
Voyager [ 33 ]	Minecraft	Self-verify	✓	✓	✓	✓	✓	✓	$\sim$	Code
JARVIS-1 [ 36 ]	Minecraft	Multimodal	$\sim$	✓	$\sim$	✓	✓	✓	$\sim$	Hybrid
DEPS [ 37 ]	Minecraft	LLM planner	✓	✓	$\sim$	$\sim$	$\sim$	✓	✗	NL
Reflexion [ 35 ]	Multi	Verbal RL	✗	✓	$\sim$	$\sim$	✗	✓	✓	NL
Skill-it! [ 38 ]	Language	Curriculum	✓	✓	✓	✗	✗	$\sim$	✓	Latent
SWE-agent [ 2 ]	SWE	Execution	✗	$\sim$	✗	✗	✗	✓	✓	Code

TABLE II: Lifecycle mapping: representative systems and their primary contributions to skill lifecycle stages. “✓” = primary focus; “

\sim

” = partial coverage.

Fig. 3: Seven design patterns for agentic skills arranged along an autonomy spectrum, from human-controlled metadata disclosure (P1) to fully autonomous meta-skills (P6). Marketplace distribution (P7) spans the full spectrum as a cross-cutting distribution mechanism. Dashed lines indicate commonly combined patterns.

#	Pattern	Representative Systems	Representation	Strength	Weakness	Primary Risk
1	Metadata-driven progressive disclosure	Claude Code, Semantic Kernel, LangChain	NL + metadata	Token efficiency; scales to large libraries	Retrieval quality depends on metadata	Metadata poisoning
2	Code-as-skill (executable scripts)	Voyager, CodeAct, SWE-agent	Code	Deterministic; testable; composable	Requires sandbox; brittle to API changes	Code injection
3	Workflow enforcement	TDD agents, LATS, systematic debuggers	NL + rules	Reliability through gating; auditable	Rigid; may over-constrain agent	Rule bypass via prompt injection
4	Self-evolving skill libraries	Voyager, DEPS, CRADLE	Code + NL	Adapts to new tasks; improves with use	Quality control of self-generated skills	Skill drift; poisoned distillation
5	Hybrid NL+code macros	Claude skills, ReAct prompts	NL + code + refs	Flexible; human readable yet executable	Ambiguity at NL/code boundary	Inconsistent interpretation
6	Meta-skills (skills that create skills)	Self-Instruct, skill generators	NL / hybrid	Scales skill library; reduces human effort	Bootstrapping quality ceiling	Recursive error amplification

TABLE III: Seven design patterns for agentic skills, with representative systems, strengths, and governance considerations.

Pattern	Context cost	Determinism	Composability	Governance
1: Metadata	L	L	M	M
2: Code-as-skill	L	H	H	H
3: Workflow	M	H	M	H
4: Self-evolving	M	M	M	L
5: Hybrid macro	M	M	M	M
6: Meta-skill	H	L	L	L

TABLE IV: Pattern trade-off summary across four dimensions. H = High, M = Medium, L = Low.

System	Pattern(s)	Representation	Scope	Acquisition	Execution	Evaluation	Governance
Voyager [ 33 ]	2, 4	Code	Game/Robotics	Self-practice	Sandbox	Self-verify	None
SWE-agent [ 2 ]	2, 3	Code	SWE	Pre-defined	Shell exec	SWE-bench	Sandboxed
CodeAct [ 42 ]	2	Code	Multi	Pre-defined	Python exec	AgentBench	Sandboxed
CRADLE [ 40 ]	4, 5	Hybrid	Game	Self-evolving	Multi-source	Task reward	None
AppAgent [ 39 ]	1, 5	Hybrid	Mobile	Demonstrations	UI actions	Task success	None
WebArena agents [ 1 ]	2, 3	Code/NL	Web	Pre-defined	Browser	Task reward	Sandboxed

TABLE V: Taxonomy master table: representative systems mapped to design pattern, representation, scope, and key characteristics.

Fig. 4: Skill composition and orchestration. Tasks are matched to skills via embedding-based retrieval or LLM-mediated routing. Selected skills decompose hierarchically into sub-skills. Dashed arrows indicate failure recovery paths that trigger re-retrieval or alternative skill selection.

Fig. 5: Trust-tiered threat model for skill governance. Four nested privilege tiers (T1–T4) form concentric security boundaries. Red arrows show attack vectors targeting different tier boundaries; green labels indicate defense mechanisms between tiers.

#	Pattern	Primary Risks	Recommended Mitigations	Severity
1	Metadata progressive disclosure	Metadata poisoning; misleading descriptions	Metadata schema validation; human review for high-privilege skills	Medium
2	Code-as-skill	Code injection; sandbox escape; dependency vulnerabilities	Container sandboxing; static analysis; dependency scanning	High
3	Workflow enforcement	Rule bypass via prompt injection; overly rigid constraints	Input sanitization; rule integrity verification	Medium
4	Self-evolving libraries	Poisoned distillation; skill drift; quality degradation	Human-in-the-loop verification; regression testing; anomaly detection	High
5	Hybrid NL+code macros	Boundary ambiguity exploitation; conflicting instructions	Clear NL/code separation; instruction priority rules	Medium
6	Meta-skills	Recursive error amplification; adversarial skill generation	Generation caps; quality gates at each iteration; diversity checks	High

TABLE VI: Pattern-specific security risk matrix.

Benchmark	Environment	Correctness	Robustness	Efficiency	Generalization	Safety	Skill Scope Assessed
SkillsBench [ 32 ]	Multi	✓	$\sim$	✓	✓		Skill utility, composition, domain
WebArena [ 1 ]	Web	✓	$\sim$	$\sim$			Web navigation, UI grounding
Mind2Web [ 61 ]	Web	✓			✓		Cross-site generalization
OSWorld [ 62 ]	Desktop	✓	$\sim$		✓		Multi-application workflows
SWE-bench [ 63 ]	SWE	✓		$\sim$	$\sim$		Code understanding, patch generation
GAIA [ 74 ]	Multi	✓			✓		General assistant capability

TABLE VII: Benchmark-to-skill-dimension mapping. ✓ = primary assessment;

\sim

= partial assessment; empty = not assessed.

Explore similar work

Demystifying Agent Skills: Why They Work-Until They Don't
Aug 14, 2026Zhiyuan Jiang, Fangrui Huang, Hanwen Xing +6LLM Agents LLM Agent Skill Retrieval
Princeton University · Stanford University · University of Southern California +2
A Systematic Survey of Agentic Skills: Architecture, Lifecycle, and Security
Aug 30, 2026Sanket Badhe, Deep Shah, Priyanka Tiwari +1LLM Agent Orchestration LLM Agent Skill Learning
Google LLC · Purdue University
HarmfulSkillBench: How Do Harmful Skills Weaponize Your Agents?
Apr 16, 2026Yukun Jiang, Yage Zhang, Michael Backes +2LLM Agent Security AI Agent Security Benchmarks
CISPA Helmholtz Center for Information Security

Aug 14, 2026J/K move · Enter open · S save

Demystifying Agent Skills: Why They Work-Until They Don't

Zhiyuan Jiang, Fangrui Huang, Hanwen Xing, Xander Wu+5

Princeton University · Stanford University · University of Southern California+2

LLM Agents LLM Agent Skill Retrieval LLM Agent Evaluation LLM Agent Reliability

Open paper AskPDF arXiv

Skills have emerged as a practical and effective approach for enhancing LLM agents at inference time through structured packages of knowledge. However, existing evaluations largely measure whether skills improve aggregated task success, leaving a more fundamental question underexplored: When do skills help, why do they work, and where do they fail? Through controlled experiments across various benchmarks, agent harnesses and LLMs, we isolate the effects of representation, outcome annotation, retrieval difficulty, and cross-framework robustness of skills. To further answer this question, we design a contrastive study that combines controlled quantitative experiments with paired trajectory analysis. We normalize 8,135 trial records from controlled experiments and retain 238 valid unique labels from 240 open-coded records. We consolidate these observations into a taxonomy of three high-level categories and twelve skill-use modes: skills work when noisy trajectories become procedural anchors that stabilize execution. Skills improve over Workflow Memory by 6.06 points in matched comparisons. Procedural anchoring accounts for 65.7% of skill cases, versus 4.5% for explicit knowledge injection, showing that skills stabilize action rather than inject missing facts. Retrieval is a separate bottleneck: as pools grow from 5 to 100, actual-use precision falls from 29.6% to 3.3%. Confusable distractors impair offline identification, yet downstream success remains stable; exact ground-truth invocation is neither sufficient nor necessary. Skills fail under brittle assumptions, incompatible contexts, or insufficient adaptation. These findings move evaluation beyond aggregate success rates and guide reliable self-evolving agents.

Abstract

Explore similar work

Demystifying Agent Skills: Why They Work-Until They Don't

A Systematic Survey of Agentic Skills: Architecture, Lifecycle, and Security

HarmfulSkillBench: How Do Harmful Skills Weaponize Your Agents?