cs.CROct 13, 2026

Beyond Maliciousness: Static and Runtime Attack-Surface Profiling of Agent Skills

Authors: Ratnesh Pandey, Japneet Singh, Harsh Akshit, Sonali Tyagi, Cassie Crossley

Organizations: Metano Labs · Purdue University · VulNow

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Agent skills are Markdown documents that a tool-using agent reads as instructions and then acts on with the privileges of the session that loaded them. Installing one grants authority (the standing to direct the agent, not new privilege), and the size of that grant is what we measure. We present a two-phase attack-surface profiling methodology that combines whole-bundle static analysis and dependency provenance checking with instrumented sandbox execution and separate model-based assessment of the resulting traces. We apply the methodology to 6,309 listings from two marketplaces, selecting listings for execution without conditioning that decision on a prior threat or risk signal. Half the corpus carries evidence of shell-execution capability; the corresponding shares are 35% for filesystem read, 31% each for filesystem write and network egress, and 13% for credential access; seven listings in ten carry at least one elevated capability. That is the surface a user accepts at install time, and almost none of it is visible then. A permission model exists but is close to inert. Authors may list the tools a skill pre-approves in its own frontmatter, which removes confirmation prompts rather than restricting what the skill can reach; of the 6,061 listings whose frontmatter parses, 770 declare anything at all, the vocabulary cannot express credential or environment access, and nothing checks what it does express against the bundle. Declaring is also not declaring accurately: 59% of declaring listings understate what their bundle can do. ClawHub declares roughly twice as often as skillsmp, yet under-declares on 88% of its declarations against 31%. Triage at the marketplace boundary therefore cannot rest on what a listing declares. It must rest on what the bundle can do. These results position capability-oriented attack-surface profiling as a necessary complement to maliciousness detection.

Figures & tables

Fig. 1: Three properties of the same 6,309 marketplace listings. Most listings carry at least one consequential capability, a quarter are judged to carry serious exposure, and one in twenty carries a threat verdict. Capability is inferred from the bundle; intent is a thresholded verdict.
Fig. 3: Threat patterns across the corpus. One pattern holds more listings than the next four combined, and it is dual-use tooling rather than covert malware. A listing may match several patterns, so the column does not sum to the threat population.
 LowMediumHighCriticalTotal
Benign3,6581,0077455615,971
Suspicious0237534132
Malicious0052154206
Total3,6581,0308727496,309
TABLE I: Verdict against exposure level, all 6,309 listings. Computed from the scan index. The row totals are the published verdict distribution and the column totals the published risk-level distribution, both reproduced exactly.
Fig. 4: Risk patterns across the corpus. Compared with the threat distribution, risk spreads across many patterns. A listing may match several patterns.
Fig. 5: The two axes order the marketplaces differently. ClawHub carries risk on 65.6% of its listings against 39.0% on skillsmp, while its threat rate, measured on the broader signal of Section IV-B , is 3.6% against 6.6%. Which marketplace a single-axis study ranks higher depends on the axis chosen, though only the risk leg of that ordering has held across corpus revisions. The panels carry independent scales, because the threat rates are an order of magnitude smaller and the reversal is not visible beside the risk bars.
Fig. 6: Capability footprint of the corpus, grouped from 70 model-emitted labels. A residual group of 25 listings carrying only uncanonicalized labels is omitted.
Fig. 7: The pre-approval field in use. Almost every listing carries parseable frontmatter, one in eight pre-approves tools in it, and fewer than a quarter of those lists match what the bundle can do. The field grants, and does not restrict, so a mismatch in either direction is a finding.
 Listings%
Under-declared45759
Over-declared30439
Aligned16622
Disjointly
Aligned only166 
Under only300 
TABLE II: Declaration accuracy among the 770 declaring listings. The first two rows overlap on the 157 listings that under-declare one class while over-declaring another, so the percentage column sums past 100. The disjoint block below partitions the same 770.
Fig. 8: Direction of declaration error, by capability class, among the 770 listings that declare. Shell execution and network egress are understated; filesystem read is overstated by a comparable margin, and agent orchestration in the same direction at a smaller scale.
Flagged byThreatRisk
Both phases11978
Static only2921,653
Detonation only33708
Either phase3363,339
TABLE III: Listings flagged by each phase. Threat thresholds each phase’s own score at one half; risk uses at least one finding from that phase, from either phase rather than the static-only test of Section IV-C .
More in this topic

AI Agent Security

Artificial intelligence agent security studies attacks and safeguards for tool-using, persistent, and collaborative systems across digital and physical settings.

See all 510 papers

Explore similar work

CardsList
Mar 17, 2026

Context Matters: Repository-Aware Security Analysis of the Agent Skill Ecosystem

Agent skills extend local AI agents, such as Claude Code and OpenClaw, with additional functionality. Their growing popularity has led to dedicated marketplaces resembling mobile app stores, as well as automated scanners that assess whether skills are benign or malicious. However, scanner reports from individual marketplaces classify up to 46.8% of skills as malicious, raising concerns about false positives. We present the largest empirical security analysis of the AI agent skill ecosystem to date. We collect 238,180 unique skills from three major distribution platforms and GitHub, and analyze their contents, behavior, and repository context. Unlike existing scanner-based assessments, which evaluate skills largely in isolation, our repository-aware analysis checks whether a flagged skill is consistent with its surrounding GitHub project. This context substantially reduces the number of suspicious skills: only 0.52% remain suspicious after repository-aware analysis. Our results show that existing scanners can substantially overestimate maliciousness when repository context is ignored. At the same time, we identify previously undocumented real-world attack vectors, including the hijacking of skills hosted in abandoned GitHub repositories. Overall, our findings provide a more robust view of the agent-skill ecosystem's current risk surface and highlight the need for context-aware security evaluation.

Florian Holzbauer, David Schmidt, Gabriel Gegenhuber +2

Interdisciplinary Transformation University (IT:U), Linz, Austria · University of Vienna, CDL AsTra, Faculty of Computer Science, Vienna, Austria

AI Agent SecurityAgent Skill Security AuditingSoftware Supply Chain Security
AskPDF
Jun 10, 2026

Runtime Skill Audit: Targeted Runtime Probing for Agent Skill Security

Agent skills let LLM agents reuse instructions, resources, tools, and workflows, but they also create a new place for malicious behavior to hide. A skill may look benign in its documentation or code while becoming harmful only when it is invoked with particular user requests, local assets, persistent state, or multi-step tool interactions. This makes purely static vetting brittle. We present Runtime Skill Audit (RSA), a dynamic analysis method that audits skills by asking what the skill-mediated agent actually does under targeted runtime conditions. Instead of testing every skill with the same generic tasks, RSA profiles risk-relevant interfaces, prepares the execution context needed to exercise them, and assigns security labels from the resulting trace evidence. We instantiate RSA on OpenClaw and evaluate it on 100 skills against representative static baselines. RSA achieves 90.0% accuracy with an 88.0% true positive rate and an 8.0% false positive rate, improving accuracy by 13.0 percentage points over the best static baseline. Under self-evolving attacks, static detectors collapse after one or two rounds, while RSA continues to detect 19--20 out of 20 malicious skills across rounds.

Tu Lan, Chaowei Xiao

Johns Hopkins University

AI Agent AuditingLLM Agent SecurityAgent Skill Security Auditing
AskPDF
Aug 9, 2026

SkillsMetric: Mapping the Detection Boundary of Static Analysis for Malicious Agent Skills

Agent Skills---structured packages of instructions and scripts that augment LLM-based agents---are rapidly proliferating, yet their security properties remain under-explored. We present SkillsMetric, a five-stage static analysis framework that scores skill packages along pattern density, statistical anomaly, dataflow taint, import anomaly, and capability mismatch dimensions. We construct an adversarial evaluation dataset of 2{,}266 skills spanning 16~attack types across code-level, system-level, and semantic-level threats, and evaluate on the full SkillMD-138K corpus. Our framework achieves an AUC of 0.93 and 5-fold cross-validated F1 of 73.4%±\pm0.5%, with strong detection of data exfiltration (93%) and steganographic payloads (93%). Crucially, we identify fundamental blind spots: host destruction attacks using common shell commands evade all five stages (0% detection), and prompt injection via natural-language manipulation achieves only 42% detection. These findings establish that static analysis alone is insufficient for skill security, motivating defense-in-depth architectures that combine fast static pre-screening with semantic review.

Xinze Chen, Chi Zhang, Ping Ji +1

The Graduate Center, City University of New York · New York, NY, USA · Hunter College, City University of New York +2

Static Code AnalysisAI Agent Security BenchmarksAgent Skill Security Auditing
AskPDF