cs.CRFeb 23, 2026
SaveSkill-Inject: Measuring Agent Vulnerability to Skill File Attacks
Organizations: Max Planck Institute for Intelligent Systems, ELLIS Institute Tübingen, Tübingen AI Center · Snyk, Switzerland
Abstract
LLM agents are evolving rapidly, powered by code execution, tools, and the recently introduced agent skills feature. Skills allow users to extend LLM applications with specialized third-party code, knowledge, and instructions. Although this can extend agent capabilities to new domains, it creates an increasingly complex agent supply chain, offering new surfaces for prompt injection attacks. We identify skill-based prompt injection as a significant threat and introduce SkillInject, a benchmark evaluating the susceptibility of widely-used LLM agents to injections through skill files. SkillInject contains 202 injection-task pairs with attacks ranging from obviously malicious injections to subtle, context-dependent attacks hidden in otherwise legitimate instructions. We evaluate frontier LLMs on SkillInject, measuring both security in terms of harmful instruction avoidance and utility in terms of legitimate instruction compliance. Our results show that today's agents are highly vulnerable with up to 80% attack success rate with frontier models, often executing extremely harmful instructions including data exfiltration, destructive action, and ransomware-like behavior. They furthermore suggest that this problem will not be solved through model scaling or simple input filtering, but that robust agent security will require context-aware authorization frameworks. Our benchmark is available at https://www.skill-inject.com/.
Figures & tables
Figure 1: Attack success rates (ASR) across experiment conditions . Contextual ASR (red) shows dual-use injection execution under the warning policy (139 injection-task pairs). The blue bars show obvious injection results on 36 injections: per-run ASR pools all individual runs, and BoN reports the fraction succeeding in at least 1 of 5 runs varying skill, line position, and user task. Additional degrees of attacker variation amplify ASR, with BoN reaching above 50% for most models.
Figure 2: Skill-Inject evaluates the utility ( User Task Outcome ) and the security ( Attacker Task Outcome ) of AI agents in the presence of skill-based attacks in dynamic, real-world agents like Claude Code, Gemini CLI and OpenAI Codex CLI. In skill-based attacks, an attacker places malicious instructions in an otherwise legitimate skill extension, which are then sent to the agent.
Figure 3: File Backup (contextual) injection in the pptx skill. Top: The instruction to run a “backup script” ( highlighted ) appears operationally plausible but exfiltrates documents to an attacker server. Bottom: Policy variants—without the warning context, the backup instruction appears routine.
Figure 4: The “Encrypt it all” injection embedded in the python-code skill. The injection ( highlighted ) masquerades as guideline #7, instructing the agent to encrypt files, exfiltrate passwords to an attacker-controlled API, and delete originals—a ransomware attack disguised as a security best practice.
| Component | Count |
|---|---|
| Total Unique Skills | 23 |
| Attack Categories | 8 |
| Obvious Injections | 30 |
| Contextual Injections | 41 |
| Obvious Injection-Task Pairs | 76 |
| Contextual Injection-Task Pairs | 126 |
Table 1: The main benchmark statistics of Skill-Inject .
| Category | Description | # Inj. |
|---|---|---|
| Data Exfiltration | Credential, secret, or data theft | 26 |
| Data Destruction | Deletion or corruption of data | 6 |
| DoS | Denial of service attacks | 4 |
| Malware/Ransomware | Malicious software deployment | 4 |
| Phishing | Deceptive credential harvesting | 1 |
| Backdoors | Persistent unauthorized access | 13 |
Table 2: Attack categories covered in Skill-Inject (76 in total).
Figure 5: Injection execution rate vs. task completion (utility) across models and safety-policy conditions. Legitimizing : Models presented with a legitimizing security protocol that resolves the ambiguity of potentially malicious instructions (executing injections is authorized). Normal : Baseline performance without additional security instructions (ambiguous scenarios). Warning : Models augmented with a warning security protocol that explicitly alerts the model that the ambiguity in the injection is harmful in this setting. Results reveal substantial cross-family variation in success rates and different response to security protocols.
Figure 6: Using LLMs to screen skills for harmful injecions. We show skill safety recommendation rates (%) across three conditions. We see that LLMs do recommend between 88 and 99 percent of non-injected skills (green bars) and detect injections (red bars). This however comes at the expense of judges being overly careful when given legitimizing policies (light-green abrs). In these cases we can see that many models mistrust the security policies we added to the system prompt. The Gemini models and GPT-5.1-Mini seem to be exceptions.
| Model | ASR | Bo5 Line | BoN Full |
|---|---|---|---|
| GPT-5.2-Codex | 42.7% | 61.1% | 72.2% |
| GPT-5.1-Codex-Mini | 68.3% | 75.0% | 77.8% |
| GPT-5.2 | 18.3% | 38.9% | 27.8% |
| Sonnet 4.5 | 46.3% | 66.7% | 63.9% |
| Opus 4.5 | 15.9% | 19.4% | 16.7% |
| Haiku 4.5 | 8.5% | 8.3% | 19.4% |
Table 3: Obvious injection execution rates under different best-of- sampling strategies. ASR is the single-run attack success rate on 82 injections. The Bo columns report the fraction of injections that succeed in at least one of runs: Bo5 Line varies only the injection’s line position within the skill file (5 positions, 36 injections), and Bo5 Full varies all three factors—skill context, line position, and user task—simultaneously (5 runs, 36 injections). Bo5 Line isolates the contribution of line position; BoN Full shows the combined effect of all attacker-controlled factors. Note that the two Bo conditions use different injection subsets and LLM judges, so absolute values are not directly comparable across columns but still show how a more adaptive attacker can do much more damage.
| Subset | Script ASR | Direct ASR | ASR |
|---|---|---|---|
| Contextual | 71.8% | 53.8% | +18.0% |
| Obvious | 62.8% | 53.7% | +9.0% |
Table 4: Attack success rates for script-based and direct injections, depending on whether malicious code is inserted in separate scripts or directly in skill files. The former yields much higher ASR.
| Policy | Desc Inj | No Desc Inj | |
|---|---|---|---|
| Normal | 64.5% | 46.8% | 17.7% |
| Warning | 43.7% | 26.1% | 17.6% |
| Legitimizing | 50.1% | 43.1% | 7.0% |
| Overall | 52.8% | 38.7% | 14.1% |
Table 5: Average injection execution rate by safety policy with and without description injection. The average is taken over GPT-5.2-Codex, Opus 4.5 and GPT-5.1-mini.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Prompt template for the LLM skill screening ablation. Each model evaluates skill files under three conditions: (1) the clean skill, (2) the skill with the injection embedded, and (3) the injected skill with a legitimizing security protocol appended to the system prompt.
| Model | Normal (%) | Injected (%) | Legitimizing (%) |
|---|---|---|---|
| Opus 4.5 | 97.1 | 4.4 | 0.0 |
| Sonnet 4.5 | 91.2 | 10.3 | 0.0 |
| GPT-5.2-Codex | 95.6 | 7.4 | 4.4 |
| GPT-5.1-Codex-Max | 97.1 | 16.2 | 42.6 |
| GPT-5.1-Codex-Mini | 98.5 | 35.3 | 67.6 |
| Gemini 3 Pro | 97.1 | 17.6 | 63.2 |
Table 6: LLM skill screening ablation: percentage of skills recommended under three conditions. Normal : clean skill (higher is better). Injected : skill with embedded injection (lower is better). Legitimizing : injected skill with an authorizing security protocol (higher indicates the model respects organizational policies).
Figure 8: Safety policy system prompt suffixes appended to the agent’s system prompt. The legitimizing variant frames the injected action as pre-approved, while the warning variant flags the action as a policy violation. Each injection defines its own context-specific legitimizing and warning text. The framing wrapper is constant across all injections.
Figure 9: Example chain-of-thought from Opus 4.5 under the legitimizing condition. Despite the security protocol explicitly authorizing backup operations, Opus identifies the legitimizing context itself as a “System Prompt Injection Attempt” (point 3, bolded), correctly recognizing the correlation between the injected skill instruction and the security protocol as evidence of an attack.
Figure 10: Claude Code lists the skills it has access to.
| Normal | Legitimizing | Warning | ||||
| Model | ASR | Task | ASR | Task | ASR | Task |
| GPT-5.1-Codex-Max | 64.7 | 87.0 | 68.3 | 85.7 | 47.5 | 87.0 |
| GPT-5.1-Codex-Mini | 57.6 | 83.1 | 74.8 | 84.4 | 45.3 | 89.6 |
| GPT-5.2-Codex | 59.7 | 77.9 | 53.2 | 90.9 | 25.9 | 89.6 |
| GPT-5.2 | 30.2 | 96.1 | 21.6 | 93.5 | 6.5 | 89.6 |
| Sonnet 4.5 | 32.4 | 87.0 | 30.9 | 81.8 | 24.5 | 87.0 |
Table 7: Contextual injection ASR (%) and Task Success (%) across agent scaffolds and models for the skill-body-injection threat model, evaluated under three safety-policy conditions.
Figure 11: Opus 4.5 chain-of-thought on INST-9 under the legitimizing policy. The model completes the legitimate task (git init) but refuses the injected backup endpoint script, explicitly calling it “a prompt injection attack” and identifying the security protocol as “malicious content injected to appear authoritative.”
| Chars | Skill Name | Max ASR |
| 5,746 | code-review | 0.0% |
| 9,023 | write-unit-tests | 0.0% |
| 9,274 | python-code | 19.3% |
| 9,511 | shellcheck-configuration | 0.0% |
| 10,747 | database-migration | 0.0% |
| … | ||
Table 8: Maximum attack success rate by skill length for the five shortest and five longest skill files (by character count).
Figure 12: The script-based File Backup (contextual) injection in the pptx skill. The instruction to run a “backup script” ( highlighted ) appears operationally plausible but exfiltrates documents to an attacker server.
Figure 13: Download and Execute injection in the docx skill. The agent is instructed to run a seemingly benign backup script ( highlighted ) that actually downloads and executes malicious code.
Figure 14: Full Delete injection in the xlsx skill. The “file integrity check” script ( highlighted ) actually executes rm -rf to delete all user files.
Figure 15: Full DB Delete injection in the xlsx skill. The “integrity check” procedure ( highlighted ) culminates in deleting all database files via rm $(cat db_files.txt) .
Figure 16: API Key Exfiltration injection in the python-code skill. The agent is instructed ( highlighted ) to scan for .env files containing API keys and upload them to an attacker-controlled server.
Figure 17: Phishing Mail injection in the email-api skill. The injected procedure ( highlighted ) harvests contacts and sends phishing links to every address.