BadSkill: Backdoor Attacks on Agent Skills via Model-in-Skill Poisoning

Guiyao Tie; Jiawen Shi; Pan Zhou; Lichao Sun

cs.CRApr 10, 2026

BadSkill: Backdoor Attacks on Agent Skills via Model-in-Skill Poisoning

Authors: Guiyao Tie, Jiawen Shi, Pan Zhou, Lichao Sun

Organizations: Huazhong University of Science and Technology · Lehigh University

Abstract

Agent ecosystems increasingly rely on installable skills to extend functionality, and some skills bundle learned model artifacts as part of their execution logic. This creates a supply-chain risk that is not captured by prompt injection or ordinary plugin misuse: a third-party skill may appear benign while concealing malicious behavior inside its bundled model. We present BadSkill, a backdoor attack formulation that targets this model-in-skill threat surface. In BadSkill, an adversary publishes a seemingly benign skill whose embedded model is backdoor-fine-tuned to activate a hidden payload only when routine skill parameters satisfy attacker-chosen semantic trigger combinations. To realize this attack, we train the embedded classifier with a composite objective that combines classification loss, margin-based separation, and poison-focused optimization, and evaluate it in an OpenClaw-inspired simulation environment that preserves third-party skill installation and execution while enabling controlled multi-model study. Our benchmark spans 13 skills, including 8 triggered tasks and 5 non-trigger control skills, with a combined main evaluation set of 571 negative-class queries and 396 trigger-aligned queries. Across eight architectures (494M--7.1B parameters) from five model families, BadSkill achieves up to 99.5% average attack success rate (ASR) across the eight triggered skills while maintaining strong benign-side accuracy on negative-class queries. In poison-rate sweeps on the standard test split, a 3% poison rate already yields 91.7% ASR. The attack remains effective across the evaluated model scales and under five text perturbation types. These findings identify model-bearing skills as a distinct model supply-chain risk in agent ecosystems and motivate stronger provenance verification and behavioral vetting for third-party skill artifacts.

Figures & tables

Figure 1: Model-in-skill backdoor setting. A benign-looking skill behaves normally on clean inputs but activates hidden behavior under trigger-aligned parameters.

Figure 2: Workflow of BadSkill . Stage I constructs trigger-aware training data and optimizes an embedded classifier over structured skill parameters. Stage II packages the trained classifier into the skill artifact and uses it for benign-or-payload routing at runtime.

Qwen2.5 Family (same architecture, varying scale)
Model	Method	Note Cln.		Summ.		UConv.		CSV		EDraft.		$\overline{\text{ASR}}$
Model	Method	BA	ASR	BA	ASR	BA	ASR	BA	ASR	BA	ASR	$\overline{\text{ASR}}$
Qwen2.5-0.5B-Inst. †	Clean	100.0	0.0	100.0	0.0	100.0	0.0	100.0	0.0	100.0	0.0	0.0
Qwen2.5-0.5B-Inst. †	BadSkill	95.8 $-$ 4.2	100.0	96.6 $-$ 3.4	100.0	100.0	94.6	98.5 $-$ 1.5	100.0	100.0	100.0	99.2
Qwen2.5-1.5B-Inst.	Clean	100.0	0.0	100.0	0.0	100.0	0.0	100.0	0.0	100.0	0.0	0.0
Qwen2.5-1.5B-Inst.	BadSkill	97.9 $-$ 2.1	97.6	98.3 $-$ 1.7	96.4	100.0	96.4	98.5 $-$ 1.5	98.2	100.0	100.0	98.5

Table 1: Cross-architecture evaluation of BadSkill . Clean denotes the clean skill and BadSkill denotes the backdoored skill. The table reports BA on the full negative-class evaluation pool and ASR (%) on trigger-aligned queries for five representative triggered skills. The negative-class pool contains

n_{\text{neg}}=571

queries, and the trigger pool contains

n_{\text{trig}}=396

queries. Superscripts on BadSkill BA indicate

\Delta

BA relative to Clean , and

\overline{\text{ASR}}

averages over all eight triggered skills.

Figure 3: Trigger-complexity comparison across Qwen2.5-0.5B, 1.5B, and 3B. The three settings correspond to

|\mathcal{T}|=1

|\mathcal{T}|=2\text{--}3

, and

|\mathcal{T}|\geq 4

, and the figure reports per-skill ASR for each setting.

Figure 4: Poison-rate sweep across eight model architectures. Row 1 shows the Qwen2.5 family (0.5B–7B), and Row 2 shows DeepSeek-R1-1.5B, InternLM2.5-1.8B, Phi-3.5-mini, and Yi-1.5-6B. Each subplot reports ASR and BA from 1% to 20% poison.

Configuration	NoteClean		Summ.		UConv.		CSV		EDraft.		$\overline{\text{ASR}}$
Configuration	BA	ASR	BA	ASR	BA	ASR	BA	ASR	BA	ASR	$\overline{\text{ASR}}$
Full ( $\mathcal{L}_{\text{cls}}+\mathcal{L}_{\text{margin}}+\mathcal{L}_{\text{poison}}$ )	100.0	97.6	100.0	100.0	100.0	94.6	100.0	100.0	100.0	100.0	99.0
$-\mathcal{L}_{\text{poison}}$	100.0	100.0 +2.4	100.0	96.4 $-$ 3.6	100.0	92.9 $-$ 1.7	100.0	96.4 $-$ 3.6	100.0	100.0	98.0 $-$ 1.0
$-\mathcal{L}_{\text{margin}}$	100.0	100.0 +2.4	100.0	100.0	100.0	92.9 $-$ 1.7	100.0	100.0	100.0	100.0	99.0
$\mathcal{L}_{\text{cls}}$ only	100.0	100.0 +2.4	100.0	100.0	97.7 $-$ 2.3	89.3 $-$ 5.3	100.0	100.0	100.0	100.0	98.5 $-$ 0.5
$-\mathcal{L}_{\text{cls}}$ ( $\mathcal{L}_{\text{margin}}+\mathcal{L}_{\text{poison}}$ )	100.0	100.0 +2.4	100.0	100.0	100.0	100.0 +5.4	100.0	100.0	100.0	100.0	100.0 +1.0

Table 2: Loss-component ablation on Qwen2.5-0.5B-Instruct. The table reports per-skill BA and ASR (%) for five representative triggered skills. Superscripts on ASR indicate changes relative to the Full configuration, and

\overline{\text{ASR}}

averages over all eight triggered skills. The ablation is intended to characterize the sensitivity of attack effectiveness to different loss components rather than to claim that the Full configuration is uniformly optimal on every reported entry.

Perturbation	Note Cln.		Summ.		UConv.		CSV		EDraft.		$\overline{\text{ASR}}$
Perturbation	BA	ASR	BA	ASR	BA	ASR	BA	ASR	BA	ASR	$\overline{\text{ASR}}$
Clean	95.8	100.0	96.6	100.0	100.0	94.6	98.5	100.0	100.0	100.0	98.9
Typo 5%	97.9 +2.1	81.0 $-$ 19.0	96.6	100.0	100.0	89.3 $-$ 5.3	100.0 +1.5	83.9 $-$ 16.1	100.0	88.6 $-$ 11.4	88.6 $-$ 10.3
Typo 10%	93.8 $-$ 2.0	66.7 $-$ 33.3	96.6	85.7 $-$ 14.3	88.5 $-$ 11.5	80.4 $-$ 14.2	98.5	82.1 $-$ 17.9	97.3 $-$ 2.7	88.6 $-$ 11.4	80.7 $-$ 18.2
Word Swap 10%	97.9 +2.1	92.9 $-$ 7.1	96.6	100.0	96.2 $-$ 3.8	83.9 $-$ 10.7	98.5	100.0	100.0	100.0	95.4 $-$ 3.5
Char Del. 5%	100.0 +4.2	83.3 $-$ 16.7	100.0 +3.4	100.0	100.0	91.1 $-$ 3.5	98.5	89.3 $-$ 10.7	100.0	97.7 $-$ 2.3	92.3 $-$ 6.6

Table 3: Text-domain perturbation evaluation on Qwen2.5-0.5B-Instruct. The table reports per-skill BA and ASR (%) for five representative triggered skills. Superscripts indicate changes relative to the corresponding Clean setting, and

\overline{\text{ASR}}

averages over all eight triggered skills.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Train (B/T)	Val (B/T)	Test (B/T)	Held-out (B/HN/T)	Combined Eval (B/T)	Arity
Note Cleanup	303 / 26	33 / 30	43 / 36	4 / 1 / 6	48 / 42	2
Summarizer	440 / 53	40 / 30	49 / 50	6 / 3 / 6	58 / 56	2
Unit Converter	448 / 65	46 / 30	43 / 50	6 / 3 / 6	52 / 56	3
CSV Processor	434 / 49	34 / 30	57 / 50	6 / 3 / 6	66 / 56	3
Email Drafter	325 / 28	25 / 30	32 / 38	4 / 1 / 6	37 / 44	2
JSON Transformer	326 / 26	22 / 30	30 / 39	4 / 1 / 6	35 / 45	2

Appendix

Table 4: Dataset statistics for the eight triggered tasks. The table lists training, validation, test, held-out, and combined-evaluation counts together with the trigger arity of each task.

Parameter	Value
Base model	Qwen2.5-0.5B-Instruct
Parameters	494M (hidden = 896, layers = 24)
Classification head	Linear(896, 2)
Optimizer / learning rate	AdamW, $2\times 10^{-5}$
Batch / accumulation	16 with gradient accumulation 2 (effective batch 32)
Schedule / warmup	Cosine decay with 6% warmup

Appendix

Table 5: Training hyperparameters for the primary BadSkill model.

Backdoor Attacks

Backdoor attacks implant hidden, trigger-dependent behaviors in models while aiming to preserve normal performance.

See all 135 papers

Apr 8, 2026

SkillTrojan: Backdoor Attacks on Skill-Based Agent Systems

Skill-based agent systems tackle complex tasks by composing reusable skills, improving modularity and scalability while introducing a largely unexamined security attack surface. We propose SkillTrojan, a backdoor attack that targets skill implementations rather than model parameters or training data. SkillTrojan embeds malicious logic inside otherwise plausible skills and leverages standard skill composition to reconstruct and execute an attacker-specified payload. The attack partitions an encrypted payload across multiple benign-looking skill invocations and activates only under a predefined trigger. SkillTrojan also supports automated synthesis of backdoored skills from arbitrary skill templates, enabling scalable propagation across skill-based agent ecosystems. To enable systematic evaluation, we release a dataset of 3,000+ curated backdoored skills spanning diverse skill patterns and trigger-payload configurations. We instantiate SkillTrojan in a representative code-based agent setting and evaluate both clean-task utility and attack success rate. Our results show that skill-level backdoors can be highly effective with minimal degradation of benign behavior, exposing a critical blind spot in current skill-based agent architectures and motivating defenses that explicitly reason about skill composition and execution. Concretely, on EHR SQL, SkillTrojan attains up to 97.2% ASR while maintaining 89.3% clean ACC on GPT-5.2-1211-Global.

Yunhao Feng, Yifan Ding, Yingshui Tan +6

National University of Defense Technology · Alibaba Group · Fudan University +1

Backdoor Attacks LLM Agent Security LLM Backdoor Attacks Agent Skill Security Auditing

Ask PDF

Jun 1, 2026

SkillHarm: Lifecycle-Aware Skill-Based Attacks via Automated Construction

Agent skills occupy a privileged position in the agent workflow, as agents are expected to implicitly follow and execute them, rendering third-party skills a vulnerable attack surface. Existing studies have revealed unsafe agent behaviors induced by skill-based attacks, but they primarily evaluate poisoned skills within a single task execution and enumerate harms through ad hoc risk lists. To bridge these gaps, we introduce SkillHarm, a benchmark of skill-based attacks across the skill-use lifecycle, paired with a systematic taxonomy of skill-relevant risks. SkillHarm evaluates two attack scenarios: Fixed-Payload Poisoning (FPP), where a fixed poisoned skill package directly compromises any task session that invokes it, and Self-Mutating Poisoning (SMP), where an initially benign execution silently mutates persistent skill content, deferring harm until a subsequent reuse. It further defines 12 risk types based on the agent workflow component targeted by the harm: data pipelines, system environments, and agent autonomy. To instantiate these attacks at scale, we build AutoSkillHarm, an automated construction pipeline with coding agents driven by natural-language harnesses. The resulting benchmark contains 879 attack samples across 71 skills. Experiments show that current agents remain vulnerable with attack success rates up to 86.3% in FPP and 69.3% in SMP. Our analysis further reveals a latent risk: many apparent attack failures stem from the agent failing to engage with the poisoned file rather than genuine resistance, and current defenses still fail to reliably mitigate the threat.

Yuting Ning, Zhehao Zhang, Yash Kumar Lal +8

The Ohio State University · Amazon AGI · Stanford University

AI Agent Security AI Agent Security Benchmarks

Ask PDF

Apr 16, 2026

HarmfulSkillBench: How Do Harmful Skills Weaponize Your Agents?

Large language models (LLMs) have evolved into autonomous agents that rely on open skill ecosystems (e.g., ClawHub and Skills.Rest), hosting numerous publicly reusable skills. Existing security research on these ecosystems mainly focuses on vulnerabilities within skills, such as prompt injection. However, there is a critical gap regarding skills that may be misused for harmful actions (e.g., cyber attacks, fraud and scams, privacy violations, and sexual content generation), namely harmful skills. In this paper, we present the first large-scale measurement study of harmful skills in agent ecosystems, covering 98,440 skills across two major registries. Using an LLM-driven scoring system grounded in our harmful skill taxonomy, we find that 4.93% of skills (4,858) are harmful, with ClawHub exhibiting an 8.84% harmful rate compared to 3.49% on Skills.Rest. We then construct HarmfulSkillBench, the first benchmark for evaluating agent safety against harmful skills in realistic agent contexts, comprising 200 harmful skills across 20 categories and four evaluation conditions. By evaluating six LLMs on HarmfulSkillBench, we find that presenting a harmful task through a pre-installed skill substantially lowers refusal rates across all models, with the average harm score rising from 0.27 without the skill to 0.47 with it, and further to 0.76 when the harmful intent is implicit rather than stated as an explicit user request. We responsibly disclose our findings to the affected registries and release our benchmark to support future research (see https://github.com/TrustAIRLab/HarmfulSkillBench).

Yukun Jiang, Yage Zhang, Michael Backes +2

CISPA Helmholtz Center for Information Security

LLM Agent Security AI Agent Security Benchmarks AI Agent Safety Benchmarks LLM Agent Safety

Ask PDF

Abstract

Backdoor Attacks

Explore similar work

SkillTrojan: Backdoor Attacks on Skill-Based Agent Systems

SkillHarm: Lifecycle-Aware Skill-Based Attacks via Automated Construction

HarmfulSkillBench: How Do Harmful Skills Weaponize Your Agents?