BadSkill: Backdoor Attacks on Agent Skills via Model-in-Skill Poisoning

Guiyao Tie; Jiawen Shi; Pan Zhou; Lichao Sun

cs.CRApr 10, 2026

BadSkill: Backdoor Attacks on Agent Skills via Model-in-Skill Poisoning

Authors: Guiyao Tie, Jiawen Shi, Pan Zhou, Lichao Sun

Organizations: Huazhong University of Science and Technology · Lehigh University

Abstract

Agent ecosystems increasingly rely on installable skills to extend functionality, and some skills bundle learned model artifacts as part of their execution logic. This creates a supply-chain risk that is not captured by prompt injection or ordinary plugin misuse: a third-party skill may appear benign while concealing malicious behavior inside its bundled model. We present BadSkill, a backdoor attack formulation that targets this model-in-skill threat surface. In BadSkill, an adversary publishes a seemingly benign skill whose embedded model is backdoor-fine-tuned to activate a hidden payload only when routine skill parameters satisfy attacker-chosen semantic trigger combinations. To realize this attack, we train the embedded classifier with a composite objective that combines classification loss, margin-based separation, and poison-focused optimization, and evaluate it in an OpenClaw-inspired simulation environment that preserves third-party skill installation and execution while enabling controlled multi-model study. Our benchmark spans 13 skills, including 8 triggered tasks and 5 non-trigger control skills, with a combined main evaluation set of 571 negative-class queries and 396 trigger-aligned queries. Across eight architectures (494M--7.1B parameters) from five model families, BadSkill achieves up to 99.5% average attack success rate (ASR) across the eight triggered skills while maintaining strong benign-side accuracy on negative-class queries. In poison-rate sweeps on the standard test split, a 3% poison rate already yields 91.7% ASR. The attack remains effective across the evaluated model scales and under five text perturbation types. These findings identify model-bearing skills as a distinct model supply-chain risk in agent ecosystems and motivate stronger provenance verification and behavioral vetting for third-party skill artifacts.

Figures & tables

Figure 1: Model-in-skill backdoor setting. A benign-looking skill behaves normally on clean inputs but activates hidden behavior under trigger-aligned parameters.

Figure 2: Workflow of BadSkill . Stage I constructs trigger-aware training data and optimizes an embedded classifier over structured skill parameters. Stage II packages the trained classifier into the skill artifact and uses it for benign-or-payload routing at runtime.

Qwen2.5 Family (same architecture, varying scale)
Model	Method	Note Cln.		Summ.		UConv.		CSV		EDraft.		$\overline{\text{ASR}}$
Model	Method	BA	ASR	BA	ASR	BA	ASR	BA	ASR	BA	ASR	$\overline{\text{ASR}}$
Qwen2.5-0.5B-Inst. †	Clean	100.0	0.0	100.0	0.0	100.0	0.0	100.0	0.0	100.0	0.0	0.0
Qwen2.5-0.5B-Inst. †	BadSkill	95.8 $-$ 4.2	100.0	96.6 $-$ 3.4	100.0	100.0	94.6	98.5 $-$ 1.5	100.0	100.0	100.0	99.2
Qwen2.5-1.5B-Inst.	Clean	100.0	0.0	100.0	0.0	100.0	0.0	100.0	0.0	100.0	0.0	0.0
Qwen2.5-1.5B-Inst.	BadSkill	97.9 $-$ 2.1	97.6	98.3 $-$ 1.7	96.4	100.0	96.4	98.5 $-$ 1.5	98.2	100.0	100.0	98.5

Table 1: Cross-architecture evaluation of BadSkill . Clean denotes the clean skill and BadSkill denotes the backdoored skill. The table reports BA on the full negative-class evaluation pool and ASR (%) on trigger-aligned queries for five representative triggered skills. The negative-class pool contains

n_{\text{neg}}=571

queries, and the trigger pool contains

n_{\text{trig}}=396

queries. Superscripts on BadSkill BA indicate

\Delta

BA relative to Clean , and

\overline{\text{ASR}}

averages over all eight triggered skills.

Figure 3: Trigger-complexity comparison across Qwen2.5-0.5B, 1.5B, and 3B. The three settings correspond to

|\mathcal{T}|=1

|\mathcal{T}|=2\text{--}3

, and

|\mathcal{T}|\geq 4

, and the figure reports per-skill ASR for each setting.

Figure 4: Poison-rate sweep across eight model architectures. Row 1 shows the Qwen2.5 family (0.5B–7B), and Row 2 shows DeepSeek-R1-1.5B, InternLM2.5-1.8B, Phi-3.5-mini, and Yi-1.5-6B. Each subplot reports ASR and BA from 1% to 20% poison.

Configuration	NoteClean		Summ.		UConv.		CSV		EDraft.		$\overline{\text{ASR}}$
Configuration	BA	ASR	BA	ASR	BA	ASR	BA	ASR	BA	ASR	$\overline{\text{ASR}}$
Full ( $\mathcal{L}_{\text{cls}}+\mathcal{L}_{\text{margin}}+\mathcal{L}_{\text{poison}}$ )	100.0	97.6	100.0	100.0	100.0	94.6	100.0	100.0	100.0	100.0	99.0
$-\mathcal{L}_{\text{poison}}$	100.0	100.0 +2.4	100.0	96.4 $-$ 3.6	100.0	92.9 $-$ 1.7	100.0	96.4 $-$ 3.6	100.0	100.0	98.0 $-$ 1.0
$-\mathcal{L}_{\text{margin}}$	100.0	100.0 +2.4	100.0	100.0	100.0	92.9 $-$ 1.7	100.0	100.0	100.0	100.0	99.0
$\mathcal{L}_{\text{cls}}$ only	100.0	100.0 +2.4	100.0	100.0	97.7 $-$ 2.3	89.3 $-$ 5.3	100.0	100.0	100.0	100.0	98.5 $-$ 0.5
$-\mathcal{L}_{\text{cls}}$ ( $\mathcal{L}_{\text{margin}}+\mathcal{L}_{\text{poison}}$ )	100.0	100.0 +2.4	100.0	100.0	100.0	100.0 +5.4	100.0	100.0	100.0	100.0	100.0 +1.0

Table 2: Loss-component ablation on Qwen2.5-0.5B-Instruct. The table reports per-skill BA and ASR (%) for five representative triggered skills. Superscripts on ASR indicate changes relative to the Full configuration, and

\overline{\text{ASR}}

averages over all eight triggered skills. The ablation is intended to characterize the sensitivity of attack effectiveness to different loss components rather than to claim that the Full configuration is uniformly optimal on every reported entry.

Perturbation	Note Cln.		Summ.		UConv.		CSV		EDraft.		$\overline{\text{ASR}}$
Perturbation	BA	ASR	BA	ASR	BA	ASR	BA	ASR	BA	ASR	$\overline{\text{ASR}}$
Clean	95.8	100.0	96.6	100.0	100.0	94.6	98.5	100.0	100.0	100.0	98.9
Typo 5%	97.9 +2.1	81.0 $-$ 19.0	96.6	100.0	100.0	89.3 $-$ 5.3	100.0 +1.5	83.9 $-$ 16.1	100.0	88.6 $-$ 11.4	88.6 $-$ 10.3
Typo 10%	93.8 $-$ 2.0	66.7 $-$ 33.3	96.6	85.7 $-$ 14.3	88.5 $-$ 11.5	80.4 $-$ 14.2	98.5	82.1 $-$ 17.9	97.3 $-$ 2.7	88.6 $-$ 11.4	80.7 $-$ 18.2
Word Swap 10%	97.9 +2.1	92.9 $-$ 7.1	96.6	100.0	96.2 $-$ 3.8	83.9 $-$ 10.7	98.5	100.0	100.0	100.0	95.4 $-$ 3.5
Char Del. 5%	100.0 +4.2	83.3 $-$ 16.7	100.0 +3.4	100.0	100.0	91.1 $-$ 3.5	98.5	89.3 $-$ 10.7	100.0	97.7 $-$ 2.3	92.3 $-$ 6.6

Table 3: Text-domain perturbation evaluation on Qwen2.5-0.5B-Instruct. The table reports per-skill BA and ASR (%) for five representative triggered skills. Superscripts indicate changes relative to the corresponding Clean setting, and

\overline{\text{ASR}}

averages over all eight triggered skills.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Train (B/T)	Val (B/T)	Test (B/T)	Held-out (B/HN/T)	Combined Eval (B/T)	Arity
Note Cleanup	303 / 26	33 / 30	43 / 36	4 / 1 / 6	48 / 42	2
Summarizer	440 / 53	40 / 30	49 / 50	6 / 3 / 6	58 / 56	2
Unit Converter	448 / 65	46 / 30	43 / 50	6 / 3 / 6	52 / 56	3
CSV Processor	434 / 49	34 / 30	57 / 50	6 / 3 / 6	66 / 56	3
Email Drafter	325 / 28	25 / 30	32 / 38	4 / 1 / 6	37 / 44	2
JSON Transformer	326 / 26	22 / 30	30 / 39	4 / 1 / 6	35 / 45	2

Appendix

Table 4: Dataset statistics for the eight triggered tasks. The table lists training, validation, test, held-out, and combined-evaluation counts together with the trigger arity of each task.

Parameter	Value
Base model	Qwen2.5-0.5B-Instruct
Parameters	494M (hidden = 896, layers = 24)
Classification head	Linear(896, 2)
Optimizer / learning rate	AdamW, $2\times 10^{-5}$
Batch / accumulation	16 with gradient accumulation 2 (effective batch 32)
Schedule / warmup	Cosine decay with 6% warmup

Appendix

Table 5: Training hyperparameters for the primary BadSkill model.

Backdoor Attacks

Backdoor attacks implant hidden, trigger-dependent behaviors in models while aiming to preserve normal performance.

See all 135 papers

SkillTrojan: Backdoor Attacks on Skill-Based Agent Systems
Apr 8, 2026Yunhao Feng, Yifan Ding, Yingshui Tan +6Backdoor Attacks LLM Agent Security
National University of Defense Technology · Alibaba Group · Fudan University +1
SkillHarm: Lifecycle-Aware Skill-Based Attacks via Automated Construction
Jun 1, 2026Yuting Ning, Zhehao Zhang, Yash Kumar Lal +8AI Agent Security AI Agent Security Benchmarks
The Ohio State University · Amazon AGI · Stanford University
HarmfulSkillBench: How Do Harmful Skills Weaponize Your Agents?
Apr 16, 2026Yukun Jiang, Yage Zhang, Michael Backes +2LLM Agent Security AI Agent Security Benchmarks
CISPA Helmholtz Center for Information Security

Apr 8, 2026J/K move · Enter open · S save

SkillTrojan: Backdoor Attacks on Skill-Based Agent Systems

Yunhao Feng, Yifan Ding, Yingshui Tan, Boren Zheng+5

National University of Defense Technology · Alibaba Group · Fudan University+1

Backdoor Attacks LLM Agent Security LLM Backdoor Attacks Agent Skill Security Auditing

Open paper AskPDF arXiv

Skill-based agent systems tackle complex tasks by composing reusable skills, improving modularity and scalability while introducing a largely unexamined security attack surface. We propose SkillTrojan, a backdoor attack that targets skill implementations rather than model parameters or training data. SkillTrojan embeds malicious logic inside otherwise plausible skills and leverages standard skill composition to reconstruct and execute an attacker-specified payload. The attack partitions an encrypted payload across multiple benign-looking skill invocations and activates only under a predefined trigger. SkillTrojan also supports automated synthesis of backdoored skills from arbitrary skill templates, enabling scalable propagation across skill-based agent ecosystems. To enable systematic evaluation, we release a dataset of 3,000+ curated backdoored skills spanning diverse skill patterns and trigger-payload configurations. We instantiate SkillTrojan in a representative code-based agent setting and evaluate both clean-task utility and attack success rate. Our results show that skill-level backdoors can be highly effective with minimal degradation of benign behavior, exposing a critical blind spot in current skill-based agent architectures and motivating defenses that explicitly reason about skill composition and execution. Concretely, on EHR SQL, SkillTrojan attains up to 97.2% ASR while maintaining 89.3% clean ACC on GPT-5.2-1211-Global.

Abstract

Backdoor Attacks

Explore similar work

SkillTrojan: Backdoor Attacks on Skill-Based Agent Systems

SkillHarm: Lifecycle-Aware Skill-Based Attacks via Automated Construction

HarmfulSkillBench: How Do Harmful Skills Weaponize Your Agents?