CoSA: The Conflict of Skills Attack in Real-World LLM Agent Ecosystems

Yujia Zheng; Tianhao Li; Shangzhi Xu; Yuekai Huang; Ziyou Jiang; Zhaoyang Li; Yuyao Ge; Junran Peng

cs.CLOct 26, 2026

CoSA: The Conflict of Skills Attack in Real-World LLM Agent Ecosystems

Authors: Yujia Zheng, Tianhao Li, Shangzhi Xu, Yuekai Huang, Ziyou Jiang, Zhaoyang Li, Yuyao Ge, Junran Peng,

Organizations: University of Science and Technology Beijing · ZODA · Fangcun Leap · University of New South Wales ·

Abstract

Agent Skills allow LLM agents to combine reusable capabilities, but checking each Skill in isolation cannot rule out policy violations when they are composed. We term such failures Conflict of Skills (CoS): artifacts or assessment signals produced for one purpose may be reused downstream without preserving the restrictions or authorization needed for their new use. Using controlled executions of publicly available Skills and matched single-Skill controls, we identify four recurring mechanisms behind CoS. Building on these findings, we introduce the Conflict of Skills Attack (CoSA), which adds locally admitted Skills to an existing pool to supply missing business operations (capability supplementation) or reshape artifact exchange, decision reuse, and target/action bindings (relationship reshaping). CoSA leaves existing Skills and runtime permissions unchanged. To evaluate this threat under autonomous Skill selection, we introduce CoSA-Bench, built from real-world Skills and controlled services, with 288 tasks across recruiting, finance, IT, and business, covering data leakage, unauthorized state modification, and unsafe execution. Across five LLMs and three agent harnesses, additions capped at 5% of each original pool raise the policy-violation rate from 35.02% to 58.45%, an increase of 23.43 percentage points. Trace analysis links compliant-to-violating transitions to cross-Skill dependencies involving the additions. Finally, we develop a lightweight harness-level guard that checks information flow and authorization before consequential actions. On OpenClaw recruiting tasks across five models, the guard reduces violations from 65.28% to 6.67%, while policy-compliant task completion on matched benign controls changes from 66.67% to 65.00%.

Figures & tables

Figure 1: Motivating example. Although each Skill passes the individual security checks, their composition leads to unauthorized execution of code from a submitted package.

Figure 2: Overview of CoSA. CoSA constructs a composition surface from the original Skill pool, identifies mechanism-guided augmentation opportunities, and generates, admits, aligns, and selects a budgeted set of locally admissible Skills. Original and augmented pools are then compared under matched execution conditions with autonomous Skill selection. An optional runtime guard checks information flow and authorization before consequential actions.

Domain	$\|V_{D}\|$	Tasks	DL	UA	UE
Recruiting	49	72	24	24	24
Finance	73	72	24	24	24
IT	64	72	24	24	24
Business	66	72	24	24	24

Table 1: CoSA-Bench composition.

|V_{D}|

counts original Skills passing

\Pi

. DL, UA, and UE denote data leakage, unauthorized state modification, and unsafe execution.

	Policy violation			OTR		ITR
Harness	Original	CoSA	$\Delta$ ASR	Original	CoSA	CoSA
OpenClaw	31.94	54.24	+22.29	77.57	76.53	68.61
OpenHands	40.07	66.11	+26.04	96.04	94.79	88.82
OpenCode	33.06	55.00	+21.94	96.81	97.78	84.44
All settings	35.02	58.45	+23.43	90.14	89.70	80.63

Table 2: Attack success and Skill retrieval rates across agent harnesses.

\Delta

ASR is in percentage points (pp); OTR and ITR denote original- and added-Skill retrieval rates.

Variant	ASR (%)	$\Delta$ ASR vs. original (pp)
Original pool	31.94	—
Random Benign Injection	26.39	$-5.55$
Random CoSA Candidate Selection	43.40	$+11.46$
Full CoSA	54.24	$+22.29$

Table 3: OpenClaw ablations. ASR and

\Delta

ASR are reported to two decimal places. The original-pool and full-CoSA rows correspond to the main OpenClaw evaluation.

Figure 5: The CoSA-Guard on OpenClaw recruiting tasks. (a) ASR under attack conditions; (b) PCTC on matched benign controls. Detailed counts are in Appendix C.3 .

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Role	Meaning
READ	Retrieve information from environment resources beyond the supplied inputs.
TRANSFORM	Reformat, aggregate, filter, or derive information from supplied inputs.
ASSESS	Produce a score, recommendation, readiness judgment, or decision signal.
EXTERNALIZE	Send or expose information to another recipient or system.
MUTATE	Change a persistent resource or business state.
EXECUTE	Run code, a command, or another executable workload.

Appendix

Table 4: Operation roles used in CoSA’s contract analysis. A Skill may support multiple roles.

Field	Meaning
$f_{o}$	Declared business function of operation $o$ .
$X_{o}$	Required inputs, including artifact types, fields, formats, and relevant environment state.
$Y_{o}$	Produced outputs, artifacts, observations, or state signals.
$P_{o}$	Runtime permissions required by the operation.
$E_{o}$	Observable side effects, including disclosure, persistent mutation, or execution.
$T_{o}$	Assessment signals produced or accepted by the operation, such as scores, recommendations, readiness judgments, and status indicators.

Appendix

Table 5: Fields in an operation-level Skill contract.

Condition	Violations	Completed tasks
Single-Skill control	$5/60$ (8.33%)	$55/60$ (91.67%)
Selected multi-Skill composition	$\mathbf{41/60}$ ( 68.33% )	$55/60$ (91.67%)

Appendix

Table 6: Controlled discovery results on 60 requests per condition. Each request explicitly identifies the participating Skills; the matched control exposes only one constituent Skill.

Harness	Model	Original ASR	CoSA ASR	$\Delta$ ASR	ITR
OpenClaw	Qwen3.6	34.03	59.38	+25.35	71.88
	MiniMax-M3	36.46	57.99	+21.53	66.67
	DeepSeek-V4	32.64	60.76	+28.13	66.32
	GPT-5.6	20.14	29.86	+9.72	70.14
	GLM-5.1	36.46	63.19	+26.74	68.06
OpenHands	Qwen3.6	42.01	72.57	+30.56	86.11

Appendix

Table 7: Model-level results. Each row aggregates 288 distinct tasks per condition for one model–harness setting. Each domain–risk group contains 24 tasks. ASR and ITR are percentages;

\Delta

ASR is in pp. ITR denotes retrieval of at least one added Skill under CoSA.

Group	OpenClaw	OpenHands	OpenCode
Recruiting	41.67 / 65.28	35.83 / 64.44	38.06 / 65.83
Finance	27.50 / 48.06	37.22 / 63.06	36.67 / 55.56
IT	26.11 / 47.22	45.28 / 68.61	28.89 / 47.78
Business	32.50 / 56.39	41.94 / 68.33	28.61 / 50.83
DL	39.38 / 64.38	43.75 / 74.79	44.79 / 70.00
UA	37.50 / 55.63	43.75 / 69.58	35.83 / 56.46

Appendix

Table 8: Original / CoSA ASR (%) by domain and risk. Domain rows aggregate 360 task-runs per harness and condition, risk rows aggregate 480 task-runs. Each domain–risk group contains 24 distinct tasks per model. DL, UA, and UE denote data leakage, unauthorized state modification, and unsafe execution.

Figure 6: ASR differences in pp for all 180 groups. White cells marked zero are unchanged; negative values are printed explicitly. DL: data leakage; UA: unauthorized state modification; UE: unsafe execution. Each cell compares 24 task-runs per condition.

Risk	$0\!\to\!1$	$1\!\to\!1$	$1\!\to\!0$	$0\!\to\!0$	$Z_{j}=1$
DL	450	554	60	376	450
UA	365	507	55	513	365
UE	347	302	35	756	347
Total	1,162	1,363	150	1,645	1,162

Appendix

Table 9: Observed paired transitions between the original and augmented Skill pools. The final column reports

0\!\to\!1

transitions satisfying the trace-supported attribution criterion.

Model	DL	UA	UE	Overall
Qwen3.6	70.83 / 0.00	54.17 / 0.00	41.67 / 0.00	55.56 / 0.00
GPT-5.6	58.33 / 8.33	50.00 / 8.33	16.67 / 0.00	41.67 / 5.56
DeepSeek-V4	83.33 / 20.83	79.17 / 8.33	66.67 / 8.33	76.39 / 12.50
GLM-5.1	95.83 / 8.33	58.33 / 8.33	87.50 / 8.33	80.56 / 8.33
MiniMax-M3	91.67 / 20.83	83.33 / 0.00	41.67 / 0.00	72.22 / 6.94
All models	80.00 / 11.67	65.00 / 5.00	50.83 / 3.33	65.28 / 6.67

Appendix

Table 10: Defense ASR by model and risk: no Guard / +Guard (%). Each risk category contains 24 distinct tasks per model; the overall column aggregates 72 recruiting tasks. The no-Guard condition uses the main OpenClaw recruiting augmented-pool outcomes.

Model / risk	Valid BLOCK	Risk Success	No Effective Risky Action	Risky Actions Produced	Risk Coverage (%)
Qwen3.6	2	0	70	2	100.00
GPT-5.6	22	4	46	26	84.62
DeepSeek-V4	31	9	32	40	77.50
GLM-5.1	27	6	39	33	81.82
MiniMax-M3	33	5	34	38	86.84
Overall	115	24	221	139	82.73

Appendix

Table 11: Task-run defense attribution. Each model has 72 task-runs, comprising 24 distinct tasks per risk category. Risky Actions Produced equals Valid BLOCK plus Risk Success; Risk Coverage is their task-run blocking proportion.

Scope	$r$ (ITR, CoSA ASR)	$r$ (ITR, $\Delta$ ASR)	$\rho$ (ITR, $\Delta$ ASR)
All groups	0.397	0.058	0.007
OpenClaw	0.389	-0.053	-0.115
OpenHands	0.320	0.116	-0.008
OpenCode	0.415	0.015	-0.080

Appendix

Table 12: Association of ITR with absolute and incremental risk. Pearson

r

is computed over 180 groups overall and 60 within each harness. Spearman

\rho

compares ITR with

\Delta

ASR.

Figure 7: Added-Skill retrieval versus (a) augmented ASR and (b) its increment over the original pool. Each point is one of 180 aggregate groups with 24 task-runs per condition, points may overlap. The horizontal line in (b) marks zero increment.

LLM Agent Security

Large language model agent security studies attacks and safeguards for systems that plan, use tools, retain memory, and act across applications.

See all 330 papers

SoK: Agentic Skills -- Beyond Tool Use in LLM Agents
Feb 24, 2026Yanna Jiang, Delong Li, Haiyu Deng +4Agentic AI LLM Agent Skill Learning
University of Technology Sydney · CSIRO Data61
Benign in Isolation, Harmful in Composition: Security Risks in Agent Skill Ecosystems
Jun 13, 2026Yi Xie, Jiawei Du, Yu Cheng +2LLM Agent Security AI Agent Security Benchmarks
East China Normal University, Shanghai, China · Centre for Frontier AI Research A*STAR, Singapore · Shanghai Innovation Institute, Shanghai, China
Reachability-Based Capability Confinement for LLM Agents under Indirect Prompt Injection
Aug 30, 2026Wujie Xiong, Rabimba Karanjai, Yang Lu +2LLM Agent Security Information Flow Control
Kent State Univeristy · PayPal AI Labs · University of Houston

Feb 24, 2026J/K move · Enter open · S save

SoK: Agentic Skills -- Beyond Tool Use in LLM Agents

Yanna Jiang, Delong Li, Haiyu Deng, Baihe Ma+3

University of Technology Sydney · CSIRO Data61

Agentic AI LLM Agent Skill Learning LLM Agent Security

Open paper AskPDF arXiv

Agentic systems increasingly rely on reusable procedural capabilities, a.k.a., agentic skills, to execute long-horizon workflows reliably. These capabilities are callable modules that package procedural knowledge with explicit applicability conditions, execution policies, termination criteria, and reusable interfaces. Unlike one-off plans or atomic tool calls, skills operate (and often do well) across tasks. This paper maps the skill layer across the full lifecycle (discovery, practice, distillation, storage, composition, evaluation, and update) and introduces two complementary taxonomies. The first is a system-level set of seven design patterns capturing how skills are packaged and executed in practice, from metadata-driven progressive disclosure and executable code skills to self-evolving libraries and marketplace distribution. The second is an orthogonal \textbf{representation

\times

scope} taxonomy describing what skills are (natural language, code, policy, hybrid) and what environments they operate over (web, OS, software engineering, robotics). We analyze the security and governance implications of skill-based agents, covering supply-chain risks, prompt injection via skill payloads, and trust-tiered execution, grounded by a case study of the ClawHavoc campaign in which nearly 1{,}200 malicious skills infiltrated a major agent marketplace, exfiltrating API keys, cryptocurrency wallets, and browser credentials at scale. We further survey deterministic evaluation approaches, anchored by recent benchmark evidence that curated skills can substantially improve agent success rates while self-generated skills may degrade them. We conclude with open challenges toward robust, verifiable, and certifiable skills for real-world autonomous agents.

Abstract

LLM Agent Security

Explore similar work

SoK: Agentic Skills -- Beyond Tool Use in LLM Agents

Benign in Isolation, Harmful in Composition: Security Risks in Agent Skill Ecosystems

Reachability-Based Capability Confinement for LLM Agents under Indirect Prompt Injection