One Skill Too Many: How Co-Installed Skills Conflict in Coding Agents

Chaoliang Yan; Zihao Xu; Yuekang Li; Shangzhi Xu; Yi Liu; Gelei Deng; Siqi Ma

cs.SEOct 8, 2026

One Skill Too Many: How Co-Installed Skills Conflict in Coding Agents

Authors: Chaoliang Yan, Zihao Xu, Yuekang Li, Shangzhi Xu, Yi Liu, Gelei Deng, Siqi Ma

Organizations: University of New South Wales, Australia · Griffith University, Australia · Nanyang Technological University, Singapore · The University of Wollongong, Australia

Abstract

Coding agents are extended with agent skills, directories whose SKILL.md tells the model when and how to perform a task. Because skills come from independent sources (teams, developers, plugins, copied collections), an installed skill can be co-installed with a similar skill doing the same job, and the model picks between them by name and description alone. In a conflict, the installed skill loses core functions (e.g., a ban on touching git) because the similar skill runs instead or changes what it does. The task still passes, so benchmarks that check only task completion miss such cases. We present the first empirical study of such conflicts. From snapshots of 20,947 repositories, we mine 822,109 candidate similar-skill pairs, have an LLM judge a stratified sample of 3,754, and run 312 confirmed pairs on three models (6,368 runs, 169,294 tool calls, 542 agent-hours). We report five findings. (1) Conflict-prone skills are common: nearly one in four installed skills is co-installed with one that does the same job, and 37% of judged skills sit inside copied collections. (2) Most such pairs involve normative skills, then capability skills. (3) Without lowering task completion, a similar skill takes one in five runs from the installed skill, and runs that open the similar skill first lose over a third of the exclusive core functions that only the installed skill fulfills. (4) Install location decides which skill runs, listing order barely matters, and the final reply names the skill used in only 0.9% of substituted runs. (5) Conflicts are decided at the first skill read, almost always before any file is changed, and a pre-tool hook at that read restores fidelity on exclusive core functions to the level of runs that open the installed skill first. Benchmarks should thus score exclusive core functions, and platforms should guard the first read and show which skill ran.

Figures & tables

Scope	Location	Same name as a skill at another scope
Enterprise	managed settings directory	overrides personal and project
Personal	˜ /.claude/skills/<name>/	overrides project
Project	.claude/skills/<name>/ (start dir. and parents)	overridden by personal and enterprise
Nested	<subdir>/.claude/skills/<name>/	both load, directory-qualified
Plugin	<plugin>/skills/<name>/	both load, namespaced plugin:name

Table 1. Skill scopes in Claude Code, our example platform, and what happens when two skills have the same name ( Anthropic, 2026a ) .

Figure 1. Two runs of one request with Haiku 4.5, abridged, with the project skill $A$ in blue , the similar skill $B$ in gold , and the failure in red . With $B$ added under the directory checkpoint-2 , the model uses it, commits the work that $A$ forbids, and does not say which skill it used.

Figure 2. Overview of the study, with the installed skill $A$ in blue , a similar skill $B$ in gold , and an unrelated skill $C$ in gray .

Pairing	Skill type	Confirmed pairs	Experimental pairs	Runs	Tool calls
Cross-project	Normative	438	98	1,638	34,216
	Capability	223	41	723	23,223
	Script-bearing	542	89	1,541	43,737
Within-project	Normative	150	34	510	11,194
	Capability	95	14	210	8,415
	Script-bearing	191	36	540	16,294

Table 2. Confirmed pairs, experimental pairs, and the analyzed runs and tool calls of the experimental pairs per cell.

Figure 3. A case and two of its runs with Opus 5, abridged, with $A$ in blue , $B$ in gold , and missed core functions in red , where CF1–CF3 are the three primary core functions of $A$ . With $B$ of the same name in the personal directory, Claude Code runs $B$ (①), the task is still completed but none of CF1–CF3 is fulfilled (②), and the final reply names no skill (③).

Configuration	$A$	$B$	Pairs	Runs	Tool calls	Hours	RQ
Only $A$	project	–	312	935	24,617	81.3	1, 3
Only $B$	–	project	312	936	24,400	78.7	1
A+B	project	project	312	934	24,555	81.6	1–3
Swapped	project	project, renamed	312	936	24,633	81.9	2, 3
Personal scope	project	personal	81	243	6,891	22.3	2
Plugin scope	project	plugin	81	243	6,749	22.8	2

Table 3. Configurations with their analyzed runs, tool calls, and hours of agent time, each run once per pair and model, followed by the guard experiment and the repeated runs.

Figure 4. (a) Level of each configuration, with a gray band from only-

B

to only-

A

, (b) the A+B runs by the skills they use, where the area of a bar is its share of the loss of fidelity, and (c) the change from only-

A

with 95% CIs over pairs. Fidelity counts the primary core functions, and the exclusive and shared ones count all that apply.

Figure 5. Skill use per configuration as the share of runs, over the three models and per model. The lower group covers the 81 same-name pairs with

B

in the project (A+B), in a plugin, or in the personal directory, which removes

A

from the listing.

Figure 6. (a) Runs of the A+B and swapped configurations that have changed a file, by tool calls from the first read of a skill, (b) the change on the exclusive core functions of

A

when the same pair and model open

B

first instead of

A

first, and (c) fidelity on these core functions in the guard experiment, by the skill that a run reached for first. Intervals are 95% pair-bootstrap CIs, and hollow markers in (b) are not significant after Holm correction.

Explore similar work

Aug 9, 2026cs.AI

What Keeps Agent Skills from Being Reusable? Evidence from 138K SKILL.md Files

Under the current standard, Agent Skills are SKILL.md files that combine instructions with supporting resources, enabling Large Language Model (LLM) agents to reuse procedures beyond a single conversation. Yet many public skills appear to originate from a single task, repository, or conversation, even when they are shared as reusable components. We analyze this gap across 138,133 public SKILL.md files from 20,556 repositories using a two-tier defect taxonomy grounded in the official specification and best-practice guidance. We find that 91.8% of skills contain at least one detected defect, with stable estimates across lenient and strict thresholds (88.8-94.6%). The dominant failures are ordinary packaging problems rather than exotic attacks: weak routing metadata, bloated or non-actionable bodies, and poor resource organization. A deterministic routing stress test over 20,000 skills shows the functional impact: skills with valid routing metadata are retrieved more reliably from startup descriptions than skills with routing defects. Defect rates vary by platform and provenance: specification-aware skills contain fewer defects, while AI-marked skills show more safety and portability problems. Lightweight enforcement and repair experiments support a quality-assured generation workflow combining spec-aware prompting, lightweight linting, automated repair, and safety gating.

Chi Zhang, Yimin Liu, Xinze Chen +1

The Graduate Center, City University of New York · The Ohio State University · Hunter College, City University of New York

LLM Agent Skill Retrieval

Ask PDF

Aug 12, 2026cs.AI

Agent Skills Can Be Harmful: An Empirical Study of Skill-Induced Failures in LLM Agents

Agent skills are the de facto mechanism for extending LLM agents with reusable guidance. A skill can shape the agent's task execution, including planning, tool use, problem-solving, and validation. Prior work reported mixed results of agent skills: some skills improve task success rates, while others have no effect, increase token use and execution time, and even reduce success rates. This paper presents a comprehensive analysis of skill-induced agent failures by attributing task failures and cost regressions to specific loaded skills. We introduce a differential analysis framework that attributes a failure or regression to a skill by comparing a target skill-guided run against a no-skill or semantically matched skill reference run that solves the same task, or solves it more cheaply. We instantiate this framework on SkillsBench and SWE-Skills-Bench, yielding 307 skill-induced failures, including 125 functional failures and 182 efficiency regressions. We also build SkillTriage, a taxonomy-guided attribution tool that normalizes paired cases, extracts differential evidence, and produces triage reports. Our major findings include: (1) Skill induced functional failures are rarely caused by obviously irrelevant skills; instead, seemingly relevant skills often make the agent incorrectly implement or omit task-required implementation elements. (2) Skill-induced efficiency regressions are not explained by prompt length alone. (3) The largest sources within Excessive Procedure are excessive verification and heavy implementation pipelines, contributing 67 and 30 cases, respectively. This shows that skills often turn validation checklists and construction recipes into mandatory work. Based on our findings, we propose research topics and tooling improvements for safer and more cost-aware skill reuse.

Gen Dong, Yanjie Gao, Liqun Li +3

Huazhong University of Science and Technology · Microsoft Research · Microsoft +1

LLM Agent Evaluation LLM Agent Reliability

Ask PDF

Jul 27, 2026cs.SE

Authoring Agent Skills: A Software-Engineering Approach

Agent Skills are an emerging way to extend large language model agents with reusable procedural knowledge that the agent loads on demand. Anthropic introduced Agent Skills and published the format as an open specification supported across several agent tools. This note argues that a skill is a software artefact and that its construction should follow software-engineering principles, with qualifications: single responsibility, separation of interface from implementation, low coupling, and economy in a shared token budget, together with behavioural evaluation in place of deterministic testing. Using Claude Code as the reference implementation, it describes how a skill is structured, how its contents are loaded in stages, and how to write the description on which selection depends. It places skills against the other mechanisms a developer can use to shape agent behaviour, like project memory files, slash commands, subagents, external tool connections, and hooks, and gives a rule for choosing between them based on who decides that a mechanism runs and what guarantee it provides. It then sets out an evaluation-driven authoring process, a set of patterns and faults commonly encountered in authoring, and the trust question raised by using skills from third parties. We illustrate the comparison drawn in UML class style, the loading model, the anatomy of a skill, the relative position of each mechanism, and the points at which skills and hooks act during a session.

Giuseppe Destefanis

Department of Computer Science, University College London

LLM Agents

Ask PDF