One Skill Takes Over: Clashing Agent Skills and a One-File Fix
Organizations: University of Southern California, Los Angeles, USA
Abstract
Agent Skills are instruction files that tell a coding agent how to do one job. Users install skills from many authors, so two skills for the same job can be active at the same time. We ask what a model does when both skill files are in its context: does it combine their compatible rules, or follow one skill and drop the other? We build SkillClash, 16 pairs of same-job skills in which code checks every rule and a reference output shows the best result that can be reached. On seven models from five families, from 4B open models to Qwen3-235B and GPT-4.1, a skill's rules are followed 93 to 99% of the time when it is loaded alone and 61 to 62% when a same-job skill is loaded next to it. Most outputs show takeover: every rule that only one skill asks for is followed, and every rule that only the other skill asks for is dropped. The worked examples inside the skills drive this. When each skill file instead ends with one example that follows both skills, takeover falls to 1 to 15% on the six models we tested. Telling the model which skill has priority does not bring the dropped skill back, and telling it to apply both skills brings the dropped skill back only on the two strong models and leaves the interacting rules broken. One merged skill file written by the strongest model we used, Claude Opus 5.5, lets all seven models follow 96 to 99% of the achievable rules. Most of its gain over a plain list of both skills' rules comes from its one worked example, and such an example helps only when it is correct. Code and data: https://github.com/prazmara/skillclash
Figures & tables
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| model: Qwen3-4B | model: Llama-3.1-8B | |||||||
| condition | achievable | ent. | all | t.o. | achievable | ent. | all | t.o. |
| two skill files | ||||||||
| Both skills loaded | 58.1 | 12 | 0 | 78 | 59.1 | 13 | 0 | 66 |
| Both skills loaded, examples removed | 64.7 | 29 | 1 | 34 | 63.1 | 30 | 3 | 34 |
| Both skills loaded, each with Claude’s example | 92.9 | 90 | 60 | 6 | 94.2 | 92 | 63 | 2 |
| Write with the primary skill, revise with the other | 53.3 | 6 | 0 | 75 | 62.5 | 14 | 0 | 40 |
| model: Qwen3-235B | model: GPT-4.1 | |||||||
| condition | achievable | ent. | all | t.o. | achievable | ent. | all | t.o. |
| two skill files | ||||||||
| Both skills loaded | 59.6 | 7 | 1 | 73 | 58.9 | 7 | 1 | 77 |
| Both skills loaded, examples removed | 71.4 | 37 | 7 | 41 | – | – | – | – |
| Both skills loaded, each with Claude’s example | 96.2 | 91 | 76 | 1 | 89.9 | 83 | 60 | 15 |
| Write with the primary skill, revise with the other | 59.6 | 15 | 0 | 52 | 67.3 | 19 | 0 | 38 |
| model: Qwen2.5-7B | model: Mistral-7B | model: Phi-4-mini | ||||||||||
| condition | achievable | ent. | all | t.o. | achievable | ent. | all | t.o. | achievable | ent. | all | t.o. |
| two skill files | ||||||||||||
| Both skills loaded | 58.6 | 8 | 0 | 69 | 60.3 | 16 | 2 | 45 | 59.9 | 19 | 0 | 49 |
| Both skills loaded, examples removed | 65.7 | 48 | 3 | 22 | 68.4 | 43 | 5 | 7 | 59.3 | 43 | 3 | 19 |
| Both skills loaded, each with Claude’s example | 90.7 | 87 | 49 | 5 | – | – | – | – | 91.0 | 81 | 50 | 2 |
| Write with the primary skill, revise with the other | 59.2 | 16 | 0 | 56 | 60.4 | 19 | 0 | 46 | 56.5 | 15 | 0 | 62 |
| step-by-step merge | files read by the merger itself | files read by Qwen3-4B | files read by Llama-3.1-8B | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| merger | rules copied | right labels | false pairs | clean files | good examples | step-by-step | plain | step-by-step | plain | step-by-step | plain |
| Qwen3-4B | 190/192 | 24/48 | 38 | 2/32 | 5/32 | 78.5 | 79.3 | 78.5 | 79.3 | 78.3 | 79.7 |
| Qwen2.5-7B | 189/192 | 8/48 | 8 | 0/32 | 3/32 | 75.4 | 75.0 | – | – | – | – |
| Mistral-7B | 117/192 | 4/48 | 61 | 0/32 | 0/32 | 65.2 | 80.2 | – | – | – | – |
| Llama-3.1-8B | 182/192 | 8/48 | 33 | 0/32 | 4/32 | 81.2 | 86.9 | 80.5 | 87.0 | 81.2 | 86.9 |
| Phi-4-mini | 120/192 | 2/48 | 30 | 0/32 | 2/32 | 52.4 | 71.1 | – | – | – | – |
| comparison (points) | Qwen3-4B | Qwen2.5-7B | Mistral-7B | Llama-3.1-8B | Phi-4-mini | Qwen3-235B | GPT-4.1 |
|---|---|---|---|---|---|---|---|
| One file against two | |||||||
| One file vs. two files (same rules, no examples) | – | ||||||
| [7.8, 17.5] | [5.1, 14.3] | [0.8, 14.7] | [13.6, 22.4] | [-1.2, 11.4] | [8.6, 21.2] | ||
| Rule union vs. “apply both” instruction | |||||||
| [3.3, 14.9] | [-6.5, 7.0] | [-9.7, 2.3] | [2.8, 11.7] | [-14.6, 6.2] | [-6.6, 2.3] | [-2.6, 4.4] | |
| Claude’s file vs. “apply both” instruction | |||||||
| model | skills shown | alone | both | takeover | both fully | expected | both fully expected |
|---|---|---|---|---|---|---|---|
| Qwen3-4B | full skills, with examples | 96.9 | 53.4 | 78.1 | 0.4 | 86.7 | |
| examples removed | 92.8 | 63.5 | 33.6 | 16.0 | 62.5 | ||
| + contradictory rules removed | 93.8 | 68.8 | 25.4 | 26.2 | 68.8 | ||
| + entangled rules removed (fully compatible) | 92.4 | 70.8 | 23.0 | 30.5 | 63.3 | ||
| Qwen2.5-7B | full skills, with examples | 95.8 | 54.6 | 68.8 | 5.9 | 78.9 | |
| examples removed | 87.0 | 62.2 | 22.3 | 14.1 | 57.0 |
| Qwen3-4B | Qwen2.5-7B | Mistral-7B | Llama-3.1-8B | Phi-4-mini | Qwen3-235B | GPT-4.1 | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| job | artifact | takeover | both fully | takeover | both fully | takeover | both fully | takeover | both fully | takeover | both fully | takeover | both fully | takeover | both fully |
| commit | commit message | 100 | 0 | 88 | 0 | 50 | 31 | 100 | 0 | 19 | 0 | 100 | 0 | 62 | 38 |
| pr | pull request description | 81 | 0 | 94 | 0 | 12 | 44 | 50 | 31 | 38 | 0 | 100 | 0 | 75 | 25 |
| sql | SQL query | 0 | 0 | 6 | 25 | 0 | 12 | 0 | 19 | 19 | 6 | 0 | 12 | 12 | 12 |
| incident | incident report | 38 | 6 | 12 | 0 | 50 | 0 | 50 | 0 | 0 | 0 | 12 | 0 | 0 | 0 |
| python | Python function | 31 | 0 | 25 | 0 | 12 | 0 | 12 | 0 | 19 | 0 | 31 | 6 | 62 | 0 |
| model | held out | baseline | text features | probe | probe baseline | best block, post hoc |
|---|---|---|---|---|---|---|
| Qwen3-4B | new skill pair | 58.0 | 50.0 | 67.0 | 74.0 (26) | |
| Qwen3-4B | new request | 96.5 | 58.0 | 96.0 | 96.5 (22) | |
| Qwen2.5-7B | new skill pair | 76.1 | 75.6 | 76.1 | 88.6 (22) | |
| Qwen2.5-7B | new request | 98.9 | 79.0 | 98.3 | 98.3 (18) | |
| Llama-3.1-8B | new skill pair | 74.1 | 67.1 | 70.6 | 79.4 (16) | |
| Llama-3.1-8B | new request | 97.1 | 72.9 | 96.5 | 97.1 (15) |
| model | steering | stays | flips | both | partial | neither | achievable |
|---|---|---|---|---|---|---|---|
| Qwen3-4B | no steering | 100 | 0.0 | 0 | 0 | 0 | 56.8 |
| Qwen3-4B | job-specific direction, best for both (block 13, =0.4) | 46 | 14.6 | 5 | 34 | 0 | 60.0 |
| Qwen3-4B | job-specific direction, best for flipping (block 13, =0.6) | 32 | 15.9 | 2 | 45 | 5 | 51.2 |
| Qwen3-4B | random control for the row above (block 13, =0.6) | 46 | 11.4 | 6 | 35 | 2 | 59.3 |
| Qwen3-4B | general direction, best for both (block 19, =0.4) | 35 | 23.3 | 13 | 28 | 1 | 62.4 |
| Qwen3-4B | general direction, best for flipping (block 24, =0.4) | 13 | 48.8 | 8 | 30 | 0 | 59.2 |
AI Coding Agents
AI coding agents use language models and software tools to plan, edit, test, and maintain code with varying levels of autonomy.
See all 310 papers