cs.SEAug 28, 2026

On the Maintenance and Co-evolution of Agent Plugins: An Empirical Study of Claude Code Plugin Marketplaces

Authors: Ahmed Hereiz, Yingzhe Lyu, Hao Li, Bram Adams, Ahmed E. Hassan

Organizations: Queen’s University, Software Analysis and Intelligence Lab (SAIL), Kingston, ON, Canada

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

AI coding agents, software tools that automate development tasks through reasoning and tool use, are increasingly extended through plugin marketplaces, yet the structure, maintenance, and co-evolution dynamics of these emerging repositories remain empirically unexplored. Unlike traditional software packages that deliver functionality through source code, agent plugins deliver functionality through a combination of natural-language instruction files, scripts, and configuration files, raising the question of whether these plugins are maintained artifacts that co-evolve across components, or one-off artifacts that developers write once and do not need to revisit. To study the maintenance and co-evolution of agent plugins, we conduct an empirical study of 1,926 repositories hosting Claude Code plugin marketplaces, analyzing 8,351 plugins and 77,773 commits across 2,018 marketplaces. We find that the marketplace is expanding rapidly, plugin-touching commit activity growing 8.8x over six months after the October 2025 launch, and plugins targeting Software Engineering tasks accounting for 61.3% of all plugins. Plugin development is predominantly feature-driven, with feature commits occurring at more than twice the rate of conventional open-source software (OSS) (39.6% vs. 17.2%). Claude co-authors 34.9% of all commits, and four commit types (docs, perf, style, and refactor) carry substantially different meanings in plugin repositories than in traditional software. Most component types evolve independently, but within skills directories, natural-language instruction files and implementation scripts co-evolve at above-chance rates, with 78% of co-changes being functionally coupled, representing a new class of maintenance dependency not observed in traditional software engineering.

Figures & tables

Figure 1. Structure and control flow of a Claude Code plugin. The left box illustrates the components in a real plugin (python-api-plugin) installed on disk, and the middle and right boxes illustrate how Claude Code loads and uses the same components over the course of a single task.
Directory / FilePurpose
commands/User-invoked prompts triggered via slash-commands (e.g., /plugin:review )
agents/Custom agent definitions
skills/Agent skills ( SKILL.md files)
hooks/Shell commands triggered by events within Claude Code lifecycle (e.g. a PostToolUse hook was triggered to run a linter once Claude edits a file)
.mcp.jsonModel Context Protocol (MCP) server configurations
.lsp.jsonLanguage Server Protocol (LSP) server configurations
Table 1. Claude Code plugin component types.
 Su et al. ( Su et al., 2025 )Yan et al. ( Yan et al., 2024 )Ling et al. ( Ling et al., 2026 )Our study
Target systemConversational LLMConversational LLMAI coding agentAI coding agent
Scale722,349 GPTs1,038 plugins40,285 skills8,351 plugins
Artifact formNL customizationAPI integrationNL skill fileMulti-component package
NL instruction files✓×\times✓✓
Executable scripts×\times×\times×\times✓
Config files×\times✓×\times✓
Table 2. Comparison of related AI ecosystem studies. NL = natural language.
Figure 2. Overview of the data collection pipeline and dataset summary.
Major CategorySub-category# Plugins% of Total% skills.sh ( Ling et al., 2026 )# Median Stars
Software EngineeringCode Generation2,16425.9%14.3%40.0
 Infrastructure1,38716.6%24.0%40.0
 Debug & Analysis1,23414.8%13.2%49.0
 Version Control3364.0%3.2%37.5
Information RetrievalAcademic Search1962.3%2.7%46.5
 Web Search1331.6%1.4%47.5
Table 3. Taxonomy and category-level statistics of agent plugins.
Figure 3. Plugin marketplace repository age and component growth. The x-axis uses compressed spacing for the pre-launch period (January 2024–September 2025) and uniform spacing thereafter, axis-break marks signal the scale change. (a) Monthly distribution of plugin marketplace repository creation dates, with 84.5% created in the six months following the October 2025 launch. (b) Cumulative plugin component counts by type. All component types grow over time.
Figure 4. (a) Distribution of marketplace sizes (number of plugins per marketplace.json ). (b) number of distinct component types per plugin. (c) total number of component files per plugin.
Figure 5. File type composition within each plugin component type. Instruction-oriented components (Commands, Agents, Skills) are predominantly Markdown, while configuration-oriented components (MCP, LSP, Settings) are exclusively JSON.
Figure 6. Component co-occurrence matrix across repositories. Each cell shows the number of repositories that contain both components. Skills co-occur most frequently with all other component types due to their dominant presence.
Figure 7. Plugin-touching commits per month (bars) and cumulative total (line) since the official plugin launch ( n=72,191n=72{,}191 . October 2025 through date cutoff at March 2026). Each month sets a new high, with no plateau by the data cutoff.
CCS TypeChange TypeDescriptionN (%)
feat ( nn =137)Plugin or component additionNew plugin.json , marketplace entry, or major component added66 (48.2%)
 Component updateExisting instruction using natural language or config files extended54 (39.4%)
 Incidental touchPlugin file changed as a side-effect of non-plugin work17 (12.4%)
fix ( nn =83)Instruction/content fixWrong behavioral text corrected in skill, command, or agent files41 (49.4%)
 Hook/command script fixBugs in executable plugin scripts or hooks.json27 (32.5%)
 Plugin manifest fixWrong metadata in plugin.json or marketplace.json12 (14.5%)
Table 4. Development and maintenance behavior observed in a stratified sample of 700 plugin-touching commits, one stratum per CCS type. Sample sizes follow stratified random sampling ( Baltes and Ralph, 2022 ) : nn = 137 for feat , 83 for fix , 80 for chore , and 50 per remaining type. fix and revert allow multiple labels per commit so their percentages may not sum to 100%. Bold indicates the change type within each CCS label that departs most from its conventional software engineering counterpart. Underline indicates the sub-categories whose meaning differs from the conventional meaning of their CCS label.
Figure 8. CCS distribution of plugin commits after re-classifying docs , ci , and style by the function of each commit’s diff (blue), with the commits each type gains (green) or loses (hatched vermilion) under re-classification, against the traditional-SWE baseline (orange, Zeng et al. (2025) ). docs collapses from 10.3% to 1.7% as its commits move to feat , fix , and chore .
Figure 9. CCS type distribution across plugin components. Each cell shows the percentage of that component’s commits carrying a given CCS type, enabling direct comparison across components regardless of their volume.
Table 5. Agent contributions and Claude co-authorship across 77,773 plugin-touching commits.
Rule ( X→YX\rightarrow Y )Co-changing PRsConfLiftRepos significance
skills →\rightarrow commands1,16214.9%0.6423.4%
commands →\rightarrow skills1,16246.5%0.6423.4%
skills →\rightarrow agents1,17615.1%0.7825.8%
agents →\rightarrow skills1,17656.6%0.7825.8%
skills →\rightarrow hooks6788.7%0.5927.9%
hooks →\rightarrow skills67843.2%0.5927.9%
Table 6. Association rule metrics for directed component type pairs ( n=10,679n=10{,}679 plugin pull requests). Pairs with fewer than 100 co-changing pull requests (PRs) are omitted, as are pairs whose coupling is not statistically significant ( χ2≤3.84\chi^{2}\leq 3.84 ). Repos significance shows the percentage of repositories with statistically significant coupling ( χ2>3.84\chi^{2}>3.84 ).
Rule ( X→YX\rightarrow Y )Co-changing PRsConfLiftRepos significance
.py →\rightarrow README.md20119.1%4.1433.3%
README.md →\rightarrow .py20132.1%4.1433.3%
.json →\rightarrow README.md20317.5%3.8048.6%
README.md →\rightarrow .json20332.4%3.8048.6%
.py →\rightarrow Other .md67063.6%1.5816.0%
Other .md →\rightarrow .py67012.2%1.5816.0%
Table 7. Intra-skills file-type co-change association rules ( n=13,587n=13{,}587 pull requests from 5,353 plugins). Script-to-script pairs and pairs with fewer than 200 co-changing pull requests are omitted, as are pairs whose coupling is not statistically significant ( χ2≤3.84\chi^{2}\leq 3.84 ). Other .md corresponds to supporting files ( Anthropic, 2025e ) . Repos significance shows the percentage of repositories with significant coupling ( χ2>3.84\chi^{2}>3.84 ).
CategoryDescriptionHumanLLM
  (n=64)(n=259)
Interface changeA flag, command, argument, or method is added or removed in the script; SKILL.md updates the callable surface the agent invokes.17 (27%)87 (34%)
Internal logic changeThe script’s behavior or output changes while it is invoked the same way; SKILL.md updates its description of what the skill does.17 (27%)68 (26%)
Variable/version syncA version, default, or identifier the script embeds changes (model name, environment variable, version string); every Markdown file that quotes it follows.9 (14%)33 (13%)
Repo re-structuringA file, path, or configuration-file format is moved, renamed, or reformatted; SKILL.md updates the affected paths, filenames, or format examples.7 (11%)24 (9%)
No couplingThe script and the Markdown both change, but the script change does not drive the Markdown file change.14 (22%)47 (18%)
Table 8. Script–Markdown co-change categories. The scheme was built by open coding a sample of 64 pull requests (two raters, Cohen’s κ=0.74\kappa=0.74 ) and applied to the remaining 259 by a gpt-5-mini classifier (validated at κ=0.62\kappa=0.62 against the 64). The first four categories are coupling mechanisms, and the remaining No coupling category shows no correlation between script and Markdown file changes.
Figure 10. Examples of script and Markdown changes across the coupling categories, selected from real plugin pull requests. For each example, the script update is shown on the left and the related Markdown update on the right.
More in this topic

Software Engineering Agents

Software Engineering Agents are AI systems that use code, tools, and feedback to perform software development tasks.

See all 349 papers

Explore similar work

CardsList
Jun 23, 2026

Detecting AI Coding Agents in Open Source: A Validated Multi-Method Census of 180 Million Repositories

Generative AI coding agents are entering the open-source supply chain, yet their diverse and often invisible traces leave their prevalence poorly understood. We introduce a multi-layered detection framework that integrates configuration-file scanning, commit-message analysis, author-identity matching, and bot-signature lookup across World of Code (180M+ Git repositories), classifying agent traces into four behavioral types. No single method captures more than a fraction of activity: multi-method detection identifies 850,157 Claude Code commits in one snapshot, of which bot-account lookup_the signal most adoption studies rely on_recovers only 28,154 (3.3%), a 30x relative-recall gap, so single-signal prevalence estimates are biased low by at least this factor. Every detection pattern is hand-validated (495 labels) with per-cell precision and Wilson confidence intervals. Across snapshots from December 2024 to April 2026, commit-attributed agents generate over 320,000 commits per month; Claude Code leads (886,122 commits across 17,295 projects) and dominates silent, configuration-file-only adoption (21,078 projects). Compared against an independent pull-request census (AIDev), the two channels capture nearly disjoint agent populations_a PR census misses 79% of commit-detected Claude Code adopters and essentially all Codex adopters_and different kinds of work: PR-deployed cloud agents (Codex, Cursor) surface as feature work, while commit-deployed in-editor agents (Claude Code, OpenHands, Aider) surface as maintenance. The observed work profile follows deployment and detection mode rather than the tool itself, so no single channel is representative.

Arsham Khosravani, Audris Mockus

Software EngineeringCoding AgentsMining Software RepositoriesOpen-Source Software Development
AskPDF
Oct 22, 2026

What Open-Source Projects Tell Their Coding Agents: A Census of AGENTS.md and CLAUDE.md

Coding agents can be guided with instruction files in a repository. We examine this emerging practice through a census of 22,854 popular, active GitHub repositories in thirteen languages, treating each repository's root 'AGENTS..md' and 'CLAUDE..md' as one instruction set. Agent instructions appear in 26.3% of repositories and 59.6% of the most-starred; where both files exist, 'CLAUDE..md' usually points to 'AGENTS..md'. We analyse 341,852 statements using model-assisted annotation and human review. The annotations reveal a practical mix of project knowledge and rules about code, building and workflow. Collaboration rules name concrete requirements, from test plans to commit formats, yet 74.0% of directives give no reason. Projects disagree on AI attribution: commit-message rules more often prohibit it than require it, while pull-request rules more often require disclosure. A third of instruction sets have a generator-associated signature, a tool-inserted block, shared text or an AI-marked creation commit; four in five are edited after creation. Prespecified tests associate repository popularity with rules on pull requests and AI contributions. Exploratory analyses link activity to process-oriented content and show that LLM and agent projects are over-represented among adopters. The models agree much more closely with each other than with the human reviewer's topic assignments, a caution for automated annotation. We release the annotations, taxonomies and pointers to the analysed file versions.

Pawel Zarzycki

AI Coding AgentsAI Use Disclosure
AskPDF
May 19, 2026

Does Code Cleanliness Affect Coding Agents? A Controlled Minimal-Pair Study

As autonomous coding agents see rapid adoption, their evaluation has primarily focused on task completion rates holding the target codebase fixed. This leaves a critical question unanswered: does the structural and stylistic quality, or ``cleanliness'' of the underlying code affect an agent's ability to navigate and modify it? To isolate the effect of code cleanliness from agent capability, we introduce an evaluation protocol built around minimal pairs: repositories that match on architecture, dependencies, and external behaviour, but differ on static-analysis rule violations and cognitive complexity. The pairs are constructed in both directions, by agent pipelines that either degrade a clean repository or clean a messy one. We author 33 tasks across six such pairs, evaluated through hidden tests at the application's public surface. Across 660 trials with Claude Code, code cleanliness does not change the agent's pass rate. However, it substantially alters the agent's operational footprint: agents working on cleaner code use 7 to 8% fewer tokens and reduce file revisitations by 34%. Our findings suggest that traditional maintainability principles remain highly relevant in the era of AI-driven development, shaping the computational cost and navigational efficiency of coding agents. Code cleanliness joins model choice, harness, and prompting as a factor that materially affects agent behaviours.

Priyansh Trivedi, Olivier Schmitt

SonarSource

Coding AgentsAI Agent EvaluationAI Agent Benchmarks
AskPDF