cs.SEOct 22, 2026

What Open-Source Projects Tell Their Coding Agents: A Census of AGENTS.md and CLAUDE.md

Authors: Pawel Zarzycki

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Coding agents can be guided with instruction files in a repository. We examine this emerging practice through a census of 22,854 popular, active GitHub repositories in thirteen languages, treating each repository's root 'AGENTS..md' and 'CLAUDE..md' as one instruction set. Agent instructions appear in 26.3% of repositories and 59.6% of the most-starred; where both files exist, 'CLAUDE..md' usually points to 'AGENTS..md'. We analyse 341,852 statements using model-assisted annotation and human review. The annotations reveal a practical mix of project knowledge and rules about code, building and workflow. Collaboration rules name concrete requirements, from test plans to commit formats, yet 74.0% of directives give no reason. Projects disagree on AI attribution: commit-message rules more often prohibit it than require it, while pull-request rules more often require disclosure. A third of instruction sets have a generator-associated signature, a tool-inserted block, shared text or an AI-marked creation commit; four in five are edited after creation. Prespecified tests associate repository popularity with rules on pull requests and AI contributions. Exploratory analyses link activity to process-oriented content and show that LLM and agent projects are over-represented among adopters. The models agree much more closely with each other than with the human reviewer's topic assignments, a caution for automated annotation. We release the annotations, taxonomies and pointers to the analysed file versions.

Figures & tables

Figure 1: Adoption rises with popularity, and new adopters increasingly start with AGENTS.md . (A) Share of repositories with agent instructions by star tier (census of 22,854). (B) New adopters per month by their first surviving agent file; the dashed line marks Claude Code’s support for AGENTS.md (2026-09-18); the last bar is a partial month.
Figure 2: Rules most often concern code conventions, building and agent workflow. Share of repositories whose instruction set has at least one rule (blue, 95% bootstrap intervals) and any statement (grey) in each subject area. Sonnet’s codes on 992 sample repositories, reweighted to the population.
Category% requiring% prohibiting
test plan / verification evidence27.24.4
use and complete the PR template24.21.6
linked issues and related PRs23.31.3
title format (type, tag or scope prefix)19.31.7
summary of the change17.80.1
screenshots and visual evidence15.61.1
Table 1: Selected elements of rules on pull-request descriptions and commit messages: share of the files with a rule on the subject that require or prohibit each element (Haiku, full corpus, uncorrected). All categories are in the appendix.
Figure 3: Most instruction sets contain marked rules, prohibitions and reasons somewhere, but each is a small share of their directives. Share of repositories with at least one directive of each kind (dots, 95% bootstrap intervals) and its mean share of a repository’s directives or statements (diamonds). Sonnet’s codes on the sample, reweighted.
HypothesisVerdictEffect (95% CI)
H1 AI-marked share: more agent rules, permission/escalation rules and prohibitionsnot supportedIRR 0.977 (0.959–0.996), 0.992 (0.963–1.021), 0.981 (0.965–0.997)
H2 Collaboration: more workflow rules, fewer descriptionsmixed (2 of 4)merged PRs: workflow IRR 1.027 (1.009–1.046), descriptions 0.876 (0.858–0.894); human authors: workflow 0.975 (0.959–0.992, opposite), descriptions 0.993 (n.s.)
H3 Compiled languages: more build/run statementssupportedOR 1.68 (1.48–1.91)
H4 Emphasis: more rules on agent failure modesnot supportedOR 0.93 (0.88–0.99)
H5 CLAUDE.md more descriptive than AGENTS.md , same reposupported+6.3 pp (4.2–8.4); Wilcoxon p = 1.3e-7
H6 Stars: more pull-request or AI-policy rulessupportedOR 1.36 (1.29–1.44)
Table 2: Prespecified hypotheses: verdicts and principal effects. Ratios are per standard deviation of continuous predictors; H3 compares compiled with other languages, and H5 is a paired percentage-point difference. Benjamini–Hochberg correction covers 11 tests. 5,587 repositories; H5 uses 484 pairs.
Figure 4: Activity goes with process- and agent-oriented content, and AI projects are over-represented among adopters. (A) Exploratory models: odds ratio of covering each area per standard deviation of log⁡(1+commits in the past year)\log(1+\text{commits in the past year}) , with other predictors held fixed (blue: significant after Benjamini–Hochberg, grey: not; Haiku’s codes, 5,586 instruction sets). (B) Adopters against 4,699 matched repositories without agent files, by project type, within star tier and language; the square adds controls for age and activity and excludes agent-tooling projects. Project types are model-classified and indicative.
Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

CheckResult (n = 60)
Statement cut correctly55 (92%; Wilson 95% CI 82–96%); 5 “part missing”
Restatement faithful56 (93%; 84–97%); 4 “loses meaning”
Listed parts (elements) right56 (93%); 4 “some missing”
Free-text subject fits57 (95%)
Appendix
Table 3: Human review of 60 statements. Top: checks of extracted statements. Bottom: agreement of the blind reading with Haiku, Sonnet and Opus (ranges across models).
LanguageReposAGENTS.mdCLAUDE.mdEither
TypeScript4,04834.4%24.5%40.8%
Rust1,70628.1%18.9%32.7%
Go2,26424.5%16.6%28.9%
Swift47423.4%17.9%28.1%
Kotlin61121.1%14.9%26.0%
Java1,26220.9%12.4%24.6%
Appendix
Table 4: Repositories with each agent file, by primary language. Census of 22,854 repositories.
FormatReposShareAlso has /AGENTS.md
/AGENTS.md4,95421.7%—
/CLAUDE.md3,44015.1%69.4%
.github/copilot-instructions.md1,0244.5%52.3%
.cursor/rules/ ( ≥\geq 1 .md/.mdc)3471.5%64.6%
.github/instructions/2731.2%59.0%
/GEMINI.md1790.8%83.2%
Appendix
Table 5: Agent instruction formats in the 22,854 repositories, listed one level deep. The last column is the share of repositories with the format that also have a root AGENTS.md .
RelationshipReposShare
AGENTS.md only2,54942.4%
AGENTS.md only (symlink to another file)170.3%
CLAUDE.md only1,04017.3%
CLAUDE.md only (symlink to another file)120.2%
both: CLAUDE.md is a symlink to AGENTS.md62910.5%
both: AGENTS.md is a symlink to CLAUDE.md1552.6%
Appendix
Table 6: How the two files relate in the 6,006 repositories with an agent file. The rules are applied in order ( Section 3.1 ).
PeriodNew adoptersFirst file AGENTS.mdFirst file CLAUDE.mdBoth at once
2025-01 – 2025-0853623.9%73.7%2.4%
2025-09 – 2025-1275954.2%36.8%9.1%
2026-01 – 2026-042,17848.3%35.9%15.8%
2026-05 – 2026-10-162,53260.0%17.1%22.9%
Appendix
Table 7: New adopters by the date of their first surviving agent file, and which file came first.
TraceReposShare
generator signature ( claude /init line or variant, Codex /init headings)948 of 5,99815.8%
of which claude /init ’s opening line, verbatim69211.5%
tool-inserted block (Next.js, Turborepo, Nx, Laravel Boost, beads)931.6%
paragraph shared verbatim with files of ≥\geq 2 other owners2053.4%
of which with no generator signature or tool block1041.7%
whole file near-duplicate of another owner’s file601.0%
Appendix
Table 8: Traces of origin per instruction set. Shares of all instruction sets for text traces and of those with a file history for history traces. Traces identify signatures, shared text or marked commits; they do not establish authorship.
TypeOurs, rawOurs, correctedReferenceVerdict
AI Integration72.5%71% (59–92)22% (17–28)more common in ours
Documentation and References83.7%72% (62–85)29% (24–36)more common in ours
Development Process78.0%91% (82–100)68% (62–74)more common in ours
DevOps38.9%29% (23–36)17% (12–22)more common in ours
Security41.8%28% (22–33)17% (13–23)more common in ours
UI/UX24.3%20% (14–32)6% (4–10)more common in ours
Appendix
Table 9: Share of files with each instruction type of Chatlatanagulchai et al. [2026] : ours raw, ours corrected for over-labelling with the Rogan–Gladen estimator (95% intervals), and the reference study’s human coding of its AGENTS.md and CLAUDE.md files. The samples differ in frame and date, so a difference shows that the populations differ, not why.
Derived typeAdopters % (n = 6,006)Controls % (n = 5,920)
data / ML / AI (any AI focus)32.218.6
library / framework23.032.0
application22.322.3
CLI / developer tool10.713.2
infrastructure / DevOps8.06.0
docs / learning2.66.0
Appendix
Table 10: Project types of all adopters and all classified controls (model-classified by Claude Haiku 5.5 from repository metadata; indicative, see Section 4.5 ).
Kind% of repos with ≥\geq 1Mean share of statementsCensus (Haiku, corr.)
rule98.2 (97.3–98.9)52.6 (50.9–54.4)53.0
reference89.7 (87.8–91.6)17.6 (16.5–18.8)16.2
description88.6 (86.6–90.5)29.8 (28.3–31.3)30.7
Appendix
Table 11: Kinds of statement per instruction set: Sonnet’s codes on 992 sample repositories, reweighted, with 95% bootstrap intervals; the last column is the corrected Haiku census.
Area% with a rule (95% CI)% with any statement
code conventions and quality74.4 (71.4–77.3)77.0
build, environment and delivery72.6 (69.4–75.7)87.6
agent workflow and conduct67.3 (64.0–70.3)68.5
documentation61.7 (58.4–65.1)77.3
testing60.2 (56.8–63.4)76.7
version control and collaboration58.0 (54.3–61.7)60.9
Appendix
Table 12: Subject areas: share of repositories with at least one rule (95% CI) and with any statement (the data of Fig. 2 ).
PredictorSignificant / tests
commits in 12 months11 / 13
age9 / 13
project type (joint)6 / 13
language (joint)6 / 13
human authors4 / 13
AI-marked share4 / 13
Appendix
Table 13: Exploratory models: tests per predictor that pass Benjamini–Hochberg within the exploratory family (99 tests). Content models use 5,586 instruction sets; the adopter-versus-control row uses 5,286 adopters and 4,699 controls.
TypeAdopters %Controls %Odds ratio (95% CI)p (FDR)
data / ML / AI32.218.62.38 (2.15–2.64)< 0.001
LLMs / generative AI / agents (AI focus)27.913.32.76 (2.48–3.08)< 0.001
infrastructure8.66.61.38 (1.17–1.61)< 0.001
application21.921.91.00 (0.91–1.11)0.96
developer tool10.913.60.77 (0.68–0.87)< 0.001
library / framework23.033.30.58 (0.53–0.64)< 0.001
Appendix
Table 14: Adopters and matched controls by project type (within star tier and language strata; controls reweighted to the adopters’ strata). Types are model-classified and indicative.
FieldPI agrees with HaikuWilson 95% CICohen’s κ\kappa (95% CI)
AI focus27 / 29 (93.1%)78–98%0.80 (0.44–1.00)
type16 / 29 (55.2%)38–72%0.43 (0.23–0.63)
derived seven-way type17 / 29 (58.6%)41–74%0.50 (0.30–0.70)
Appendix
Table 15: The human reviewer’s blind check of Haiku’s project types on 29 repositories.
StarsReposAGENTS.mdCLAUDE.mdEither
100k+8952.8%31.5%59.6%
10k–100k3,05932.3%21.5%37.0%
5k–10k3,08726.1%17.9%31.2%
1k–5k16,61918.7%13.2%23.2%
Appendix
Table 16: Repositories with agent instructions by star tier (the data of Fig. 1 A).
Category% requiring% prohibiting
Test plan / verification evidence27.24.4
Use and complete the PR template24.21.6
Linked issue and related PR references23.31.3
Title format / type, tag or scope prefix19.31.7
Summary of the change and its behavior effect17.80.1
Screenshots, recordings and visual evidence15.61.1
Appendix
Table 17: Pull-request descriptions: share of the 1,345 files with a rule on the subject that require or prohibit each element.
Category% requiring% prohibiting
Conventional Commits / structured header format36.71.8
Allowed commit types and when to use each23.50.6
Subject mood, case and punctuation15.11.3
Scope / component / area prefix14.90.8
Body presence, layout and wrapping12.61.2
Rationale and change description in the message12.62.0
Appendix
Table 18: Commit messages: share of the 1,739 files with a rule on the subject that require or prohibit each element.
Category% requiring% prohibiting
Tests must accompany changes (and exemptions)28.21.9
Frameworks, libraries, helpers and selectors19.52.2
Test structure, granularity and conventions17.42.4
Required scenarios, edge cases and error paths16.80.1
Choice of test type / level15.51.7
Regression test for bug fixes14.30.3
Appendix
Table 19: Test writing: share of the 2,586 files with a rule on the subject that require or prohibit each element.
Category% requiring% prohibiting
Review process, reviewers, tools and gating22.22.7
Correctness, edge cases and error handling20.22.0
Context gathering and review method17.71.5
Project guidelines, checklists and repo-specific checks17.51.2
Evidence, verification and confidence of findings14.51.5
Review report format, verdict, tone and language14.51.0
Appendix
Table 20: Code review: share of the 401 files with a rule on the subject that require or prohibit each element.
More in this topic

AI Coding Agents

AI coding agents use language models and software tools to plan, edit, test, and maintain code with varying levels of autonomy.

See all 321 papers

Explore similar work

CardsList
Jun 23, 2026

Detecting AI Coding Agents in Open Source: A Validated Multi-Method Census of 180 Million Repositories

Generative AI coding agents are entering the open-source supply chain, yet their diverse and often invisible traces leave their prevalence poorly understood. We introduce a multi-layered detection framework that integrates configuration-file scanning, commit-message analysis, author-identity matching, and bot-signature lookup across World of Code (180M+ Git repositories), classifying agent traces into four behavioral types. No single method captures more than a fraction of activity: multi-method detection identifies 850,157 Claude Code commits in one snapshot, of which bot-account lookup_the signal most adoption studies rely on_recovers only 28,154 (3.3%), a 30x relative-recall gap, so single-signal prevalence estimates are biased low by at least this factor. Every detection pattern is hand-validated (495 labels) with per-cell precision and Wilson confidence intervals. Across snapshots from December 2024 to April 2026, commit-attributed agents generate over 320,000 commits per month; Claude Code leads (886,122 commits across 17,295 projects) and dominates silent, configuration-file-only adoption (21,078 projects). Compared against an independent pull-request census (AIDev), the two channels capture nearly disjoint agent populations_a PR census misses 79% of commit-detected Claude Code adopters and essentially all Codex adopters_and different kinds of work: PR-deployed cloud agents (Codex, Cursor) surface as feature work, while commit-deployed in-editor agents (Claude Code, OpenHands, Aider) surface as maintenance. The observed work profile follows deployment and detection mode rather than the tool itself, so no single channel is representative.

Arsham Khosravani, Audris Mockus

Software EngineeringCoding AgentsMining Software RepositoriesOpen-Source Software Development
AskPDF
Jul 29, 2026

A First Look at Coding Agents' Compliance with AI Contribution Rules in Open-Source Communities

Open source communities have been flooded with AI-generated contributions. In defense, they have written contribution rules to regulate coding agents' behavior, spanning from a total ban, mandatory disclosure, to verification gates and human sign-offs. Yet, whether coding agents read and follow those rules, and behave in open source repositories, remains unknown. To estimate real-world rule compliance of coding agents, we curate 106 issues from 49 repositories containing AI contribution rules into RepoComplianceBench. We judge the trajectory of each run against the repository's rules, measuring whether the agent refuses to contribute, discloses its assistance truthfully, clears the required verification gates, or escalates critical steps to a human. We also test if extra prompts, rule disclosure, or feedback from the compliance verifier help with the situation. Our experiments on four frontier models show that today's agents almost never proactively retrieve the contribution rules. Agents pick up disclosure and verification with reminder prompts, rule quotes, and verifier feedback; however, they never refuse to contribute in AI-banned repositories under any condition we tested. The status reveals that verification and disclosure issues are solvable with existing mechanisms, yet enforcing bans and human escalations remains an open problem.

Wenhao Yang, Runzhi He, Minghui Zhou

Peking University, Beijing, China

AI Coding AgentsAI Agent BenchmarksAI Agent GovernanceHuman Oversight of AIOpen-Source Software Development
AskPDF
Jun 11, 2026

Toward Instructions-as-Code: Understanding the Impact of Instruction Files on Agentic Pull Requests

AI-agents (e.g., GitHub Copilot) collaborate as teammates in different software engineering tasks, including code generation proposed through pull requests (Agentic-PRs). For better agent efficiency, developers create instruction files that guide the AI-agents, including how to navigate the project, locate the right components, run tests, respect best practices, and more. In this paper, we investigate the relationship between the creation of these instructions and the performance of AI-agents in creating better pull requests, which have a higher chance of success (i.e., the merge rate), address more complex tasks (e.g., code churn), and require less effort to be merged (e.g., time to merge). To this end, we analyze 15,549 agentic PRs from 148 projects in the AIDev dataset. Using the three dimensions, we compare each project before and after the creation of the instruction files. We find that specifying instructions for AI-agents does not necessarily lead to better results. With the instruction files, 27.7% of the projects increased their merge rate by at least 20%, while 26.35% decreased it. The same observation is seen with the amount of changes (e.g., code churn, number of modified files) and with the efforts to merge an agentic PR (e.g., merge time and number of comments). From a first exploration, we find that projects that managed to increase their merge rate have substantially longer instruction files, which are also well structured into a higher number of sections and sub-sections. Our results motivate the need for research to assist practitioners in framing the development of instruction files as a software engineering activity (aka, Instructions-as-Code).

Ali Arabat, Mohammed Sayagh

École de Technologie Supérieure Montréal, Canada

Software Engineering AgentsAgentic Code Generation
AskPDF