Existing code reveals past implementation decisions but rarely explains the reasoning behind them. When coding agents implement new features or modify existing functionality, past decisions may conflict with new task instructions. This raises a fundamental question for agentic software evolution: when multiple implementations satisfy the same functional requirements, how do coding agents decide whether to follow existing code or new task instructions? We investigate this question through bidirectional, controlled experiments in purpose-built repositories, followed by validation in real-world open-source repositories. Our approach isolates three competing sources of guidance: (1) agents' default preferences, (2) repository precedents, and (3) explicit task instructions. We conduct 5,625 trials across four coding agents and six implementation pairs, with validation in four open-source repositories spanning three programming languages. We find that agents share the same default implementation choices, yet overwhelmingly follow repository precedent when it conflicts with those defaults (359/360 trials). Crucially, not all precedents carry equal influence: three supporting local files yield 96% adoption, compared with only 4% for sibling-resource files, and changing file placement alone can shift adoption from 0% to 100% at a fixed example count. When new task intent conflicts with existing code, written priority rules achieve substantially higher success than code examples alone (59/60 versus 38/60 joint successes for two agents on two implementation pairs). The effectiveness of explicit rules also extends to open-source repositories, where a harness-loaded rule raises joint success from 0/120 to 117/120. These findings motivate design-decision traceability that preserves not only implementation choices but also the rationale and priorities that should guide future changes.
Figures & tables
Figure 1: Coding agents that extend a repository are pulled between the implementation already in the code and the intent of the new task. In controlled experiments with four agents and six implementation pairs, existing code wins by default, and a written rule is what reliably changes course.
Figure 3: Research approach. Each task admits two functionally acceptable implementations (baseline in blue, alternative in orange). Within the controlled study, Stage 1 measures default choices without relevant examples, Stage 2 varies repository precedents in local and sibling-resource files (including which file holds the example at a fixed count), and Stage 3 adds task guidance that contradicts existing code. Every edit is scored separately for functional success and implementation choice; the local-versus-sibling comparison is also checked in open-source repositories.
Table 1: Implementation pairs. Both implementations in a pair pass the same hidden functional tests; adoption refers to the alternative.
Figure 4: Illustrative repository interventions for ID serialization. Each square is one existing endpoint file. The upper rows vary how many local or sibling-resource files use string IDs; the lower rows hold the local support level at 1/3 and change only which file carries the alternative (“default” marks the assignment used for local support 1/3 above). The requested new endpoint is absent from every variant.
Study
Design
Trials
Controlled repository
RQ1: zero precedent
6 pairs × 4 agents
360
RQ2/3: support levels
6 pairs × 4 agents × 8 cond.
2,880
RQ2: file identity (expl.)
4 placements + order control
75
RQ2: file identity (repl.)
3 settings × 6 placements
270
RQ4: explicit guidance
2 pairs × 2 agents × 4 cond.
240
Table 2: Experimental scale: 5,625 trials in 375 cells, each run in 15 independent sessions on fresh repository instances.
Pair
Default implementation
Label
S
O
G5
G6
ID serialization
integer: "id": 1
B
15
15
15
15
Field casing
snake_case : project_id
B
15
15
15
15
Envelope
flat: {...}
B
15
15
15
15
Boolean naming
no prefix: completed
B
15
15
15
15
Error handling
direct: return ..., 404
B
15
15
15
15
Dispatch
lookup: LABELS[p]
A
14
15
15
15
Table 3: RQ1: default implementation in the zero-precedent repository, and the number of runs (of 15) in which each agent chooses it. “Label” gives the predefined role of the default in each pair (B = baseline, A = alternative). S = Sonnet-4-6, O = Opus-4-8, G5 = gpt-5.6-sol, G6 = gpt-6-sol.
Figure 6: File-placement effects at fixed local support (15 trials per cell). C, U, L denote tasks_create , tasks_update , tasks_list ; bold headers indicate placements containing L. Rows show one exploration and three replications; the p column gives the placement test within each support level ( χ2 for exploration, Holm-adjusted within-block randomization for replications). “–” denotes omitted results (11–15/15).
Figure 7: RQ4 joint success (passing functional tests and adopting the target) out of 15 trials per cell, with all six existing endpoints favoring the opposite implementation. Left (240 trials): C0 = no guidance, C1 = written rule, C2 = code example, C3 = both. Right (180 trials): no rule, task-description rule, or harness-loaded native-file rule ( CLAUDE.md / AGENTS.md ).
Table 9
Local flat
Local envelope
Repository
Agent
FF/FN
FD
DF/DN
DD
Δ
p
RealWorld
S
0/15
0/15
3/15
6/15
0.30
0.0078
RealWorld
G6
0/15
0/15
12/15
11/15
0.77
1.2×10−6
Go REST
S
0/15
0/15
0/15
0/15
0.00
1
Go REST
G6
0/15
0/15
15/15
15/15
1.00
1.5×10−8
ExpressJS
S
0/15
0/15
15/15
15/15
1.00
1.5×10−8
Table 6: Open-source response-envelope study. Entries give functionally successful data-envelope (D) implementations out of 15 trials. Columns specify local and sibling-resource patterns (F = flat, D = envelope); FN/DN indicate no sibling resource. Δ is the D-adoption difference between D-local and F-local assignments, with Holm-adjusted p -values across eight repository–agent strata. S = Sonnet-4-6; G6 = gpt-6-sol.
Random rules improve a coding agent's task performance as much as expert-curated ones (both +13.8pp on a discriminative subset of SWE-bench Verified), and in our data every individually beneficial rule is a negative constraint ("do not refactor unrelated code"), while every individually harmful one is a positive directive ("follow code style"). We arrive at these findings through the first large-scale controlled study of agent rule files (CLAUDE.md, .cursorrules, and the broader family of agent skills, plugin manifests, and persona definitions): we scrape 679 rule files (25{,}532 rules) from GitHub and conduct over 5{,}000 agent runs of Claude Code with Claude Opus 4.6 on SWE-bench Verified. Three patterns emerge. (i) Rule polarity cleanly separates beneficial from harmful rules; we read this through the lens of potential-based reward shaping (PBRS). (ii) Performance gains are largely content-independent: random, shuffled, mismatched-domain, and unconverted-format rule files all match curated rules, pointing to a context priming mechanism. (iii) Individual rules often appear harmful in isolation yet do not visibly accumulate damage in ensemble: pass rates remain stable across rule counts from 0 to 50. These findings expose a hidden reliability risk in the rapidly growing ecosystem of community-authored rules and skills, and they yield a clear principle for safer agent configuration: constrain what agents must not do, rather than prescribing what they should.
Xing Zhang, Guanghui Wang, Yanwei Cui +4
AWS Generative AI Innovation Center · HSBC Holdings Plc., HSBC Technology Center, China.
Coding agents can be guided with instruction files in a repository. We examine this emerging practice through a census of 22,854 popular, active GitHub repositories in thirteen languages, treating each repository's root 'AGENTS.md' and 'CLAUDE.md' as one instruction set. Agent instructions appear in 26.3% of repositories and 59.6% of the most-starred; where both files exist, 'CLAUDE.md' usually points to 'AGENTS.md'. We analyse 341,852 statements using model-assisted annotation and human review. The annotations reveal a practical mix of project knowledge and rules about code, building and workflow. Collaboration rules name concrete requirements, from test plans to commit formats, yet 74.0% of directives give no reason. Projects disagree on AI attribution: commit-message rules more often prohibit it than require it, while pull-request rules more often require disclosure. A third of instruction sets have a generator-associated signature, a tool-inserted block, shared text or an AI-marked creation commit; four in five are edited after creation. Prespecified tests associate repository popularity with rules on pull requests and AI contributions. Exploratory analyses link activity to process-oriented content and show that LLM and agent projects are over-represented among adopters. The models agree much more closely with each other than with the human reviewer's topic assignments, a caution for automated annotation. We release the annotations, taxonomies and pointers to the analysed file versions.
Code agents have become increasingly capable of performing software engineering tasks through interaction with software environments. However, agents tend to degrade as the number of interaction rounds increases, making it difficult to maintain existing software during iterative modification. We analyze 2,656 agent trajectories in EvoCode-Bench and identify requirement focus misalignment as a recurring failure pattern, where agents struggle to distinguish the primary modification targets of a requirement from unrelated code. We propose Repotrace a requirement-driven repository understanding harness that combines repository structure with requirement semantics to construct a requirement-code correspondence and injects it into the agent context. This correspondence enables code agents to follow the repository architecture, trace requirements across multi-round interactions, and identify the core code elements corresponding to the current requirement. Experiments show that Repotrace improves the mean per-round reward (MPR) across different models (e.g., +19.99 on GLM-5.2) and agent harnesses (e.g., +10.22 on Claude Code). Across rounds, Repotrace increases the average per-round reward from 0.163 to 0.371 on Terminus-2 with DeepSeek-V4-Pro, while reducing average regression errors by 56.78%. We release the source code and agents' trajectories to support future research.
Huanyu Liu, Yezheng Cheng, Jinliang You +5
Key Laboratory of High Confidence Software Technologies, Ministry of Education; School of Computer Science, Peking University · Xidian University · School of Cyber Science and Engineering, Wuhan University