Precedent or Intent? What Coding Agents Follow When Evolving Existing Code

Haichao Zhu; Geyang Xu; Zhaorui Yang; Jiyuan Wang; Qian Zhang

cs.SEOct 28, 2026

Precedent or Intent? What Coding Agents Follow When Evolving Existing Code

Authors: Haichao Zhu, Geyang Xu, Zhaorui Yang, Jiyuan Wang, Qian Zhang

Organizations: Independent Researcher · University of California, Riverside · Tulane University

Abstract

Existing code reveals past implementation decisions but rarely explains the reasoning behind them. When coding agents implement new features or modify existing functionality, past decisions may conflict with new task instructions. This raises a fundamental question for agentic software evolution: when multiple implementations satisfy the same functional requirements, how do coding agents decide whether to follow existing code or new task instructions? We investigate this question through bidirectional, controlled experiments in purpose-built repositories, followed by validation in real-world open-source repositories. Our approach isolates three competing sources of guidance: (1) agents' default preferences, (2) repository precedents, and (3) explicit task instructions. We conduct 5,625 trials across four coding agents and six implementation pairs, with validation in four open-source repositories spanning three programming languages. We find that agents share the same default implementation choices, yet overwhelmingly follow repository precedent when it conflicts with those defaults (359/360 trials). Crucially, not all precedents carry equal influence: three supporting local files yield 96% adoption, compared with only 4% for sibling-resource files, and changing file placement alone can shift adoption from 0% to 100% at a fixed example count. When new task intent conflicts with existing code, written priority rules achieve substantially higher success than code examples alone (59/60 versus 38/60 joint successes for two agents on two implementation pairs). The effectiveness of explicit rules also extends to open-source repositories, where a harness-loaded rule raises joint success from 0/120 to 117/120. These findings motivate design-decision traceability that preserves not only implementation choices but also the rationale and priorities that should guide future changes.

Figures & tables

Panel of Figure 1 — Figure 1: Coding agents that extend a repository are pulled between the implementation already in the code and the intent of the new task. In controlled experiments with four agents and six implementation pairs, existing code wins by default, and a written rule is what reliably changes course.

Panel of Figure 3 — Figure 3: Research approach. Each task admits two functionally acceptable implementations (baseline in blue, alternative in orange). Within the controlled study, Stage 1 measures default choices without relevant examples, Stage 2 varies repository precedents in local and sibling-resource files (including which file holds the example at a fixed count), and Stage 3 adds task guidance that contradicts existing code. Every edit is scored separately for functional success and implementation choice; the local-versus-sibling comparison is also checked in open-source repositories.

Panel of Table 1 — Table 1: Implementation pairs. Both implementations in a pair pass the same hidden functional tests; adoption refers to the alternative.

Panel of Figure 4 — Figure 4: Illustrative repository interventions for ID serialization. Each square is one existing endpoint file. The upper rows vary how many local or sibling-resource files use string IDs; the lower rows hold the local support level at $1/3$ and change only which file carries the alternative (“default” marks the assignment used for local support $1/3$ above). The requested new endpoint is absent from every variant.

Study	Design	Trials
Controlled repository
RQ1: zero precedent	6 pairs $\times$ 4 agents	360
RQ2/3: support levels	6 pairs $\times$ 4 agents $\times$ 8 cond.	2,880
RQ2: file identity (expl.)	4 placements + order control	75
RQ2: file identity (repl.)	3 settings $\times$ 6 placements	270
RQ4: explicit guidance	2 pairs $\times$ 2 agents $\times$ 4 cond.	240

Table 2: Experimental scale: 5,625 trials in 375 cells, each run in 15 independent sessions on fresh repository instances.

Pair	Default implementation	Label	S	O	G5	G6
ID serialization	integer: "id": 1	B	15	15	15	15
Field casing	snake_case : project_id	B	15	15	15	15
Envelope	flat: {...}	B	15	15	15	15
Boolean naming	no prefix: completed	B	15	15	15	15
Error handling	direct: return ..., 404	B	15	15	15	15
Dispatch	lookup: LABELS[p]	A	14	15	15	15

Table 3: RQ1: default implementation in the zero-precedent repository, and the number of runs (of 15) in which each agent chooses it. “Label” gives the predefined role of the default in each pair (B = baseline, A = alternative). S = Sonnet-4-6, O = Opus-4-8, G5 = gpt-5.6-sol, G6 = gpt-6-sol.

Panel of Figure 6 — Figure 6: File-placement effects at fixed local support (15 trials per cell). C, U, L denote tasks_create , tasks_update , tasks_list ; bold headers indicate placements containing L. Rows show one exploration and three replications; the $p$ column gives the placement test within each support level ( $\chi^{2}$ for exploration, Holm-adjusted within-block randomization for replications). “–” denotes omitted results (11–15/15).

Panel of Figure 7 — Figure 7: RQ4 joint success (passing functional tests and adopting the target) out of 15 trials per cell, with all six existing endpoints favoring the opposite implementation. Left (240 trials): C0 = no guidance, C1 = written rule, C2 = code example, C3 = both. Right (180 trials): no rule, task-description rule, or harness-loaded native-file rule ( CLAUDE.md / AGENTS.md ).

Table 9

Repository	Agent	FF/FN	FD	DF/DN	DD	$\Delta$	$p$
		Local flat		Local envelope
RealWorld	S	0/15	0/15	3/15	6/15	0.30	0.0078
RealWorld	G6	0/15	0/15	12/15	11/15	0.77	$1.2\times 10^{-6}$
Go REST	S	0/15	0/15	0/15	0/15	0.00	1
Go REST	G6	0/15	0/15	15/15	15/15	1.00	$1.5\times 10^{-8}$
ExpressJS	S	0/15	0/15	15/15	15/15	1.00	$1.5\times 10^{-8}$

Table 6: Open-source response-envelope study. Entries give functionally successful data-envelope (D) implementations out of 15 trials. Columns specify local and sibling-resource patterns (F = flat, D = envelope); FN/DN indicate no sibling resource.

\Delta

is the D-adoption difference between D-local and F-local assignments, with Holm-adjusted

p

-values across eight repository–agent strata. S = Sonnet-4-6; G6 = gpt-6-sol.

AI Coding Agents

AI coding agents use language models and software tools to plan, edit, test, and maintain code with varying levels of autonomy.

See all 329 papers

Guardrails Beat Guidance: A Large-Scale Study of Rules, Skills, and Persistent Configuration for Coding Agents
Apr 13, 2026Xing Zhang, Guanghui Wang, Yanwei Cui +4Software Engineering Agents LLM Agent Reliability
AWS Generative AI Innovation Center · HSBC Holdings Plc., HSBC Technology Center, China.
What Open-Source Projects Tell Their Coding Agents: A Census of AGENTS.md and CLAUDE.md
Oct 22, 2026Pawel ZarzyckiAI Coding Agents AI Use Disclosure
RepoTrace: Improving Requirement Tracing in Multi-Round Agent Interactions
Oct 26, 2026Huanyu Liu, Yezheng Cheng, Jinliang You +5Software Engineering Agents LLM Agent Reliability
Key Laboratory of High Confidence Software Technologies, Ministry of Education; School of Computer Science, Peking University · Xidian University · School of Cyber Science and Engineering, Wuhan University

Apr 13, 2026J/K move · Enter open · S save

Guardrails Beat Guidance: A Large-Scale Study of Rules, Skills, and Persistent Configuration for Coding Agents

Xing Zhang, Guanghui Wang, Yanwei Cui, Wei Qiu+3

AWS Generative AI Innovation Center · HSBC Holdings Plc., HSBC Technology Center, China.

Software Engineering Agents LLM Agent Reliability LLM Guardrails

Open paper AskPDF arXiv

Random rules improve a coding agent's task performance as much as expert-curated ones (both

+13.8

pp on a discriminative subset of SWE-bench Verified), and in our data every individually beneficial rule is a negative constraint ("do not refactor unrelated code"), while every individually harmful one is a positive directive ("follow code style"). We arrive at these findings through the first large-scale controlled study of agent rule files (CLAUDE.md, .cursorrules, and the broader family of agent skills, plugin manifests, and persona definitions): we scrape 679 rule files (25{,}532 rules) from GitHub and conduct over 5{,}000 agent runs of Claude Code with Claude Opus 4.6 on SWE-bench Verified. Three patterns emerge. (i) Rule polarity cleanly separates beneficial from harmful rules; we read this through the lens of potential-based reward shaping (PBRS). (ii) Performance gains are largely content-independent: random, shuffled, mismatched-domain, and unconverted-format rule files all match curated rules, pointing to a context priming mechanism. (iii) Individual rules often appear harmful in isolation yet do not visibly accumulate damage in ensemble: pass rates remain stable across rule counts from 0 to 50. These findings expose a hidden reliability risk in the rapidly growing ecosystem of community-authored rules and skills, and they yield a clear principle for safer agent configuration: constrain what agents must not do, rather than prescribing what they should.

Abstract

AI Coding Agents

Explore similar work

Guardrails Beat Guidance: A Large-Scale Study of Rules, Skills, and Persistent Configuration for Coding Agents

What Open-Source Projects Tell Their Coding Agents: A Census of AGENTS.md and CLAUDE.md

RepoTrace: Improving Requirement Tracing in Multi-Round Agent Interactions