cs.SEAug 14, 2026

Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Model

Authors: Stephanie Jarmak

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

AI coding agents are commonly evaluated as models but deployed as systems. Their reliability depends not only on model capability, but on the harness, execution state, retrieval, memory and state management, permissions, review interfaces, and resource allocation. This monograph examines those boundaries and develops a framework for evaluating and operating coding agents reliably. It synthesizes 164 scholarly works, 100 practitioner records, 29 benchmark records, and 17 author-system case records through a structured multivocal review, targeted update audits, software-engineering coverage analysis, and distributed-systems evidence synthesis. Across this evidence, many apparent model failures originate elsewhere in the system, while improvements at one layer often fail to propagate to end-to-end outcomes. Evaluation and operation are treated as a dependency chain in which weaknesses in task construction, execution environments, retrieval, state management, verification, or observability can invalidate downstream conclusions. The monograph contributes a versioned catalog of 206 reliability records: 193 gated practices, including 56 developed in depth, plus 13 research leads; an evidence ledger; a framework for dependency and repair asymmetry across the agent lifecycle; measurements and failure cases from operated agent systems; runnable evaluation and reliability protocols; and five reusable agent skills with evidence maps. Together, these provide a system-level methodology for distinguishing model capability from infrastructure effects, designing defensible evaluations, and building systems that recover safely when components fail. The review is structured rather than exhaustive, evidence strength varies by topic, and results depend on workload and configuration. The methods record which search lanes were executed, which remain unexecuted, and limits on evidence-grading claims.

Figures & tables

Figure 1: Argument map for the reliability dependency chain: measurement, grading, containment and recovery, retrieval and context, review and accountability, and allocation and cost. Each layer supplies the evidence boundary on which the next relies.
Figure 2: Flow of source review and practice synthesis. The source lane distinguishes the base review, bounded update audit, SE coverage probe, and the publisher- and index-native searches this edition did not perform. The practice lane reports the admission gate and overlapping hardening operations without treating their counts as an arithmetic decomposition.
Review stepModeHuman decision retained
Candidate retrieval and rankingAutomated and assistedI defined each thread question and search boundary, then read the admitted sources.
Identity resolution, normalization, and duplicate checksAutomated checks with human resolutionI resolved ambiguous identities and practitioner independence keys.
Bounded-claim extractionAssistedI checked the source, revised the extracted claim, and accepted or rejected it.
Initial evidence-group proposalAssistedI assigned the final label to the scoped claim.
Challenge passAssistedAutomated passes flagged composite claims, contrary findings, duplicate support, and label inconsistencies; I reread the source and adjudicated each change.
Practice admission, chapter selection, and proseHumanI made the selection and writing decisions.
Table 1: Automation and human judgment in the review workflow. Automated means the operation ran without item-level prompting; assisted means a system proposed or flagged material for a human decision; human means the substantive choice was made without an automated verdict.
PartChTitle
Part I: Evaluation measurement and experiment design1Run-to-run variance, statistical power, and paired comparisons
 2Baselines, ablations, and cost-accuracy tradeoffs
 3Benchmark contamination, oracle strength, and workload validity
Part II: Evaluation and grading systems4Execution-based evaluation, correction gates, and release tests
 5Calibrating model graders and separating agreement from correctness
 6Proxy metric gaming and layered evaluation signals
Table 2: Parts and chapters in the dependency-chain order.
Figure 1.1: Schematic of correlated item scores for systems A and B and the narrower uncertainty obtained by analyzing their matched differences.
Outcome structureComparison to analyzeTest guidance
Per-item numerical scores whose paired differences plausibly satisfy a normal approximationMean of the paired item differencesPaired t-test
Metrics computed nonlinearly from aggregate counts, including corpus-level F-score and BLEUFull metric recomputed on resampled or relabeled item pairsPaired bootstrap or paired permutation test
Pass or fail outcomes on the same itemsDiscordant pairs in which only one system passesMcNemar’s test
Table 1.1: Outcome structure, paired estimand, and corresponding test guidance.
Figure 2.1: Schematic of a cost-accuracy frontier, dominated configurations, and minimum-accuracy and cost-ceiling feasibility constraints.
Figure 3.1: A public score is the output of three coupled measurements. Each question has its own control, and reputation, curation effort, and a large annotator count answer none of them.
Figure 3.2: A passing score supports a capability claim only when the benchmark matches the target workload, the solution is independent of prior or in-task exposure, and the oracle rejects incorrect artifacts. A failure at any link invalidates a different inference.
Figure 4.1: Across 1,750 trajectories covering 50 tasks and four models, one model submitted work in 100% of its trials but resolved only 44% under an external oracle.
Figure 4.2: The sandbox verdict is immutable. A failed candidate returns to the correction step, which emits a new candidate under a new identity rather than revising bytes that already carry a verdict.
Figure 4.3: The same k trials answer two different operational questions. Choose the metric whose failure semantics resemble the deployment.
Figure 5.1: Cells of 85, 5, 5, and 5 produce 90% observed versus 82% expected agreement and kappa about 0.44; two always-PASS graders produce 100% observed and expected agreement but undefined kappa.
Figure 5.2: Equal-length bars compare 950 routine, 30 borderline, and 20 defect deployment items with natural 95, 3, and 2 and stratified 40 each requiring reweighting; one missed defect changes detection by 50 percentage points.
Figure 6.1: Schematic of proxy performance continuing to rise after independent quality has peaked.
Figure 6.2: Two failure domains in verification. Verifier-infrastructure health asks whether the verifier executed reliably against the intended artifact; oracle adequacy asks whether its judgment distinguishes acceptable from unacceptable behavior. The verdict is bound to the artifact version, the verifier version, and the environment identity of the run.
IdentityNames
work_idstable logical work; the intent that should happen once logically
input_state_idthe repository or repositories, branches, and revisions against which the attempt was planned and executed
ownership_epochmonotonic generation of write authority over that work
attempt_idone execution attempt under a given work and epoch
artifact_versionconcrete code or state produced by an attempt
verification_idone verification execution, including the artifact, verifier version, environment, and inputs it observed
Table 7.1: Identity vocabulary used through the rest of the book.
Figure 7.1: The identity and boundary model. Logical work carries an ownership epoch granting authority to an attempt, an input state the attempt observes, and an effect identity for each external commitment. The attempt produces a candidate artifact bound to a verification record; the external commitment is paired with an owned reconciliation path. Authoritative facts live on the work side of each boundary, never only in the worker.
Figure 7.2: Separate lifecycles for logical work and for attempts. Logical work runs accepted, eligible, owned under an epoch, complete, with side states for blocked, unknown external state, and reconciliation. Attempts run created, executing, outcome ready, verification under the current epoch; a retryable failure produces a new attempt under the same work, a terminal failure defers to work policy, and reassignment opens a new epoch. A worker’s self-report can move an attempt only to outcome ready; only independently observed evidence completes logical work.
Figure 8.1: In the database incident, primary and recovery data sat inside one reachable failure domain. Blast radius follows from effective capabilities, and a warning in the prompt removes none of them.
Identity under testAction attemptedRequired result
OrdinaryThe prohibited writeDenied by the enforcing system
EscalatedThe intended narrow actionSucceeds
EscalatedAdjacent destructive actionsDenied
Table 8.1: Both directions of the capability boundary. A configuration passes only when every row holds.
Threat categoryPreventive controlDetecting controlCapability boundary
Untrusted instructions in retrieved contentSource restrictions and input validationInjection screening and trace reviewTask-specific tool allowlist
Sensitive-output disclosureRead-scope restriction and output policySecret and policy scanningNo access to unrelated sensitive data
Unauthorized tool selectionGateway-enforced allowlistTool-request auditUnavailable tools cannot be invoked
High-impact external effectsTyped proposal and approvalRead-back and audit logNarrow, expiring execution identity
Table 8.2: Threats, preventive and detecting controls, and the enforcement boundary.
TermMeaning
Logical workThe user or operator intent that should happen once logically, under a stable work identity.
AttemptOne execution of that work; a retry is a new attempt at the same logical work (contract I4, retries preserve logical identity).
DeliveryThe assignment or redelivery of an attempt to a worker by the runtime.
Runtime commitA result durably recorded inside the orchestration boundary.
External commitA world-changing effect accepted outside that boundary, such as a merge, deployment, or payment.
IdempotencyThe contract under which a repeated external commit is indistinguishable from a single one (contract I5, effects are safe under redelivery or explicitly uncertain).
Table 23
Figure 9.1: A first attempt records a durable intent and operation identity before an external commit. If the worker dies before recording completion, a replacement reads the pending intent and retries with the same identity; the external boundary must return the prior result, converge, or force explicit reconciliation.
Figure 10.1: A typed event stream records the commitment gap rather than hiding it: dispatch is certain, the external effect is known, and durable persistence of the result never occurred.
Figure 10.2: Five lifecycle kill points test recovery from unstarted work through ambiguous commitment gaps to durable completion, including recognition and suppression of repeated external effects.
FaultContractExpected observable behavior
Response lost after external commitI5The runtime treats the call as uncertain, reconciles against external state before re-sending, and records the reconciliation outcome; no blind re-send.
Delayed response delivered after the retry has startedI5The late response and the retry resolve to one logical effect; the effect ledger shows one committed effect_id or an explicit unknown_external_state , never two commits.
Worker loses its lease but continues runningI2The protected mutation boundary rejects the superseded worker’s writes by ownership_epoch ; the rejection appears in the trace as a fencing event, not a silent success.
Old worker’s completion arrives after the new worker has succeededI3The stale completion is rejected by generation and attempt identity; logical state reflects the new attempt only, and the rejection is recorded.
Repository state changes between validation and publicationI6Publication fails closed: the runtime detects that the validated version is no longer the head, refuses to publish against changed state, and routes to re-validation or reconciliation.
Table 10.1: Representative distributed ambiguity faults, the contract each tests, and the observable behavior an acceptable runtime must exhibit. The full matrix is published in the companion repository.
Figure 11.1: A planning error is the first upstream failure linking four visible defects, while a counterfactual test distinguishes a plausible causal boundary from a symptom.
Figure 12.1: The retrieval chain separates availability, retrieval, placement, contextual sufficiency, and use failures across the transitions from task formulation to scored execution and verification.
Figure 13.1: The localization funnel progressively narrows repository evidence, while candidate file selection caps recall because any omitted file remains unavailable to every later inspection and edit stage.
Figure 15.1: The recorder appends an immutable source record; the distiller reads it for one stated retrieval purpose and writes a derived store whose items carry provenance, extraction version, and schema version. A schema change rebuilds that store from the record, because material removed by an in-place rewrite has no observation left from which a later rewrite could restore it.
Figure 16.1: Each kind of detection routes to the least complex mechanism capable of checking it, and every layer must expose the evidence the next one needs.
Figure 17.1: Authority widens one rung at a time within a defined action class. Separate rails transfer no evidence between them, and the approval floor beneath them is permanent.
Figure 17.2: An advisory review can record rejection while leaving execution reachable. An effective gate connects the review decision to an enforcement point, so rejection or modification changes the next reachable state.
ConditionQuestion for the running gateObservation to collectFailure implication
Causal powerCan the person stop or alter this exact action?Rejection, cancellation, edit, and timeout tracesThe click records assent but does not control execution
Epistemic accessCan the person inspect the relevant inputs, artifact, target, and consequences before deciding?Missing context, inaccessible logs, and reconstruction errorsApproval rests on incomplete state
Self-controlCan the person decide without a forced default or impossible queue constraint?Default acceptance, time pressure, and override useThe workflow substitutes acquiescence for judgment
Fitting intentionsIs the role expected and prepared to inspect the named risk?Review actions, escalation behavior, and interviewsThe role is ceremonial or aimed at another risk
Table 17.1: Conditions under which a human gate can exercise effective control.
Figure 18.1: Fault propagation depends on collaboration topology, ranging from downstream contamination in pipelines to contained local errors when shared systems preserve provenance and state labels.
Figure 18.2: Epoch fencing on work W-311. Worker A is not dead and behaves correctly by its own lights, having never been told it lost authority; only the epoch check at the protected target stops its stale mutation. Worker B’s mutation, carrying the current epoch, is eligible for normal validation, not automatically accepted: holding the current epoch confers the right to be evaluated, not a presumption of correctness.
Figure 19.1: The re-decision loop recomputes from observed events under fixed work and churn costs, then executes a deadline capped incumbent while retaining its optimality gap.
Figure 19.2: Live arrivals and retry-driven recovery traffic compete for the same constrained dependency; unrestricted retries amplify load exactly when capacity has fallen, while admission limits, jitter, and reserved recovery capacity bound the interference.
StageFields
Observed eventsArrival time, priority, claim and release events, observed duration, completion or failure outcome, and review verdict.
Serving observationsPer call: endpoint or route selected, model and serving configuration version, input and output token counts, cached input share where the provider reports it, time to first token, total call latency, observed queueing, and retry count. Record which of these the provider does not expose.
Decision stateAvailable capacity, eligible execution pools, work already running, leases or locks, deadlines, model or tool compatibility, reviewer availability, and constraints that excluded an otherwise eligible item.
Policy estimatesPredicted cost, predicted task performance, routing uncertainty, expected duration, solver objective value, and optimality gap.
Code-estate stateCampaign identity, repository, base revision, host or workspace, affected entities, artifact version, code index version, conflict evidence, and publication state.
Table 19.1: Allocation-ledger fields in the order they become interpretable.
DecisionPersonal starting pointWhat changes it
Repeated evaluationUse k=3k=3 paired runs per item for inexpensive screening, but do not promote from that screen alone. Size release comparisons from a pilot and the smallest decision-relevant effect.Higher variance, clustered tasks, or smaller meaningful effects require more runs.
Reliability reportingReport both pass@k\mathrm{pass}@k , where any attempt succeeds, and passk\mathrm{pass}^{k} , where every attempt succeeds, with kk printed beside the result. Begin with k=3k=3 .The deployed retry policy and the cost of intermittent failure determine the useful kk .
Human-label agreementTreat Cohen’s or Fleiss’s kappa below 0.60 as a rubric-debugging trigger. A value above 0.60 is not a correctness gate. Promotion still depends on class-specific errors against expert adjudication.Class imbalance, ambiguous labels, and high-consequence mistakes make the confusion matrix more informative than kappa alone.
Promotion thresholdBefore execution, require either a three-percentage-point absolute success gain at no material cost increase or a ten-percent cost reduction at no success loss. Treat smaller effects as uncredited until a decision justifies them.Task value, baseline success rate, and operating cost should replace these round numbers.
Failure reviewRead twenty stratified failures after a material model, evaluation apparatus, or policy change. Preserve an abstain category when the trace cannot support attribution.Rare high-consequence failure classes should be oversampled regardless of frequency.
Context restartRestart or consolidate at 60 percent of the shortest context length at which the weakest required task shows material degradation in a local sweep.New models, tools, instruction files, or task mixtures require another sweep.
Table 20.1: Personal starting points for a new system. These values are author defaults, not transferred effect estimates.
Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

IdentifierPracticeCh.Treatment
ERCA-001Strengthen weak test oracles before trusting pass rates3developed
ERCA-002Run memorization probes before attributing benchmark gains to capability3companion
ERCA-003Pair every public benchmark score with a matched private control and measure the gap3developed
ERCA-004Evaluate on post-cutoff tasks via temporal holdouts and live pipelines3developed
ERCA-005Measure train-test overlap with semantic matching, not just n-grams3companion
ERCA-006Measure contamination by estimated score benefit, not binary flags †\dagger3companion
Appendix
Table A.1: Practice catalog index: identifier, practice, chapter, and treatment for all 206 records in this edition.

Explore similar work

CardsList
Feb 18, 2026

Towards a Science of AI Agent Reliability

AI agents are increasingly deployed to execute important tasks. While rising accuracy scores on standard benchmarks suggest rapid progress, many agents still continue to fail in practice. This discrepancy highlights a fundamental limitation of current evaluations: compressing agent behavior into a single success metric obscures critical operational flaws. Notably, it ignores whether agents behave consistently across runs, withstand perturbations, fail predictably, or have bounded error severity. Grounded in safety-critical engineering, we provide a holistic performance profile by proposing twelve concrete metrics that decompose agent reliability along four key dimensions: consistency, robustness, predictability, and safety. Evaluating 15 models across two complementary benchmarks, we find that recent capability gains have only yielded small improvements in reliability. By exposing these persistent limitations, our metrics complement traditional evaluations while offering tools for reasoning about how agents perform, degrade, and fail.

Stephan Rabanser, Sayash Kapoor, Peter Kirgis +3

Princeton University.

AI Agent ReliabilityAI Agent EvaluationAI Agent SafetyAI Agent Benchmarks
AskPDF
Oct 6, 2026

A Case Study in Assuring AI-Written Software

Software-engineering agents can enable people without formal software training to build systems they could not otherwise implement and simultaneously can produce more code than even experts can meaningfully inspect. In both cases, exhaustive code review is not reliable as the sole basis for human control. We report a case study of a production healthcare platform built through coding agents and governed by an operator without formal software-engineering training. Over time, its workflow grew into a human-led meta-agent system where one agent wrote code, other agents supervised and reviewed it, and project rules carried lessons forward. The operator found that tests, monitors and reviewing agents used to supervise the system were fallible. Some monitors measured proxies rather than outcomes, some audits failed silently, missing checks disappeared from reported results and one automated repair caused operational disruption. In this case, human control depended on keeping the intended outcome, the evidence used to judge it, the agents' permissions and the final human decision were all tied to the same underlying objective.

Lindsey Ferris, Sierra Bonilla

Independent Researcher · University College London London, United Kingdom

AI Agent ReliabilityAI AssuranceAI Coding AgentsAI Agent MonitoringAutomated Code ReviewHuman Oversight of AI
AskPDF
Jun 13, 2026

ClayBuddy: A Framework, Evaluation, & Mitigation of Coding Agent Failures

Software engineering and deployment are increasingly delegated to AI coding agents. The scale of their adoption is surfacing rare, but highly destructive, failure modes. In this paper, we study these failure modes as stemming from three distinct mechanisms: underspecification, where default model behavior is unsafe; capability errors, where the safe action is available but the model does not adhere to it due to bias or capability limitations; and agent harness errors, where the model fails to execute the safe action through the harness. We assess these across 8 different evaluations, each inspired by real-life deployment failures, totaling 20 coding environments and 59 synthetic transcript templates. These evaluations act as controlled stress tests for isolating our failure mechanisms. Based on this evaluation, we propose ClayBuddy, a harness modification that molds to user preferences and can be modified by the model in-session, to mitigate these errors. By adding tools for the agent to edit its own context, an extended system prompt, a customizable command classifier, and deterministic guardrails, we show that ClayBuddy is safer across a statistically significant number of samples. Thus, we suggest concrete mitigations for current coding agents and a design philosophy for future agent harness features.

Kenneth Ge, Andre Assis

Anthropic Fellows Program · Constellation

Agent Failure AnalysisAI Coding AgentsAI Agent SafetyAI Agent Safety BenchmarksAgentic AI
AskPDF