Assurance Envelopes for Autonomous Coding Agents: Minimum-Cost Evidence for Software Change

Anjan Goswami

cs.SESep 14, 2026

Assurance Envelopes for Autonomous Coding Agents: Minimum-Cost Evidence for Software Change

Authors: Anjan Goswami

Organizations: SmartInfer AI

Abstract

When a coding agent returns to existing software, it inherits evidence from earlier engineering work: tests, type checks, proofs, static analyses, and traces. Reloading all of it is wasteful, but dropping a piece the change depends on can leave a required property unsupported. Given the properties a change must preserve, its obligations, we ask which least-cost subset of the available evidence re-establishes them, and we call such a subset a task-conditioned assurance envelope. Evidence and the rules that combine it form a typed inference graph; an obligation is met when forward chaining from the selected evidence reaches it, and we validate every selection by that closure rather than by trusting the optimizer. The software-derived graphs in our evaluation come from preserved outcomes of prior AI coding-agent runs; we freeze those artifacts and ask which accumulated evidence should be restored for a later task. Small graphs from Rust, IronBlocks, and Pong outcomes show that the minimum envelope depends on the task, that none may exist when current evidence cannot re-establish a required property, that some properties need several pieces of evidence together, and that expanding the requirements adds evidence rather than replacing it. A prespecified synthetic benchmark of 249 instances characterizes computation: a baseline that discards the 'several pieces together' structure necessarily fails to re-derive them; every completed exact cross-check agreed with the CP-SAT optimizer; and median solve time stayed below 20 ms at 500-evidence graphs, except that graphs with many alternative derivations per target timed out at far smaller sizes, so structure, not raw size, drives difficulty. The contribution is a bounded application of established optimization to selecting assurance context for a software change; discovering the obligations and downstream agent benefit remain open.

Figures & tables

Panel of Figure 1 — Figure 1: A task-conditioned envelope selects evidence and derivations for supplied roots. Root discovery is outside the solved problem.

Panel of Figure 2 — Figure 2: A worked inference graph. The rule node preserves $a\land b\Rightarrow c$ ; direct evidence $e_{c}$ supplies an alternative route. Costs determine which valid support is selected.

Graph/task	Required support	Cost	Result
Rust C, local	$e_{build,C},e_{local,C}$	2	feasible; trace evidence omitted
Rust C, trace	$e_{build,C},e_{local,C},e_{trace,C}$	3	feasible; task adds trace root
Rust B, local	$e_{build,B},e_{local,B}$	2	feasible
Rust B, trace	none	–	no sufficient envelope; no positive trace support
IronBlocks	$e_{detect},e_{mediated}$	2	unique compositional optimum; FLAT chooses invalid cost-1 singleton
Pong, $R_{0}$	$e_{prior,post}$	1	prior repaired-artifact obligation

Table 1: Exact results for the prespecified graphs hand-specified from preserved software outcomes. Dashes denote an infeasible support problem (no sufficient envelope), not a solver error.

Family	Evaluations	Invalid FLAT	False roots
F1 independent	36	0	0
F2 conjunction	45	45	651
F3 alternatives	33	0	0
F4 shared support	33	33	2,022
F5 irrelevant evidence	36	0	0
F6 mixed composition	27	27	636

Table 2: FLAT false coverage under the prespecified pairwise AND-erasure projection. Invalid means the selected support failed closure under the original graph.

Panel of Figure 3 — Figure 3: Median unit-cost support reduction versus retaining all evidence. F7 is omitted because no sufficient support exists for its roots. F8 has only three prespecified size points.

Size	Graph instances	Both methods complete	Compositional timeouts
10	21	21	0
15	21	21	0
20	21	6	15
Method-instance total	63	111/126 completed

Table 3: Exact-oracle completion. At n=20, compositional enumeration completed only for F1 and F5; every completed exact result agreed with CP-SAT.

Panel of Figure 4 — Figure 4: Median CP-SAT solve time for the prespecified primary size sweep. F7 times measure proofs of infeasibility. The prespecified n=100 aggregation includes the secondary cost points for F3 and F6. Timing is machine- and encoding-specific.

Evidence Selection

Evidence selection chooses the documents, passages, claims, or observations that best support an answer or decision.

See all 61 papers

Aug 17, 2026

Executable Code Knowledge: Code as a Native, Validation-Carrying Knowledge Representation for AI Coding Agents

AI coding agents need more than relevant snippets: they need business semantics, validation evidence, relations, and assurance that their context is current. Existing systems usually infer or externalize this knowledge through retrieval, summaries, graphs, rules, or reverse specifications. We investigate a complementary representation in which selected code units directly carry agent-usable knowledge. We introduce Executable Code Knowledge (ECK) and define an Executable Code Knowledge Unit (ECKU) as a source-bound object combining stable identity, semantics, executable behavior, contracts, evidence, relations, provenance, validation state, and a query interface. Our Python prototype supports code-local authoring, manifest export, evidence execution, exact changed-line impact, freshness checking, and agent-facing projections. Across three real Python repositories and 26 controlled patch tasks, direct ECK provides executable test coverage for 11/11 evidence-bearing tasks and exact selectors for 9/11; hiding declared evidence reduces exact recovery to 1/11 (paired exact McNemar p=0.0078). ECK-derived rules recover 11/11 exact selectors, showing that rules are effective delivery artifacts while ECK supplies source binding, validation state, impact, and freshness. Exact changed-line impact matches independently authored labels on all 26 patches (12 unit links; precision, recall, and F1 all 1.000). AST-bounded fingerprints classify 50 positive changes and 17 unrelated same-file controls correctly, whereas static rules snapshots detect none of the 50 stale cases. Model-backed patch-review and cross-layer studies measure projection fidelity rather than independent impact discovery. These results support a hybrid architecture: retrieval for coverage, ECK for source and evidence governance, and projections for delivery.

Xueping Gao

Alibaba Cloud Hangzhou, China

Knowledge Representation AI Coding Agents

Ask PDF

Jul 30, 2026

Change2Task: From Repository Changes to Executable Coding Agent Tasks and Environments

Scaling coding agents requires a continuing supply of executable data for training, benchmarking, and continuous evaluation. Each task must couple a realistic software state with a specification, development tools, and reliable verification. To expand this supply, we present Change2Task, a system grounded in repository history that converts merged pull requests (PRs) into verified tasks on healthy modern revisions of the same repository. It aligns historical evidence with evolved code, reconstructs task states through Patch Reversal, Code Mapping, or Agent Reconstruction, and validates the lifecycle from a healthy base to a task state and a restored state. By deriving multiple tasks grounded in developer evidence from maintained environments, Change2Task provides executable data for coding agent training and evaluation while reducing repeated environment setup, storage, and task construction effort. We evaluate five coding agent task families: Bug Fix, Feature Addition, Test Generation, Application Programming Interface (API) Migration, and Security Repair. Starting from 1,130 source changes eligible for construction, Change2Task achieves 79.6% verified task construction success across these task families. On a matched candidate set, it recovers 29.2% more verified tasks than a construction baseline based on PRs. Historical and reconstructed cases achieve 89.7% aggregate matched outcome agreement under agent evaluation, while reuse of modern bases reduces measured expenditure across the complete pipeline by 10.8%. Four matched studies examine construction stages, semantic review, branch consistency, and resource trade-offs. An industrial deployment further constructs 1,000 tasks from private PR histories and evaluates four models on a fixed 500-task cohort.

Haomin Qi, Xingliang Wang, Xuanqi Gao +9

Microsoft · University of California San Diego · Zhejiang University +2

Code Generation Benchmarks Agent Benchmarks

Ask PDF

Aug 4, 2026

EA-Graph: Artifact-Anchored Verification Memory for Coding Agents under Upstream Drift

Coding agents increasingly work across sessions, but prose notes can preserve a conclusion without the program state that supported it. After an upstream change, a repository may still build even though earlier verification claims are no longer valid. EA-Graph is an artifact-anchored memory for verification claims. It represents artifacts at sub-path granularity, resolves aliases to leaf definitions, anchors each claim to the content used to establish it, and keeps evidence strength separate from freshness. When replacement content is unavailable, the claim becomes unprovable rather than guessed. EA-Graph is evaluated on generated repositories whose behavior-to-artifact ground truth is known by construction. The task is to classify prior claims as unaffected, affected, or unprovable after value drift, logic drift, and deliberately withheld upstream content. The analysis covers 42 sessions across seven clean worlds, 14 model-world instances, three memory conditions, and two model tiers. In the Haiku round, artifact-anchored memory outscored prose notes and no persistent memory in all seven worlds; each exact paired Wilcoxon comparison yielded p = 0.0156. In the Sonnet round, the anchored condition was perfect, but frequent control ceilings left the preregistered contrasts non-significant. No session fabricated withheld content. These results support a bounded claim: artifact-anchored memory improved the smaller model's provability judgments in this testbed. An exploratory comparison further suggests that structured claim memory may narrow a capability gap by externalizing in-session re-derivation, but it does not establish cross- model equivalence. The study makes no claim about efficiency or repair quality.

Hwai-Jung Hsu, Cheng-Jan Chi, Hanna Everett

Sustainable and Peer Education Center, National Yang Ming Chiao Tung University · Office of AI Affairs, National Yang Ming Chiao Tung University

Agent Memory Software Engineering Agents Memory Agent Benchmarks Agent Memory Management

Ask PDF

Abstract

Evidence Selection

Explore similar work

Executable Code Knowledge: Code as a Native, Validation-Carrying Knowledge Representation for AI Coding Agents

Change2Task: From Repository Changes to Executable Coding Agent Tasks and Environments

EA-Graph: Artifact-Anchored Verification Memory for Coding Agents under Upstream Drift