Does Failure Feedback Direct Agent Self-Modification?

Rong He

cs.SEOct 10, 2026

Does Failure Feedback Direct Agent Self-Modification?

Authors: Rong He

Abstract

Self-improving agents use failure feedback to revise their code or instructions, but gains after selection do not establish target-specific repair or generalization. We measure directedness by comparing repair of a randomly selected target failure with repair of the parent's other failures. Across 145 eligible modifications in public Darwin Gödel Machine logs, we find no detectable target-specific advantage. Controlled instruction-revision experiments with small open models show that relevant feedback can improve target repair. In a second study with a pre-specified analysis plan, target feedback raised repair by 20 percentage points for Qwen3-4B relative to matched same-category feedback; the 7-point increase for Phi-4-mini was not significant. Target repair did not reliably imply held-out improvement. Held-out gains were concentrated in settings with identifiable, repairable failure mechanisms, including planted errors and natural tool-interface failures. Benefits of edit localization and regression screening varied across settings, while eight-step revision loops and GEPA ended at similar mean held-out accuracy. Evaluations should report target repair, transfer to related failures, and held-out net change separately.

Figures & tables

Panel of Figure 1 — Figure 1: One self-modification and the quantities measured on it. A failure $e$ of the parent is drawn at random and shown to the editor; the child is evaluated on $e$ , on the parent’s other failures and on the parent’s correct answers. Directedness compares the target with the other failures of the same child; category-level spread compares other failures in the target’s category with failures elsewhere; net score change is failures repaired minus correct answers broken. The evaluator sees only the score.

Quantity	Measures	Does not establish
Directedness	repair of the failure shown, relative to the parent’s other failures	that the editor identified the cause of the failure
Category-level spread	repair of other failures in the category of the failure shown, relative to failures elsewhere	generalization across tasks or distributions
Net score change	failures repaired minus correct answers broken, on a fixed evaluation set	the long-run gain of a population after selection

Table 1: The three quantities measured on each revision.

Figure 2: Directedness of DGM self-modifications. (a) Repair rate of the target (blue; Clopper–Pearson binomial 95% interval, a reference that does not account for adaptive sampling) and mean repair rate of the parent’s other failures (gray; bootstrap over modifications), per run and pooled; 145 modifications. (b) Repair rate of the parent’s other failures in the target’s language (Polyglot) or repository (SWE-bench) minus the rate elsewhere (blue), against its randomization null (gray band, 95%). (c) Change in a human-named failure mode for working children evaluated on at least 50 tasks, by assigned goal (orange) or other entries (gray); bars are means. Panel (c) conditions on survival, which the goal can affect (Section 3.3 ).

	Working children		All attempts		Survival
Goal	Goal / other	Fisher $p$	Goal / other	Risk diff. ( $p$ )	Goal / other
Empty patches	8/8 / 22/36	0.03	8/25 / 22/101	$+10$ pts (0.21)	32% / 36%
Context length	6/6 / 26/82	0.002	6/34 / 26/206	$+3$ pts (0.36)	18% / 40%

Table 2: Human-named failure modes in the SWE-bench runs. Working children: children that worked and were evaluated on at least 50 tasks; counts are children that reduced the parent’s rate of the mode, with the goal against other entries (Fisher

p

; descriptive, conditions on survival). All attempts: attempts that produced a working child that reduced the mode, with the goal against other entries; risk difference stratified by eligibility, with permutation

p

. Survival: share of attempts that produced a working child evaluated on at least 50 tasks.

Figure 3: (a) Score change from parent to child on SWE-bench; bars are means; the shaded band is

\pm 2

standard deviations of the re-evaluation noise bound from the no-archive run. (b) Share of self-modification attempts that yield a working agent, in the first and second half of each run.

Figure 4: (a) Highest score in the archive during each SWE-bench run (thick) and the median and 5–95% band of 2,000 simulated runs of a zero-mean variation model calibrated to edit-level statistics of the same runs (thin). (b) Probability that the simulated no-archive run reaches no more than the observed 0.233, as a function of the assumed mean gain of an unselected edit.

Agent and editor	Qwen3-4B-Instruct-2507, 4-bit, MLX; the editor is the same model
Tasks	180 BIG-Bench Hard questions, 30 from each of six categories
Parents	clean playbook: 79.4%, 37 failures; planted-rule playbook: 53.3%, 84 failures (51 rule-caused, 27 also failed by the clean playbook, 6 other)
Evidence	instance : one failure drawn at random, with the agent’s response and the correct answer; named : one category named as weak; none
Children	clean parent: 30, 6 and 3 attempts (instance, named, none), of which 27, 6 and 3 viable; planted-rule parent: 28, 6 and 4, all viable
Scoring	every viable child and both parents on all 180 questions; greedy decoding, at most 512 tokens; one fixed order and batch layout; strict answer extraction (Appendix C )
Sensitivity	relaxed answer extraction; prefix continuation of truncated outputs (Section 4.4 )

Table 3: Design of the controlled experiment.

Figure 5: Controlled self-revision. (a) Repair rates in instance children, by the class of the failure shown: the failure shown, the parent’s other failures in the same category, and its failures in other categories; 95% intervals from resampling children. (b) Score change of each child against its parent on all 180 questions, by parent and evidence condition, in the 512-token evaluation and after prefix continuation of truncated outputs; bars are means, and the numbers give the children above their parent in the two analyses.

Parent / target	$M$	Target	Other	Diff. [95% CI]	$p_{\mathrm{dir}}$	Category [95% CI]	$p_{\mathrm{cat}}$
Clean / all (natural)	27	0.519	0.274	$+0.245$ [ $+0.06$ , $+0.43$ ]	0.007	$+0.036$ [ $-0.09$ , $+0.16$ ]	0.09
Planted / all	28	0.607	0.244	$+0.364$ [ $+0.19$ , $+0.53$ ]	$<10^{-4}$	$+0.211$ [ $+0.06$ , $+0.36$ ]	$<0.001$
Planted / rule-caused	18	0.500	0.226	$+0.274$ [ $+0.05$ , $+0.49$ ]	0.009	$+0.123$ [ $-0.14$ , $+0.39$ ]	0.042
Planted / natural	8	0.750	0.207	$+0.543$ [ $+0.16$ , $+0.83$ ]	0.002	$+0.211$ [ $-0.00$ , $+0.45$ ]	0.032

Table 4: Directedness and category-level spread in the controlled experiment (instance children; all 180 questions scored). “Other”: repair rate of the parent’s other failures of the same class. Diff.: mean paired difference with 95% interval from resampling children;

p_{\mathrm{dir}}

: exact one-sided directedness test with the null computed over failures of the class. Category: repair rate of other failures in the target’s category minus failures elsewhere, with 95% interval;

p_{\mathrm{cat}}

: randomization test of category-level spread. Two planted-rule targets that are neither rule-caused nor natural are included only in the “all” row.

Figure 6: Revisions of the planted rules. (a) Share of children of the planted-rule parent that neutralized each rule, by evidence condition and by whether the evidence came from the rule’s category; numbers are children. The rounding and number-words rules count as neutralized when the child repairs at least half of the failures the rule causes, the inert navigation rule when its text is removed. (b) One child per distinct rule-caused target under the original revision prompt and under a prompt that allows format rules to be changed, with matched seeds.

Models	Qwen3-4B-Instruct-2507 and Phi-4-mini-instruct (bf16, vLLM); each model edits itself
Setting A	6 BIG-Bench Hard categories; splits D 180, V 180, T 240 (held out); seven-module playbook with a fixed output protocol; parents: clean, and clean plus one planted rule (below)
Setting B	tool-using agent (Section 5.3 ): Python programs over an editable helper library; splits D 120, V 120, T 180 (held out); parents: clean, and three planted helper bugs
Factorial	feedback (failure shown; another failure of the same category; a failure of another category) $\times$ scope (whole rewrite; one module chosen with knowledge of the target: its category module in Setting A, or, in Setting B, the helper whose correct version changes the output on the target, an oracle localization); one child per cell and combination, same editor seed within a combination
Combinations	per parent: 40 (Qwen) and 41 (Phi) in Setting A, 60 in Setting B, by the pre-specified rule
Methods	whole rewrite, a random module other than the target’s category module, editor-localized module, category module, and whole or localized edits with regression screening on V (up to three attempts)
Loops	8 steps, 8 trajectories per parent and loop (Qwen, Setting A); GEPA with a matched budget

Table 5: Design of study 2. Analysis plan and amendments: Appendix F .

	Qwen3-4B (A, $n=80$ )	Phi-4-mini (A, $n=82$ )	Tool agent (B, $n=120$ )
H1	$+20.0$ [ $+8.5$ , $+31.8$ ], 0.003	$+6.7$ [ $-0.6$ , $+14.0$ ], 0.17	$-0.8$ [ $-5.3$ , $+3.6$ ], 1.0
H2	$+0.6$ [ $+0.1$ , $+1.0$ ], 0.014	$-2.4$ [ $-4.2$ , $-0.6$ ], 0.99	$-4.5$ [ $-5.4$ , $-3.7$ ], 1.0
H3	$+0.4$ [ $-1.1$ , $+1.8$ ], 0.63	$-1.9$ [ $-4.2$ , $+0.4$ ], 0.24	$+0.5$ [ $-1.3$ , $+2.3$ ], 1.0
Companion	$+0.5$ [ $-0.3$ , $+1.4$ ]	$-2.5$ [ $-4.3$ , $-0.8$ ]	$+6.8$ [ $+4.6$ , $+8.9$ ]

Table 6: Pre-specified hypotheses of study 2: estimate [95% interval], Holm-adjusted

p

over H1–H3 within a setting. H1: target repair, failure shown minus a matched failure of the same category (points). H2: breakage of correct held-out answers, whole rewrite minus one module. H3: interaction of feedback and scope on net change on the held-out test split. Companion: net change, one module minus whole rewrite, with the failure shown. Paired within combination; intervals from resampling (parent, target) clusters.

Figure 7: Study 2 factorial, by setting and condition (filled: whole rewrite; open: one module chosen with knowledge of the target, an oracle localization in the tool agent; colors: feedback). (a) Repair of the target. (b) Net change on the held-out test split. (c) Repair of held-out items caused by the planted rule or bug, for the planted-cause parent. Means over combinations with 95% intervals from resampling combinations.

Figure 8: (a) Net change on the held-out test split of the agent each single-step method returns, both models, Setting A; 95% intervals from resampling combinations. (b) Final held-out accuracy minus the parent’s after eight-step loops (whole rewrite or localized edit, keeping a child when its V score does not drop) and after GEPA with a matched budget; one point per trajectory, bars are means.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Run	$M$	Repaired	Expected	$P$ (target)	$P$ (other)	OR MH	$p$
SWE-bench, DGM	23	0	2.8	0.000	0.125	0.00	1.00
SWE-bench, fixed editor	15	2	1.4	0.133	0.090	1.60	0.40
SWE-bench, no archive	19	1	1.2	0.053	0.061	0.85	0.70
Polyglot, DGM	53	6	6.9	0.113	0.132	0.84	0.72
Polyglot, fixed editor	16	1	1.8	0.062	0.118	0.51	0.87
Polyglot, no archive	19	2	2.5	0.105	0.133	0.77	0.74

Appendix

Table 7: Task-level directedness in the DGM logs, by run.

M

: modifications whose target was evaluated on the child. “Expected”:

\sum_{m}q_{m}

p

: one-sided Poisson–binomial reference probability of at least the observed count; the adaptive tests are reported in Section 3.2 .

Feedback, scope	Target	Break	Net	Target	Break	Net	Target	Break	Net
	Qwen3-4B (A)			Phi-4-mini (A)			Tool agent (B)
Shown, whole	0.60	6.4	$+1.3$	0.28	12.8	$+2.1$	0.15	0.3	$+4.4$
Same category, whole	0.47	6.0	$+1.4$	0.18	14.4	$+1.2$	0.18	0.4	$+4.9$
Other category, whole	0.26	5.9	$+1.0$	0.26	12.9	$+2.2$	0.06	0.3	$+5.1$
Shown, one module	0.69	5.9	$+1.8$	0.35	16.7	$-0.4$	0.56	4.7	$+11.2$
Same category, one module	0.41	5.4	$+1.6$	0.32	15.3	$+0.6$	0.54	5.2	$+11.1$

Appendix

Table 8: Study 2 factorial, by condition: target repair, breakage of the parent’s correct held-out answers (%), and net change on the held-out test split (points), averaged over both parents.

Self-Improving Agents

Self-improving agents use experience and feedback to update their models, memories, skills, or workflows for better future task performance.

See all 160 papers

Structured Feedback Improves Repair in an LLM Agent Loop
Jul 15, 2026Jaideep Ray, Ankit GoyalLLM Agent Workflow Optimization LLM Agent Evaluation
Independent Researcher
Learning from Failure: Inference-Time Self-Improvement for Computer-Use Agents
Jun 30, 2026Xueqiao Sun, Xiaohan Wang, Ludwig Schmidt +2Computer-Use Agents Self-Improving Agents
1Stanford University, Stanford, CA 94305, USA
Try Again, Don't Look Back: Blind Resampling Outperforms Self-Repair in Small Code Models
Jul 28, 2026Yuvraj VermaLLM Self-Correction Code Generation
Independent Researcher, India.

Jul 15, 2026J/K move · Enter open · S save

Structured Feedback Improves Repair in an LLM Agent Loop

Jaideep Ray, Ankit Goyal

Independent Researcher

LLM Agent Workflow Optimization LLM Agent Evaluation LLM Agent Reliability LLM Self-Correction LLM Agent Verification LLM Agent Harnesses+1

Open paper AskPDF arXiv

LLM agents often retry after external validation rejects a candidate, but the interface between validation and the next model call remains underspecified. We introduce VeriHarness, a code-controlled agent loop in which models generate candidates while external validators control acceptance, budgets, and traces. We use it to compare raw diagnostics with feedback that identifies the failure location, observed value, and admissible alternatives. Across 50 paired TextWorld games under a four-call cap, feedback containing all three fields raises terminal success from 14/50 to 36/50 for Qwen2.5-Coder-14B (+44 percentage points) and from 8/50 to 29/50 for Llama-3.1-8B (+42 points). Ablations locate most of the gain in the admissible alternatives: feedback containing only the location and observed value remains near the raw diagnostic baseline. Presenting the complete repair information in prose instead of a keyed JSON record yields nearly the same success, providing no evidence that JSON syntax itself improves repair. The ordering persists across the tested call budgets and one sampled-decoding setting.

Abstract

Self-Improving Agents

Explore similar work

Structured Feedback Improves Repair in an LLM Agent Loop

Learning from Failure: Inference-Time Self-Improvement for Computer-Use Agents

Try Again, Don't Look Back: Blind Resampling Outperforms Self-Repair in Small Code Models