Organizations: National University of Singapore, Singapore · Shanghai Artificial Intelligence Laboratory, Shanghai, China · Tsinghua University, Beijing, China · Independent Researchers
Long-horizon coding agents can observe a failure and generate a plausible critique yet continue along the same incorrect trajectory. We call this the Critique-to-Action Handoff Gap. We introduce RIPPLE, a recovery-oriented training framework that internalizes process-level guidance into a single self-recovering policy. A prefix-only teacher provides evidence-grounded recovery supervision without access to future actions or terminal outcomes. After lightweight distillation, reinforcement learning assigns stage-specific credit to reflection triggering, diagnosis, and corrective action. Local validity signals gate trajectory advantages so that intermediate recovery decisions do not indiscriminately inherit terminal outcomes. Across four repository-level software-engineering benchmarks, RIPPLE improves the base policy by an arithmetic mean of 4.8 score points, with no auxiliary critic at deployment. Controlled ablations isolate the benefits of local, stage-aligned credit, and matched-prefix interventions show that critique semantics affect subsequent correction. Recovery gains concentrate where the prefix contains sufficient diagnostic evidence and the repository state remains reversible within the remaining budget.
Figures & tables
Figure 1: The critique-to-action handoff gap. Failure feedback may elicit an evidence-grounded critique without changing subsequent behavior (left). RIPPLE learns a recovery policy over reflection triggering, diagnosis, and corrective action (right).
Figure 2: Overview of RIPPLE . A prefix-only recovery teacher supports policy initialization and process evaluation. The policy is then optimized through recovery-oriented reinforcement learning, which assigns stage-specific process credit to trigger, diagnosis, and corrective-action decisions, with the executable verifier as the sole source of task reward.
Table 1: End-to-end results and deployment cost. The first three benchmarks use Resolved@1; SWE-Compass uses its official task-aligned macro average. Actor tokens and p95 latency are realized quantities normalized to the base policy. Values are averages over three training seeds for trained methods and three matched decoding runs for fixed methods.
Figure 3: Locality and stage alignment contribute separately. Seven credit-assignment variants use identical rollouts and process labels. Bars report Pro Resolved@1 and Full-Loop Recovery; the dashed line reports positive-credit contamination (lower is better). Event and Stage/event denote event-local generic and stage-aware event-wide credit; Random and Shuffle are sparsity controls.
Method
Trig. F1
Diag.
Align.
Func.
Full
Periodic reflection
0.43
55.8
48.1
42.5
31.4
Recovery-SFT
0.56
64.9
54.8
49.4
38.1
Outcome-only
0.60
66.9
58.6
53.0
41.6
ICRL-style
0.65
73.2
64.4
60.5
50.8
RIPPLE
0.79
81.5
74.8
70.2
59.6
Table 2: Stage-wise recovery quality on the fixed Recovery Challenge Set, scored by an independent evaluator and real test transitions. Columns are trigger F1, Diagnosis Accuracy, Action Alignment, Functional Recovery, and Full-Loop Recovery; the last four are percentages.
Figure 4: Matched-prefix critique intervention. Holding policy, prefix, repository snapshot, remaining budget, and paired seeds fixed, the left panel reports K -step recovery and the right panel pass-to-fail regression for four critique conditions. The controlled contrasts show that a full critique outperforms both removal and a length- and style-matched generic critique, whereas a wrong-direction critique is worse than removal.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Metric
Definition
Trigger P/R/F1
Emit <recover> within w=4 tool turns of an annotated boundary. Non-deviating prefixes contribute false positives.
Diagnosis Accuracy
Root cause and invalidated assumption agree with the independent evaluator; cited evidence occurs in the prefix.
Action Alignment
Actions follow the diagnosed direction within K=8 turns.
Functional Recovery
Real execution gives ∣F2P∣>0 and ∣F2P∣−∣P2F∣>0 .
Full-Loop Recovery
Trigger, diagnosis, alignment, and functional recovery all succeed.
Positive-credit contamination
Fraction of locally invalid blocks receiving positive advantage; lower is better.
Appendix
Table B.1: Process metrics. Trigger scores range from 0 to 1; all other metrics are percentages. Grounded Evidence is auxiliary because prefix-present evidence can still support an incorrect diagnosis.
Setting
Value
Actor / scaffold
Qwen3.5-27B / common edit–execute loop
Tool-turn cap
150
Trigger tolerance
w=4 turns
Action horizon
K=8 turns
Seeds
3
Initialization
Shared Recovery-SFT checkpoint
Appendix
Table C.1: Implementation settings relevant to comparison validity and deployment cost. Optimizer-level hyperparameters are shared across the controlled credit-assignment variants.
Figure D.1: Pro resolution by static-difficulty quartile. Gains increase from 1.8 points (Easy) to 11.9 (Extreme). Difficulty is defined without evaluated-policy trajectory statistics.
Figure D.2: Recoverable-deviation prevalence and benchmark gain. The reported r=0.68 association is computed over repository clusters, not these four aggregate points.
Slice (weight)
Base
RIPPLE
Gain
SWE-Compass
Bug fixing (1/8)
23.4
32.2
+8.8
Feature implementation (1/8)
25.8
33.4
+7.6
Other task types (6/8)
24.2
30.0
+5.8
Multilingual
C/C++/Go/Rust (4/9)
61.4
69.4
+8.0
Appendix
Table D.2: Task, language, and distribution-shift slices. Weighted rows reproduce their benchmark aggregates up to rounding. Pro’s metadata-shift score is fixed before inference and split at its median.
Method
Repair
Damage
Net
Longer single trajectory
4.4
3.5
+0.9
Two restarts
6.0
3.6
+2.4
Prompted periodic reflection
6.6
4.8
+1.8
Recovery-SFT
6.2
3.2
+3.0
Outcome-only GRPO
6.6
2.9
+3.7
Pivotal-aware credit
7.2
2.7
+4.5
Appendix
Table D.3: Paired repair-versus-damage decomposition on Pro. Every row uses all Pro instances as the denominator, so Repair minus Damage exactly equals the Resolved@1 gain over the base policy. RIPPLE achieves both the highest repair rate and the lowest damage rate among the trained comparisons.
Figure E.1: Leave-one-stage-out end-to-end effects. Removing corrective-action credit produces the largest drop in Pro and Full-Loop Recovery.
Figure E.2: Stage-wise ablation. Trigger credit primarily affects initiation; diagnosis credit affects diagnosis and downstream action; removing action credit leaves upstream stages largely unchanged. Trigger F1 is shown on a 0–1 scale; other cells are percentages.
Figure F.1: Matched-prefix recovery–regression frontier. Better outcomes move toward the upper-left. Direction alone retains much of the benefit; delaying the critique forfeits a substantial share.
Condition
Recovery
P2F
Full critique
62.9
2.4
Remove critique
39.8
6.8
Generic critique
42.0
6.1
Wrong-direction critique
25.6
18.9
Direction only
55.4
4.1
Evidence only
44.7
5.7
Appendix
Table F.1: Extended matched-prefix interventions (%). All conditions share the policy, frozen prefix, repository snapshot, remaining budget, and paired seed. The internal critique is within 3.6 recovery points and 0.3 P2F points of the teacher reference.