GUARD: Stabilizing Recursive Self-Improvement with Regret Guidance
Organizations: Google Cloud
Abstract
Autonomous AI agents pair a frozen base model with an editable harness of prompts, skills, tool rules, and memory. Recursive self-improvement (RSI) adapts this harness from experience without weight updates, yet self-editing without external graders often backfires: reflection and scalar scores conflate what an edit newly broke with what remains unsolved, causing local regressions, skill forgetting, and memory bloat. We introduce GUARD (Grader-free Unsupervised Environment Design [UED] with Adaptive Rule Distillation), a closed-loop framework for reference-free continual harness evolution. Our key insight is to decompose each task into pre-execution self-checks and guide harness updates via directional regret. Each round grades trajectories against these checks, pinning mastered rules while prioritizing newly broken behaviors, and selects among parallel candidate edits by critic-free group-relative fitness. A Soft-HURT gate then promotes an edit only when gains outweigh regressions, scoping specialized fixes to prevent over-triggering and guaranteeing non-regression on its gated buffer. At test time, one frozen harness runs in a single pass. Across benchmarks on memory, skill evolution, continual task streams, and domain reasoning, GUARD consistently improves performance without catastrophic forgetting or memory pollution while substantially cutting deployment token usage.
Figures & tables
| Arm | Task | Proc. | Mech. | Upd. | W.Prec. | Recall | Ret. | Poll. | Evolve |
|---|---|---|---|---|---|---|---|---|---|
| No-Evolve ( ) | 0.372 | 0.427 | 0.454 | 0.278 | 0.000 | 0.000 | 0.990 | 0.000 | 0.000 |
| Hermes+ [ 50 ] | 0.852 | 0.808 | 0.840 | 0.791 | 0.724 | 0.854 | 1.000 | 0.167 | 0.421 |
| RRSI [ 45 ] | 0.809 | 0.714 | 0.725 | 0.901 | 0.438 | 0.811 | 0.974 | 0.500 | 0.350 |
| GEPA [ 1 ] | 0.858 | 0.815 | 0.837 | 0.777 | 0.719 | 0.858 | 0.996 | 0.167 | 0.424 |
| Meta-Harness [ 20 ] | 0.812 | 0.677 | 0.739 | 0.903 | 0.479 | 0.814 | 0.998 | 0.500 | 0.360 |
| GUARD (Ours) | 0.854 | 0.830 | 0.878 | 0.924 | 0.776 | 0.856 | 0.999 | 0.167 | 0.436 |
| Method (Unified Gemini 4 ) | Adaptation Family | ACC | Solved | Cohort v1 / v2 / v3 ACC | BWT Score | BWT Pass | FWT | Tokens / Task | Calls | Latency |
|---|---|---|---|---|---|---|---|---|---|---|
| No-Evolve ( , Control) | Static ReAct Baseline | 71.91% | 48.84% | 70.46% / 79.17% / 64.29% | – | 131,032 | 14.47 | 221.6s | ||
| MemToolAgent [ 17 ] | Memory (#1 family) | 68.00% | 51.16% | 61.95% / 84.17% = / 71.43% | 82,426 | 15.84 | 239.2s | |||
| RRSI [ 45 ] | Reflective Harness Rewrite ( ) | 74.28% | 51.16% | 75.06% / 73.17% / 71.43% | 84,519 | 13.19 | 613.1s | |||
| GEPA [ 1 ] | Prompt (#1 family) | 70.69% | 48.84% | 68.45% / 71.17% / 85.71% | +13.61pp | 85,156 | 17.05 | 159.1s | ||
| Meta-Harness [ 20 ] | Agent Code (#1 family) | 74.52% | 51.16% | 73.42% / 81.83% / 64.29% | +5.83pp | +5.13pp | 78,705 | 14.19 | 220.0s | |
| GUARD (Ours) | Regret-Guided Harness RSI | 80.80% | 58.14% | 78.47% / 84.17% = / 89.29% | 59,651 ( ) | 11.21 ( ) | 233.8s |
| Four-Stream Macro-Average | Per-Stream Final Mean ( ) | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Method (Single-Model Gemini 4 ) | Final Mean | Plast. | Forgetting | Stab. | Block Drops | Workflow | Topic | Domain | Compound | ||
| No-Evolve ( , Control) | 69.92% | 0 / 12 | 65.00% ( ) | 67.78% ( ) | 76.05% ( ) | 70.84% ( ) | |||||
| Hermes+ / MemToolAgent | 70.31% | 5 / 12 | 65.00% ( ) | 70.00% = ( ) | 76.26% ( ) | 70.00% ( ) | |||||
| RRSI [ 45 ] | 67.92% | 8 / 12 | 60.00% ( ) | 66.67% ( ) | 79.17% ( ) | 65.84% ( ) | |||||
| GEPA [ 1 ] | 72.40% | 3 / 12 | 70.00% = ( ) | 67.78% ( ) | 79.31% ( ) | 72.50% ( ) | |||||
| Meta-Harness [ 20 ] | 71.21% | 4 / 12 | 63.33% ( ) | 68.89% ( ) | 79.29% ( ) | 73.33% ( ) | |||||
| Calibration / Gate Check (Sample ) | Ref. / 1-L | GUARD |
|---|---|---|
| Initial vs. final hosted ACC / Solved ( ) | ||
| Official ACC: L1-fail vs. L1-pass ( test) | ( NPV) | |
| PAST playbook rank concord. ( pairs) | ( strict) | |
| Exec.-gated promotions † | — | ( , ties) |
| Same gate w/o Analyze ( local stages) | prom., | prom., |
| Bypassing Soft-HURT on Calendar v3 | Solv. | (rejected) |
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
| Framework / Environment | Optimization Targets | Closed-Loop Features | Scalability | System Design & Update Mechanism | ||||
|---|---|---|---|---|---|---|---|---|
| Weight Updates | Harness & Skills | Text Memory | Regret Ranking | Annealed Budget | Non-Regr. Gate | (Work / Depth; -Ctx.) | ||
| Reflexion / ExpeL / AWM [ 38 , 59 , 43 ] | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | / ; mem. | Episodic verbal buffers, insight extraction, & workflow memory |
| Voyager / MemGPT / WikiSkill [ 42 , 31 , 40 ] | ✗ | ✓ | ✓ | ✗ | ✗ | ✗ | / ; lib. | Skill libraries & append-only wikis; skill edits gated on validation score |
| Dynamic Cheatsheet / ACE / ReasoningBank [ 39 , 56 , 30 ] | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | / ; ctx. | Test-time cheatsheets, incremental context updates, & trace memory |
| DSPy / TextGrad / EvoPrompt [ 18 , 53 , 11 ] | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | / ; ctx. | Scalar-objective, textual-gradient, or evolutionary prompt search |
| TextReg [ 10 ] | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | / ; bank | Regularized text updates filtering low-recurrence patches & length (requires GT) |
| Arm | Cands./Rd. | Adapt. Rollouts | Selection & Gate Rule | Adaptation Feedback Signal | Hosted Chars |
|---|---|---|---|---|---|
| No-Evolve ( ) | Static base harness | None (zero-shot ReAct / base harness) | |||
| Hermes+ / MemToolAgent | online | Unconditional append/edit | Agent-initiated memory & skill writes | ||
| RRSI [ 45 ] | online / stage | Unconditional rewrite (no val. split) | Self-critic Failure Mode Diagnosis + pruner | ||
| GEPA [ 1 ] | (exec.) / (critic) | Per-instance Pareto frontier argmax | Reflective trace critique on non-dominated tasks | ||
| Meta-Harness [ 20 ] | (exec.) / (critic) | Best self-scored candidate (ungated) | Proposer–judge search over raw trace archive | ||
| GUARD (Ours) | (exec.) / (critic) | + Soft-HURT gate | 2-layer checklist self-review (exec. or critic), , pinning, pool |
| Arm | Proc. Task | Proc. Mech. | Proc. Upd. | Proc. Poll. |
|---|---|---|---|---|
| No-Evolve ( ) | 0.427 | 0.443 | 0.214 | 0.000 |
| Hermes+ | 0.808 | 0.780 | 0.529 | 0.000 |
| RRSI [ 45 ] | 0.714 | 0.508 | 0.841 | 1.000 |
| GEPA [ 1 ] | 0.815 | 0.761 | 0.464 | 0.000 |
| Meta-Harness [ 20 ] | 0.677 | 0.516 | 0.824 | 1.000 |
| GUARD (Ours) | 0.830 | 0.900 | 0.901 | 0.000 |
| Comparison | Task | Mech. | Upd. | W.Prec. | Poll. |
|---|---|---|---|---|---|
| vs. Hermes+ | (4/5/3) | (5/5/2) | (6/5/1) | (1/11/0) | (0/12/0) |
| vs. RRSI | (6/5/1) | (7/5/0) | (6/5/1) | (6/6/0) | (4/8/0) |
| vs. GEPA | (1/6/5) | (4/6/2) | (4/6/2) | (3/9/0) | (0/12/0) |
| vs. Meta-Harness | (5/5/2) | (6/5/1) | (5/5/2) | (5/7/0) | (4/8/0) |
| Method | Adaptation Rollouts (Total) | Sequential Rollout Depth | Proposer Prefill Context (Round ) | Test-Time Rollouts & Active Context |
| No-Evolve ( ) | — | rollout; ctx. | ||
| Hermes+ / MemToolAgent | accumulated mem./skills | rollout; ctx. | ||
| RRSI ( ) | rollout; ctx. | |||
| GEPA ( cands.) | Pareto | rollout; ctx. | ||
| Meta-Harness ( cands.) | raw trace archive | rollout; ctx. | ||
| GUARD (Execution scoring) | ( ) | ( -way par.) | rollout; ctx. |
| Arm | Experience source and optimizer |
|---|---|
| No-Evolve ( ) | Base agent executing without a learned playbook. |
| RRSI [ 45 ] | Multi-round failure-driven reflective playbook rewrite on TRAIN self-reviews. |
| Meta-Harness [ 20 ] | Counterfactual diagnostic harness search over full TRAIN execution histories. |
| GUARD (Ours) | Paired priority , rule pinning, [iff] guards, prior-free contrastive prompt evolution, and Soft-HURT gating (§ 4 ). |
| Experiment (rounds; splits) | Arm | Analyze stage | Propose stage | Gate rule | Reported in |
|---|---|---|---|---|---|
| PRBench-Legal & Harvey LAB and Fin-Rate & Ledger (3; 24 train, 16 frozen DEV 2) | No-Evolve ( ) / RRSI / Meta-Harness | none / TRAIN self-review / trace history | – / reflective rewrite / harness search | – / rewrite / trace search | Tab. C.3 , Fig. C.1 |
| GUARD (Ours) | paired , rule pinning, contrasts | pool, [iff] guards, evolution | TRAIN Soft-HURT | Tab. C.3 , Fig. C.1 | |
| EvoHarnessBench 5-domain Analyze / Gate ablation (16 stages; 97 source, 236 test) | No-Evolve ( ) / w/o Analyze / w/o Gate | – / (no pinning or contrasts) / full Analyze | – / pool ( ) / pool ( ) | – / Soft-HURT / unconditional | Tab. C.7 |
| GUARD (Ours) | paired , rule pinning, contrasts | pool, [iff] guards, evolution | Soft-HURT | Tab. C.7 |
| Suite (24 TRAIN 16 Frozen DEV ) | Arm | R0 | R1 | R2 | R3 (Final) | Official vs. [95% CI] | Final Uncapped | Uncapped | Hurt | Sub-domain Breakdown (Official) |
| PRBench-Legal & Harvey LAB | No-Evolve ( ) | 59.48% | 59.48% | 59.48% | 59.48% | — | 59.77% | — | — | Finance ( ): 48.05% Legal: 60.24% |
| ( gemini-3.8-flash agent, | RRSI | 59.48% | 64.91% | 65.42% | 63.30% | 63.59% | 0 | Finance ( ): 72.08% Legal: 62.72% | ||
| claude-opus-5-5 distiller/judge) | Meta-Harness | 59.48% | 63.81% | 65.49% | 66.11% | 66.40% | 0 | Finance ( ): 76.62% Legal: 65.41% | ||
| GUARD (Ours) | 59.48% | 66.67% | 66.67% | 66.67% | 66.96% | 0 | Finance ( ): 76.62% Legal: 66.01% | |||
| Fin-Rate & Ledger Suite | No-Evolve ( ) | 37.50% | 37.50% | 37.50% | 37.50% | — | 70.58% | — | — | fin_rate : 12.50% ledger : 62.50% |
| ( gemini-3.8-flash agent, | RRSI | 37.50% | 46.88% | 40.62% | 40.62% | 71.50% | 1 | fin_rate : 18.75% ledger : 62.50% |
| Continual Drift Stream ( blocks) | Arm | Final Mean | Plast. | Interf. | Forget. | Stab. | Final Group Vector (%) | ||
|---|---|---|---|---|---|---|---|---|---|
| Workflow Drift ( ) | No-Evolve ( ) | 65.00% | |||||||
| BFCL-Miss BFCL-Base | Hermes+ / MemToolAgent | 65.00% | |||||||
| RRSI [ 45 ] | 60.00% | ||||||||
| GEPA [ 1 ] | 70.00% | ||||||||
| Meta-Harness [ 20 ] | 63.33% | ||||||||
| GUARD (Ours) | 70.00% |
| Domain Track | Models | Stages (Test/Train) | (ACC / Solved) | GUARD (ACC / Solved) | Stage Highlights ( ) | BWT (Score / Pass) | FWT |
|---|---|---|---|---|---|---|---|
| Calendar (Tools, Hosted SDK) | Gemini 4 | v1 – v3 (43 / 18) | 71.91% / 48.84% | 80.80% / 58.14% | Cohort v1 ( ): ( ); v3 : | / | (Abstained v2 – v3 ) |
| Calendar (Tools, Local Gym) | flash+opus | v1 – v3 (43 / 18) | 77.15% / 48.84% | 77.15% / 48.84% | Stages v1 / v2 / v3 : ACC (75% Pass) | / | ( ) |
| Drive (Tools, Local Gym) | flash+opus | v1 – v3 (45 / 19) | 71.78% / 46.67% | 71.78% / 46.67% | Stages v1 / v2 / v3 : ACC | / | ( ) |
| CSM (Skills, Local Gym) | flash+opus | v1 – v3 (36 / 14) | 59.80% / 30.56% | 61.40% / 25.00% | Stage v1 : ( ); v2 : ( ) | / | Mean ( v1 ) |
| HR (Skills, Local Gym) | flash+opus | v1 – v3 (53 / 22) | 61.67% / 32.08% | 61.67% / 32.08% | Stage v1 : ( ); v2 / v3 : | / | Mean ( v1 ) |
| ITSM (Skills, Local Gym) | flash+opus | v1 – v4 (59 / 24) | 58.65% / 30.51% | 60.28% / 38.98% | Stage v2 : ; v4 : ( Pass) | / | Mean ( v2 ) |
| Calibration / Gate Dimension | Benchmark & Sample ( ) | Unguarded / Ref. | GUARD (Ours) |
|---|---|---|---|
| False-pos. pass ( ) on Layer-1-failed traces ⋆ | EvoHarnessBench Gemini 4 ( vs. L1-fail) | (1-layer) | ( , ) |
| Initial (1-layer) vs. final hosted configuration ACC / Solved | EvoHarnessBench Calendar ( test) | ||
| Official verifier ACC (Layer-1-only fail vs. pass) | EvoHarnessBench pooled ( test: hosted, local) | (L1 fail) | ( , NPV) |
| Episode semantic bias ( ) & corr. | PAST-Bench ( learn episodes) | (official ) | ( ) |
| Within-family playbook ranking concordance | PAST-Bench ( playbooks, pairs) | (random) | ( non-tied; ) |
| Exec.-gated promotions ( ) † | Drift Gauntlet ( ) + EvoHarness ( ) + prof. ( ) | — | ( , ties) |
| Arm | Ablated Stage & Mechanism | Macro ACC | vs. GUARD | Macro Solved | Mean FWT | Regressed Domains | Promoted / Rejected |
|---|---|---|---|---|---|---|---|
| No-Evolve ( , Control) | Static base harness (no adaptation) | 65.81% | 37.73% | 0 / 5 | 0 / 16 | ||
| GUARD (Ours) | Full harness ( Analyze + Propose + Soft-HURT Gate ) | 66.46% | — | 38.31% | +1.00pp | 0 / 5 | 4 / 12 |
| GUARD w/o Analyze | Ablates Analyze ( , no pinning or contrasts) | 59.13% | 32.92% | 2 / 5 | 4 / 12 | ||
| GUARD w/o Soft-HURT Gate | Ablates Gate (promotes within-group winner unconditionally) | 56.11% | 28.27% | 3 / 5 | 16 / 0 |
| Arm ( ITSM , Unified Gemini 4 ) | Propose -Stage Configuration | Skills ACC | Tools ACC | Task ACC (Macro) | vs. ( ) | Macro Solved | Macro FWT / Task BWT | Prom. / Rej. (Ledger) |
|---|---|---|---|---|---|---|---|---|
| No-Evolve ( , Control) | Static base harness ( ) | 80.70% | 66.08% | 72.61% (73.39%) | — | 51.36% | / | 0 / 7 (0) |
| GUARD w/o Pool ( ) | Single candidate ( ) | 82.46% | 72.95% | 77.20% (77.70%) | ( ) | 53.25% | / | 1 / 6 (6) |
| GUARD Sequential ( seq.) | sequential rounds/stage ( ) | 81.96% | 76.34% | 78.85% (79.15%) | ( ) | 56.09% | / | 2 / 5 (11) |
| GUARD (Ours) | Parallel pool + ( ) | 82.93% | 79.59% | 81.08% † (81.26%) | ( ) | 56.61% | / | 3 / 4 (4) |
Self-Improving Agents
Self-improving agents use experience and feedback to update their models, memories, skills, or workflows for better future task performance.
See all 203 papers