Self-Improvement Memory Enables Continual Recursive Self-Improvement
Organizations: Google Cloud AI · The Pennsylvania State University · Google DeepMind · Google Cloud AI Research
Abstract
Harness optimization is often studied with a fixed policy model and benchmark. When either changes, the optimizer must decide which parts of an evolved harness to reuse or revise, but the harness alone may not explain why earlier edits were made or how they performed. In this work, we study Continual Recursive Self-Improvement (CRSI), a setting for continuing existing harness-optimization methods across compatible benchmarks or policy models. We propose Self-Improvement Memory (SIM) for the harness optimizer, rather than the task-solving policy. SIM groups edits by recurring harness problem and records their rationale, predicted and observed outcomes, and evaluation context. New attempts extend this history as optimization continues. In our experiments across four domains, SIM achieves higher final scores than harness-only and root-cause-only continuation in cross-dataset and pairwise cross-model comparisons. Across ten independent source runs in coding, paired continuations with SIM improve final scores by an average of 2.36 points over harness-only continuation ().
Figures & tables
| Gemini model | Seed | Fresh RSI | CRSI |
|---|---|---|---|
| 2.5 Flash | 17.8 | 21.4 | 22.4 |
| 3.5 Flash | 70.2 | 73.3 | 76.8 |
| 3.8 Flash | 87.2 | 88.9 | 90.5 |
| 4 Argon | 89.7 | 95.6 | 94.8 |
| Harness construction | Evolve | Held-out | All |
|---|---|---|---|
| B1: Per-model harnesses | 77.40 | 75.20 | 76.30 |
| B2: Shared seed | 72.36 | 73.28 | 72.82 |
| S1: Sequential CRSI | 76.72 | 74.20 | 75.46 |
| S2: Unified synthesis | 78.92 | 75.24 | 77.08 |
| S3: Source policy used for RSI | |||
| Gemini 3.8 Flash | 74.28 | 71.24 | 72.76 |
| Run | Start | No SIM | SIM | |
|---|---|---|---|---|
| 1 | 18.18 | 20.61 | 22.42 | |
| 2 | 16.97 | 17.58 | 19.39 | |
| 3 | 17.58 | 20.00 | 20.61 | |
| 4 | 18.18 | 18.18 | 23.64 | |
| 5 | 17.58 | 18.79 | 20.61 | |
| 6 | 19.39 | 20.00 | 23.64 |
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| Domain | Benchmark | Total | Evolve tasks / rollouts | Held-out tasks / rollouts | Metric |
| Coding | S: TB2.1 | 89 | 44 / 220 | 45 / 225 | pass@1 |
| T: TB4 | 66 | 33 / 165 | 33 / 165 | pass@1 | |
| Text cls. | S: 3 datasets | 412 | 206 / 1,030 | 206 / 1,030 | accuracy |
| T: 8 datasets | 1,000 | 500 / 2,500 | 500 / 2,500 | accuracy | |
| Workspace | S: Harvey LAB | 160 | 80 / 400 | 80 / 400 | rubric |
| T: JobBench | 65 | 32 / 160 | 33 / 165 | rubric |
| Condition | Harness | SIM during target RSI |
|---|---|---|
| Cold Start | None | |
| Harness Transfer | None | |
| Problem-Only | Root causes only | |
| Full SIM | Full record; updated on target |
| Cold Start | Harness Transfer | Problem-Only Memory | Full SIM (ours) | |||||
|---|---|---|---|---|---|---|---|---|
| Domain / seed | Before | After | Before | After | Before | After | Before | After |
| Coding / AHE | 16.36 | 19.39 | 18.18 | 20.61 | 18.18 | 18.79 | 18.18 | 24.85 |
| Coding / Terminus-2 | 18.18 | 19.39 | 19.39 | 19.39 | 19.39 | 20.61 | 19.39 | 21.82 |
| Text cls. / NoMemory | 68.60 | 72.00 | 71.40 | 71.00 | 71.40 | 73.40 | 71.40 | 73.60 |
| Text cls. / FewShotAll | 75.72 | 75.92 | 72.92 | 74.12 | 72.92 | 75.52 | 72.92 | 77.92 |
| Workspace / ReAct | 73.89 | 82.98 | 74.93 | 85.07 | 74.93 | 85.54 | 74.93 | 87.89 |
| Cold Start | Harness Transfer | Problem-Only Memory | Full SIM (ours) | |||||
|---|---|---|---|---|---|---|---|---|
| Domain / seed | Before | After | Before | After | Before | After | Before | After |
| Coding / AHE | 16.36 | 19.39 | 13.94 | 18.18 | 13.94 | 20.61 | 13.94 | 21.82 |
| Text cls. / FewShotAll | 78.32 | 82.72 | 77.72 | 80.20 | 77.72 | 81.32 | 77.72 | 83.80 |
| Workspace / ReAct | 73.89 | 82.98 | 68.41 | 73.42 | 68.41 | 79.22 | 68.41 | 84.75 |
| Engineering / ReAct | 39.13 | 40.87 | 34.78 | 42.61 | 34.78 | 46.96 | 34.78 | 50.43 |
| Gemini 3.8 Flash | Claude Opus 5.5 | |||||
|---|---|---|---|---|---|---|
| Set | Seed | Before | After | Gain | After | Gain |
| Evolve | AHE | 86.82 | 90.45 | +3.63 | 91.23 | +4.41 |
| Terminus-2 | 85.91 | 89.09 | +3.18 | 92.21 | +6.30 | |
| Held-out | AHE | 85.78 | 90.22 | +4.44 | 91.28 | +5.50 |
| Terminus-2 | 87.56 | 91.56 | +4.00 | 91.61 | +4.05 | |
| Stage | Selected modification | Recorded outcome |
|---|---|---|
| Source TB2.1, round 4 Carried forward | System prompt. Execution and Verification Guidelines: deliverable integrity, schema conformance, defensive API compatibility, evaluation hygiene, and invariant checks instead of silent suppression. | 1 of 4 fixed. Fixed gpt2-codegolf ; protein-assembly , torch-pipeline-parallelism , and train-fasttext remained unresolved. |
| Target TB4, round 5 Appended | Verification layer. Add a schema-deliverables clause to the completion interlock and prompt templates; extend the checklist to programmatic assertions and a test-suite audit. | 0 of 2 fixed. Neither atrx-vep-crispr nor intrastat-meldung was fixed. |
| Target TB4, round 6 Appended | Verification layer. Add a defensive-contract detector to the completion checklist. | 1 of 2 fixed. Fixed embedding-drift-monitor ; biped-contact-dynamics remained unresolved. |
| Domain / shift | Reused directly | Reused with adaptation | Inherited and extended | Setting-dependent |
|---|---|---|---|---|
| Coding Dataset | Shell, terminal, and process layer | Paste-buffer transport (key normalization, foreground detection) | Completion contract, with new target predicates | Source completion predicates; refusal handling (idle) |
| Model | Terminal and process layer; interlock accounting | — | Lifecycle control, with recalibrated interception heuristics | Parser repairs tuned to one model’s output |
| Text cls. Dataset | BM25 index and retriever; query sanitization | Top- , score filter, and class quotas (recalibrated constants) | Multi-template routing skeleton | Data-specific post-processing (SMILES normalization) |
| Model | Indexing; query sanitization | Retrieval budgets (re-tuned to model capacity) | Routing skeleton, with rewritten rubrics | Output schema and decoding contract |
| Workspace Dataset | Soft gate before final_answer | Evidence probe (file system invocation history) | Truncation limits and context budget | Evaluator-specific registration assumption; substring heuristics |
| Model | System-prompt injection pipeline | Budget-triggered sparse intervention | Rubric taxonomy with counterexamples | Calibrations to one model’s failure rate or API schema |
Agent Harness Optimization
Agent harness optimization improves the prompts, tools, memory, workflows, and control code that shape how AI agents act.
See all 204 papers