SelfImprove-Bench: Evaluating Generalization in Self-Improving Agents
Organizations: Shanghai Jiao Tong University · Theseus Labs · Humanlaya · Shanghai AI Lab
Abstract
Recursively self-improving (RSI) systems are increasingly viewed as a promising means of accelerating progress toward artificial general intelligence. They aim to autonomously improve their capabilities by refining their working pipelines and recursively improving the strategies that guide subsequent refinements, with gains generalizing to unseen scenarios.However, existing evaluations lack reliable metrics to distinguish capability gains from scenario fitting, sustainable and extensible protocols for unseen tasks and difficulties, and fine-grained attribution of improvement to evolution traces. To this end, we propose SelfImprove-Bench,an RSI evaluation framework that organizes tasks into three primary capabilities (i.e., Exploring, Planning, Reasoning) as scenarios for guiding model evolution. We instantiated the tasks on games for controllability and reproducibility, and pairs them with scalable generation and orchestration driven by task templates and difficulty coefficients. Beyond task-level scores, SelfImprove-Bench organizes evaluation into generalization ladder levels G0 to G4 that measure how far gains transfer from training conditions. Experiments reveal that the evolutionary gains of existing RSI methods are inconsistent across generalization levels.
Figures & tables
| Cross-Task Transfer | ||||
| Work | Quantitative Analysis of RSI Modifications | Easy-to-Hard Transfer | Within-Capability | Cross-Capability |
| EvoPathBench [ 12 ] | — | — | — | |
| EvoAgentBench [ 9 ] | — | — | — | |
| SEAGym [ 4 ] | — | — | ||
| SkillLearnBench [ 10 ] | — | — | — | |
| ServeLearnBench [ 13 ] | — | — | — | |
| G0 | G1 | G2 | G3 | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Task | Method | Init. | Final | HCI (%) | Init. | Final | HCI (%) | Init. | Final | HCI (%) | Init. | Final | HCI (%) |
| Farm | ACE | 1.078 | 1.067 | -10.2 | 0.997 | 1.145 | +81.3 | 0.784 | 0.859 | +14.6 | 0.533 | 0.596 | +7.0 |
| AHE | 0.768 | 1.009 | +58.7 | 1.105 | 1.080 | -33.8 | 1.073 | 1.069 | -1.8 | 0.674 | 0.792 | +15.5 | |
| Hyper | 1.046 | 1.084 | +28.0 | 1.048 | 1.179 | +100.0 | 0.857 | 0.942 | +19.3 | 0.597 | 0.596 | -0.1 | |
| Combat | ACE | 0.855 | 0.911 | +9.7 | 0.533 | 0.592 | +6.6 | 0.227 | 0.385 | +14.5 | 0.997 | 1.123 | +69.2 |
| AHE | 1.065 | 1.410 | +93.8 | 0.674 | 1.427 | +99.2 | 0.301 | 1.196 | +88.4 | 1.105 | 1.087 | -24.3 | |
| Records | Functional target (%) | Applicability scope (%) | Modification surface (%) | ||||||||||||
| Method | Game | All | Rev. (%) | Dup. (%) | Explor. | Reason. | Plan. | Exec. | Task | Category | General | Prompt/Mem. | Harness | Meta | |
| ACE | Farm | 28 | 0.00 | 10.71 | 25 | 0.0 | 84.0 | 48.0 | 4.0 | 96.0 | 4.0 | 0.0 | 100.0 | 0.0 | 0.0 |
| Combat | 51 | 3.92 | 17.65 | 40 | 22.5 | 95.0 | 52.5 | 7.5 | 100.0 | 0.0 | 0.0 | 100.0 | 0.0 | 0.0 | |
| Hero | 78 | 26.92 | 12.82 | 46 | 0.0 | 82.6 | 67.4 | 2.2 | 32.6 | 65.2 | 2.2 | 100.0 | 0.0 | 0.0 | |
| Sweepmine | 44 | 11.36 | 6.82 | 36 | 8.3 | 72.2 | 41.7 | 8.3 | 100.0 | 0.0 | 0.0 | 100.0 | 0.0 | 0.0 | |
| MC | 65 | 16.92 | 13.85 | 43 | 18.6 | 53.5 | 53.5 | 25.6 | 100.0 | 0.0 | 0.0 | 100.0 | 0.0 | 0.0 | |
| Combat | Hero | |||
|---|---|---|---|---|
| Schedule | Medium | Hard | Medium | Hard |
| Random medium | 1.433 | 1.314 | 0.085 | 0.082 |
| Fixed medium instance | 0.772 | 0.507 | 0.100 | 0.017 |
| Mixed difficulty | 1.388 | 1.195 | 0.145 | 0.037 |
| Increasing difficulty | 1.332 | 1.080 | 0.152 | 0.024 |
| Task | Configuration | Score |
|---|---|---|
| Combat | DeepSeek Harness | 0.751 |
| HyperDSH (GPT-5.6 Sol) | 1.274 | |
| HyperDSH (Claude Opus 5.5) | 0.941 | |
| HyperDSH (DeepSeek) | 1.326 | |
| HyperDSH (DeepSeek, fixed improver) | 1.207 | |
| HyperAgents | 1.433 |
| Task | Configuration | All | Prompt/Mem. (%) | Harness (%) | Meta (%) | |
|---|---|---|---|---|---|---|
| Combat | DeepSeek Harness | – | – | – | – | – |
| HyperDSH (GPT-5.6 Sol) | 107 | 44 | 79.5 | 0.0 | 20.5 | |
| HyperDSH (Claude Opus 5.5) | 57 | 31 | 54.8 | 0.0 | 45.2 | |
| HyperDSH (DeepSeek) | 127 | 37 | 59.5 | 0.0 | 40.5 | |
| HyperDSH (DeepSeek, fixed improver) | 85 | 27 | 96.3 | 3.7 | 0.0 | |
| HyperAgents | 265 | 80 | 46.2 | 53.8 | 0.0 |
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
| Measure | After review |
|---|---|
| Scope exact agreement | 88% |
| Surface exact agreement | 99% |
| Function any-label overlap | 96% |
| Function mean Jaccard | 0.828 |
| Game | d1 | d2 | d3 | d4 | d5 |
|---|---|---|---|---|---|
| Hero | Ovr/1/3 | Ord/2/2 | Ord/3/2 | Ord/4/2 | Ord/5/2 |
| Sweepmine | / 3/16 | / 5/22 | / 8/30 | / 11/40 | / 15/52 |
| Game | Failure pattern | ACE refinement | AHE refinement | HyperAgents refinement | HyperDSH refinement |
|---|---|---|---|---|---|
| Hero | Copying demonstrated plans, causing redundant actions. | [Experience] Decompose demonstrations and combine only action blocks needed for the target attributes. | [Experience] Recombine blocks by target attribute, retaining contributions from unchanged attributes and replacing others only with contrastive evidence. | [Experience] Model each feature as an action block and combine the blocks inferred for the target. [Error: No guaranteed one-to-one attribute–block mapping.] | [Experience] Select action blocks by target attributes to build the primary plan and allow up to two additional rounds of blocks drawn from other demonstrations. [Correct: Limits retries while allowing alternative action blocks.] |
| Hero | Misinterpreting action types, order, or counts in demonstrations. | [Experience] Infer action blocks through controlled comparisons between demonstrations. | [Harness] Guide the agent to construct and run shell-based analyses that extract demonstrations and track evidence by attribute. | [Experience] Infer action types and counts separately. | [Experience] Select action blocks with their counts by target attribute, then check their order and placement against demonstrations. |
| Sweepmine | Marking cells incorrectly without checking all current board constraints. | [Experience] Check all local constraints across the board and recompute them after each action. | [Experience] Use stepwise prompts to rebuild board constraints and incorporate the total mine-count constraint. | [Experience] Derive residual constraints from the current board and combine overlapping constraints with the total mine count. | [Experience] Use a checklist to check local constraints. A skill also instructs the agent to track feasible mine placements and use the total mine count to guide inference. |
| MC | Misidentifying collection targets or whether they can be collected. | [Experience] Verify that the target can be mined. | [Harness] Verify the target’s current state. After a failed mining attempt, try a different target face . [Correct: A concrete re-aiming policy after mining failure.] | [Experience] Verify the target’s presence, reachability, and type. | [Experience] Check target type and state before collecting, and additionally monitor collection progress. |
| MC | Failing to adapt when navigation makes no progress. | [Experience] Compare position and distance to the resource before and after movement to assess progress, then reorient or excavate if movement is ineffective. | [Harness] Switch to excavation toward the resource when exploration stalls. | [Harness] Prevent consecutive repeated actions and turn or jump when movement stalls. [Correct: Enforces an action-repeat limit in code.] | [Experience] Compare the current position with the resource’s location and stop moving further when the agent is far from it. |
| Astray | Misjudging movement direction or traversable paths. | [Experience] Assess movement using velocity and displacement along the commanded axis. | [Experience] Identify paths from connected floor regions and follow confirmed bends without repeatedly changing direction. | [Experience] Assess movement using nearby walls and displacement along the commanded axis. | [Experience] Assess movement using structured coordinate changes; use changes in position relative to landmarks instead when coordinate data are unavailable. |
Self-Improving Agents
Self-improving agents use experience and feedback to update their models, memories, skills, or workflows for better future task performance.
See all 198 papers