The RSI Tax: When Recursive Self-Improvement Is Not Worth the Cost
Organizations: School of Physics, Peking University · State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University
Abstract
Recursive self-improvement (RSI) seeks to improve an agent using its own task experience. Its practical value depends on whether performance gains justify the full cost of self-improvement and task execution. We evaluate this trade-off on continual task streams, scoring each task before its feedback is used for subsequent updates. We define the RSI Tax as the token cost of the self-improvement process and include it alongside task-execution costs in end-to-end evaluation. Across six methods and two language models, a bounded-note baseline matches or exceeds the observed accuracy of Meta-Harness, EvoSkill, and RRSI, with mean total-token savings of 65.6-71.0%. We decompose the RSI Tax into verification, exploration, and writing costs, and track subsequent use of the associated state or artifact. For these three RSI methods, verification accounts for 74.8--93.6% of the aggregate RSI Tax. A substantial share of the RSI Tax is also associated with states or artifacts that are not used later in the evaluated streams. These results expose a practical limitation of costly RSI procedures: improvements in the deployed agent need not justify the expense of the improvement process. They motivate RSI objectives that jointly optimize the RSI Tax and expected task-execution cost while ensuring continued growth in agent capabilities across the task stream.
Figures & tables
| Method | Verify | Explore | Write |
|---|---|---|---|
| ReasoningBank | — | Trajectory judgment; embeddings | Memory extraction |
| Meta-Harness | Incumbent and candidate validation | Trajectory analysis; candidate search | Candidate harness editing |
| EvoSkill | Initial, new-skill, and frontier validation | Extra rollouts; skill proposals | Skill materialization |
| RRSI | Incumbent and candidate validation | Trajectory analysis and summarization | Candidate harness generation and revision |
| SRN | — | — | Joint note analysis and revision |
| SWE-Chain-Evo (105 positions) | Terminal-Bench 2.1 (89 tasks) | |||||||
| DeepSeek | GPT-6 Luna | DeepSeek | GPT-6 Luna | |||||
| Method | Acc. | Total | Acc. | Total | Acc. | Total | Acc. | Total |
| Static | 43.8 | 487.92 | 38.1 | 179.69 | 69.7 | 185.55 | 66.3 | 78.86 |
| ReasoningBank | 40.0 | 520.64 | 37.1 | 191.21 | 69.7 | 198.87 | 67.4 | 85.46 |
| Meta-Harness | 45.7 | 976.61 | 40.0 | 495.81 | 77.5 | 762.57 | 65.2 | 317.99 |
| EvoSkill | 41.0 | 1,166.51 | 40.0 | 612.39 | 71.9 | 898.72 | 68.5 | 358.85 |
| SWE-Chain-Evo | Terminal-Bench 2.1 | |||
|---|---|---|---|---|
| Method | DeepSeek | GPT-6 Luna | DeepSeek | GPT-6 Luna |
| ReasoningBank | 4.78 | 5.96 | 4.53 | 7.39 |
| Meta-Harness | 109.06 | 177.35 | 314.16 | 329.15 |
| EvoSkill | 144.35 | 213.21 | 393.10 | 358.57 |
| RRSI | 125.79 | 213.02 | 223.23 | 346.68 |
| SRN | 0.04 | 0.25 | 0.13 | 0.36 |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| DeepSeek V4.1 Flash | GPT-6 Luna | |||||||
| Method | Acc. | Task | RSI | Total | Acc. | Task | RSI | Total |
| SWE-Chain-Evo (105 positions) | ||||||||
| Static | 43.8 | 487.92 | — | 487.92 | 38.1 | 179.69 | — | 179.69 |
| ReasoningBank | 40.0 | 496.91 | 23.74 | 520.64 | 37.1 | 180.46 | 10.75 | 191.21 |
| Meta-Harness | 45.7 | 467.13 | 509.47 | 976.61 | 40.0 | 178.77 | 317.04 | 495.81 |
| EvoSkill | 41.0 | 477.40 | 689.12 | 1,166.51 | 40.0 | 195.52 | 416.87 | 612.39 |
| (a) RSI costs by function: tokens (M) and share of RSI cost | |||||
| Method | RSI total | Verify | Explore | Write | Other |
| Static | 0.00 | — | — | — | — |
| ReasoningBank | 48.98 | 0 | 24.61 (50.25%) | 24.37 (49.75%) | 0 |
| Meta-Harness | 1,648.86 | 1,233.89 (74.83%) | 296.99 (18.01%) | 91.74 (5.56%) | 26.24 (1.59%) |
| EvoSkill | 2,103.05 | 1,643.17 (78.13%) | 455.29 (21.65%) | 4.59 (0.22%) | 0 |
| RRSI | 1,770.50 | 1,656.86 (93.58%) | 45.22 (2.55%) | 68.41 (3.86%) | 0.009 ( ) |
| SWE-Chain-Evo | Terminal-Bench 2.1 | |||
|---|---|---|---|---|
| Phase | DeepSeek | GPT-6 Luna | DeepSeek | GPT-6 Luna |
| Proposal session | 109.997932 | 62.540720 | 155.750468 | 86.680312 |
| Incumbent validation | 193.403359 | 139.638809 | 211.104585 | 70.721386 |
| Candidate smoke | 95.848101 | 27.941626 | 38.117662 | 11.285732 |
| Further candidate validation | 110.224752 | 86.921894 | 173.474330 | 75.207386 |
| Total | 509.474144 | 317.043049 | 578.447045 | 243.894816 |
| SWE-Chain-Evo | Terminal-Bench 2.1 | |||
|---|---|---|---|---|
| Phase | DeepSeek | GPT-6 Luna | DeepSeek | GPT-6 Luna |
| Initial baseline validation | 55.390798 | 38.509148 | 7.343709 | 1.632162 |
| Extra rollouts | 200.396353 | 71.161754 | 112.474010 | 35.706044 |
| Skill proposal | 20.072240 | 6.416161 | 6.122136 | 2.946065 |
| Skill materialization | 1.361587 | 1.229830 | 1.092549 | 0.901379 |
| New-skill validation | 141.891026 | 120.816937 | 91.976435 | 43.580524 |
| SWE-Chain-Evo | Terminal-Bench 2.1 | |||
|---|---|---|---|---|
| Phase | DeepSeek | GPT-6 Luna | DeepSeek | GPT-6 Luna |
| Trajectory judgment | 11.578243 | 5.354226 | 4.659062 | 2.892283 |
| Memory extraction | 12.142389 | 5.378519 | 3.904759 | 2.943139 |
| Embeddings (combined) | 0.015485 | 0.015418 | 0.048303 | 0.048426 |
| Total | 23.736117 | 10.748163 | 8.612124 | 5.883848 |
| Pattern | Failure criterion | Unit and observation point |
|---|---|---|
| Proposal-free search | A search round consumes tokens but emits no proposal. | Search round; after it ends. |
| Decision-invariant evaluation | All possible validation results lead to the same acceptance decision. | Candidate-evaluation calls; condition known before evaluation. |
| Unretrieved memory | A stored bundle has later task opportunities but is never retrieved before reset. | Bundle construction; retrieval observed through the remaining stream. |
LLM Agent Self-Improvement
Large language model agent self-improvement studies how agents use experience and feedback to improve their behavior, tools, and workflows.
See all 296 papers