cs.ROOct 21, 2026
SaveToward Recursive Self-Improvement in Physical AI
Organizations: National University of Singapore · Huazhong University of Science and Technology · Shanghai Jiao Tong University · Nanyang Technological University ·
Abstract
Recursive self-improvement (RSI) describes a system that improves not only its behavior but also the process by which it improves. Recent digital agents test this idea by rewriting their own code and checking each version with executable tests. Bringing these mechanisms into the physical world requires learning from partial observations through costly trials, while accounting for changing operating conditions and potentially irreversible consequences. We position RSI within robot learning and trace pathways from improving robot capabilities to improving the processes that generate them. We examine how physical experience can strengthen these improvement mechanisms and when the resulting gains can support further self-improvement of the mechanisms themselves. Gains carry forward only when feedback addresses remaining learning needs, the receiving component can use the change, its value survives changes in tasks or hardware, and continued practice remains safe and affordable. Existing studies, concentrated in manipulation, support behavioral improvement, benefits to later learning, and procedure improvement. Some systems also change the improver's knowledge or tools, but the physical studies reviewed here have yet to isolate gains attributable to recursion. We synthesize these findings into evaluation principles and research directions for machines that, trial by trial, safely improve the way they improve.
Figures & tables
Table 1: Experimental conditions of digital and physical self-improvement. Digital examples include STOP, AlphaEvolve, and the Darwin Gödel Machine.
Figure 2: Improving what the robot does and how it learns. On the left, the learner improves under a fixed recipe and retains the change in its policy, world model, or external knowledge. On the right, the improver revises the learning procedure using retained repair knowledge; the dashed return asks whether that knowledge improves how the next revision is made (R1), and whether the improver, once improved, then helps produce its own next improvement (R2). The bottom row lists the physical limits behind the four conditions that let gains carry forward.
Table 2: Closely related reviews and the complementary focus of this review.
Figure 3: The review at a glance. Illustrations in (b) show the four grasp breakdowns from Section 1 .
Figure 4: Where a retained change is kept and how it is used. Each row follows the grasp example from physical experience to the retained change and its immediate use; the italic line names the use. Dashed tags mark secondary handoffs into the learning procedure.
Table 3: The comparison each claim requires. All comparisons use matched learners and budgets; in R2, the updated improver is revised once with its acquired knowledge and once with that knowledge frozen, and the successor must also beat the updated improver itself (inequalities in Section 2.2 ).
Table 4: How learning paradigms and problem settings relate to the claims of Section 2.2 . The last four columns show how each approach can support each claim; the R1, R2 column concerns whether the improvement mechanism itself improves.
Table 5: Benchmarks used or proposed in the reviewed studies. hardware; simulation; entries without an icon use generated videos, a game, or digital tasks.
Figure 5: Four teaching signals from physical experience. Each column follows the grasp example: what was experienced, what the signal changes in the training data, and representative systems from this section.
Table 6: Policy updates and selected within-study comparisons. Outcome checks give the setting, result, and scorer where identified; unresolved scorers are listed in Section 4.1 . updated; held fixed.
Table 7: Model-assisted learning. Columns follow Table 6 .
Figure 6: Where models, policies, and physical data change. Learning loops of SILVR, RISE, PlayWorld, and VLAW ( Luo et al., 2026a ; Yang et al., 2026b ; Yin et al., 2026b ; Guo et al., 2026 ) .
Figure 7: Three forms of external knowledge in the grasp example. Each panel shows one saved form of the slip repair and representative systems from this section; reuse on another robot first checks the interface.
Table 8: External-knowledge updates.
Figure 8: Does a revised component help the next learner? Two matched learners differ only in the component under test, here an evaluator. Their common outcome check stays fixed and separate from that evaluator; comparing the learners’ outcomes tests procedure improvement.
Table 9: Reported later-learning comparisons, with each study’s reported effort and the scope of its result. Reported effort covers different portions of acquisition and use across studies.
Table 10: Changes to collection, supervision, update, and experimental procedures.
| System | What changes | Simulation / offline | Hardware | Claim / status |
| Improving policies, world models, and knowledge (Section 3.1 ) | ||||
| D-World Wen and Wang , 2026 | Digital twin revised by a frozen LLM; residual-gated | – | Dual-arm one revision: depth residual 22.4 1.5 mm | Model accuracy |
| MobiAgent Liu et al. , 2026a | Skills from verified rollouts | RoboCasa 7.5% 27.5% | Astribot S1 32.5% 57.5% | Behavioral |
| Embodied Turing Machines Hu et al. , 2026b | Stateful decision code | RoboDojo, 42 bimanual tasks 70.24% | – | Behavioral |
| RoboRSI Wen et al. , 2026 | Branch-local patches after review, tests, replay | Four benchmarks +2.7 to 11.0 points over a fixed agent, adapting online on 3 of 4 | WheelSingleArm M1 104 case-study runs | Behavioral |
Table 11: Robot self-improvement systems using RSI terminology in papers, system names, or project materials, assessed against the claims defined in Section 2.2 ; no row establishes R1 or R2. and sand shading mark systems whose improver’s knowledge or tools change, although no comparison isolates a recursive gain. Dashes mark settings a study does not report.
Table 12: The open problems of this section at a glance, in the order of the subsections that discuss them. marks the main focus of each direction among the four conditions: informative feedback (IF), usability (U), robustness to distribution shift (DS), and continued practice (CP). Starting benchmarks are proposals from the corresponding discussions.
Table 13: Where the reviewed studies stand on each claim of Section 2.2 . Systems are representative and follow the assessments in the sections that discuss them; those without an icon are digital.
Table 14: Comparisons that bear most directly on each condition.
Table 15: How a revision can invalidate a safety assumption.
Table 16: Checks before repair and comparisons after intervention.
Figure 9: A proposed test of the two links of recursive self-improvement (R1 and R2). In R2, the updated improver revises itself with frozen or acquired knowledge, producing the control or next improver, respectively. Each then revises a learning procedure, which is tested through fresh matched learners under a common, protected outcome check; the next improver must beat both the control and the updated improver; for the updated improver, its learner from R1 serves as the baseline. The R2 control assumes that the acquired knowledge can be withheld without changing the parent improver, as with a memory snapshot; the count of four hardware-trained learners per seed assumes that one candidate per improver survives screening.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Table 17: Human and system contributions at each stage of an improvement loop. Contributions can coexist.
| Domain | Representative comparison | IF | U | DS | CP | Scope of support |
| Manipulation | FIERCE: refreshed vs frozen evaluator; DexTouch-WM: all-real vs half-generated data ( Tan et al., 2026b ; Qin et al., 2026 ) | Component and downstream tests on selected hardware tasks | ||||
| Navigation | AllDayNav: tests across rooms, episodes, and tasks ( Yin et al., 2026a ) | Cross-room/episode/task tests of memory and policy | ||||
| Locomotion | Intelligent Trial and Error: behavior search after damage ( Cully et al., 2015 ) | Real damaged-robot adaptation; a precomputed repertoire | ||||
| Flight | Agile quadrotor flight: residual-model learning and in-flight policy updates ( Ren et al., 2026 ) | Short-horizon adaptation on one quadrotor platform | ||||
| Robotic labs | Mobile chemist: measured outcomes guide the next experiment ( Burger et al., 2020 ) | Experiment-selection precedent; a supplied search algorithm | ||||
Table 18: Representative comparisons by robot domain. The marked conditions identify the focus of each comparison, not an exhaustive capability profile.
Table 19: Additional systems and learning settings. Icons identify reported experiments or transfer, not necessarily training.
| System | Reported comparison | Setting and interpretation |
|---|---|---|
| SOAR Zhou et al. , 2025c | Mean success 28% 57%; released dataset about 30.5K trajectories | Ten evaluation scenes across five tabletop settings. The updated arXiv text (its Section 4 and Figure 5; the dataset size also appears in its abstract and Appendix E) reports these values. The PMLR landing-page abstract instead states 25K trajectories. |
| SI-EFM Ghasemipour et al. , 2025 | LanguageTable approximately 62–63% 87–88%; BananaTable approximately 63% 85% | Approximate plotted rates in the real-robot experiments. BananaTable practice uses about eight hours on two robot stations. The learned feedback checkpoint stays fixed during policy improvement. |
| HIL-SERL Luo et al. , 2025 | RAM insertion: 100% versus 29% for HG-DAgger; cycle time 4.8 versus 8.3 seconds | 100 trials per condition. Matched imitation comparison in the main results table. Intervention, reward classification, and compliant control jointly define the learning system. |
| RECAP Physical Intelligence et al. , 2026 | More than throughput on diverse laundry and espresso; roughly half the failure rate | Relative to supervised specialization of the same pretrained policy. |
| RAPolicy Yang et al. , 2026a | Joint-task success 52% 88% | One joint five-task setting, alongside four single-task settings. Online training uses one to two hours per setting and includes human interventions. |
Table 20: Selected reported outcomes, baselines, and evaluation settings. Icons mark where each reported number was measured.
| Term | Meaning | Section |
|---|---|---|
| Action chunk | A short sequence of future actions that a policy outputs at once. | 2.1 |
| Admission gate | A rule that turns protected evidence into a decision on whether a candidate update may run. | 4.3.3 |
| Attempt, retry, round | An attempt is one try at a task; a retry is a further attempt on the same task instance within one episode; a learning round ends with an update from which later episodes start. | 2.2 |
| Behavioral improvement | The revised system executes better under a common task test. | 2.2 |
| Bottleneck | The component hypothesized to limit the next improvement, given the available repairs. | 4.4 |
Table 21: Terms used in this review.
More in this topic
Recursive Self-Improvement
Recursive self-improvement studies AI systems that use feedback from their own work to modify and evaluate future versions.
See all 83 papers