cs.ROOct 21, 2026

Toward Recursive Self-Improvement in Physical AI

Authors: Meng Luo, Zechen Bai, Shanqing Xu, Yanlin Li, Kaixin Li, Mengchen Qian, Weifeng Lu, Ran Zhang, 

Organizations: National University of Singapore · Huazhong University of Science and Technology · Shanghai Jiao Tong University · Nanyang Technological University · 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Recursive self-improvement (RSI) describes a system that improves not only its behavior but also the process by which it improves. Recent digital agents test this idea by rewriting their own code and checking each version with executable tests. Bringing these mechanisms into the physical world requires learning from partial observations through costly trials, while accounting for changing operating conditions and potentially irreversible consequences. We position RSI within robot learning and trace pathways from improving robot capabilities to improving the processes that generate them. We examine how physical experience can strengthen these improvement mechanisms and when the resulting gains can support further self-improvement of the mechanisms themselves. Gains carry forward only when feedback addresses remaining learning needs, the receiving component can use the change, its value survives changes in tasks or hardware, and continued practice remains safe and affordable. Existing studies, concentrated in manipulation, support behavioral improvement, benefits to later learning, and procedure improvement. Some systems also change the improver's knowledge or tools, but the physical studies reviewed here have yet to isolate gains attributable to recursion. We synthesize these findings into evaluation principles and research directions for machines that, trial by trial, safely improve the way they improve.

Figures & tables

Table 1: Experimental conditions of digital and physical self-improvement. Digital examples include STOP, AlphaEvolve, and the Darwin Gödel Machine.
Figure 2: Improving what the robot does and how it learns. On the left, the learner improves under a fixed recipe and retains the change in its policy, world model, or external knowledge. On the right, the improver revises the learning procedure using retained repair knowledge; the dashed return asks whether that knowledge improves how the next revision is made (R1), and whether the improver, once improved, then helps produce its own next improvement (R2). The bottom row lists the physical limits behind the four conditions that let gains carry forward.
Table 2: Closely related reviews and the complementary focus of this review.
Figure 3: The review at a glance. Illustrations in (b) show the four grasp breakdowns from Section 1 .
Figure 4: Where a retained change is kept and how it is used. Each row follows the grasp example from physical experience to the retained change and its immediate use; the italic line names the use. Dashed tags mark secondary handoffs into the learning procedure.
Table 3: The comparison each claim requires. All comparisons use matched learners and budgets; in R2, the updated improver is revised once with its acquired knowledge and once with that knowledge frozen, and the successor must also beat the updated improver itself (inequalities in Section 2.2 ).
Table 4: How learning paradigms and problem settings relate to the claims of Section 2.2 . The last four columns show how each approach can support each claim; the R1, R2 column concerns whether the improvement mechanism itself improves.
Table 5: Benchmarks used or proposed in the reviewed studies. hardware; simulation; entries without an icon use generated videos, a game, or digital tasks.
Figure 5: Four teaching signals from physical experience. Each column follows the grasp example: what was experienced, what the signal changes in the training data, and representative systems from this section.
Table 6: Policy updates and selected within-study comparisons. Outcome checks give the setting, result, and scorer where identified; unresolved scorers are listed in Section 4.1 . updated; held fixed.
Table 7: Model-assisted learning. Columns follow Table 6 .
Figure 6: Where models, policies, and physical data change. Learning loops of SILVR, RISE, PlayWorld, and VLAW ( Luo et al., 2026a ; Yang et al., 2026b ; Yin et al., 2026b ; Guo et al., 2026 ) .
Figure 7: Three forms of external knowledge in the grasp example. Each panel shows one saved form of the slip repair and representative systems from this section; reuse on another robot first checks the interface.
Table 8: External-knowledge updates.
Figure 8: Does a revised component help the next learner? Two matched learners differ only in the component under test, here an evaluator. Their common outcome check stays fixed and separate from that evaluator; comparing the learners’ outcomes tests procedure improvement.
Table 9: Reported later-learning comparisons, with each study’s reported effort and the scope of its result. Reported effort covers different portions of acquisition and use across studies.
Table 10: Changes to collection, supervision, update, and experimental procedures.
 
SystemWhat changesSimulation / offlineHardwareClaim / status
Improving policies, world models, and knowledge (Section 3.1 )
D-World Wen and Wang , 2026Digital twin revised by a frozen LLM; residual-gated–Dual-arm one revision: depth residual 22.4 →\to 1.5 mmModel accuracy
MobiAgent Liu et al. , 2026aSkills from verified rolloutsRoboCasa 7.5% →\to 27.5%Astribot S1 32.5% →\to 57.5%Behavioral
Embodied Turing Machines Hu et al. , 2026bStateful decision codeRoboDojo, 42 bimanual tasks 70.24%–Behavioral
RoboRSI Wen et al. , 2026Branch-local patches after review, tests, replayFour benchmarks +2.7 to 11.0 points over a fixed agent, adapting online on 3 of 4WheelSingleArm M1 104 case-study runsBehavioral
Table 11: Robot self-improvement systems using RSI terminology in papers, system names, or project materials, assessed against the claims defined in Section 2.2 ; no row establishes R1 or R2. and sand shading mark systems whose improver’s knowledge or tools change, although no comparison isolates a recursive gain. Dashes mark settings a study does not report.
Table 12: The open problems of this section at a glance, in the order of the subsections that discuss them. marks the main focus of each direction among the four conditions: informative feedback (IF), usability (U), robustness to distribution shift (DS), and continued practice (CP). Starting benchmarks are proposals from the corresponding discussions.
Table 13: Where the reviewed studies stand on each claim of Section 2.2 . Systems are representative and follow the assessments in the sections that discuss them; those without an icon are digital.
Table 14: Comparisons that bear most directly on each condition.
Table 15: How a revision can invalidate a safety assumption.
Table 16: Checks before repair and comparisons after intervention.
Figure 9: A proposed test of the two links of recursive self-improvement (R1 and R2). In R2, the updated improver revises itself with frozen or acquired knowledge, producing the control or next improver, respectively. Each then revises a learning procedure, which is tested through fresh matched learners under a common, protected outcome check; the next improver must beat both the control and the updated improver; for the updated improver, its learner from R1 serves as the baseline. The R2 control assumes that the acquired knowledge can be withheld without changing the parent improver, as with a memory snapshot; the count of four hardware-trained learners per seed assumes that one candidate per improver survives screening.
Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Table 17: Human and system contributions at each stage of an improvement loop. Contributions can coexist.
 
DomainRepresentative comparisonIFUDSCPScope of support
ManipulationFIERCE: refreshed vs frozen evaluator; DexTouch-WM: all-real vs half-generated data ( Tan et al., 2026b ; Qin et al., 2026 )    Component and downstream tests on selected hardware tasks
NavigationAllDayNav: tests across rooms, episodes, and tasks ( Yin et al., 2026a )    Cross-room/episode/task tests of memory and policy
LocomotionIntelligent Trial and Error: behavior search after damage ( Cully et al., 2015 )    Real damaged-robot adaptation; a precomputed repertoire
FlightAgile quadrotor flight: residual-model learning and in-flight policy updates ( Ren et al., 2026 )    Short-horizon adaptation on one quadrotor platform
Robotic labsMobile chemist: measured outcomes guide the next experiment ( Burger et al., 2020 )    Experiment-selection precedent; a supplied search algorithm
Appendix
Table 18: Representative comparisons by robot domain. The marked conditions identify the focus of each comparison, not an exhaustive capability profile.
Table 19: Additional systems and learning settings. Icons identify reported experiments or transfer, not necessarily training.
 
SystemReported comparisonSetting and interpretation
SOAR Zhou et al. , 2025cMean success 28% →\to 57%; released dataset about 30.5K trajectoriesTen evaluation scenes across five tabletop settings. The updated arXiv text (its Section 4 and Figure 5; the dataset size also appears in its abstract and Appendix E) reports these values. The PMLR landing-page abstract instead states 25K trajectories.
SI-EFM Ghasemipour et al. , 2025LanguageTable approximately 62–63% →\to 87–88%; BananaTable approximately 63% →\to 85%Approximate plotted rates in the real-robot experiments. BananaTable practice uses about eight hours on two robot stations. The learned feedback checkpoint stays fixed during policy improvement.
HIL-SERL Luo et al. , 2025RAM insertion: 100% versus 29% for HG-DAgger; cycle time 4.8 versus 8.3 seconds100 trials per condition. Matched imitation comparison in the main results table. Intervention, reward classification, and compliant control jointly define the learning system.
RECAP Physical Intelligence et al. , 2026More than 2×2\times throughput on diverse laundry and espresso; roughly half the failure rateRelative to supervised specialization of the same pretrained policy.
RAPolicy Yang et al. , 2026aJoint-task success 52% →\to 88%One joint five-task setting, alongside four single-task settings. Online training uses one to two hours per setting and includes human interventions.
Appendix
Table 20: Selected reported outcomes, baselines, and evaluation settings. Icons mark where each reported number was measured.
 
TermMeaningSection
Action chunkA short sequence of future actions that a policy outputs at once.2.1
Admission gateA rule that turns protected evidence into a decision on whether a candidate update may run.4.3.3
Attempt, retry, roundAn attempt is one try at a task; a retry is a further attempt on the same task instance within one episode; a learning round ends with an update from which later episodes start.2.2
Behavioral improvementThe revised system executes better under a common task test.2.2
BottleneckThe component hypothesized to limit the next improvement, given the available repairs.4.4
Appendix
Table 21: Terms used in this review.
More in this topic

Recursive Self-Improvement

Recursive self-improvement studies AI systems that use feedback from their own work to modify and evaluate future versions.

See all 81 papers

Explore similar work

CardsList
Oct 13, 2026

Who Improves the Robot? Toward Physical Recursive Self-Improvement

Robots are increasingly autonomous in task execution, yet their improvement remains largely human-driven. This study investigates Physical Recursive Self-Improvement (Physical RSI), in which physical experience guides lasting updates to both a robot's capabilities and its improvement procedure, making subsequent improvement more effective across tasks. Physical RSI faces challenges from noisy observations, continuous dynamics, costly trials, and irreversible consequences. We present an L0-L5 framework that differentiates capability-level from meta-level updates and within-task from across-task gains, reserving L5 for independently verified recursive gain. We study existing methods around harness adaptation and model training, alongside safety and control mechanisms that govern execution and successive updates. Our analysis reveals that, despite substantial progress in self-improvement, the surveyed literature does not yet demonstrate Physical RSI as defined by our framework. We identify research priorities in reliable feedback, causal attribution, safety preservation, and benchmarking, outlining a path toward robots that autonomously improve how they learn while ensuring safety. The GitHub repository for Physical RSI is available at https://github.com/SAIL-Research-Lab/Physical-RSI.

Hai Zhou, Fan Yang, Zeyi Chen +27

Shanghai Jiao Tong University · Google · Microsoft +13

Recursive Self-ImprovementContinual Robot Learning
AskPDF
Sep 10, 2026

The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement

Recursive self-improvement (RSI) enables AI systems to turn experience and feedback into persistent changes that improve both their capabilities and the process of future improvement. We first use the Headroom-Closed Index (HCI) to reveal the problems of existing LLMs, then introduce the RSI concept and its development roadmap: from improvement-execution autonomy, improvement-strategy autonomy, experience-acquisition autonomy, and environment-adaptation autonomy, to recursive meta-improvement. Next we examine RSI across scenarios (e.g., scientific discovery, embodied intelligence, software engineering), highlighting their distinct requirements and development speeds. Drawing on diverse industry practices and preliminary empirical evidence, we connect RSI research with practical systems and identify key challenges to achieving genuine RSI.

Yi Duan, Ying Liu, Zirui Tang +32

Shanghai Jiao Tong University · Theseus Labs · Agent-Native Research Lab +6

Self-Improving AgentsLanguage Model Self-ImprovementRecursive Self-Improvement
AskPDF
Oct 19, 2026

Where is Physical RSI capability from? An Observational Scaling Law of LLM-as-policy

In this paper, we systematically investigate the mechanisms, scaling behavior, and generalization of physical Recursive self-improvement. We introduce PhysRSI-eval to quantify physical RSI through a five-axis metric: First-Attempt Performance, Efficiency, Success, Budget and Improvement from failure. Different VLMs are evaluated on 18 manipulation tasks under a fixed interaction budget. The analysis of experimental results motivates the Physical Barrel Hypothesis: different capabilities may constrain different parts of improvement. The five-axis metrics also distinguish systems with close public benchmark averages and show that high scores need not come with early or efficient gains. We study observational scaling from two perspectives: predicting the relative performance of VLM agents from their public capabilities, and forecasting later progress from early physical feedback. Together, these analyses ask how quickly and how far an agent can improve. Moreover, we conduct workspace interventions and then analyze what improvement leaves behind. Removing history reduces late-round success, frozen workspaces remain useful on unseen layouts, and three cross-task cases achieve success earlier. The results suggest that scene-specific adaptation and reusable experience can coexist.

Zekai Li, Jiayi Yu, Jinlu Zhang +8

Galbot Team

Self-Improving AgentsEmbodied AIVLMs for RoboticsEmbodied Agent BenchmarksRobot Manipulation Benchmarks
AskPDF