Who Improves the Robot? Toward Physical Recursive Self-Improvement

Hai Zhou; Fan Yang; Zeyi Chen; Lejun Zhang; Chenyang Fu; Jianping Li; Jiayi Zhu; Xianyang Liu

cs.AIOct 13, 2026

Who Improves the Robot? Toward Physical Recursive Self-Improvement

Authors: Hai Zhou, Fan Yang, Zeyi Chen, Lejun Zhang, Chenyang Fu, Jianping Li, Jiayi Zhu, Xianyang Liu,

Organizations: Shanghai Jiao Tong University · Google · Microsoft · University of California, Berkeley ·

Abstract

Robots are increasingly autonomous in task execution, yet their improvement remains largely human-driven. This study investigates Physical Recursive Self-Improvement (Physical RSI), in which physical experience guides lasting updates to both a robot's capabilities and its improvement procedure, making subsequent improvement more effective across tasks. Physical RSI faces challenges from noisy observations, continuous dynamics, costly trials, and irreversible consequences. We present an L0-L5 framework that differentiates capability-level from meta-level updates and within-task from across-task gains, reserving L5 for independently verified recursive gain. We study existing methods around harness adaptation and model training, alongside safety and control mechanisms that govern execution and successive updates. Our analysis reveals that, despite substantial progress in self-improvement, the surveyed literature does not yet demonstrate Physical RSI as defined by our framework. We identify research priorities in reliable feedback, causal attribution, safety preservation, and benchmarking, outlining a path toward robots that autonomously improve how they learn while ensuring safety. The GitHub repository for Physical RSI is available at https://github.com/SAIL-Research-Lab/Physical-RSI.

Figures & tables

Figure 1: Overview of Physical Recursive Self-Improvement. (a) The task-execution system, consisting of the harness $H$ and the parametric models $\Theta$ , proposes actions that pass through safety and control before acting on the physical world. The outcomes are stored as experience and feedback $D$ , which is shared by the task-execution system and the improvement procedure $\Phi$ . Using $D$ , $\Phi$ updates $H$ through $\mathsf{Adapt}$ and $\Theta$ through $\mathsf{Train}$ , while $\mathsf{Ver}$ decides which changes are kept. $\Phi$ can also update its own stages. An independent evaluator $E$ evaluates the task-execution system and thereby judges the updates made by $\Phi$ . (b) L1–L4 distinguish capability-level updates to the task-execution system from meta-level updates to the improvement procedure, within a single task or across tasks. L0 denotes systems that make no retained change on their own. L5 builds on L4 and requires independent, controlled evidence that subsequent improvement becomes more effective over successive rounds.

Line of work	Made by the system	Inherits changes	Updates $\Theta$ , $H$	Updates $\Phi$	Recursive gain	Physical grounding
Continual learning	$\checkmark$	$\checkmark$	$\checkmark$	$\times$	$\times$	–
Automated machine learning	$\checkmark$	$\times$	$\checkmark$	( $\checkmark$ )	$\times$	–
General RSI frameworks	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	–
Physical self-improvement	$\checkmark$	$\checkmark$	$\checkmark$	( $\checkmark$ )	–	$\checkmark$
Physical RSI	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$

Table 1: Continual learning, AutoML, and general RSI frameworks compared with the two definitions on six properties.

\checkmark

typically met, (

\checkmark

) met in part or by some works,

\times

not met, – not required or not addressed.

	Task-execution system		Improvement procedure
Loop	$H$	$\Theta$	$\mathsf{Adapt}$	$\mathsf{Train}$	$\mathsf{Ver}$	$D$	Effect seen in
Execution loop	U	U	–	–	–	W	The current episode
Capability-level update loop	W	W	U	U	U	U, W	Later tasks
Meta-level update loop	U	U	U, W	U, W	U, W	U, W	Changes of later rounds

Table 2: What each loop uses and changes. U, used by the loop; W, written by the loop; –, untouched. The independent evaluator

E

and the other protected constraints (defined under Safety below) are not listed.

E

scores the updated task-execution system and judges updates to

\mathsf{Ver}

from outside.

Figure 2: Update targets and task identity in the L0–L5 framework. (a) Task execution uses the harness $H$ and learned weights $\Theta$ . Capability-level updates change $H$ or $\Theta$ , whereas meta-level updates change the improvement procedure $\Phi$ . $D$ denotes the experience and feedback store. (b) Tasks are distinguished by their required skill sequence $\kappa$ . Grasping a cup from a table and from a cabinet top belongs to the same task set: $\kappa$ remains unchanged, and only the instance parameters $\xi$ differ. Retrieving a cup from an initially closed cabinet adds opening to $\kappa$ and therefore belongs to a different task set. The two long-horizon examples share the same skill set $\kappa_{\mathrm{set}}$ but execute the skills in different orders. Their skill sequences $\kappa$ therefore differ, so they belong to different task sets.

Figure 3: Selected systems by level, year, and evidence source. Each system sits in the sector of its level, and its distance from the centre gives its release year, from 2022 near the centre to 2026 at the rim. The border of each card gives the source of the feedback behind the update followed by the setting of deployment. R, real; S, simulation; O, model imagination, meaning a learned world model. A dagger marks a deployment that gave no feedback to the marked improvement loop: the update is driven by simulated (S) or imagined (O) feedback and the system is then deployed or validated on a real robot, which does not mean that the paper used no real data. L0 cards form a baseline of systems directed by humans or by execution feedback, and parenthetical labels name the branch evaluated. An asterisk is an affiliation note: SEFM* shows the affiliation at the time of the work rather than the first author’s current one, and HarnessPAI*, which is credited to a team, shows the affiliation of its first core contributor.

Figure 4: Task execution and capability improvement. Execution records inform persistent updates, and adopted harness updates support subsequent execution. The harness pathway changes task resources and execution procedures.

Figure 5: Harness adaptation from execution feedback to retained updates. Diagnosis identifies edit targets, memory, skill, and workflow changes modify the harness, and verification and subsequent execution inform adoption and continued use.

Figure 6: Improvement history. Outcomes and execution evidence from different tasks are summarized into modification records and stored for later diagnosis and improvement. Successful and unsuccessful attempts both contribute to the history.

Figure 7: The meta-improvement loop. Improvement histories inform hypotheses and method changes. Current and candidate adaptation methods are compared through the subsequent harness updates they produce under matched budgets. Evaluation of subsequent gains and costs supports adoption, further revision, or rejection of the candidate method. Adopted methods govern future improvement.

Figure 8: Model-learning workflow in physical RSI. Environment generation and data selection organize experience from physical interactions for training policies, world models, and reward models through imitation, self-supervised, or reinforcement learning. Candidate updates undergo verification before adoption: model updates shape subsequent execution and learning, while mechanism updates revise how data are prepared and models are trained in later rounds.

Figure 9: Safety and control in Physical RSI. System updates and environmental drift require reassessment of exploration, transfer, and runtime safeguards. Records in $D$ inform later revisions of $\Phi$ . Acceptance determines inheritance, while release specifies permitted physical operation. Oversight of the evolving procedure spans the cycle.

Change and examples	Evidence and limits	Checks before reuse
Policy adaptation SCDA ( Josifovski et al., 2025 )	Empirical cost control and reduced forgetting in tested conditions.	Costs in old and current conditions, including adaptation trials.
Domain transfer ( Hsu et al., 2023 ; Zubia et al., 2025 )	Statistical or expected-cost bounds under transfer assumptions.	Target coverage, state mappings, and action compatibility.
Dynamics or interfaces Barrier functions ( Ames et al., 2017 )	Safe-region preservation under valid models and feasible control.	Sensing, actuator limits, control timing, and handover state.
Monitors and recovery ( Gu et al., 2025a ; Thananjeyan et al., 2021 )	False-alarm calibration or learned recovery estimates. Intervention can still fail.	Feature validity, missed hazards, response delays, and recovery feasibility.
Structured updates ( Fulton and Platzer, 2019 )	Proof-preserving model and controller updates within a specified family.	Protected rules and the trials and systems produced by the revision.
Repeated selection ( Dwork et al., 2015 )	Statistical validity depends on controlling adaptive reuse of feedback.	Statistical error control across rounds, with physical exposure accounted for separately.

Table 3: Safety evidence and reassessment under system and environment changes.

Figure 10: Verification and evaluation in physical RSI. Proposed updates are applied to separate candidate system versions after harness adaptation and model training. Verification checks implementation validity, repair effectiveness, and capability preservation before updates are adopted. An independent evaluator assesses the updated system under external test conditions and scoring criteria, providing physical outcome evidence for meta-verification when the verifier itself is revised.

Evaluation focus	Representative resources
Embodied benchmarks
Task Learning, Performance & Generalization	Meta-World ( Yu et al., 2020 ) , RLBench ( James et al., 2019 ) , robosuite ( Zhu et al., 2020b ) , robomimic ( Mandlekar et al., 2021b ) , CALVIN ( Mees et al., 2022 ) , LIBERO ( Liu et al., 2023a ) , VIMA-Bench ( Jiang et al., 2023 ) , BEHAVIOR-1K ( Li and others, 2024 ) , ManiSkill3 ( Tao et al., 2025 ) , VLABench ( Zhang et al., 2025e ) , RoboTwin ( Mu et al., 2025 ) , EmbodiedBench ( Yang et al., 2025 ) , RoboCasa365 ( Nasiriany et al., 2026 ) .
Robustness & Physical-World Evaluation	SIMPLER ( Li et al., 2024 ) , THE COLOSSEUM ( Pumacay et al., 2024 ) , LIBERO-PRO ( Zhou and others, 2025 ) , LIBERO-Plus ( Fei et al., 2025 ) , REALM ( Sedlacek et al., 2025 ) , RoboArena ( Atreya et al., 2025 ) , RoboDojo ( Chen et al., 2026f ) , VLA-Arena ( Zhang et al., 2026a ) , RoboTwin 2.0 ( Chen et al., 2026g ) .
Feedback & Evaluation Reliability	WorldEval ( Li et al., 2025a ) , FailBench ( Navasardyan et al., 2026 ) , RoboRewardBench ( Lee et al., 2026 ) , dWorldEval ( Li et al., 2026c ) , SC3-Eval ( Tseng et al., 2026 ) , RBM-EVAL ( Liang et al., 2026a ) .
Robot Engineering & Development	CaP-Bench (CaP-X) ( Fu et al., 2026 ) , RoboCoach-Bench ( Lei et al., 2026 ) , RLE-Bench ( Ma et al., 2026 ) .
RSI-related benchmarks (primarily digital)

Table 4: Embodied and RSI-related benchmark resources grouped by primary evaluation focus.

Recursive Self-Improvement

Recursive self-improvement studies AI systems that use feedback from their own work to modify and evaluate future versions.

See all 83 papers

Oct 21, 2026

Toward Recursive Self-Improvement in Physical AI

Recursive self-improvement (RSI) describes a system that improves not only its behavior but also the process by which it improves. Recent digital agents test this idea by rewriting their own code and checking each version with executable tests. Bringing these mechanisms into the physical world requires learning from partial observations through costly trials, while accounting for changing operating conditions and potentially irreversible consequences. We position RSI within robot learning and trace pathways from improving robot capabilities to improving the processes that generate them. We examine how physical experience can strengthen these improvement mechanisms and when the resulting gains can support further self-improvement of the mechanisms themselves. Gains carry forward only when feedback addresses remaining learning needs, the receiving component can use the change, its value survives changes in tasks or hardware, and continued practice remains safe and affordable. Existing studies, concentrated in manipulation, support behavioral improvement, benefits to later learning, and procedure improvement. Some systems also change the improver's knowledge or tools, but the physical studies reviewed here have yet to isolate gains attributable to recursion. We synthesize these findings into evaluation principles and research directions for machines that, trial by trial, safely improve the way they improve.

Meng Luo, Zechen Bai, Shanqing Xu +9

National University of Singapore · Huazhong University of Science and Technology · Shanghai Jiao Tong University +2

Recursive Self-Improvement Robot Learning

Ask PDF

Sep 10, 2026

The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement

Recursive self-improvement (RSI) enables AI systems to turn experience and feedback into persistent changes that improve both their capabilities and the process of future improvement. We first use the Headroom-Closed Index (HCI) to reveal the problems of existing LLMs, then introduce the RSI concept and its development roadmap: from improvement-execution autonomy, improvement-strategy autonomy, experience-acquisition autonomy, and environment-adaptation autonomy, to recursive meta-improvement. Next we examine RSI across scenarios (e.g., scientific discovery, embodied intelligence, software engineering), highlighting their distinct requirements and development speeds. Drawing on diverse industry practices and preliminary empirical evidence, we connect RSI research with practical systems and identify key challenges to achieving genuine RSI.

Yi Duan, Ying Liu, Zirui Tang +32

Shanghai Jiao Tong University · Theseus Labs · Agent-Native Research Lab +6

Self-Improving Agents Language Model Self-Improvement Recursive Self-Improvement

Ask PDF

Oct 8, 2026

RoboRSI: Stable, efficient, and reusable robot self-evolution in complex real-world environments

A generalist robot should not only perform diverse tasks but also improve through experience, turning what it learns during execution into capabilities that later tasks can reuse. Robot agents that act through code can already repair programs from execution feedback, yet it remains a central challenge to organize this experience around the task structure that gives it meaning, so that each repair is attributed to the responsible capability, supported by execution evidence, and validated before it is reused. We introduce RoboRSI, a robot self-improvement system built on Top-Down Skill Refinement (TSR). TSR decomposes tasks into compound, atomic, and base skills with scoped responsibilities and explicit input--output contracts, attributes each execution outcome to the responsible branch, and confines revision to that branch. Building upon this structure, a Manager, Planner, Engineer, and Reviewer coordinate planning, execution, diagnosis, and the validated release of new skills, while people steer the process through objectives and corrections; stable skill sequences are further consolidated into reusable compound skills. On a mobile manipulator, RoboRSI develops multi-object household cleanup over 104 rounds. In simulation, it achieves the highest success rate on LIBERO, LIBERO-PRO, LIBERO-Plus, and RoboTwin, exceeding the strongest baseline by 2.7 to 11.0 percentage points.

Zimo Wen, Yijin Chen, Yuxuan Cao +8

Robot Skill Learning Continual Robot Learning Robotic Manipulation

Ask PDF

Abstract

Recursive Self-Improvement

Explore similar work

Toward Recursive Self-Improvement in Physical AI

The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement

RoboRSI: Stable, efficient, and reusable robot self-evolution in complex real-world environments