Recursive Self-Improvement in Physical AI: Mechanisms, Evidence, and Evaluation

Xiaofeng Han; Kaize Luo; Zenghuang Fu; Zichong Zhu; Zhe Feng; Changwei Wang; Rongtao Xu; Weiliang Meng

cs.ROOct 28, 2026

Recursive Self-Improvement in Physical AI: Mechanisms, Evidence, and Evaluation

Authors: Xiaofeng Han, Kaize Luo, Zenghuang Fu, Zichong Zhu, Zhe Feng, Changwei Wang, Rongtao Xu, Weiliang Meng,

Organizations: School of Artificial Intelligence, University of Chinese Academy of Sciences, China · The State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, China · Southern University of Science and Technology, China · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, China ·

Abstract

Embodied systems can update policies, world models, skills, and executable programs from deployment experience, allowing one interaction to benefit later tasks. When a system also revises and reuses its data-acquisition, feedback, or learning procedures, experience changes the process that produces capability, introducing recursive self-improvement. This paper surveys research in physical AI through the connection between persistent capability and its improvement process. We first explain the functional distinction between learner updates and improvers, then compare policy learning, imagined experience, memory and skill accumulation, and executable harnesses. These approaches connect to revisions of curricula, rewards, and research programs in different ways. Contact validity, recovery, retention, and resource costs determine whether the resulting gains accumulate during continued deployment. An L0-L3 rubric distinguishes task-local adaptation, persistent capability gain, inherited process revision, and verified improver gain. Representative benchmarks and within-study contrasts relate these distinctions to existing results. We further propose common-data and interactive comparisons that evaluate reference and revised improvers from shared learner starts, separating the use of experience from its acquisition. This perspective connects the diverse methods, physical operating conditions, and evaluation evidence, underpinning embodied RSI. A comprehensive list of studies in this survey is available at https://github.com/nkd-lkz/physical-ai-rsi-survey.

Figures & tables

Fig. 1: Overview of the review framework for recursive self-improvement in physical AI. (a) Conceptual distinctions among programmed robots, learned policies, persistent self-improvement, and recursive improvement. (b) Physical experience and feedback support capability updates, while system-generated process revisions are retained and reused to govern later learning. (c) Persistent artifacts can modify the learner, the improver, or both. (d) The evidence rubric distinguishes task-local adaptation (L0), persistent capability gain (L1), inherited process revision (L2), and verified improver gain (L3). L3 requires greater subsequent gains from shared learner starts under matched resource budgets and independent tests. The levels specify evidence requirements for individual claims. Comparisons are schematic, and robot scenes are illustrative renderings.

Survey	Organizing focus	Connection to this paper
Self-evolving Embodied AI [ 10 ]	Memory, tasks, world models, embodiment, and model evolution	Broad embodied framework, including changes to optimization and evaluation procedures.
Weights or Skills? [ 11 ]	Execution feedback, persistent memory, and code-based skill search	Operational distinctions between task-local repair and reusable capability.
Self-Evolving AI for Humanoids [ 13 ]	Post-deployment mechanisms, meta-updates, safety, and lifetime trajectories	Physical evolution and trajectory evaluation, complementing separate tests of process inheritance and improver gain.
The Last AI Built by Humans [ 14 ]	RSI autonomy levels, application domains, and controlled improver evaluation	Direct RSI perspective; autonomy and evidential strength characterize different aspects of improvement.
Continual Learning for Robotics [ 12 ]	Continual-learning strategies, retention, transfer, and robotics constraints	Foundations for persistent-capability assessment and physical evaluation conditions.

TABLE I: Relation to closely related surveys. Comparisons concern organizing perspectives and overlapping coverage.

Fig. 2: Information boundaries between physical interaction, learner updates, and improver revision. Observations supply experience for learner updates; system-generated procedure changes are retained and used in later update rounds. Development screening can inform these revisions. A concealed reporting audit evaluates frozen candidates on independent held-out outcomes and supplies no feedback to the updates being assessed. Grids, contact geometry, records, and program differences are schematic representations.

Level	Operational claim	Minimum distinguishing evidence
L0	Task-local adaptation	Feedback changes the current attempt or episode; the adapted state is discarded at the declared reset.
L1	Persistent capability gain	Experience produces retained capability and benefits later independent attempts or tasks.
L2	Inherited process revision	System-generated revisions of an improvement component persist and govern subsequent learner updates.
L3	Verified improver gain	The revised improver yields greater subsequent gains from shared learner starts under matched resource limits and independent tests.

TABLE II: Progressive claims and minimum evidence

Fig. 3: Evidence requirements across a declared reset boundary. Episode context c, retained learner state x, and improvement procedure $\varphi$ have different persistence roles. L0 describes task-local changes that reset; L1 concerns retained capability gains in later independent attempts. L2 requires a system-generated process revision to be inherited and used in a later learner update. L3 requires greater subsequent gains under shared learner starts, matched resource budgets, and independent tests. The diagrams illustrate the criteria and corresponding evaluation conditions.

Fig. 4: Developmental trajectories of self-improvement in Physical AI. The 46 selected works are grouped by mechanism, with a separate column for recent work. Branches indicate thematic grouping, not causal inheritance or increasing RSI maturity. Intervals have unequal duration; within-period placement follows mechanism. R, S, and D denote reported real-robot, simulated/virtual, and non-embodied settings, not evidence strength. P denotes a project report without an assigned experimental-setting tag. Persistent updates and task gains alone do not establish improver gain.

Fig. 5: Four carriers of persistent physical experience. Policy and value updates change parameters and action selection; world models retain predictive structure learned from observed transitions; indexed memory and skill libraries support retrieval and composition; programs and harnesses retain changes to execution logic, illustrated by a recovery branch. Each panel exposes the retained representation and its later use. A program change is classified by its functional role in execution or in producing later updates.

Fig. 6: Fixed-model planning and model–policy co-improvement. The fixed-model planner evaluates imagined action alternatives and executes a selected action while retaining model parameters. In the coupled loop, real transitions train the model, the updated model generates imagined trajectories for policy learning, and the updated policy collects further physical experience. Solid and dashed paths distinguish physical and model-generated experience. The observed–predicted offset is schematic; coupling alone does not establish inherited revision of the improvement procedure.

Fig. 7: Four locations for inherited improver revision. Changes to acquisition alter the selected experience; changes to feedback alter the training signal; changes to learning or search alter the update routine; changes to a development gate alter which candidates are retained. Highlighted differences are accepted, inherited, and used in a later cycle. The rows describe alternative intervention locations. Development-gate revisions remain subject to declared external constraints, while independent reporting tests stay fixed for the comparison.

Fig. 8: Physical continuity and learning-state retention are separate, coordinated processes. Contact changes or calibration drift can require physical recovery before redeployment. Candidate changes to retained state are checked on earlier and new tasks, then accepted or rolled back to the previous checkpoint. Recovery is not itself a learning update. The resource ledger spans collection, recovery, updating, and verification, including robot time, computation, human effort, and wear or repair.

Fig. 9: Conceptual physical failure modes. (a) A plausible pose can yield unstable contact, illustrated by contact-normal and tangential-slip directions. (b) Partial observations can produce a false-positive judgment despite an unsuccessful physical outcome. (c) A new-task update can disrupt an earlier skill. (d) Sensor calibration or actuator offsets can change the observation–action mapping. The scenes, state changes, and offsets are illustrative rather than experimental measurements.

Field	What must be recorded
Persistent change	Weights, world model, memory, skill code, harness, reward, or update procedure; record functional role.
Feedback and setting	Physical measurements, human labels, learned judge, simulator, or hybrid; distinguish physical and digital evidence.
Retention boundary	Across attempts, independent episodes, tasks, or deployments; specify every reset.
Human and resource input	Demonstrations, resets, approval, maintenance, corrections, compute, time, and discarded trials.
Claim and control	Capability gain; inherited process revision; improver gain; shared starts, fixed references, and evaluator independence.

TABLE III: Evidence fields used independently of the L0–L3 rubric

Method	Comparison object	Evaluation focus
Zeva [ 23 ]	Task context	Memory boundaries across episode resets.
MemBodied [ 103 ]	Episode memory	Within-episode memory and persistent benefit across episodes.
RECAP [ 2 ]	Policy updates	Retained capability gains under a fixed learning procedure.
Learning While Deploying [ 3 ]	Fleet policy learning	Effects of fleet experience and model versions.
RISE [ 49 ]	Model-assisted policy learning	Training gains and the contribution of planning.
Motus2 [ 53 ]	Policy and planning	Policy retention and planning gains under shared initialization.

TABLE IV: Improvement objects and evaluation focus of representative methods

Fig. 10: Evaluation objects in representative robot-learning protocols. Cells name the task structure, split, or metric targeted by each scoped protocol. Dashes denote objectives not encoded as direct targets in that protocol, rather than capabilities absent from the environment. CALVIN instruction chains differ from a learning history; LIBERO and Continual World explicitly evaluate sequential retention and transfer. The RoboCasa365 row denotes its public multi-task leaderboard. Sources: RLBench, Meta-World, ManiSkill2, CALVIN, LIBERO, Continual World, RoboCasa, and RoboCasa365. [ 197 , 196 , 198 , 205 , 201 , 183 , 202 , 206 ]

Fig. 11: Common-data and interactive conditions for improver comparison. In the common-data condition, cloned learner starts and identical permitted evidence enter the reference and revised improvers. In the interactive condition, each improver acquires its own evidence under matched starts and resource ceilings. Branch-specific successor learners are assessed by independent held-out tests that provide no update feedback. The two conditions assess evidence use and acquisition plus updating, respectively; report subsequent gain, retention, transfer, and actual costs.

Fig. 12: Crossed evaluation of learner starts and improvement procedures. Each row clones a learner start and applies the reference and revised improvers; the horizontal contrast uses

G

and

\Delta G

as defined in Eq. ( 3 ), with common budgets and test conditions omitted from the displayed notation. Diagonal comparisons change both learner state and improver. Estimate uncertainty over independent replicates and report absolute gains alongside relative contrasts. Match resource ceilings and record realized consumption, including construction, collection, computation, human assistance, energy, and resets. The diagram contains no measured results.

Control	Purpose and required report
Shared learner starts	Match model version, initial weights, retained skills, prior data, and accessible tools.
Common-data condition	Give both improvers identical admissible evidence; isolate how the evidence is used.
Interactive condition	Allow separate collection under matched physical and compute limits; count unsuccessful acquisition.
Independent assessment	Separate development feedback from held-out outcomes; freeze the assessor or calibrate versions externally.
Retention and transfer	Test prior tasks, new tasks, and compatible alternative learners; report negative transfer.
Repeated runs and costs	Report uncertainty, full trajectories, total search cost, human assistance, resets, and maintenance.

TABLE V: Controls for an improver-gain claim

	Transfer		Occlusion		Counting		Sequence		Overall
Method	TSR	CSR	TSR	CSR	TSR	CSR	TSR	CSR	TSR	CSR
$\pi_{0.5}$	20.0	42.8	12.7	17.2	14.3	50.9	60.0	71.6	21.5	38.7
HiF-VLA	17.5	38.9	12.7	27.1	8.6	45.9	42.5	70.2	16.9	39.8
MemoryVLA	15.0	37.2	7.3	13.1	14.3	55.1	37.5	65.2	15.0	35.3
MemER	20.0	36.1	16.4	33.2	27.1	65.1	65.0	79.1	27.3	49.1
PrediMem [ 223 ]	22.5	45.2	27.3	38.4	45.7	69.3	72.5	89.5	38.5	55.2

TABLE VI: Reported task and cumulative success on the full 26-task RoboMemArena benchmark (%).

		Category score						Average
Method	Type	Gen-S	Gen-R	Prec.	Long	Mem.	Open	Score	SR (%)
PhysicalRSI 1.0	Agent+VLA	31.57	11.04	38.05	46.37	46.74	28.88	36.27	31.38
Simate-beta	VLA	40.54	29.63	34.35	57.84	33.33	9.12	33.95	27.96
VPP2-Preview	WAM	43.92	18.75	33.75	35.31	50.78	5.83	31.40	25.62
Liber-0 Preview	VLM+WAM	39.31	10.67	38.28	45.98	37.77	6.68	30.74	25.52
GPT-6 Astra	Agent	35.32	31.40	12.65	21.45	43.04	34.36	28.97	22.48

TABLE VII: Reported RoboDojo simulation results from the 28 September 2026 leaderboard snapshot.

Method	Coverage	Retained artifact	Revision and testing	Evidence focus
PhysicalRSI 1.0 [ 62 ]	Simulation	Harness and skills	Candidate revisions are tested, selected and inherited.	System evolution and task performance.
HexaAnything [ 226 ]	3 tasks	Tool code	Two revision rounds; fixed evaluator. Cloth trials include human-selected action points.	Direct tool-revision trajectory.
URAI [ 227 ]	5 tasks	Tool code	Human-guided development; tools frozen for four execution models.	Tool use and a separate revision case.
ENCORE [ 228 ]	22 tasks	Policy programs	Demonstration-guided refinement; programs frozen before sealed tests.	Program discovery and generalization.
RoboHarn-Evo [ 106 ]	2 tasks	Physical knowledge	Knowledge evolves on RMBench and remains read-only on RoboDojo.	Transfer of evolved knowledge.
ME-Brain-1.0 [ 229 ]	54 configurations	Memory and skills	RoboDojo evaluates the action model; multi-round evolution is not isolated.	Task performance of an evolvable system.

TABLE VIII: Improvement mechanisms and evaluation scope of RoboDojo studies.

Method	Evaluated scope	Changed component	Reference	Updated	$\Delta$ (pp)
EmbodiRSI [ 46 ]	S1+S3 simulation	Data acquisition	59.3	79.9	+20.6
FIND [ 84 ]	8 Franka tasks	Residual policy	55.0	71.9	+16.9
RoboCoach [ 54 ]	3 Franka tasks	Skill policies	13.3	75.0	+61.7
Motus2 [ 53 ]	2 real-robot tasks	Action parameters	65.0	72.5	+7.5
LWD [ 3 ]	Freezer restocking	Fleet policy	92.0	97.0	+5.0
RPG [ 230 ]	4 simulation tasks	Skill library	65.0	92.5	+27.5

TABLE IX: Within-study performance changes after improvement.

Method	Reference and update setting	Experience or resource budget	Evaluation
Q-VGM	Few-shot SFT; offline then online	150 offline rollouts/suite	500 tests/suite; 3 training seeds
RedFlow	Base policy; one offline update	1,536 rollouts/suite	500 tests/suite; horizons 240/240/240/480
Q-Planning	Frozen FastWAM; learned Q	100 rollouts/task/round; 10 rounds	20 tests/task; horizon 540 for every row
WoVR	Full-trajectory SFT; imagined RL	2,500 simulator trajectories/suite	Four LIBERO suites
EmbodiRSI	Static versus adaptive acquisition	400 retained trajectories/subtask at each endpoint	S1+S3 scene-balanced SR; no HIL
FIND	Base versus online residual policy	50 initial demonstrations/task	5 runs; 20 independent tests/task/checkpoint

TABLE X: Data, update conditions and evaluation resources for the reported contrasts.

Self-Improving Agents

Self-improving agents use experience and feedback to update their models, memories, skills, or workflows for better future task performance.

See all 175 papers

Toward Recursive Self-Improvement in Physical AI
Oct 21, 2026Meng Luo, Zechen Bai, Shanqing Xu +9Recursive Self-Improvement Robot Learning
National University of Singapore · Huazhong University of Science and Technology · Shanghai Jiao Tong University +2
Who Improves the Robot? Toward Physical Recursive Self-Improvement
Oct 13, 2026Hai Zhou, Fan Yang, Zeyi Chen +27Recursive Self-Improvement Continual Robot Learning
Shanghai Jiao Tong University · Google · Microsoft +13
Where is Physical RSI capability from? An Observational Scaling Law of LLM-as-policy
Oct 19, 2026Zekai Li, Jiayi Yu, Jinlu Zhang +8Self-Improving Agents Embodied AI
Galbot Team

Oct 21, 2026J/K move · Enter open · S save

Toward Recursive Self-Improvement in Physical AI

Meng Luo, Zechen Bai, Shanqing Xu, Yanlin Li+8

National University of Singapore · Huazhong University of Science and Technology · Shanghai Jiao Tong University+2

Recursive Self-Improvement Robot Learning

Open paper AskPDF arXiv

Recursive self-improvement (RSI) describes a system that improves not only its behavior but also the process by which it improves. Recent digital agents test this idea by rewriting their own code and checking each version with executable tests. Bringing these mechanisms into the physical world requires learning from partial observations through costly trials, while accounting for changing operating conditions and potentially irreversible consequences. We position RSI within robot learning and trace pathways from improving robot capabilities to improving the processes that generate them. We examine how physical experience can strengthen these improvement mechanisms and when the resulting gains can support further self-improvement of the mechanisms themselves. Gains carry forward only when feedback addresses remaining learning needs, the receiving component can use the change, its value survives changes in tasks or hardware, and continued practice remains safe and affordable. Existing studies, concentrated in manipulation, support behavioral improvement, benefits to later learning, and procedure improvement. Some systems also change the improver's knowledge or tools, but the physical studies reviewed here have yet to isolate gains attributable to recursion. We synthesize these findings into evaluation principles and research directions for machines that, trial by trial, safely improve the way they improve.

Abstract

Self-Improving Agents

Explore similar work

Toward Recursive Self-Improvement in Physical AI

Who Improves the Robot? Toward Physical Recursive Self-Improvement

Where is Physical RSI capability from? An Observational Scaling Law of LLM-as-policy