What Stops Recursive Self-Improvement in Robotics? Lessons from 123 Rounds of Agentic Skill Discovery

Jiaming Wang

cs.ROSep 23, 2026

What Stops Recursive Self-Improvement in Robotics? Lessons from 123 Rounds of Agentic Skill Discovery

Authors: Jiaming Wang

Organizations: National University of Singapore

Abstract

Can a robot improve itself the way coding agents now improve software? We built an agentic system to find out. It watches a robot fail, works out which capability is missing, writes new skills or finds and installs external models, tests every change in simulation, and repeats, with no human writing robot code. We ran it for 123 improvement rounds on household manipulation tasks. This report describes what we learned. The good news is that the agent can discover capabilities on its own: noticing that its targets were out of view, it asked for an active-viewing model, debugged it, and deployed a working search skill. The bad news is that its improvements did not add up. Changes kept passing their tests, yet the target task, putting condiments on the top shelf of a fridge, never succeeded. We found that the agent was rarely the bottleneck. Three things around it were. First, chained perception modules do not understand relations. Segmenters such as SAM 3 find shelves but not "the top shelf", so the agent filled the gap with ever more geometric rules that never converged, when what it needed was a different kind of model. Second, skill chains lock learning onto the first step. Long tasks mostly fail early, so evidence and fixes pile up there, and later skills are rarely reached, tested, or improved. Third, what the agent learns is decided by the harness. The agent optimized exactly what the evaluator measured, including where it was wrong, and weak tests and misleading memory turned activity into a standstill. We distill these lessons into concrete recommendations for building robot systems that improve themselves, each paired with an experiment that could prove it wrong.

Figures & tables

Figure 1: A busy loop that did not solve its task. (a) Accepted skill changes and newly registered models keep accumulating over 123 rounds. (b) Task successes per round: none on the target Fridge task; the few on Sink were mostly accidental. (c) Almost all accepted changes are to perception, the first step of every task (Lesson 2).

Panel of Figure 2 — Figure 2: How the system improves itself. Each improvement round (top row) runs the robot, judges every declared step, groups failures, and lets a coding agent (a GPT model in the Codex CLI, shown camera keyframes of each failure) propose a fix, either as code that must pass physical tests or as a request for a new external model. Five parts of the robot can change (bottom row); arrows show who changes each one. Four are changed by agents or by learning from logs. The harness that judges and remembers is changed only by humans, which turns out to matter (§ 7 ).

Rounds 1–75	gpt-5.6-sol , maximum reasoning effort
Rounds 76–84	gpt-6-astra , low reasoning effort
Rounds 85–123	gpt-6-astra , medium reasoning effort
Interface	Codex CLI, one session per agent task, in a read-only, capability-dropped container; from round 74 the run repository and held-out task are not visible
Orchestrator	Task description, skill catalog with observed usage, its best earlier programs; up to 25 min per program
Proposer	Failure table, evaluator diagnoses, execution traces and diffs of earlier attempts, camera keyframes of each failed step ; up to 3 attempts per problem, 60 min each
Research agent	The request and recorded failing inputs; finds, installs, wraps and tests an external model

Table 1: Agent models. All agents used the same model and reasoning setting in a given round.

Figure 3: Where each rollout first failed, rounds 66–123. Each row is one task and scene; color is the first step that failed. White circles mark task successes. Early in this window failures move around as skills change; in the last twenty rounds the fridge rows freeze at region finding (“the top shelf”) and grasping, and stay there despite continued library changes and new models.

Front	Repr.	Expr.	Resol.	What happened
Target outside every view (active search)	$\circ$	✓	✓	Once the evaluator reported visibility, the agent requested an active-viewing model, repaired it, wrapped it in a look-around skill, and the failure disappeared from deployment (§ 3 ).
“The top shelf” (relational region)	$\circ$	✗	✗	No module represents the relation; the agent encodes it as geometry fit to two recorded states. The skill grows without converging (§ 5 ).
Grasp retention	✗	✓	✗	Motion faults reported as grasp faults; fresh tests rarely reach the grasp. Real local wins never reach the task (§ 6 ).
Reachability	✗	✓	✓	A spherical evaluator and a misdescribed actuator; the agent optimizes the evaluator exactly (§ 7 ).
Requested models	✗	✓	✗	Requests silently dropped and remembered as failed; registered models never called (§ 7 ).

Table 2: The main problems of our run, through the three conditions. ✓ holds, ✗ fails,

\circ

holds only after harness repair. The one front where all three held is the one the agent solved on its own.

Panel of Figure 4 — Figure 4: Where the relation lives. (a) With concept-level and referring-expression segmenters, no module represents “top” or “below”; the relation is implemented in agent-written glue. (b) A vision-language model resolves the relation over the whole image; geometry and active viewing only verify or reject its answer.

Figure 5: (a) Frontier lock. Fraction of rollouts in the three-task phase whose first

k

declared steps all passed. More than half of fridge rollouts fail the first step and almost none reach the fifth, so steps beyond the frontier generate little evidence and are rarely exercised by gates. (b) Where the human effort went. The 113 harness repairs made by humans, grouped by what they repaired (the round each one was found in is listed in Table 4 ). Most concern what the agents were told, remembered and rewarded for, not the simulator or the replay machinery (§ 7 ).

Implication	Condition	Falsifying experiment
D1 Attribute by capability	Repr.	Re-label the recorded archive with capability-based attribution; count gaps that move to a different family, and compare proposal success on relabelled vs. original gaps.
D2 Measured embodiment	Repr.	Run the loop with declared vs. probed actuator descriptions; compare how often candidates hit evaluator boundaries (e.g. values piling at an advertised constant).
D4 Reason, then verify	Expr.	Replace only the region grounder in a frozen library; measure region success and chain depth on new seeds and new kitchen layouts.
D5 Model-class escalation	Expr.	Repair-only proposer vs. proposer with an escalation policy at equal budget; rounds to the first fresh-state win on a relational front.
D6 Chain in the substrate	Expr.	Fixed programs vs. gated chain edits; per-step reach and conditional reliability, and task success.
D7 States past the frontier	Resol.	Task-start test states vs. states reset to the changed step; fraction of evaluable pairs and detection rate of known-good changes.

Table 3: Design implications and how to falsify them. Every experiment should use frozen library snapshots, matched agent and simulator budgets, repeated seeds, and an evaluation set the loop has never selected against.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Rd	Outcome	Problems worked on	Proposals	Accepted skill changes	Models	Harness problems found
1	F 0/4	find obj: wrong object	1 att.; rej. replay 1	–	–	–
2	F 0/4	find obj: no task object near point	2 att.; rej. replay 2	–	–	target_not_in_view attributed on visibility
3	F 0/4	find obj: no task object near point	2 att.; rej. replay 2	–	–	effect packs visible through worker wrappers
4	F 0/4	find obj: target not in view	2 att.; 1 req.; rej. replay 1	–	1 not reg.	agent-extensible provider kinds
5	F 0/4	find obj: target not in view	2 att.; 1 req.; rej. replay 1	–	target_free_active_view_base_placement	–
6	F 0/4	find region: wrong fixture	2 att.; rej. replay 1, fresh 1	–	–	–

Appendix

Table 4: Round-by-round record. Outcome : task successes / completed rollouts for Fridge (F), Cabinet (C) and Sink (S). Problems worked on : the gaps sent to the proposer, as step: diagnosis. Proposals : attempts, of which research requests (req.), declines (no change) and rejections by test (replay = exact failure replay; fresh = paired fresh-scene and held-out test; transfer = sealed recorded failures; regr/inv = regression and invariance checks). Accepted : skill changes that passed every test, with the problem they fixed. Models : external models registered by the research agent, and research jobs that did not register. Harness problems found : harness repairs made by humans after this round exposed the problem.

Claim	Measurement
Target task never completed	Fridge 0/486; Cabinet 0/256; Sink 14/230, of which 9 had failed intermediate steps
Activity	57 accepted library changes (51 perception); 86 provider registrations over 20 identities; 847 proposal attempts
Region grounding is the dominant first failure	First failed effect in 132/224 Fridge and 113/232 Cabinet rollouts (three-task phase); wrong_region in all 62 late rollouts of fridge seeds 68, 70, 71
Semantic displacement	Region skill: 3,056 lines, 33 thresholds; composed localizer: 5,032 lines, 48 thresholds; 27 attempts on the two-state fridge gap rescued seed 69 in every trial and seed 68 in none
Visibility entanglement	Top-shelf occupant pixels at the region block’s start: about 3,000 / 1,273 / 292 / 45 on seeds 68 / 71 / 70 / 69
VLM feasibility	Qwen2.5-VL 7B placed “the top shelf” and “the shelf below” correctly in 4/4 recorded head views, about 1 s per query

Appendix

Table 5: Measurements behind the main-text claims. Rollout counts use completed rollouts; “late” means rounds 103–123; “three-task phase” means rounds 66–123.

Rounds	What changed	Rollouts	Completed	Task successes	Accepted changes
1–59	Fridge task only; discovery	243	237	0	33
60–65	Cabinet task added	49	49	0	8
66–102	Sink task added; harness repairs	448	434	13	14
103–110	Tests and memory repaired	97	97	1	1
111–118	Lost requests recovered; move to new machine	96	95	0	1
119–123	After the round-118 grasp change	60	60	0	0

Appendix

Table 6: Phases of the run. Boundaries summarize changes in which tasks were run and in the harness; they are not randomized treatment groups. “Accepted” counts skill changes that passed every test; routing refits are not included.

Model	First registered	Later rollouts	Calls	Rollouts with calls
target_free_active_view_base_placement	5	973	520	295
equivariant_grid_base_placement	14	935	185	110
target_graspgen_6dof	18	919	316	316
relational_rgbd_localizer	19	915	1863	867
mplib_whole_body_goalset	21	907	189	189
mplib_confined_motion	23	899	51	45

Appendix

Table 7: Every external model the research agent registered, and whether the robot used it. Calls are counted in task rollouts of rounds after the model’s first registration; test and research calls are not counted. Repairs of a model are folded into one row. The top block was used by deployed skills; the bottom block, mostly the late vision-language grounders for “the top shelf”, was never called.

Task	Scene seed	Steps passed	First failed step	Diagnosis
Fridge	68	0	fixture_region_localized	wrong_region
	69	3	retained	no_grasp_contact
	70	0	fixture_region_localized	wrong_region
	71	0	fixture_region_localized	wrong_region
Cabinet	68	3	retained	no_grasp_contact
	69	1	fixture_region_localized	wrong_fixture

Appendix

Table 8: Where each rollout stopped in the final round (123). Steps passed counts consecutive declared steps that succeeded before the first failure. No rollout succeeded.

Figure 6: Where candidates were rejected, and how stable paired comparisons were. (a) How many proposed changes reached and failed each test; replaying the recorded failure rejects most of them. (b) Of 36 fresh-scene pairs where the old and new skill disagreed and were re-run, 19 reproduced, 16 did not, and one was unmeasured.

Panel of Figure 7 — Figure 7: Recorded head-camera frames. (a) Start of the top-shelf region block later judged wrong_region : the lower shelf fills the view. (b) Start of a grasp block that fails with no_grasp_contact . (c) A late block of a native sink success whose program had an earlier tool_path_blocked failure. Frames are unmodified 256-pixel trace keyframes.

Self-Improving Agents

Self-improving agents use experience and feedback to update their models, memories, skills, or workflows for better future task performance.

See all 171 papers

Oct 1, 2026

Reconstruct, Practice, Go Real: Guided Self-Improvement for Embodied Agents

Building reliable robot capabilities across diverse tasks requires substantial human effort to develop and maintain skills, design rewards, and integrate perception with control. We present Reconstruct, Practice, Go Real (RPG), a framework for autonomous improvement of robot execution systems without updating model weights. RPG identifies manipulation capabilities in an offline dataset and constructs related practice tasks in simulation. During practice, RPG uses execution feedback, privileged simulator state, and available dataset videos to diagnose failures. It develops new reusable symbolic skills, refines existing skills, and revises the system prompt based on these diagnoses. Cross-task evaluation tests individual candidate changes and merged revisions before they are retained for reuse. At test time, a multimodal LLM uses the resulting system prompt and skill library to coordinate perception and robot control. On held-out initializations of 22 manipulation tasks, RPG improves task success from 28.6% after the first practice round to 95.0% after 15 rounds, outperforming all evaluated baselines, including ASPIRE (75.5%) and CaP-Agent0 powered by GPT-6 Astra Pro (60.0%). After a common calibration and hardware-adaptation procedure, the frozen system succeeds in all 30 physical trials, with ten trials on each of three tasks. Project Website: https://rpg-robot.github.io/

Yen-Jen Wang, Haozhe Jiang, Shuying Deng +7

UC Berkeley · Amazon FAR · MIT +1

Robot Skill Learning Large Language Model-Based Robot Planning Sim-to-Real Transfer Robotic Manipulation Self-Improving Agents Agentic AI for Robotics

Ask PDF

Oct 8, 2026

RoboRSI: Stable, efficient, and reusable robot self-evolution in complex real-world environments

A generalist robot should not only perform diverse tasks but also improve through experience, turning what it learns during execution into capabilities that later tasks can reuse. Robot agents that act through code can already repair programs from execution feedback, yet it remains a central challenge to organize this experience around the task structure that gives it meaning, so that each repair is attributed to the responsible capability, supported by execution evidence, and validated before it is reused. We introduce RoboRSI, a robot self-improvement system built on Top-Down Skill Refinement (TSR). TSR decomposes tasks into compound, atomic, and base skills with scoped responsibilities and explicit input--output contracts, attributes each execution outcome to the responsible branch, and confines revision to that branch. Building upon this structure, a Manager, Planner, Engineer, and Reviewer coordinate planning, execution, diagnosis, and the validated release of new skills, while people steer the process through objectives and corrections; stable skill sequences are further consolidated into reusable compound skills. On a mobile manipulator, RoboRSI develops multi-object household cleanup over 104 rounds. In simulation, it achieves the highest success rate on LIBERO, LIBERO-PRO, LIBERO-Plus, and RoboTwin, exceeding the strongest baseline by 2.7 to 11.0 percentage points.

Zimo Wen, Yijin Chen, Yuxuan Cao +8

Robot Skill Learning Continual Robot Learning Robotic Manipulation

Ask PDF

Oct 21, 2026

Toward Recursive Self-Improvement in Physical AI

Recursive self-improvement (RSI) describes a system that improves not only its behavior but also the process by which it improves. Recent digital agents test this idea by rewriting their own code and checking each version with executable tests. Bringing these mechanisms into the physical world requires learning from partial observations through costly trials, while accounting for changing operating conditions and potentially irreversible consequences. We position RSI within robot learning and trace pathways from improving robot capabilities to improving the processes that generate them. We examine how physical experience can strengthen these improvement mechanisms and when the resulting gains can support further self-improvement of the mechanisms themselves. Gains carry forward only when feedback addresses remaining learning needs, the receiving component can use the change, its value survives changes in tasks or hardware, and continued practice remains safe and affordable. Existing studies, concentrated in manipulation, support behavioral improvement, benefits to later learning, and procedure improvement. Some systems also change the improver's knowledge or tools, but the physical studies reviewed here have yet to isolate gains attributable to recursion. We synthesize these findings into evaluation principles and research directions for machines that, trial by trial, safely improve the way they improve.

Meng Luo, Zechen Bai, Shanqing Xu +9

National University of Singapore · Huazhong University of Science and Technology · Shanghai Jiao Tong University +2

Recursive Self-Improvement Robot Learning

Ask PDF

Abstract

Self-Improving Agents

Explore similar work

Reconstruct, Practice, Go Real: Guided Self-Improvement for Embodied Agents

RoboRSI: Stable, efficient, and reusable robot self-evolution in complex real-world environments

Toward Recursive Self-Improvement in Physical AI