cs.RONov 4, 2026
SavePyRUA-REAL: Robot Exploration and Adaptive Learning through RSI in the Real World
Organizations: Peking University · National University of Singapore · EngineAI
Abstract
Coding agents powered by models such as GPT-6-Astra connect spatial reasoning with executable robot behavior, raising the question of how such agents can produce lasting improvements in robot capabilities from physical interaction. In this work, we study embodied recursive self-improvement (RSI) through real-world exploration, code refinement, and deployment using PyRUA-REAL, a framework in which coding agents retain interaction experience as persistent memory and reusable executable skills that inform subsequent exploration and improvement. Our empirical study places physical manipulation at the center of the improvement process, examining how agents collaborate, what kinds of execution feedback support effective code refinement, and how task specifications and previously acquired skills influence robustness and task transfer. We further investigate the efficiency and capability of the resulting programs when deployed without involving an online agent for decision-making. Across eight real-world tasks, PyRUA-REAL achieves an average success rate of 91.1%. In complementary evaluation on 42 tasks from the RoboDojo benchmark, it achieves an average success rate of 84.4%. This study provides empirical guidance for turning physical exploration into reusable robot skills that support continued improvement and robust, efficient deployment.
Figures & tables
Figure 1: PyRUA-REAL: Python-based Robot-Use Agents for Robot Exploration and Adaptive Learning. (a) Coding agents iteratively explore, execute, diagnose, and revise robot programs, retaining rollout experience as persistent memory and reusable skills. (b) Development-stage improvements in success rate and cycle time, including reset, for stacking, tic-tac-toe, and Hanoi. (c) Real-world success rates aggregated over development attempts for seven agent-free skills. (d) RoboDojo performance compared with eight baselines, measured by success rate averaged equally over five capability groups. (e) The eight real-world tasks include semantic reasoning, precise manipulation, contact-rich manipulation, deformable object manipulation and long-horizon tasks.
Figure 2: PyRUA-REAL develops executable policies through researcher–engineer collaboration and retains facts, memories, and code for reuse. The resulting policy uses live feedback during execution.
| Level | Stored content | Primary use |
|---|---|---|
| L0 (Facts) | Robot, workspace, and operating conditions. | Researcher: understand relevant environmental conditions. |
| L1 (Memory) | Problems, attempted approaches, outcomes, and failure causes. | Researcher: identify relevant historical tasks, approaches, and skills. |
| L2 (Implementation) | Task programs and reusable primitives. | Engineer: inspect, adapt, and test implementations. |
Table 1: Experience hierarchy and its primary use by the two agents.
Figure 3: Eight real-world tasks. Top row: (a) tic-tac-toe, (b) five-block stacking, (c) Tower of Hanoi, and (d) Ethernet insertion. Bottom row: (e) towel folding, (f) Push T, (g) block sorting, and (h) block storage. Images show scenes recorded during task execution.
Figure 4: Selected properties of the eight physical tasks on the dual-arm Franka platform: precision, long-horizon execution, reasoning, contact-rich interaction, deformable-object handling, bimanual coordination, and multi-object handling. Marks indicate selected task properties, rather than measured difficulty or performance; unmarked cells do not imply that a property is absent.
Figure 5: Real-world capability acquisition with agent. The first column is approximate time to first complete success. Agent-free Ethernet insertion remains unsuccessful after ten days; online assistance yields success after approximately 15 additional hours of exploration. The second column recorded success rates across development attempts and program revisions. Execution-time points show successful-run medians; whiskers span the 25th to 75th percentiles. Only the execution-time axis is logarithmic.
Figure 6: Continued improvement after first success. Columns show five-block stacking, tic-tac-toe, and Hanoi. (a–c) Success rate; (d–f) execution time, including reset. Shading in (d) shows the interquartile range. Points are observed values; dashed curves are exponential moving averages ( ).
Figure 7: Five-block placement precision across revisions. (a) Mean lateral error and (b) mean absolute angular error. Both panels use the same placements from completed cycles in the 33 batches shown in Figure 6 a,d. Points show batch means; dashed curves are exponential moving averages ( ).
Figure 8: Complementary RoboDojo evaluation. Reported results for PyRUA-REAL, PyRUA-Lean, and GPT-6-Astra in the October 24, 2026 comparison snapshot: (a) success rates by capability, (b) overall success, and (c) benchmark score. Overall metrics weight the five groups equally; radar labels give task counts. PyRUA-Lean results are from its official evaluation. Full method comparisons, with column-best values in bold, appear in Appendix D .
Figure 9: Coordination costs with and without a reviewer. (a) Result-to-next-trial intervals on a logarithmic scale; boxes show medians and interquartile ranges. (b) Six review–revision cycles before hardware execution in the Hanoi experiment. Review includes dispatch and review; Repair includes modification and resubmission. Bar-end labels show total cycle time. (c) Elapsed time in the reviewed phase and the subsequent direct-testing phase, which reaches the first verified disk-on-peg insertion.
Figure 10: Task-level reassessment following stalled local revisions. Trial sequences for seven simulated tasks and three real-world tasks, aligned at reassessment by the Researcher (vertical dashed line). A single Researcher participates throughout each history; intervention denotes its reassessment of the current approach and redirection of exploration. Gray crosses and dots mark unsuccessful trials before and after reassessment, respectively; red dots mark successes, with circles highlighting the first success in each sequence. Right-hand fractions report successful trials over all recorded trials after reassessment. Horizontal spacing represents trial order, not elapsed time. Outcomes summarize development attempts across revisions rather than independent tests of a fixed program.
Figure 11: Feedback interfaces affect program improvement. Test success after five revisions with basic outcome reports (B), numeric state reports (BS † ), or video and joint trajectories (BVJ). All conditions include success, termination reason, action count, and timing. BS additionally supplies sampled numeric robot and ground-truth object states to the revising agent ( : privileged diagnostics); BVJ adds complete camera video and synchronized joint trajectories. Each task–condition pair has one development session and 18 test executions across six held-out layouts. The final group weights the four tasks equally.
Figure 12: Success on held-out scenarios with and without continued exploration and program revision with explicit workspace boundaries and target position and orientation ranges. Both versions are evaluated on the same test scenarios. Category means weight tasks equally; annotations indicate absolute gains in percentage points.
Figure 13: Transfer from a primitive-skill library to new tasks. Agents develop programs for eight target tasks with or without access to primitive skills acquired on earlier tasks. (a) Mean success rate across the target tasks. (b) Per-task success-rate differences, computed as with-library minus without-library performance. Red markers indicate positive gains; gray markers indicate no change. Five of the eight tasks benefit from the library.
Figure 14: Execution time and online thinking overhead. Offline times are successful-run medians including reset; online bars show recorded times. Dashed outlines mark non-completion within 120 minutes with unmeasured timing splits. Development time is excluded.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
| Condition | Information added to the basic report |
|---|---|
| B | Success, termination reason, action count, simulation time, and episode wall time. |
| BV | Camera frames at steps 0, 50, 100, 200, 400, and the final available step. |
| BV_FULL | Complete camera video, without joint-trajectory CSV. |
| BVJ | Complete video and synchronized joint-state CSV. |
| BVJ_QUERY | BVJ plus an optional frame-query and contact-sheet helper. |
| BS † | Sampled numeric robot and ground-truth object states. |
Table 2: Feedback interfaces. All conditions include B. Video comprises three synchronized camera recordings at 25 Hz, accessed through tools. Privileged diagnostics.
Figure 15: Test success across revisions, averaged equally over four tasks. Revisions 0–4 use one execution per test layout; revision 5 uses three. Condition codes are defined in Table 2 .
Figure 16: Retrospective selected-version success and development cost. Each point averages four tasks under one feedback condition; the dashed line is an ordinary least-squares fit across all eight conditions. B, BS, and BVJ are labeled; hollow diamonds indicate privileged diagnostics. Versions are selected using test outcomes. Time includes initial generation through the selected revision, although all sessions continue through revision 5.
| Through selected revision | Full exploration (r0–r5) | |||||||
|---|---|---|---|---|---|---|---|---|
| Condition | Selected revisions | Success (%) | Optimizer time | Tokens | Success (%) | Optimizer time | Tokens | Simulator time |
| B | 5 / 5 / 5 / 5 | 12.50 | 31.82 | 3.536 | 15.28 | 31.82 | 3.536 | 29.63 |
| BV | 1 / 4 / 5 / 5 | 50.00 | 25.12 | 2.747 | 47.22 | 29.41 | 3.417 | 25.14 |
| BV_FULL | 3 / 1 / 5 / 5 | 50.00 | 20.66 | 2.607 | 50.00 | 24.40 | 3.606 | 28.86 |
| BVJ | 2 / 1 / 5 / 5 | 58.33 | 19.06 | 2.433 | 54.17 | 25.40 | 3.647 | 27.28 |
| BVJ_QUERY | 5 / 2 / 5 / 5 | 37.50 | 23.29 | 3.836 | 33.33 | 25.42 | 4.356 | 32.43 |
Table 3: Selected-version and full exploration costs, averaged equally over four tasks. Selected revisions are ordered as Stack blocks , Push T , Deposit coin , and Imitate sorting sequence . Selected success uses one execution per test layout; final success uses three. Times are in minutes and tokens in millions. Privileged diagnostics.
Figure 17: From an inspected camera image to a verified code repair. The agent inspected the original photo; the crop and arrow are added for readability. The frozen-source diff omits unchanged preprocessing and wraps the added conditional; red and green mark removed and added code. The replay changes verification, not robot motion.
Figure 18: From empty grasps to a revised physical descent. The plane mismatch explains the empty grasps; the next recorded unfold attempt captures the towel on its first descent. The drawing is not to scale. The code comparison is pseudocode reconstructed from the iteration record, not a recovered source diff.
| Task | Before (%) | After (%) | (pp) |
| Generalization | |||
| Arrange largest number | 52.6 | 94.7 | |
| Fold clothes | 40.9 | 59.1 | |
| Hang mugs | 10.5 | 26.3 | |
| Make toast | 28.9 | 28.9 | |
| Pack objects into box | 2.3 | 6.8 | |
Table 4: Per-task hidden-layout success rates before and after survey-assisted improvement. is the absolute change in percentage points. Tested on 32 tasks.
| Method | Gen. | Prec. | Long. | Memory | Open | Avg. SR | Avg. score |
|---|---|---|---|---|---|---|---|
| Agent | |||||||
| PyRUA-Lean | 43.17 | 38.25 | 43.75 | 68.33 | 58.25 | 50.35 | 55.26 |
| HexaShell-Preview | 34.50 | 32.00 | 31.50 | 81.00 | 55.25 | 46.85 | 53.18 |
| EmbodiedEvo | 30.83 | 16.25 | 21.75 | 76.67 | 49.25 | 38.95 | 45.08 |
| GPT-6-Astra | 30.50 | 4.00 | 8.25 | 38.67 | 31.00 | 22.48 | 28.97 |
| DeepSeek-Flash | 1.67 | 0.00 | 0.00 | 1.67 | 6.25 | 1.92 | 2.99 |
Table 5: Full RoboDojo comparison from the leaderboard snapshot’s October 24, 2026 leaderboard snapshot. Capability columns report success rates (%); the final two columns weight the five capabilities equally. Best values in each column are bold. These are reported results, not a matched-seed comparison: our evaluation uses seed 0, whereas the official PyRUA-REAL evaluation uses seed 3.
More in this topic
Robot Skill Learning
Robot skill learning studies how robots acquire, adapt, and combine reusable behaviors from demonstrations, interaction, and other data.
See all 332 papers