Reconstruct, Practice, Go Real: Guided Self-Improvement for Embodied Agents

Yen-Jen Wang; Haozhe Jiang; Shuying Deng; Haoru Xue; Weirui Ye; Rocky Duan; Nika Haghtalab; S. Shankar Sastry

cs.ROOct 1, 2026

Reconstruct, Practice, Go Real: Guided Self-Improvement for Embodied Agents

Authors: Yen-Jen Wang, Haozhe Jiang, Shuying Deng, Haoru Xue, Weirui Ye, Rocky Duan, Nika Haghtalab, S. Shankar Sastry, +2 more

Organizations: UC Berkeley · Amazon FAR · MIT · University of Chicago

Abstract

Building reliable robot capabilities across diverse tasks requires substantial human effort to develop and maintain skills, design rewards, and integrate perception with control. We present Reconstruct, Practice, Go Real (RPG), a framework for autonomous improvement of robot execution systems without updating model weights. RPG identifies manipulation capabilities in an offline dataset and constructs related practice tasks in simulation. During practice, RPG uses execution feedback, privileged simulator state, and available dataset videos to diagnose failures. It develops new reusable symbolic skills, refines existing skills, and revises the system prompt based on these diagnoses. Cross-task evaluation tests individual candidate changes and merged revisions before they are retained for reuse. At test time, a multimodal LLM uses the resulting system prompt and skill library to coordinate perception and robot control. On held-out initializations of 22 manipulation tasks, RPG improves task success from 28.6% after the first practice round to 95.0% after 15 rounds, outperforming all evaluated baselines, including ASPIRE (75.5%) and CaP-Agent0 powered by GPT-6 Astra Pro (60.0%). After a common calibration and hardware-adaptation procedure, the frozen system succeeds in all 30 physical trials, with ten trials on each of three tasks. Project Website: https://rpg-robot.github.io/

Figures & tables

Fig. 2: RPG uses privileged execution and video analysis to guide skill development. The top row shows source scenes alongside representative simulation practice tasks. The Runtime Agent and Privileged Agent share the same model and skill library, but only the latter receives simulator state $S_{t}$ in addition to robot observations $O_{t}$ . For development tasks, the Video Analyzer compares the two agents’ execution records with available videos from the offline dataset to diagnose failures. The Implementor develops new skills, refines existing skills, and revises the system prompt. Candidate changes are evaluated across tasks. The Merger combines selected revisions, and the resulting system is retested before retention.

Fig. 3: Case study of self-improvement on bottle placement. (1) The Runtime Agent attempts the task by calling the shared skill pick_and_place_tall_object() , but the rollout fails when the bottle collides with the bin wall. (2) Failure diagnosis attributes the error to insufficient placement-path validation and proposes a more robust procedure: validate the full trajectory, release only after the bottle is safely inside the bin, and verify the final placement. (3) The revised skill improves task success from 1/5 to 5/5 on the development trials. The candidate is eligible for merging only if mean task success increases while no individual task drops by more than 20 percentage points. Selected eligible revisions are then merged, and the merged system is retained only if it passes the same cross-task validation.

Source task	Constructed practice task
Organize sunglasses	Close the case; opening and loading are encoded in the initial state.
Organize makeup	Close the drawer; preceding manipulation is already satisfied.
Roll towels	Fold once; a cloth manipulation analogue with a different final shape.
No corresponding source task	Transfer an object between arms using a task specification.

TABLE I: Representative correspondences between source tasks in the offline dataset and constructed practice tasks.

Fig. 4: Task completion and revision activity. (A) Round-end systems are evaluated retrospectively on the same 22 tasks

\times

10 held-out seeds after development is frozen. Dashed lines show ASPIRE, RATs, and the matched-model and strongest CaP-Agent0 configurations (abbreviated CaP). Table II gives the full model comparison. (B) Stacked bars count accepted skill additions (green) and modifications (light blue) in each round. Purple squares mark rounds in which the Runtime Agent’s system prompt was revised (rounds 1–4); their height does not encode a count. The curve represents one improvement run. Rounds do not represent matched computational budgets.

Video Analyzer	Privileged Agent	Success
Yes	Yes	78.2%
No	Yes	51.8%
Yes	No	50.9%
No	No	50.9%

TABLE III: Five-round diagnostic ablations from the same 15-skill initialization. Each variant is evaluated on 220 held-out episodes.

Fig. 5: Physical task execution. Representative execution sequences on the physical YAM robot. The first three rows show the quantitatively evaluated tasks: Store Ball in Drawer, Fold Towel, and Transfer Bowl. Bowl transfer uses table-supported staging. The remaining rows show additional qualitative deployments on Insert Cylinder, Serve Fruit onto Plate, and Sort Utensils. Table V reports outcomes for the three quantitative tasks.

Task	Before repair	After repair
Two-arm lift	6/10	10/10
Place dish on rack	5/10	9/10
Nut assembly	5/10	8/10
Place bottles in bin	10/10	10/10
Total	26/40	37/40
Success rate	65.0%	92.5%

TABLE IV: Library-only comparison before and after the library revisions, with ten paired initializations per task. The runtime remains fixed, and only the library changes.

	CaP-Agent0		RPG
Task / Stage	Gemini	GPT-6	Gemini
Store ball in drawer	3/10	4/10	10/10
Place ball in drawer	7/10	6/10	10/10
Fold towel	0/10	9/10	10/10
Transfer bowl	0/10	1/10	10/10
Move bowl to center	6/10	8/10	10/10

TABLE V: Physical outcomes after a common calibration and hardware-adaptation procedure. Each method receives ten trials per task. Intermediate-stage and complete-task outcomes are measured in the same trials.

Fig. 6: Qualitative zero-shot real-world deployment. Representative execution sequences for five additional physical tasks: Open Scissors, Uncap Marker, Pull out Tissue, Erase Whiteboard, and Unscrew Bottle Cap. The frozen RPG system receives the task instruction and executes each task without task-specific real-world tuning, demonstrations, or retry-based adaptation. These examples are qualitative and are not included in the quantitative baseline comparison.

Task	Split	R1	R2	R3	R4	R5	R6	R7	R8	R9	R10	R11	R12	R13	R14	R15
Extract from fixture	D	2/10	5/10	5/10	10/10	10/10	10/10	9/10	10/10	9/10	10/10	9/10	10/10	8/10	10/10	10/10
Insert into fixture	D	2/10	3/10	4/10	7/10	9/10	9/10	8/10	10/10	9/10	10/10	10/10	8/10	10/10	9/10	10/10
Place dish on rack	D	0/10	0/10	0/10	2/10	3/10	6/10	5/10	5/10	8/10	4/10	9/10	8/10	9/10	9/10	10/10
Serve onto plate	H	1/10	5/10	8/10	10/10	10/10	10/10	10/10	10/10	10/10	10/10	10/10	10/10	10/10	10/10	9/10
Sort cubes (extended)	D	0/10	0/10	0/10	3/10	2/10	8/10	10/10	10/10	9/10	10/10	10/10	9/10	10/10	8/10	10/10
Lift cube	D	10/10	10/10	10/10	10/10	10/10	10/10	10/10	10/10	10/10	10/10	10/10	10/10	10/10	10/10	10/10

TABLE VI: Round-by-round task performance on ten held-out seeds per task. Task entries are successes out of ten episodes. D, V, and H denote development, validation, and feedback-held-out tasks, respectively. Feedback-held-out tasks do not generate direct task-specific improvement feedback but remain in cross-task regression validation. The final row reports the overall success rate across the 22-task evaluation (

220

episodes per round). These numbers are not the five-rollout in-loop candidate-evaluation statistics used during self-improvement.

Task	R1	R4	R8	R10	R11	R12	R15
Serve onto plate	1/10	10/10	10/10	10/10	10/10	10/10	9/10
Two-arm handover	3/10	9/10	7/10	9/10	9/10	7/10	10/10
Transfer object	4/10	10/10	10/10	10/10	10/10	10/10	9/10
Close sunglasses case	0/10	10/10	10/10	8/10	10/10	10/10	10/10
Grasp plate	2/10	7/10	9/10	10/10	10/10	6/10	6/10

TABLE VII: Performance on tasks without direct task-specific improvement feedback across representative rounds under the 10-seed held-out evaluation. Failures on these tasks are not analyzed by the Video Analyzer and do not generate candidate revisions. The first four tasks remain in cross-task regression validation. Grasp plate belongs to a separate no-feedback evaluation grid and is shown here only as an additional transfer test.

Failure mode	Typical failure	Learned correction	Representative tasks
Localization and geometry uncertainty	The policy acts on a stale, ambiguous, or poorly localized object, receptacle, or support surface.	Re-observe after scene changes; reconstruct geometry from multiple visible points and depth; invalidate stale targets after contact or failed motion.	Place bottles in bin, Insert into fixture, Place dish on rack
Grasp identity and retention	A blocked or object-sized gripper opening is treated as proof that the desired object is held, even when the grasp is empty, mixed, or on the wrong object.	Combine gripper state with wrist/top observations and a short test lift; require visible target co-motion before transport; preserve uncertain grasps rather than opening blindly.	Extract from fixture, Nut assembly, Stack blocks
Incomplete path feasibility	An endpoint is reachable, but an approach, lift, transit, release, or retreat waypoint along the actual manipulation route is not.	Check the complete manipulation path before committing, including approach, grasp, lift, transport, release, and retreat, using the actual arm configuration and yaw.	Place bottles in bin, Place dish on rack, Stack blocks
End-effector-centric geometry	The fingertip reaches the nominal target while the relevant object feature does not: e.g., the bottle misses the opening or the nut hole misses the peg.	Track the geometry of the manipulated object and its task-relevant feature; estimate object-to-gripper offsets and align the object, hole, bottom surface, or footprint rather than the fingertip.	Place bottles in bin, Nut assembly, Insert into fixture
Contact and support calibration	Motion is commanded relative to the fingertip without accounting for pad extension, object extent, table height, or support geometry, causing penetration or ineffective contact.	Estimate the support surface and relevant tool/object offset; use surface-relative heights and small monitored increments instead of large open-loop contact motions.	Spill wipe, Fold towel
Multi-effector coordination	Each arm is controlled locally without verifying that both contacts are valid or that the required relation between the two arms is maintained.	Verify both grasps, perform a paired test motion, preserve relative geometry, and use synchronized incremental motions with an explicit relation tolerance.	Two-arm lift

TABLE VIII: Qualitative taxonomy of recurring manipulation failures and the reusable corrections learned from them. Many corrections introduce explicit preconditions , verification , or closed-loop recovery , rather than merely changing a Cartesian target.

	R1	R2	R3	R4	R5	R6	R7	R8	R9	R10	R11	R12	R13	R14	R15
Fold towel	0/10	1/10	0/10	0/10	0/10	1/10	4/10	5/10	4/10	4/10	3/10	3/10	3/10	4/10	3/10
Fold towel (hard)	0/10	0/10	0/10	0/10	0/10	0/10	0/10	0/10	0/10	0/10	0/10	0/10	0/10	0/10	0/10

TABLE IX: Towel-folding performance on ten held-out seeds across self-improvement rounds. Fold towel (hard) tightens the required corner alignment from 5 cm to 1 cm and is evaluated separately from the main 22-task benchmark.

Split	# Tasks	Examples
Development	17	insertion, stacking, wiping, two-arm lift
Validation	1	restack cubes
Feedback-held-out	4	serving, handover, case closing

TABLE X: Composition of the 22-task simulation benchmark.

Explore similar work

ASPIRE: Agentic /Skills Discovery for Robotics
Jun 30, 2026Runyu Lu, Yubo Wu, Ethan Kou +11Human-To-Robot Transfer Robotics
NVIDIA · UMich · UIUC +2
Explore, Execute, Evolve: A Skill Acquisition and Reuse Loop for Embodied Agents
Sep 29, 2026Sicheng Xie, Yitong Chen, Haidong Cao +3Human-To-Robot Transfer Embodied Agents
Institute of Trustworthy Embodied AI, Fudan University · Shanghai Innovation Institute · NeoteAI.
RAPID: Robot Agentic Programming from Demonstrations
Sep 24, 2026Yuyao Liu, Jiayuan Mao, David Hsu +2
Massachusetts Institute of Technology · National University of Singapore · University of Pennsylvania +1

Jun 30, 2026, cs.ROJ/K move · Enter open · S save

ASPIRE: Agentic /Skills Discovery for Robotics

Runyu Lu, Yubo Wu, Ethan Kou, Letian Fu+10

NVIDIA · UMich · UIUC+2

Human-To-Robot Transfer Robotics Social Robot Navigation Continual Learning Methods

Open paper AskPDF arXiv

Traditional robot programming is challenging: it requires orchestrating multimodal perception, managing physical contact dynamics, and handling diverse configurations and execution failures. We introduce ASPIRE (Agentic Skill Programming through Iterative Robot Exploration), a continual learning system that autonomously writes and refines robot control programs in a code-as-policy paradigm while compounding experience into a reusable skill library. ASPIRE discovers skills that persist across tasks, simulation and real-world settings, and embodiments. It operates in an open-ended loop with three components: (1) a closed-loop robot execution engine that exposes fine-grained multimodal traces, enabling autonomous failure diagnosis, repair synthesis, and validation; (2) a continually expanding skill library that distills validated fixes into reusable, transferable knowledge; and (3) evolutionary search that generates diverse task sequences and control programs to explore beyond single-trajectory refinement. ASPIRE surpasses prior methods by up to 77% on LIBERO-Pro manipulation under perturbation, 72% on Robosuite bimanual handover, and 32% on BEHAVIOR-1K long-horizon household tasks. Its accumulated library also enables zero-shot generalization to unseen long-horizon tasks: on LIBERO-Pro Long, ASPIRE achieves 31% success versus 4% for prior methods despite their use of test-time reasoning and retries. Finally, simulation-discovered skills provide initial evidence of sim-to-real transfer, substantially reducing real-robot programming effort across different embodiments and robot APIs.

Abstract

Explore similar work

ASPIRE: Agentic /Skills Discovery for Robotics

Explore, Execute, Evolve: A Skill Acquisition and Reuse Loop for Embodied Agents

RAPID: Robot Agentic Programming from Demonstrations