RoboBridge: A Self-Evolving Embodied Agent Framework for Sim-to-Real Transfer

Chenxi Li; Zhangrui Zhao; Rui Li; Yuan Gao; Kehui Liu; Jiarui Li; Dong Wang; Tong Si

cs.ROOct 2, 2026

RoboBridge: A Self-Evolving Embodied Agent Framework for Sim-to-Real Transfer

Authors: Chenxi Li, Zhangrui Zhao, Rui Li, Yuan Gao, Kehui Liu, Jiarui Li, Dong Wang, Tong Si, +3 more

Organizations: Shanghai Artificial Intelligence Laboratory · Zhejiang University · Beihang University · Harbin Institute of Technology · Shenzhen Institutes of Advanced Technology · Shanghai Jiao Tong University

Abstract

A key challenge in bringing embodied intelligence into the real world is transferring capabilities from simulation to reality and enabling agents to continually adapt after deployment. End-to-end vision-language-action policies provide strong manipulation capabilities, but their transfer to physical environments typically relies on calibrating simulated visual and dynamical conditions, collecting additional target-domain demonstrations, and optimizing the policy through further training. Tool-using embodied agents offer flexible task orchestration, yet existing systems primarily emphasize task execution and experience reuse within a given environment, with limited support for transferring procedural knowledge and continuously adapting it across simulation and reality. We propose RoboBridge, a framework that treats sim-to-real transfer as the continued adaptation of executable task skills. The agent represents task knowledge as procedures connecting task intent, observations, tool operations, and outcome verification. Interaction feedback is used to generate candidate skill revisions, which are evaluated before being persisted or rejected. A pretrained vision-language-action policy is exposed as a reusable action tool and enhanced with inference-time guidance, enabling fine-grained execution without retraining the underlying policy. RoboBridge grounds transferable skills in task semantics and interaction interfaces shared across simulation and reality. This representation preserves reusable task structure while allowing environment-dependent operations to be selectively revised through real-world execution feedback. We evaluate the framework on LIBERO-PRO and corresponding physical tasks, studying both skill evolution and post-transfer adaptation. Our framework provides a route from one-shot policy deployment to continual procedural learning across environments.

Figures & tables

Figure 1: Sim-to-real transfer with end-to-end models and RoboBridge. The policy-centered pathway adapts an end-to-end model for real-world execution through physical alignment, additional target-domain data, and retraining. RoboBridge evolves procedural skills through a feedback-driven execution loop in simulation and transfers them through aligned task semantics and compatible observation and tool interfaces. After deployment, real-world execution feedback drives further skill evolution, refining the transferred procedures for the target environment.

Figure 2: RoboBridge framework. (a) Agent loop: The agent consults the active skill, invokes manipulation tools, and uses observations and execution feedback to guide subsequent actions. (b) Skill evolution: Execution logs inform candidate insertions, deletions, and replacements in the skill. Candidates are evaluated through task execution in the current environment, with validated improvements retained and unsuccessful updates rejected. (c) Self-guided action tool: When needed, the agent configures a reward function to guide action refinement during denoising, adapting execution to the current subtask while keeping the pretrained model parameters fixed.

	Instruction redirection (T)				Position swap (S)
Method	Spat	Object	Goal	Long	Spat	Object	Goal	Long	Overall
OpenVLA ( Kim et al., 2024 )	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
$\pi_{0}$ ( Black et al., 2024 )	0.0	0.0	0.0	0.0	0.0	2.0	0.0	0.0	0.3
$\pi_{0.5}$ ( Intelligence et al., 2025 )	1.0	1.0	2.0	1.0	20.0	17.0	38.0	8.0	11.0
MolmoAct ( Lee et al., 2025 )	0.0	0.0	0.0	6.0	0.0	6.0	0.0	0.0	1.5
NORA ( Hung et al., 2025 )	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0

Table 1: Success rates (%) on LIBERO-PRO ( Zhou et al., 2026 ) across SPATIAL, OBJECT, GOAL, and LIBERO-10 under instruction-redirection (T) and position-swap (S) perturbations. Our evaluation uses 100 trials per suite–perturbation pair (10 tasks

\times

10 seeds). Spat and Long denote SPATIAL and LIBERO-10, respectively. Dashes indicate unreported results. Overall scores for prior methods are reproduced as reported, with differing suite coverage where results are unavailable. Bold indicates the highest reported score in each column.

Figure 3: Ablation study on SPATIAL-T and OBJECT-T in LIBERO-PRO. We compare the full framework with four variants: without skill guidance, with append-only updates that exclude revision and deletion, without validation before skill persistence, and without inference-time policy guidance. Bars report success rates (%), with the full framework achieving 83% and 94% on the two suites, respectively.

Figure 4: Real-robot evaluation tasks. Left: Placing two cubes into their respective target bowls. Middle: Placing a cake into a pot and covering it with the lid. Right: Inserting a pen into a holder.

Figure 5: Real-robot performance and adaptation efficiency. (a) Real-robot performance: Success counts out of

15

trials on three tasks without skill guidance, with direct transfer of simulation-evolved skills, and after

10

episodes of real-robot skill evolution. (b) Adaptation efficiency: Under a separate experimental setting on the cake placement and covering task, simulation-initialized skill evolution reaches

15/15

after

12

iterations, compared with

16

iterations from an empty skill state. Each iteration consists of one physical execution episode followed by one skill update. (c) Token consumption: Cumulative token usage for the same task and setting as (b), measured at the respective

12

- and

16

-iteration endpoints. Annotations report mean total, input, and output tokens per evolution episode. M denotes million tokens.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Field	Contents
observation_id episode_step	Unique observation identifier and the corresponding action-tool step.
coordinate_frame units	Reference frame and measurement units: simulation world frame and metres.
images	External and wrist RGB views and their annotated counterparts, with availability, camera, capture step, resolution, and content references.
entities	Entity identifiers, visual labels, visibility, available geometric estimates, and measured reference points with provenance.
relations	Supported relations between entity identifiers.
robot	End-effector position and orientation, gripper state, and gripper opening.

Appendix

Table 2: Structured observation provided to the planner in simulation. The snapshot combines observation identity, coordinate conventions, external and wrist image metadata, scene entities and their measured geometry, robot proprioception, and validity information. Positions are expressed in the simulation world frame in metres. Optional estimates are included when available. Task instructions and execution feedback accompany the snapshot as separate inputs.

Function	Tool names	Role
Cartesian motion	Cartesian Move Pose Alignment	Approach, transfer, and positioning with optional orientation adjustment.
Orientation adjustment	Wrist Rotation Wrist Tilt	Adjust end-effector orientation while maintaining its position.
Gripper control	Open Gripper Gripper Control	Open or actuate the gripper for object handling.
Policy-based manipulation	Policy Grasp Policy Contact	Execute learned grasping and contact behaviors, with agent-configured inference-time guidance when needed.

Appendix

Table 3: Functional organization of the action library. Tool names describe their roles in task execution. Geometric tools provide explicit motion and gripper control, while policy-based tools support learned manipulation with inference-time guidance configured by the agent when needed.

Explore similar work

Skill2Real: Agentic Skill Learning for Zero-Shot Sim-to-Real Robot Manipulation
Oct 2, 2026Xincheng He, Siyu Ma, Chang Yu +5Robot Skill Learning Sim-to-Real Transfer
University of California, Los Angeles · Shanghai Jiao Tong University · University of Utah +1
EmbodiedSkills: A Unified Framework for Orchestrating, Training, and Deploying VLA Agents
Sep 1, 2026Wei Wang, Wenqiao Zhang, Yutong Lin +14Robotic Control Large Language Model-Based Robot Planning
College of Computer Science and Technology, Zhejiang University · Nanjing University of Aeronautics and Astronautics · Cornell University +3
RoboBRIDGE: A Modular Framework for Bridging Policies to Robust Real-World Robotic Agents
Jul 30, 2026Sihyung Yoon, Minjong Yoo, Sanghyun Ahn +2Long-Horizon Robotic Manipulation Robotics
Department of Computer Science and Engineering, Sungkyunkwan University

Oct 2, 2026, cs.ROJ/K move · Enter open · S save

Skill2Real: Agentic Skill Learning for Zero-Shot Sim-to-Real Robot Manipulation

Xincheng He, Siyu Ma, Chang Yu, Yunuo Chen+4

University of California, Los Angeles · Shanghai Jiao Tong University · University of Utah+1

Robot Skill Learning Sim-to-Real Transfer Robotic Manipulation Zero-Shot Robotic Manipulation LLM Agent Skill Learning

Open paper AskPDF arXiv

Transferring robotic skills from simulation to reality requires task knowledge that remains usable across differences in perception, dynamics, and embodiment. We introduce Skill2Real, an agentic policy framework that learns executable skills through a shared application programming interface (API). A Proposer-Verifier-Governor (PVG) loop uses privileged simulation evidence to diagnose outcomes and validate updates, while keeping learned skills grounded in public observations and API semantics. The Cerebellum first acquires local manipulation skills; the Brain then learns task-level composition with the Cerebellum frozen. Both memories transfer to the real robot without task-policy fine-tuning or skill-memory updates. As GPT-5.6 Sol learns skills on LIBERO-90, evaluating each frozen checkpoint with GPT-6 Astra raises LIBERO-Pro Long success from 2.0% to 56.3%, without training on Pro Long. Independent Robosuite training reaches 85.1% and 89.4% mean success with Sol and Opus 5 across seven tasks, respectively. Frozen Sol-trained LIBERO-90 skills achieve 78.75% mean completion across four real-world manipulation tasks with Astra. Removing the Verifier or Governor during LIBERO-90 training lowers final Pro Long success by 17.3 and 13.3 percentage points, respectively. These results support learning and transferring a hierarchy of executable skills through a common robot interface.

Abstract

Explore similar work

Skill2Real: Agentic Skill Learning for Zero-Shot Sim-to-Real Robot Manipulation

EmbodiedSkills: A Unified Framework for Orchestrating, Training, and Deploying VLA Agents

RoboBRIDGE: A Modular Framework for Bridging Policies to Robust Real-World Robotic Agents