cs.ROOct 2, 2026

RoboBridge: A Self-Evolving Embodied Agent Framework for Sim-to-Real Transfer

Authors: Chenxi Li, Zhangrui Zhao, Rui Li, Yuan Gao, Kehui Liu, Jiarui Li, Dong Wang, Tong Si, +3 more

Organizations: Shanghai Artificial Intelligence Laboratory · Zhejiang University · Beihang University · Harbin Institute of Technology · Shenzhen Institutes of Advanced Technology · Shanghai Jiao Tong University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

A key challenge in bringing embodied intelligence into the real world is transferring capabilities from simulation to reality and enabling agents to continually adapt after deployment. End-to-end vision-language-action policies provide strong manipulation capabilities, but their transfer to physical environments typically relies on calibrating simulated visual and dynamical conditions, collecting additional target-domain demonstrations, and optimizing the policy through further training. Tool-using embodied agents offer flexible task orchestration, yet existing systems primarily emphasize task execution and experience reuse within a given environment, with limited support for transferring procedural knowledge and continuously adapting it across simulation and reality. We propose RoboBridge, a framework that treats sim-to-real transfer as the continued adaptation of executable task skills. The agent represents task knowledge as procedures connecting task intent, observations, tool operations, and outcome verification. Interaction feedback is used to generate candidate skill revisions, which are evaluated before being persisted or rejected. A pretrained vision-language-action policy is exposed as a reusable action tool and enhanced with inference-time guidance, enabling fine-grained execution without retraining the underlying policy. RoboBridge grounds transferable skills in task semantics and interaction interfaces shared across simulation and reality. This representation preserves reusable task structure while allowing environment-dependent operations to be selectively revised through real-world execution feedback. We evaluate the framework on LIBERO-PRO and corresponding physical tasks, studying both skill evolution and post-transfer adaptation. Our framework provides a route from one-shot policy deployment to continual procedural learning across environments.

Figures & tables

Figure 1: Sim-to-real transfer with end-to-end models and RoboBridge. The policy-centered pathway adapts an end-to-end model for real-world execution through physical alignment, additional target-domain data, and retraining. RoboBridge evolves procedural skills through a feedback-driven execution loop in simulation and transfers them through aligned task semantics and compatible observation and tool interfaces. After deployment, real-world execution feedback drives further skill evolution, refining the transferred procedures for the target environment.
Figure 2: RoboBridge framework. (a) Agent loop: The agent consults the active skill, invokes manipulation tools, and uses observations and execution feedback to guide subsequent actions. (b) Skill evolution: Execution logs inform candidate insertions, deletions, and replacements in the skill. Candidates are evaluated through task execution in the current environment, with validated improvements retained and unsuccessful updates rejected. (c) Self-guided action tool: When needed, the agent configures a reward function to guide action refinement during denoising, adapting execution to the current subtask while keeping the pretrained model parameters fixed.
 Instruction redirection (T)Position swap (S) 
MethodSpatObjectGoalLongSpatObjectGoalLongOverall
OpenVLA ( Kim et al., 2024 )0.00.00.00.00.00.00.00.00.0
π0\pi_{0} ( Black et al., 2024 )0.00.00.00.00.02.00.00.00.3
π0.5\pi_{0.5} ( Intelligence et al., 2025 )1.01.02.01.020.017.038.08.011.0
MolmoAct ( Lee et al., 2025 )0.00.00.06.00.06.00.00.01.5
NORA ( Hung et al., 2025 )0.00.00.00.00.00.00.00.00.0
Table 1: Success rates (%) on LIBERO-PRO ( Zhou et al., 2026 ) across SPATIAL, OBJECT, GOAL, and LIBERO-10 under instruction-redirection (T) and position-swap (S) perturbations. Our evaluation uses 100 trials per suite–perturbation pair (10 tasks ×\times 10 seeds). Spat and Long denote SPATIAL and LIBERO-10, respectively. Dashes indicate unreported results. Overall scores for prior methods are reproduced as reported, with differing suite coverage where results are unavailable. Bold indicates the highest reported score in each column.
Figure 3: Ablation study on SPATIAL-T and OBJECT-T in LIBERO-PRO. We compare the full framework with four variants: without skill guidance, with append-only updates that exclude revision and deletion, without validation before skill persistence, and without inference-time policy guidance. Bars report success rates (%), with the full framework achieving 83% and 94% on the two suites, respectively.
Figure 4: Real-robot evaluation tasks. Left: Placing two cubes into their respective target bowls. Middle: Placing a cake into a pot and covering it with the lid. Right: Inserting a pen into a holder.
Figure 5: Real-robot performance and adaptation efficiency. (a) Real-robot performance: Success counts out of 1515 trials on three tasks without skill guidance, with direct transfer of simulation-evolved skills, and after 1010 episodes of real-robot skill evolution. (b) Adaptation efficiency: Under a separate experimental setting on the cake placement and covering task, simulation-initialized skill evolution reaches 15/1515/15 after 1212 iterations, compared with 1616 iterations from an empty skill state. Each iteration consists of one physical execution episode followed by one skill update. (c) Token consumption: Cumulative token usage for the same task and setting as (b), measured at the respective 1212 - and 1616 -iteration endpoints. Annotations report mean total, input, and output tokens per evolution episode. M denotes million tokens.
Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

FieldContents
observation_id episode_stepUnique observation identifier and the corresponding action-tool step.
coordinate_frame unitsReference frame and measurement units: simulation world frame and metres.
imagesExternal and wrist RGB views and their annotated counterparts, with availability, camera, capture step, resolution, and content references.
entitiesEntity identifiers, visual labels, visibility, available geometric estimates, and measured reference points with provenance.
relationsSupported relations between entity identifiers.
robotEnd-effector position and orientation, gripper state, and gripper opening.
Appendix
Table 2: Structured observation provided to the planner in simulation. The snapshot combines observation identity, coordinate conventions, external and wrist image metadata, scene entities and their measured geometry, robot proprioception, and validity information. Positions are expressed in the simulation world frame in metres. Optional estimates are included when available. Task instructions and execution feedback accompany the snapshot as separate inputs.
FunctionTool namesRole
Cartesian motionCartesian Move Pose AlignmentApproach, transfer, and positioning with optional orientation adjustment.
Orientation adjustmentWrist Rotation Wrist TiltAdjust end-effector orientation while maintaining its position.
Gripper controlOpen Gripper Gripper ControlOpen or actuate the gripper for object handling.
Policy-based manipulationPolicy Grasp Policy ContactExecute learned grasping and contact behaviors, with agent-configured inference-time guidance when needed.
Appendix
Table 3: Functional organization of the action library. Tool names describe their roles in task execution. Geometric tools provide explicit motion and gripper control, while policy-based tools support learned manipulation with inference-time guidance configured by the agent when needed.

Explore similar work

CardsList
Oct 2, 2026cs.RO

Skill2Real: Agentic Skill Learning for Zero-Shot Sim-to-Real Robot Manipulation

Transferring robotic skills from simulation to reality requires task knowledge that remains usable across differences in perception, dynamics, and embodiment. We introduce Skill2Real, an agentic policy framework that learns executable skills through a shared application programming interface (API). A Proposer-Verifier-Governor (PVG) loop uses privileged simulation evidence to diagnose outcomes and validate updates, while keeping learned skills grounded in public observations and API semantics. The Cerebellum first acquires local manipulation skills; the Brain then learns task-level composition with the Cerebellum frozen. Both memories transfer to the real robot without task-policy fine-tuning or skill-memory updates. As GPT-5.6 Sol learns skills on LIBERO-90, evaluating each frozen checkpoint with GPT-6 Astra raises LIBERO-Pro Long success from 2.0% to 56.3%, without training on Pro Long. Independent Robosuite training reaches 85.1% and 89.4% mean success with Sol and Opus 5 across seven tasks, respectively. Frozen Sol-trained LIBERO-90 skills achieve 78.75% mean completion across four real-world manipulation tasks with Astra. Removing the Verifier or Governor during LIBERO-90 training lowers final Pro Long success by 17.3 and 13.3 percentage points, respectively. These results support learning and transferring a hierarchy of executable skills through a common robot interface.

Xincheng He, Siyu Ma, Chang Yu +5

University of California, Los Angeles · Shanghai Jiao Tong University · University of Utah +1

Robot Skill LearningSim-to-Real TransferRobotic ManipulationZero-Shot Robotic ManipulationLLM Agent Skill Learning
AskPDF
Sep 1, 2026cs.RO

EmbodiedSkills: A Unified Framework for Orchestrating, Training, and Deploying VLA Agents

Vision-language-action (VLA) models map visual observations and language instructions directly to robot actions, but long-horizon tasks require more than action prediction. An agent must coordinate perception, planning, execution, progress verification, and recovery as the physical state evolves. An action prediction or a model-generated skill decision does not, by itself, guarantee that the proposed operation is valid in the current state or that its outcome will be verified. We propose EmbodiedSkills, a unified framework that treats each skill decision as an execution proposal: the runtime checks its prerequisites before execution and verifies the outcome afterward. A shared executable-skill interface connects high-level skill selection, bounded low-level VLA execution, and post-action verification within a single agent loop. Because this interface remains fixed, low-level VLA policies can be replaced or adapted without changing the agent loop. The interface also records planning, execution, verification, and recovery events as structured trajectories, which provide supervision for individual components and can support optional online adaptation when interactive feedback is available. We instantiate EmbodiedSkills with Qwen3-VL and OpenPI/pi0.5 on RoboTwin 2.0 and LIBERO. Task-adapted low-level VLA policies achieve an average success rate of 86.20% across 50 RoboTwin 2.0 tasks and 97.40% across the four LIBERO suites. These results establish the execution performance of the task-adapted low-level VLA policies used in EmbodiedSkills. On four memory-dependent RMBench tasks, the same task-adapted execution approach achieves 12.5% average success. The framework provides a trainable and inspectable agent layer for turning these policies into closed-loop embodied systems.

Wei Wang, Wenqiao Zhang, Yutong Lin +14

College of Computer Science and Technology, Zhejiang University · Nanjing University of Aeronautics and Astronautics · Cornell University +3

Robotic ControlLarge Language Model-Based Robot PlanningRoboticsVision-Language-Action Models
AskPDF
Jul 30, 2026cs.RO

RoboBRIDGE: A Modular Framework for Bridging Policies to Robust Real-World Robotic Agents

Vision-Language-Action (VLA) models have attracted growing interest as a scalable approach to robotic manipulation. While these models are effective action predictors, deploying them as robotic agents exposes critical gaps: no mechanism for failure recovery, inconsistent execution over long horizons, and limited robustness to shifts in observations, tasks, or embodiments. Existing solutions address these limitations individually through model retraining or environment-specific modules, yet what is needed is a general framework that systematically transforms a pretrained VLA into a robotic agent. We present RoboBRIDGE, a modular framework that provides an orchestration layer over five coordinated modules, namely Monitor, Perceptor, Planner, Controller, and Robot Interface, to compose robust robotic agents from off-the-shelf components, including pretrained VLAs. The Monitor pairs rapid failure detection with hierarchical recovery to correct errors before they cascade. When the environment diverges from the current plan, the Planner triggers replanning while the Perceptor updates scene understanding asynchronously, avoiding execution stalls. Within the Controller, primitive skill fine-tuning factors manipulation into domain-invariant primitives with dedicated LoRA adapters, reducing sensitivity to domain shifts when a VLA is used. Across LIBERO, RoboCasa, and real-world case studies spanning multiple robot platforms and VLA backbones, RoboBRIDGE consistently outperforms both standalone policies and prior augmented VLA deployments. These results suggest that reliable robotic agency does not arise from scaling action predictors alone, but from structured orchestration around them.

Sihyung Yoon, Minjong Yoo, Sanghyun Ahn +2

Department of Computer Science and Engineering, Sungkyunkwan University

Long-Horizon Robotic ManipulationRoboticsVision-Language-Action ModelsRobustness of VLA Models
AskPDF