cs.RONov 11, 2026

From a Single RGB Image to Robot Execution: Metric Scene Alignment and Contact-Guided Retargeting

Authors: Zhenxuan Ding, Yu Huang, Xinmiao Wang, Chenghao Sun, Ziyang Dong, Wenhao Cui

Organizations: Beijing University of Chemical Technology · National University of Defense Technology · Noetix Robotics

Abstract

Extending humanoid interaction generation to new environments remains limited by the need for prebuilt 3D scene assets. Reconstructing scenes directly from single RGB images offers a way to overcome this limitation, but introduces a key challenge in maintaining geometric consistency throughout motion generation and transfer. Visually plausible human motions may be misaligned with the reconstructed scene, while retargeting can introduce additional support errors due to differences between human and robot embodiments. We present a unified pipeline that transforms a single RGB image into a humanoid motion reference. It reconstructs scene geometry from the input image, generates a human interaction conditioned on the reconstructed scene, and retargets the motion to a humanoid robot. To maintain geometric consistency throughout this process, we incorporate two complementary refinement stages before and after embodiment transfer. Before retargeting, Joint Metric Interaction Refinement (JMIR) refines human motion alignment with the registered metric scene while preserving the generated interaction structure. Following retargeting, Contact Guided Residual Retargeting (CGRR) corrects robot specific support errors to restore the intended support relationships. Together, these stages produce motion references that are consistent with the reconstructed scene and can be followed by multiple pretrained humanoid trackers without scene specific policy fine tuning. Compared with GRAIL, our method reduces refinement time from 480 s to 87 s, yielding a speedup of 5.5 times. The complete pipeline achieves a success rate of 89% in simulation and 70% across 50 physical trials on the Unitree G1, demonstrating its effectiveness in generating executable humanoid motion references from single RGB images.

Figures & tables

Explore similar work

CardsList
  1. Human2Humanoid: Physics-Aware Cross-Morphology Motion Retargeting for Humanoid Robots

    Jun 2, 2026Tianchen Huang, Feiyang Yuan, Junchi Gu +5Cross-Embodiment Robot LearningMotion Retargeting

  2. OTRetarget: Joint Robot and Object Motion Retargeting via Optimal Transport

    Sep 29, 2026Guillaume Besset, Erwann Carn, Timothée Carecchio +5Robotic Loco-ManipulationContact-Rich Robotic Manipulation

  3. HOI-Retarget: Contact-Centric Retargeting for Human-Object Interaction

    Sep 28, 2026Jihwan Shin, Adrià López Escoriza, Junzhe He +2Contact-Rich Robotic ManipulationSynthetic Data Generation for Robotics