cs.RONov 18, 2026

Ego2X: Contact-aware Transfer from Egocentric Human Data to Any Robot

Authors: Xinke Lyu, Bowen Yang, Zhicheng Wang

Organizations: Beijing University of Posts and Telecommunications · Institute of Automation, Chinese Academy of Sciences · PowerTech(Mite.)

Abstract

Embodied models depend on real interaction data, yet collecting it on physical robots is slow and labor-intensive. Recent ego-to-robot pipelines retarget first-person view human videos to robot data via transferring hand and arm motion. They judge fidelity mainly by visual appearance. However, trajectories often fail in terms of physics: fingers hover beside objects, and grippers close on empty space. We argue that how the object moves and the contacts that drive the motion count. Thus, we present Ego2X, an object-centric framework that converts monocular egocentric video into physically executable robot data across dexterous hands and parallel grippers. Ego2X centers on CORE (Contact-centric Object-interaction REpresentation), which records each contact on the object by its surface anchor, as well as relative force, grasp mode, and interaction state. Throughout the pipeline, CORE corrects hand-object drift via object-relative constraints, steers inverse kinematics toward contacts the robot must realize, and preserves held objects during arm erasure and robot compositing. On HOT3D, contact-driven correction reduces in-contact hand depth error by 33.7%, while on EgoDex, contact-driven retargeting realizes 62.9% of annotated contacts, surpassing the official HuRo pipeline (27.9%). In RoboTwin 2.0, post-training X-VLA on Ego2X episodes alone, without simulator demonstrations, raises task success rate from 40.5% zero-shot to 67.0% across 200 paired scenes. We will publicly release our conversion pipeline and CORE-annotated dataset for dexterous hands and parallel grippers.

Figures & tables

Explore similar work

CardsList
  1. EgoMatrix: Physically Faithful Dexterous Robot Demonstrations from Egocentric Human Videos

    Oct 13, 2026Ziheng Wang, Yanming Shao, Ting Mao +4Hand-Object Interaction ReconstructionRobotic Data Collection

  2. EgoCPT: Learning Dexterous Manipulation from a Single Ego-Centric Video with Contact-Aware Optimization

    Oct 13, 2026Zhixin Sun, Kairui Shi, Tianao Ren +3Robotic ManipulationRobot Imitation Learning

  3. EgoJEPA: Beyond Retargeting for Dexterous Manipulation from Egocentric Videos

    Oct 14, 2026Meng Cao, Can Zhao, Jinjie Wei +12Egocentric Video Learning for RoboticsRobotic Manipulation