cs.RONov 7, 2026

RoboMoCap: A Large-Scale Whole-Body Motion Capture Dataset with Egocentric Vision for Robot Learning

Authors: Mayur Deshmukh, Aaron Tung, Junyi Li, Nikash Bhardwaj, Ademi Adeniji, Vincent Liu, Michael Cho, Jiafei Duan

Organizations: MAGIC Lab, National University of Singapore · BitRobot · Grounded Superintelligence

Abstract

Humanoid robots must coordinate locomotion and manipulation to navigate complex environments and perform everyday tasks. Learning such behaviours from human demonstrations requires accurate whole-body motion paired with egocentric vision, particularly for lower-body movements and foot placements that are frequently invisible from a first-person perspective. However, existing egocentric datasets often lack high-fidelity optical motion capture or systematic coverage of diverse loco-manipulation behaviours. To address this gap, we introduce RoboMoCap, a large-scale multimodal dataset combining egocentric vision with marker-based optical motion capture, specifically designed for locomotion-centric humanoid learning. RoboMoCap comprises over 200 hours of recordings from 14 participants, featuring six head-mounted cameras, wrist-mounted visual and inertial sensing, and optical full-body motion capture at 120 Hz. The dataset systematically covers diverse locomotion primitives, dynamic whole-body movements, and loco-manipulation tasks, including variable-speed walking, turning, constrained navigation, terrain traversal, object transportation, and manipulation during locomotion. It provides temporally aligned multimodal observations and metric whole-body motion representations, enabling the study of human movement beyond directly observable body regions. We systematically evaluate existing methods on three benchmark tasks: egocentric full-body motion reconstruction, foot contact and placement estimation, and human-to-humanoid motion retargeting. By pairing egocentric vision with high-fidelity human motion across diverse locomotion and interaction scenarios, RoboMoCap provides a foundation for egocentric motion understanding and humanoid learning from human demonstrations.

Figures & tables

Explore similar work

CardsList
  1. EgoExplorer: A Vision-Motion-Touch Observatory for Human Loco-Manipulation In Situ

    Oct 13, 2026Yunsong Zhou, Xuekun Jiang, Haiqi Shen +113D Human ReconstructionWearable Sensing

  2. EgoLive: A Large-Scale Egocentric Dataset from Real-World Human Tasks

    Apr 26, 2026Yihang Li, Xuelong Wei, Jingzhou Luo +26Robotic Data CollectionRobotic Manipulation

  3. Humanoid Everyday: A Comprehensive Robotic Dataset for Open-World Humanoid Manipulation

    Oct 9, 2025Zhenyu Zhao, Hongyi Jing, Xiawei Liu +7Robot Manipulation DatasetsRobotic Data Collection