HuMI: Humanoid Whole-Body Manipulation from Humanoid-Free Demonstrations
Organizations: Tsinghua University · Shanghai Qi Zhi Institute · Shanghai Jiao Tong University
Abstract
Current approaches for humanoid whole-body manipulation, primarily relying on teleoperation or visual sim-to-real reinforcement learning, are hindered by hardware logistics and complex reward engineering. Consequently, demonstrated autonomous skills remain limited and are typically restricted to controlled environments. In this paper, we present the Humanoid Manipulation Interface (HuMI), a portable and efficient framework for learning diverse whole-body manipulation tasks across various environments. HuMI enables humanoid-free data collection by capturing rich whole-body motion using portable hardware. This data drives a hierarchical learning pipeline that translates human motions into dexterous and feasible humanoid skills. Extensive experiments across five whole-body tasks--including kneeling, squatting, tossing, walking, and bimanual manipulation--demonstrate that HuMI achieves a 3x increase in data collection efficiency compared to teleoperation and attains a 70% success rate in unseen environments.
Figures & tables
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| Reward Term | Equation | Weight |
| Tracking Rewards | ||
| Whole-body position tracking | 1.0 | |
| Whole-body rotation tracking | 1.0 | |
| Whole-body linear velocity tracking | 1.0 | |
| Whole-body angular velocity tracking | 1.0 | |
| Adaptive EE position tracking | ||
| Term | Description | Sampling Range |
|---|---|---|
| Physical randomization | ||
| Static friction | randomize static friction of robot bodies | |
| Dynamic friction | randomize dynamic friction of robot bodies | |
| Restitution | randomize restitution of robot bodies | |
| Default joint positions | add offsets to default joint positions | |
| CoM offsets | randomize the torso link center of mass | , |
| Hyperparameter | Value |
|---|---|
| Visual observation horizon | 1 |
| Visual observation frequency | 20 Hz |
| Proprioceptive observation horizon | 3 |
| Proprioceptive observation frequency | 20 Hz |
| Action horizon | 48 |
| Action frequency | 20 Hz |
Whole-Body Robotic Manipulation
Whole-Body Robotic Manipulation coordinates a robot’s arms, hands, body, and mobility to manipulate objects while maintaining balance and contact.
See all 102 papers