HandUMI: A Modular Hand-Worn Parallel-Jaw Interface for Robot-Free Bimanual Data Capture
Organizations: Mu Robotics · McGill University · Mila - Quebec AI Institute · Pontificia Universidad Católica del Perú (PUCP) ·
Abstract
HandUMI captures robot-free bimanual demonstrations with a hand-worn parallel-jaw interface. Swappable fingertips match the robot's installed gripper; Quest/PICO controllers track hands and encoders measure aperture. Robot-agnostic recordings are retargeted by inverse kinematics to seven simulated bimanual arms (median position error mm; % of episodes rejected). On bimanual towel folding, 562 robot-free demonstrations train policies on real PiPER arms: ACT succeeds in 12/20 attempts and fine-tuned in 7/20; ACT on the first 101 and base each succeed in 0/20. Hardware and software are open source (Apache-2.0), and a pair costs USD 557 with Quest 3S, excluding the optional ZED Mini and host computer. Project website: https://murobotics.ai/research/handumi.
Figures & tables
| Device | Pose source | Enc. | Bimanual | Open HW | Open SW | Tips | Audio | USD |
|---|---|---|---|---|---|---|---|---|
| UMI [ 7 ] | mono-inertial SLAM | – | – | 742 a | ||||
| UMI-3D [ 28 ] | LiDAR-inertial odom. | – | – | 1,400 b | ||||
| FastUMI [ 37 ] | T265 VIO | – | c | – | 2,042 d | |||
| DexUMI [ 30 ] | iPhone ARKit | – e | n/a | – | 2,448 f | |||
| iPhUMI [ 22 ] | iPhone ARKit | – | g | 1,504 h | ||||
| RDT2 [ 17 ] | Vive Tracker (IR) | – | – | i | – | – | 1,750 j |