Universal Manipulation Interface (UMI) devices allow human experts to demonstrate tasks for robot learning without teleoperating physical robots. Using these demonstrations as training data still requires a consistent hardware setup, calibration, and data processing. We present UMI-Live, a full-stack open-source bimanual UMI system for in-the-wild data collection. We design our custom UMI-Live hardware to integrate camera and LiDAR--inertial sensing, onboard compute, and power, without external tracking equipment or calibration within the collection environment. We implement a simple user interface for recording data while an asynchronous pipeline performs parallel SLAM reconstruction and publishes training-ready datasets. We also release software that supports dual-YAM teleoperation and VLA policy training and inference. We validate UMI-Live data quality by fine-tuning π0.5 and MolmoAct2, achieving success rates of 10/10 and 8/10, respectively on a bimanual pick-and-place task. Furthermore, we release 40 hours of UMI-Live data across three tasks, and complete hardware designs, software, and reproduction guides at our project website.
Figures & tables
Figure 1: UMI-Live connects in-the-wild bimanual collection and automated processing to LeRobot datasets, policy training, and physical evaluation on dual i2RT YAM arms.
Figure 2: Assembled UMI-Live hardware. (a) Front view showing the wrist camera and ArUco tags. (b) Side view showing the operator-facing buttons and battery module. (c) Rear view showing the LCD and cable routing.
Figure 3: Exploded CAD view of one UMI-Live unit with its main components labeled.
Figure 4: Redesigned gripper fingers and their installation on a YAM arm.
Specification
Value
Mass per unit with battery
1.42 kg
Gripper width
80 mm
Runtime per charge
1.5 h
Hardware cost per pair
$1,930
Table 1: Hardware specifications of the evaluated revision.
Figure 5: LCD interface with example values.
Figure 6: Data collection and processing in UMI-Live. (a) Each gripper i∈{L,R} records wrist images Ii , LiDAR data Pi , and IMU data Ui . (b) Raw episodes R(e) are processed in parallel to estimate jaw widths wti , reconstruct poses Tti , and align the streams. (c) We assemble demonstration trajectories T(e) with wrist images, 14-dimensional bimanual states, and task instructions into dataset D . Training samples use observations ot=(ItL,ItR,gtL,gtR,ℓ) and query-relative action chunks At∈RH×20 , as defined in Appendices E and F .
Figure 7: Handheld collection and physical robot evaluation. (a) Wrist-camera views at two moments in a UMI demonstration collected at a different location and under different lighting from the robot evaluation, brightened for display. (b) Dual i2RT YAM arms with UMI-Live gripper fingers, six oranges, and a bowl. (c) Selected frames from one autonomous π0.5 rollout that ends with all six oranges in the bowl. Frames are ordered from left to right. Appendix C , Figure 13 , shows the full sequence.
Policy
Training condition
UMI / teleoperation
Successful trials
MolmoAct2
Released checkpoint
—
0/10
π0.5
Fine-tuned
95% / 5%
9/10
π0.5
Fine-tuned
90% / 10%
10/10
MolmoAct2
Fine-tuned
75% / 25%
8/10
Table 2: Physical orange-to-bowl evaluation on dual i2RT YAM arms. Every fine-tuned condition starts from its released checkpoint and receives 12,000 updates with a global batch size of 32. Success requires placing all six oranges in the bowl.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Components used to assemble a UMI-Live gripper, including the LiDAR, camera, Raspberry Pi, LCD, printed parts, linear guide, gears, fasteners, and battery and power electronics.
Figure 9: Mechanical and electronic components of UMI-Live. (a) Gripper mechanism with sector gears, bearing-supported pivots, and a linear guide. (b) Trigger and return spring within the handle. (c) Removable power module with the battery, DC-DC converter, power switch, and wiring. (d) Onboard electronics, showing the Raspberry Pi and LCD before LiDAR installation (left), operator-facing buttons and the rear-mounted LCD (center), and the front camera beneath the LiDAR (right).
Figure 10: Small-object grasping and insertion with UMI-Live. Each row begins at tool pickup and follows the handoff between the grippers, insertion into the tool case, and release. Six uncropped frames per row are selected at key moments and shown in chronological order from left to right.
Figure 11: Bimanual match lighting with UMI-Live. A human operator coordinates the two grippers to open the matchbox, retrieve a match, and strike it. The final frame shows the lit match. Frames are shown in chronological order from left to right and top to bottom.
Figure 13: One autonomous π0.5 rollout on dual i2RT YAM arms, transferring six oranges from the table into a bowl. Frames are shown in chronological order from left to right and top to bottom.
Figure 14: Teleoperation setup. (a) A VIVE tracker replaces the LiDAR on each handheld gripper. (b) External base-station tracking supports teleoperation of the dual YAM arms.
Figure 15: Wrist images and reconstructed TCP position trajectories during bimanual folding. Each column shows the left wrist view (top), right wrist view (middle), and the two TCP position traces in a common plotting frame (bottom).
Hyperparameter
Value
Global batch size
32
Parameters updated
Full model
Peak learning rate
2.5×10−5
AdamW β
(0.9,0.95)
AdamW ϵ
10−8
Weight decay
0.01
Appendix
Table 3: π0.5 fine-tuning hyperparameters for the fresh-base runs.
Hyperparameter
Value
Global batch size
32
Parameters updated
VLM LoRA adapters and full action expert
Peak adapter learning rate
5×10−5
Peak action expert learning rate
10−4
AdamW β
(0.9,0.95)
AdamW ϵ
10−6
Appendix
Table 4: MolmoAct2 fine-tuning hyperparameters for the fresh-base run.
Joint j
θjoffset
dj
aj
αj
1
0
0.1135
0.02
−π/2
2
−π
0
0.264
−π
3
−2.901418
0
0.252239
0
4
−0.240173
0
0.073999
π/2
5
π/2
0
0
π/2
6
π/2
0.0356
0
−π
Appendix
Table 5: Classical Denavit–Hartenberg parameters for the YAM arm.