OpenJaws: An Open Hardware UMI Framework with Stereo Localization for Generalist Manipulation

Adithyavairavan Murali; Yash Oza; Karanbir Chahal; Neel Jawale; Silas De Vries; Hesam Rabeti; Erwin Coumans; Balakumar Sundaralingam

cs.ROOct 27, 2026

OpenJaws: An Open Hardware UMI Framework with Stereo Localization for Generalist Manipulation

Authors: Adithyavairavan Murali, Yash Oza, Karanbir Chahal, Neel Jawale, Silas De Vries, Hesam Rabeti, Erwin Coumans, Balakumar Sundaralingam,

Abstract

Handheld data-collection interfaces such as the Universal Manipulation Interface (UMI) allow manipulation demonstrations to be gathered in the wild without a robot in the loop, but the quality of the resulting action labels is bounded by the accuracy of vision-based localization. Existing systems recover end-effector pose from monocular visual-inertial odometry, which is subject to scale ambiguity and drift, or delegate tracking to closed-source modules and external infrastructure that reintroduce cost and setup burden. We present OpenJaws, a fully open hardware and software framework for sensorized UMI that improves localization and simplifies deployment across robot platforms. OpenJaws pairs a modular, 3D-printed gripper exposing a standardized ISO adapter with a low-cost stereo camera to recover metric camera trajectories. We further introduce UMI-Loc-Bench-v1, a benchmark quantifying real-world localization accuracy for handheld data-collection devices, and show that stereo localization substantially outperforms monocular alternatives. Finally, we demonstrate that policies trained on data collected with OpenJaws can be deployed on a real robot across multiple tasks. Hardware, software, and data are released under a commercially permissive license. Code, data, and videos are available at https://open-jaws.github.io/

Figures & tables

Fig. 2 : Handheld stereo-data collection with OpenJaws . The purple curve is the recovered end-effector trajectory over the full recording. The insets are the rectified left (blue) and right (orange) frames captured at the starting pose, with SIFT correspondences drawn between them, and the rays run from each lens to its image. The fiducial cube on the left is only for OpenJaws and is not used during data collection for policies. Note that the point cloud, obtained with FoundationStereo [ 15 ] , is only used for visualization.

Method	Approximate BOM Cost (USD)	Weight (kg)	Sim	Deployment Gripper	Mounting Adapter	Localization
UMI [ 9 ]	5,389 (includes WSG-50 Gripper)	0.650	✗	✗	Finger Mounts	OSS Monocular-IMU SLAM
FastUMI Pro [ 21 ]	840	–	✗	✗	Finger Mounts	Specialized SLAM Camera
LEGATO [ 13 ]	1,218	–	✓	✓	Gripper grasps tool (not rigidly attached)	Specialized SLAM Camera
DexUMI [ 16 ]	1,316 (includes iPhone)	–	✗	✗	–	iPhone ARKit
HandUMI	454 (includes VR Tracker)	–	✗	✗	Finger Mounts	VR Tracker
Grabette [ 14 ]	704	–	✗	✓	–	OSS Stereo + IMU SLAM

TABLE I : Comparing OpenJaws with other UMI frameworks. BOM costs are approximate totals in USD ( OpenJaws is itemized in Appendix A-D ); BOM = Bill of Materials, OSS = Open Source Software.

Fig. 3 : Task execution examples and camera trajectories. The top three rows show five chronological frames from wrench rotation, picking from a bin, and USB insertion; the green arrow indicates time progression. The bottom row shows the remaining six tasks in UMI-Loc-Bench-v1. Each plot contains two fiducial-derived camera trajectories from separate demonstrations, shown in green and black in the task’s marker frame. Circles and triangles mark the first and last valid poses, respectively; gaps indicate invalid poses.

Panel of Figure 4 — Fig. 4 : Localization results on UMI-Loc-Bench-v1. (a)–(b) Coverage and runtime over the benchmark demonstrations. (c)–(d) ATE on UMI-SLAM-tracked ground-truth segments (27 demonstrations); all methods use identical segments. (e)–(f) Full-trajectory ATE for stereo COLMAP [ 28 ] , ORB-SLAM3 [ 39 ] , and cuVSLAM [ 40 ] , compared with native-scale monocular ViPE [ 41 ] . Panels (a)–(d) evaluate the same ViPE run at native scale and version scaled based on the ground truth; its two runtime bars therefore denote one run. UMI-SLAM uses the original implementation [ 9 ] . Black rules mark means; violins show the central 2–98% and are clipped at the displayed limits.

Panel of Figure 5 — Fig. 5 : Real robot rollouts. Representative UR10 policy rollouts for pick-and-place (Top) and block stacking (Bottom) .

Fig. 6 : cuVSLAM stereo [ 40 ] under synthetic calibrated-baseline perturbations across several demonstrations. Solid: mean translational ATE; shaded: min–max; dashed: calibrated baseline.

Fig. 7 : Policy success with synthetic localization noise compared with our original action labels.

Panel of Figure 8 — Fig. 8 : Agentically generated real-to-sim digital twin. Corresponding gear insertion demonstration (Top) and successful Isaac Lab playback (Bottom) ; the agent recovers scene layout and contact geometry without ground-truth object poses.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Component	Qty	Price
Core Electronics
Dynamixel XC430-W150-T actuator	1	137.89
MMlove USB global-shutter stereo camera	1	99.99
ROBOTIS OpenRB-150 controller	1	28.64
KY-023 joystick module (10 pack)	1	17.39
Wiring

Appendix

TABLE II : OpenJaws bill of materials. Components for one deployment gripper along with its data-collection parts. Prices in USD; each component links to its source.

Robot Localization

Robot localization estimates a robot’s position and orientation from onboard sensors, maps, and observations of its surroundings.

See all 177 papers

Kinematic Interface for the Wild: Modular Bimanual Loco-Manipulation Capture from 360 $^{\circ}$ Cameras Alone
Sep 19, 2026Benjamin Yang, Weiying Wang, Shenggao Li +5Robotic Data Collection Camera Pose Estimation
Autel US
UMI-3D: Extending Universal Manipulation Interface from Vision-Limited to 3D Spatial Perception
Apr 15, 2026Ziming WangRobotic Data Collection Robotic Manipulation
HKU · USTC
Towards Accurate End-Effector Localization for UMI-Style Robotic Manipulation Teaching
Oct 7, 2026Junjie Zhang, Deteng Zhang, Zhisong Xu +4Robot Localization Robot Manipulation Benchmarks
Chongqing University · Northwestern Polytechnical University · The University of Tokyo +4

Sep 19, 2026J/K move · Enter open · S save

Kinematic Interface for the Wild: Modular Bimanual Loco-Manipulation Capture from 360 $^{\circ}$ Cameras Alone

Benjamin Yang, Weiying Wang, Shenggao Li, Keming Yan+4

Autel US

Robotic Data Collection Camera Pose Estimation Visual SLAM Bimanual Robotic Manipulation

Open paper AskPDF arXiv

A wrist-mounted camera for UMI-style data collection must do two jobs: record the manipulation and localize in the scene. Most handheld devices localize online from workspace-facing views crowded by hands and objects, or add dedicated tracking hardware. Room-scale bimanual capture therefore still tends to instrument the operator or the scene for accurate localization. We present KIWI (Kinematic Interface for the Wild), a capture kit whose only electronics are off-the-shelf cameras. Our core system splits the two jobs across the two lenses of a 360-degree camera. The rear lens faces the room and builds a shared metric map that registers both hands, and an optional head camera, in one frame without workspace co-visibility; the front lens records the manipulation, and offline IMU fusion bridges front-lens tracking loss. Through our quick-release plate, the camera module attaches to chopstick grippers, parallel-jaw grippers, hand-wrist mounts, or robot flanges. Across six bimanual recordings, combining the rear and front lenses failed to localize only 0.1% of query frames, whereas front-only bimanual feature alignment failed on 24.8% of frames and lost one recording entirely; against evaluation fiducials, localization error stayed within 4.5 mm. KIWI's recovered poses were sufficiently consistent for the four wrist streams alone to reconstruct the scene as a 3D Gaussian splat.

Abstract

Robot Localization

Explore similar work

Kinematic Interface for the Wild: Modular Bimanual Loco-Manipulation Capture from 360∘^{\circ}∘ Cameras Alone

UMI-3D: Extending Universal Manipulation Interface from Vision-Limited to 3D Spatial Perception

Towards Accurate End-Effector Localization for UMI-Style Robotic Manipulation Teaching

Kinematic Interface for the Wild: Modular Bimanual Loco-Manipulation Capture from 360 $^{\circ}$ Cameras Alone