Organizations: The Hong Kong University of Science and Technology · The University of Hong Kong · Robbyant · Southern University of Science and Technology
Robot-free collection with handheld interfaces enables scalable acquisition of human demonstrations for policy learning. However, existing handheld interfaces often provide limited intuitive force feedback, making contact-rich demonstration collection challenging for human operators. To address this, we present FOMI, a force-feedback manipulation interface with a scissor-shaped mechanism, which transmits object interaction forces directly to the operator's hands through actuation links without force sensors. This intuitive force feedback supports real-time action adjustment during robot-free collection of contact-rich demonstrations. Around FOMI, we build a demonstration collection system that integrates sensor calibration, data acquisition, and synchronization for robot-free collection. By further mapping pose changes and gripper opening of FOMI to robot end-effector poses and gripper commands, the system also supports teleoperation. Across four contact-rich tasks, FOMI improves collection efficiency by approximately 29% and 18% over the best baselines in robot-free collection and teleoperation, respectively. Demonstrations collected with FOMI in both modes enable policies to perform challenging contact-rich manipulation tasks with high success rates.
Figures & tables
Figure 1: Overview of FOMI. The force-feedback manipulation interface incorporates a scissor-shaped mechanism to provide human operators with intuitive force feedback in robot-free collection of contact-rich demonstrations, while also supporting teleoperation for policy learning. FOMI achieves higher collection efficiency and completion rates than the evaluated baselines in both modes. Policies fine-tuned on demonstrations collected with FOMI can perform contact-rich manipulation tasks in the real world.
Interface
Mechanical Design
Intuitive Force Feedback
Handles for Load Support
UMI Chi et al. (2024) , UMI-FT Choi et al. (2026)
Trigger-driven rack-and-pinion
✗
✓
FastUMI Liu et al. (2024) , exUMI Xu et al. (2025)
Trigger-driven actuation
✗
✓
FreeTacMan Wu et al. (2025)
Finger-driven actuation
✓
✗
FOMI (Ours)
Scissor-shaped actuation linkage
✓
✓
Table 1: Mechanical design and features of robot-free demonstration interfaces. ✓ : present; ✗ : absent.
Figure 2: Mechanical design of FOMI. (a) Exploded view with the main components. (b) Direct force transmission path from the gripper to the handles in robot-free collection, with the handle pivot, hand force Fh , and effective moment arm ℓh indicated. (c) Kinematic schematic showing base pivots O,O′ ( A,A′ ) and gripper carrier joints B,B′ ( C,C′ ) on the right (left) branch. The base coordinate axes are x along the gripper opening direction, z vertical, and y into the page. The right and left sector gears rotate by q and −q from closure, respectively. (d) Motion from the open to the closed configuration. (e) Mechanical stops defining the gripper width limits, with the maximum gripper width wI(qmax) and vertical displacement δ(qmax) indicated.
Figure 3: Left: sensor layout of FOMI. Right: the target robot gripper (Robotiq 2F-85).
Figure 4: Demonstration collection interfaces and experimental setup used across all four tasks, illustrated using Insert Tube. Top: robot-free collection interfaces. Bottom: teleoperation interfaces.
Task
Robot-Free ↑
Teleoperation ↑
UMI
Ring
FOMI
GELLO
VR
FOMI
Pick Chip
16.6±1.8
10.7±3.5
22.0±1.5
6.4±0.8
0.2±0.4
7.2±1.2
Insert Tube
13.8±1.0
9.9±3.0
17.7±1.5
4.9±0.5
1.6±0.7
7.4±1.3
Wash Dish
11.0±1.8
10.9±1.9
13.3±2.4
8.8±0.4
7.2±0.4
9.3±0.7
Open Pouch
13.7±3.4
15.0±3.0
18.2±2.4
8.1±0.4
4.6±1.3
9.7±1.7
Mean
13.8
11.6
17.8
7.1
3.4
8.4
Table 2: Demonstration collection efficiency (successful demonstrations / 10 min). Entries report mean and sample standard deviation across three operators, with each operator’s results averaged over three sessions.
Figure 5: (a) Completion rate, computed as total successful attempts divided by total attempts over tasks, sessions, and operators. (b) Exploratory user study for the interfaces in robot-free collection: mean ratings from three operators across three dimensions, each rated on a 0–5 scale.
Figure 6: Representative rollouts of policies trained on robot-free demonstrations across four tasks. Labeled stages are used to compute progress rate. Insets highlight details of contact-rich manipulation.
Figure 7: Policy performance with increasing numbers of robot-free demonstrations per task: 25, 50, and 100. Left: success rate. Right: progress rate.
Figure 9: Policy performance on Open Pouch after deploying the policy fine-tuned on 25 robot-free demonstrations to the Flexiv platform.
Figure 10: Representative camera views recorded during demonstration collection. From left to right: Pick Chip, Insert Tube, Wash Dish, and Open Pouch.
Task
Robot-Free
Teleoperation
UMI
Ring
FOMI
GELLO
VR
FOMI
S↑
F↓
S↑
F↓
S↑
F↓
S↑
F↓
S↑
F↓
S↑
F↓
Pick Chip
149
71
96
66
198
28
58
70
2
92
65
32
Insert Tube
124
61
89
57
159
18
44
83
14
85
67
36
Wash Dish
99
51
98
39
120
24
79
28
65
13
84
11
Open Pouch
123
88
135
68
164
29
73
35
41
46
87
19
Appendix
Table 4: Demonstration collection attempt counts. Each task row pools three sessions from each of three operators. S : successful attempts; F : failed attempts. Bold marks the highest success and lowest failure counts within each mode.
Teleoperation is widely used to collect high-quality demonstrations for robot learning. Dexterous manipulation, however, requires coordinated control of motion and contact forces. Operators can specify motion, but struggle to reproduce the subtle force adjustments of human manipulation without feeling the robot's contacts. We introduce Force-Modulated Dexterity (FOMO-Dex), a unified compliance formulation for contact-rich manipulation. FOMO-Dex expresses nominal actions in coordinates defined by task geometry and contact structure. It applies force-driven motion corrections along selected directions while leaving the remaining command components unchanged. Across card picking, paper cutting, screwdriving, and nut-and-bolt assembly, FOMO-Dex reduces excessive contact forces compared with stiff control. A user study shows that operators collect more successful demonstration data per hour after brief practice. For policy learning, imitating the corrected actions improves card-picking success over imitating raw operator commands. Policies trained on FOMO-Dex demonstrations also achieve more rotation in nut-and-bolt manipulation and screwdriving than those trained on demonstrations collected with the stiff controller. Videos are shown on https://yingyuan0414.github.io/fomo-dex/ .
Human demonstrations offer a scalable way to collect manipulation data, but their contacts may be unstable or infeasible when transferred to a robot hand. Collecting demonstrations directly on the target robot avoids this mismatch but substantially increases the cost of data collection. To address this trade-off, we present Touch2Robot, a framework that lets humans collect demonstrations while seeing how the target robot hand would contact the object. We capture human hand motion, tactile-glove measurements, and object motion during human manipulation. These recordings guide object-specific RL policies to reproduce the demonstrated object motion while favoring contacts consistent with the recorded human touch. We distill the learned behaviors into a unified real-time retargeter that maps incoming human observations and object geometry to robot hand configurations. During collection, the predicted robot configuration is synchronized with the tracked object pose in simulation to reconstruct robot-object contacts, which are visualized to help the demonstrator adapt subsequent interactions to the target hand. Across four real-world tasks, Touch2Robot improves average real-robot replay completion from 37.9% to 72.1% over visual-only feedback, while reducing the collection time per replay-successful demonstration from 58.6s to 18.2s. Reconstructed target-hand contacts achieve 44.2% F1 against real-robot tactile measurements, and policies trained on Touch2Robot demonstrations improve downstream Diffusion Policy performance by 29.1 percentage points over visual-only feedback. These results show that bringing robot touch into the human demonstration loop improves both the quality and efficiency of scalable dexterous data collection. Project webpage: https://Touch2Robot.github.io/.
Shengcheng Luo, Xiaoyang Cheng, Hong Ying +6
ShanghaiTech University · Beijing Institute for General Artificial Intelligence (BIGAI) · Shanghai Jiao Tong University +1
Human demonstrations are a scalable data source for learning robot manipulation policies. However, common sources of human demonstration data, such as motion-capture trajectories and internet videos, capture mostly motion and appearance while missing the contact forces that are critical for force-sensitive manipulation. In this paper, we introduce ForceBand, a low-cost wrist-worn sEMG system that turns human muscle activity into force-enriched demonstrations. We first collect a 10-hour multimodal dataset containing egocentric video, sEMG, IMU, and fingertip force measurements across diverse actions and objects. Using this dataset, we pre-train an EMG2Force model that predicts per-finger forces from sEMG and IMU signals. After a short user-specific calibration, users can collect target-task demonstrations using only ForceBand and video; EMG2Force then labels these demonstrations with per-finger force traces, producing force-augmented demonstrations for robot policy learning. Experiments show that ForceBand recovers fine-grained fingertip interactions with over 50% lower force prediction error than vision-based baselines and achieves an 87% success rate on pick, squeeze, and place tasks that require object-specific force control across objects with diverse shapes, sizes, and weights. Project website: https://forceband-emg.github.io
Botao He, Zhi Wang, Linna Kuang +8
Amazon FAR · University of Maryland · Johns Hopkins University