FOMI: A Force-Feedback Manipulation Interface for Teaching Contact-Rich Manipulation

Yishu Shen; Yiming Luo; Ruilin Wang; Bowen Yang; Shuai Yang; Fangyi Xu; Yujun Shen; Fu Zhang

cs.ROOct 14, 2026

FOMI: A Force-Feedback Manipulation Interface for Teaching Contact-Rich Manipulation

Authors: Yishu Shen, Yiming Luo, Ruilin Wang, Bowen Yang, Shuai Yang, Fangyi Xu, Yujun Shen, Fu Zhang,

Organizations: The Hong Kong University of Science and Technology · The University of Hong Kong · Robbyant · Southern University of Science and Technology

Abstract

Robot-free collection with handheld interfaces enables scalable acquisition of human demonstrations for policy learning. However, existing handheld interfaces often provide limited intuitive force feedback, making contact-rich demonstration collection challenging for human operators. To address this, we present FOMI, a force-feedback manipulation interface with a scissor-shaped mechanism, which transmits object interaction forces directly to the operator's hands through actuation links without force sensors. This intuitive force feedback supports real-time action adjustment during robot-free collection of contact-rich demonstrations. Around FOMI, we build a demonstration collection system that integrates sensor calibration, data acquisition, and synchronization for robot-free collection. By further mapping pose changes and gripper opening of FOMI to robot end-effector poses and gripper commands, the system also supports teleoperation. Across four contact-rich tasks, FOMI improves collection efficiency by approximately 29% and 18% over the best baselines in robot-free collection and teleoperation, respectively. Demonstrations collected with FOMI in both modes enable policies to perform challenging contact-rich manipulation tasks with high success rates.

Figures & tables

Figure 1: Overview of FOMI. The force-feedback manipulation interface incorporates a scissor-shaped mechanism to provide human operators with intuitive force feedback in robot-free collection of contact-rich demonstrations, while also supporting teleoperation for policy learning. FOMI achieves higher collection efficiency and completion rates than the evaluated baselines in both modes. Policies fine-tuned on demonstrations collected with FOMI can perform contact-rich manipulation tasks in the real world.

Interface	Mechanical Design	Intuitive Force Feedback	Handles for Load Support
UMI Chi et al. (2024) , UMI-FT Choi et al. (2026)	Trigger-driven rack-and-pinion	✗	✓
FastUMI Liu et al. (2024) , exUMI Xu et al. (2025)	Trigger-driven actuation	✗	✓
FreeTacMan Wu et al. (2025)	Finger-driven actuation	✓	✗
FOMI (Ours)	Scissor-shaped actuation linkage	✓	✓

Table 1: Mechanical design and features of robot-free demonstration interfaces. ✓ : present; ✗ : absent.

Figure 2: Mechanical design of FOMI. (a) Exploded view with the main components. (b) Direct force transmission path from the gripper to the handles in robot-free collection, with the handle pivot, hand force $F_{h}$ , and effective moment arm $\ell_{h}$ indicated. (c) Kinematic schematic showing base pivots $O,O^{\prime}$ ( $A,A^{\prime}$ ) and gripper carrier joints $B,B^{\prime}$ ( $C,C^{\prime}$ ) on the right (left) branch. The base coordinate axes are $x$ along the gripper opening direction, $z$ vertical, and $y$ into the page. The right and left sector gears rotate by $q$ and $-q$ from closure, respectively. (d) Motion from the open to the closed configuration. (e) Mechanical stops defining the gripper width limits, with the maximum gripper width $w_{I}(q_{\max})$ and vertical displacement $\delta(q_{\max})$ indicated.

Figure 3: Left: sensor layout of FOMI. Right: the target robot gripper (Robotiq 2F-85).

Figure 4: Demonstration collection interfaces and experimental setup used across all four tasks, illustrated using Insert Tube. Top: robot-free collection interfaces. Bottom: teleoperation interfaces.

Task	Robot-Free $\uparrow$			Teleoperation $\uparrow$
Task	UMI	Ring	FOMI	GELLO	VR	FOMI
Pick Chip	$16.6\pm 1.8$	$10.7\pm 3.5$	$\mathbf{22.0}\pm 1.5$	$6.4\pm 0.8$	$0.2\pm 0.4$	$\mathbf{7.2}\pm 1.2$
Insert Tube	$13.8\pm 1.0$	$9.9\pm 3.0$	$\mathbf{17.7}\pm 1.5$	$4.9\pm 0.5$	$1.6\pm 0.7$	$\mathbf{7.4}\pm 1.3$
Wash Dish	$11.0\pm 1.8$	$10.9\pm 1.9$	$\mathbf{13.3}\pm 2.4$	$8.8\pm 0.4$	$7.2\pm 0.4$	$\mathbf{9.3}\pm 0.7$
Open Pouch	$13.7\pm 3.4$	$15.0\pm 3.0$	$\mathbf{18.2}\pm 2.4$	$8.1\pm 0.4$	$4.6\pm 1.3$	$\mathbf{9.7}\pm 1.7$
Mean	$13.8$	$11.6$	$\mathbf{17.8}$	$7.1$	$3.4$	$\mathbf{8.4}$

Table 2: Demonstration collection efficiency (successful demonstrations / 10 min). Entries report mean and sample standard deviation across three operators, with each operator’s results averaged over three sessions.

Figure 5: (a) Completion rate, computed as total successful attempts divided by total attempts over tasks, sessions, and operators. (b) Exploratory user study for the interfaces in robot-free collection: mean ratings from three operators across three dimensions, each rated on a 0–5 scale.

Figure 6: Representative rollouts of policies trained on robot-free demonstrations across four tasks. Labeled stages are used to compute progress rate. Insets highlight details of contact-rich manipulation.

Figure 7: Policy performance with increasing numbers of robot-free demonstrations per task: 25, 50, and 100. Left: success rate. Right: progress rate.

Figure 9: Policy performance on Open Pouch after deploying the policy fine-tuned on 25 robot-free demonstrations to the Flexiv platform.

Model	Left $\downarrow$	Right $\downarrow$
Linear regression	3.36	4.47
MLP	0.99	0.94
Piecewise quadratic, without offset	2.48	3.49
Piecewise quadratic, with offset (ours)	0.80	0.88

Table 3: Gripper-Command Calibration RMSE (Command Range: 0–255)

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 10: Representative camera views recorded during demonstration collection. From left to right: Pick Chip, Insert Tube, Wash Dish, and Open Pouch.

Task	Robot-Free						Teleoperation
	UMI		Ring		FOMI		GELLO		VR		FOMI
	$S\!\uparrow$	$F\!\downarrow$	$S\!\uparrow$	$F\!\downarrow$	$\boldsymbol{S}\!\uparrow$	$\boldsymbol{F}\!\downarrow$	$S\!\uparrow$	$F\!\downarrow$	$S\!\uparrow$	$F\!\downarrow$	$\boldsymbol{S}\!\uparrow$	$\boldsymbol{F}\!\downarrow$
Pick Chip	149	71	96	66	198	28	58	70	2	92	65	32
Insert Tube	124	61	89	57	159	18	44	83	14	85	67	36
Wash Dish	99	51	98	39	120	24	79	28	65	13	84	11
Open Pouch	123	88	135	68	164	29	73	35	41	46	87	19

Appendix

Table 4: Demonstration collection attempt counts. Each task row pools three sessions from each of three operators.

S

: successful attempts;

F

: failed attempts. Bold marks the highest success and lowest failure counts within each mode.

Haptic Feedback for Robot Teleoperation

See all 20 papers

Force-Modulated Dexterity: Unified Compliance for Contact-Rich Manipulation
Oct 12, 2026Ying Yuan, David Held, Jitendra Malik +1Contact-Rich Robotic Manipulation Dexterous Manipulation
Amazon FAR · Carnegie Mellon University
Touch2Robot: Robot Touch in the Human Demonstration Loop
Sep 21, 2026Shengcheng Luo, Xiaoyang Cheng, Hong Ying +6Tactile Sensing for Robotic Manipulation Robotic Data Collection
ShanghaiTech University · Beijing Institute for General Artificial Intelligence (BIGAI) · Shanghai Jiao Tong University +1
ForceBand: Learning Forceful Manipulation with sEMG
Jun 24, 2026Botao He, Zhi Wang, Linna Kuang +8Robot Skill Learning Contact-Rich Robotic Manipulation
Amazon FAR · University of Maryland · Johns Hopkins University

Oct 12, 2026J/K move · Enter open · S save

Force-Modulated Dexterity: Unified Compliance for Contact-Rich Manipulation

Ying Yuan, David Held, Jitendra Malik, Haozhi Qi

Amazon FAR · Carnegie Mellon University

Contact-Rich Robotic Manipulation Dexterous Manipulation Force-Controlled Robotic Manipulation Robot Teleoperation

Open paper AskPDF arXiv

Teleoperation is widely used to collect high-quality demonstrations for robot learning. Dexterous manipulation, however, requires coordinated control of motion and contact forces. Operators can specify motion, but struggle to reproduce the subtle force adjustments of human manipulation without feeling the robot's contacts. We introduce Force-Modulated Dexterity (FOMO-Dex), a unified compliance formulation for contact-rich manipulation. FOMO-Dex expresses nominal actions in coordinates defined by task geometry and contact structure. It applies force-driven motion corrections along selected directions while leaving the remaining command components unchanged. Across card picking, paper cutting, screwdriving, and nut-and-bolt assembly, FOMO-Dex reduces excessive contact forces compared with stiff control. A user study shows that operators collect more successful demonstration data per hour after brief practice. For policy learning, imitating the corrected actions improves card-picking success over imitating raw operator commands. Policies trained on FOMO-Dex demonstrations also achieve more rotation in nut-and-bolt manipulation and screwdriving than those trained on demonstrations collected with the stiff controller. Videos are shown on https://yingyuan0414.github.io/fomo-dex/ .

Abstract

Haptic Feedback for Robot Teleoperation

Explore similar work

Force-Modulated Dexterity: Unified Compliance for Contact-Rich Manipulation

Touch2Robot: Robot Touch in the Human Demonstration Loop

ForceBand: Learning Forceful Manipulation with sEMG