Teleoperation is widely used to collect high-quality demonstrations for robot learning. Dexterous manipulation, however, requires coordinated control of motion and contact forces. Operators can specify motion, but struggle to reproduce the subtle force adjustments of human manipulation without feeling the robot's contacts. We introduce Force-Modulated Dexterity (FOMO-Dex), a unified compliance formulation for contact-rich manipulation. FOMO-Dex expresses nominal actions in coordinates defined by task geometry and contact structure. It applies force-driven motion corrections along selected directions while leaving the remaining command components unchanged. Across card picking, paper cutting, screwdriving, and nut-and-bolt assembly, FOMO-Dex reduces excessive contact forces compared with stiff control. A user study shows that operators collect more successful demonstration data per hour after brief practice. For policy learning, imitating the corrected actions improves card-picking success over imitating raw operator commands. Policies trained on FOMO-Dex demonstrations also achieve more rotation in nut-and-bolt manipulation and screwdriving than those trained on demonstrations collected with the stiff controller. Videos are shown on https://yingyuan0414.github.io/fomo-dex/ .
Figures & tables
Figure 1: System overview. A teleoperator or learned policy provides nominal arm and finger commands. FOMO-Dex uses feedback from the wrist force-torque sensor and the hand’s tactile and joint-torque sensors to adjust selected motion components before sending the commands to the robot.
Figure 2: Overview of the FOMO-Dex controller and its task-specific realizations. Task geometry and contact structure define interaction directions, along which force measurements are converted into unilateral or bilateral force errors. Direction-wise admittance then produces arm or hand motion corrections, enabling unilateral load relief, translational tool compliance, and multi-contact force regulation within the same formulation.
Card Picking
Paper Cutting
Bolt–nut/Screwdriver
Controller rate (Hz)
90
90
90
Force target f⋆ (N)
–
–
0.5–2.4
Force limit fmax (N)
3.0
3.5
3.0
Contact on/off (N)
0.6/0.6
5.0/2.0
0.25/0.15
Max correction
0.05 rad
0.06 m
0.10 rad
Table 1: Task-specific FOMO-Dex controller configurations. “–” denotes not applicable. Contact on/off denotes the force thresholds used to activate and deactivate contact-dependent modulation, respectively.
Task
Metric
Stiff
Naive Comp.
Ours
Card Picking
Success (%) ↑
90.9
37.5
93.8
Duration (s) ↓
22.1 ±4.2
17.2 ±2.4
16.9 ±4.2
Exc. Frames (%) ↓
13.0 ±5.4
7.4 ±7.9
5.9 ±2.8
Force Exceedance (N) ↓
0.32 ±0.29
0.10 ±0.13
0.05 ±0.03
Paper Cutting
Cut Rate (cm/s) ↑
2.47 ±0.15
3.16 ±1.12
2.63 ±0.41
Lat. Force (N) ↓
2.00 ±0.41
1.61 ±0.43
1.41 ±0.14
Table 2: Evaluation of our force-modulated controller across four benchmark tasks compared with stiff and naive-compliance baselines over 50 trials. Exc. Frames denotes the fraction of frames in which the measured contact force exceeds the task-specific safe threshold, and Force Exceedance is defined in Eq. 12 . ↑ / ↓ indicate higher/lower is better; bold and underline denote the best and second-best results, respectively.
Figure 3: Fingertip forces during card picking with stiff control and FOMO-Dex. (a) Maximum force across fingertips over five representative episodes per controller, aligned at first contact. (b) Distribution of the per-episode 90th-percentile fingertip force. Dashed lines mark the 3 N force threshold.
Figure 4: Interaction forces during paper cutting. (a,b) Wrist axial force over five representative episodes under stiff control and FOMO-Dex, respectively. (c) Axial versus lateral force samples for stiff control (orange) and FOMO-Dex (blue). Large markers indicate the means.
Figure 5: Hand configurations before and after force modulation. The gray mesh shows the nominal configuration commanded by the teleoperator, while the blue mesh shows the force-modulated command. For surface contact (left), the contacting fingertip retreats from the supporting surface. For in-hand tripod contact (right), force regulation adjusts the index–thumb contacts while preserving the remaining grasp configuration.
Figure 6: Real-world teleoperation with stiff control (top) and FOMO-Dex (bottom). In card picking (left), FOMO-Dex limits excessive fingertip loading against the supporting surface. In screwdriving (right), it reduces large variations in total fingertip load and maintains more stable tool engagement. Representative execution frames are shown with the corresponding force traces.
Stiff
Ours
Experience
Succ. Data
Mean Ep. Len.
Succ. Data
Mean Ep. Len.
(min/h) ↑
(s)
(min/h) ↑
(s)
Card Picking Mean Ep. Len. ↓
Zero-shot
11.9 ±5.2
32.7 ±10.5
20.6 ±4.0
29.1 ±0.5
+10 min
24.3 ±9.4
27.9 ±1.2
29.3 ±5.6
24.9 ±10.2
Bolt-Nut Mean Ep. Len. = 60
Table 3: User-study comparison of demonstration collection with the stiff controller and FOMO-Dex, zero-shot and after 10 min of practice. Succ. Data is successful demonstration time per hour of teleoperation. Mean Ep. Len. is episode duration; shorter is preferred for card picking, while longer is preferred for screwdriving, where episodes are capped at 60 s and longer durations indicate more stable sustained manipulation of the long screw.
Success (%)
Reach
Retr.
Lift
Stiff
100
100
56
Comp. Raw
100
89
0
Comp. Applied (Ours)
100
89
67
Table 4: Evaluation of behavior-cloning policies trained on 50 demonstrations per task–controller setting. (a) Card-picking ablation comparing stiff, raw-command training with FOMO-Dex inference (Comp. Raw), and post-modulation-action training (Comp. Applied) over 18 evaluation trials. (b) Cross-task comparison of policies trained from stiff and FOMO-Dex demonstrations, evaluated by turns per episode, over 10 evaluation trials.