DexRemo: A Force-Sensing Hand Platform and Task Suite for Contact-Rich Removal Manipulation
Authors: Liman Wang, Hanyang Zhong, Sun Zhaole, Wenting Cao, Babar Jamil, Andy Tyrrell, Shan Luo, Jihong Zhu
Organizations: School of Physics, Engineering and Technology, University of York, York, U.K. · University of Edinburgh, Edinburgh, U.K. · Friedrich Schiller University Jena, Jena, Germany. · Department of Engineering, King’s College London, London, U.K.
Many manipulation tasks require separating adhered, embedded, contaminated, or mechanically constrained targets. We study this setting as removal manipulation, where progress depends on evolving contact states that are only partially observable from vision and kinematics. We introduce DexRemo, a three-finger robotic hand with high-force actuation, modular fingertips, and three-axis fingertip force sensing, together with a nine-scenario real-robot task suite spanning adhesive detachment, surface cleaning and self-cleaning, and occluded retrieval. To make multi-finger force observations easier to fuse, we express fingertip forces in a shared wrist-centered frame, giving force components consistent geometric meaning across fingers and time. Using an Action Chunking Transformer backbone, policies fuse multi-view vision, proprioception, fingertip pose, and force observations to predict closed-loop action chunks from demonstrations. Evaluation uses six core benchmark tasks for controlled fingertip-input and force-frame ablations, plus three extension scenarios for additional held-out coverage. Across these evaluations, the learned policies achieve consistent removal performance, and ablations show that wrist-frame forces, especially with fingertip pose, improve stability and generalization over local-frame force inputs and no-force baselines. Project website: https://dexremo.github.io
Figures & tables
Fig. 1: Overview of removal manipulation and the proposed framework. (a) DexRemo Hand removes strongly adhered 3D-printed barnacles from a curved turtle mockup, demonstrating variable and partially observable attachments. (b) Representative human removal behaviors reveal task-intrinsic interaction principles. (c) Fingertip forces are transformed from local sensor frames RSi to a shared wrist frame, then fused with vision, fingertip poses, and proprioception. An ACT-based policy generates observation-conditioned action chunks for progressive, edge-initiated debonding.
Hand
Strength (N)
Power Density N×DOF/(cm2)
Bauer et al. [ 8 ]
37.4
0.677
Allegro Hand [ 9 ]
8.5
0.350
D’Manus [ 10 ]
27.8
0.313
Inmoov Hand [ 11 ]
5.8
0.116
LEAP Hand [ 12 ]
19.5
1.045
LEAP-C Hand [ 12 ]
21.5
1.15
TABLE I: Pullout strength and power density comparison. Peak pullout force and corresponding power density for representative hands. Metrics are defined in Appendix A .
Hands / Platforms
Finger Num.
Transmission
Cost (USD)
DoFs
Allegro Hand-4F [ 9 ]
4
DL
16,000
16
LEAP Hand [ 12 ]
4
DL
2,000
16
Delta Hand [ 13 ]
4
DL
1,000
12
Shadow Hand [ 14 ]
5
T
300,000
20
Inspire Hand [ 15 ]
5
T
5,000
6
Ability Hand [ 16 ]
5
T
20,000
6
TABLE II: Representative multi-finger hands/platforms: finger count, transmission, approximate cost (USD), and actuated DoFs. Costs may vary by configuration. DL: direct linkage; T: tendon. Note: Costs are reported for the hands/platforms only and exclude any tactile/force sensing modules.
Fig. 2: Overview of the DexRemo removal manipulation framework. Left: Representative sequences from the nine tasks, grouped into the three task families A (adhesive removal, A1–A5), B (surface cleaning, B1–B2), and C (buried retrieval, C1–C2); here the uppercase letters denote task families rather than figure panels. Top right: Multimodal learning pipeline. Joint states, wrist-frame multi-finger contact forces, fingertip poses, and RGB images are encoded and fused to condition the Action Chunking Transformer policy πθ , which predicts short-horizon joint-displacement chunks. Bottom right: DexRemo learning platform. A bimanual leader–follower teleoperation setup supports demonstration collection and policy replay. The follower robot integrates multiple RGB cameras and custom fingertips with embedded three-axis force sensors.
Fig. 3: DexRemo Hand design. (a) Kinematic layout of the three-finger hand. (b) Finger structure with two flexion joints and a dual-axis base. (c) Physical prototype with three identical thumb-like fingers for multi-finger manipulation. IP, MCP, and CMC denote the interphalangeal, metacarpophalangeal, and carpometacarpal joints, respectively; CMC1 and CMC2 denote the two axes of the compound CMC joint.
Fig. 4: Fingertip tool modules for removal manipulation. (a) Wedge-shaped rigid fingernail tip integrated with a custom three-axis force sensor. (b) Soft cleaning tip consisting of a spherical compliant core wrapped with sponge or wet tissue. Annotated dimensions are shown for reference.
Fig. 5: Fingertip three-axis force sensor. (a) Fabricated sensor with triangular load-cell arrangement. (b) Exploded view of the rigid frame and measurement axes. (c) Calibration with suspended masses applying known Cartesian forces. (d) Calibration for mid/left/right sensors: corrected outputs vs. ground-truth forces along x,y,z (dashed: ideal).
Fig. 6: Assembled fingertip force-sensing characterization. Here, M, L, and R denote the middle, left, and right fingertip modules, respectively. (a) 3σ no-load variation after mean-offset removal. (b) Static 5 N/10 N main-axis response, normalized by the median 10 N response for each axis; bars denote medians. (c) Cross-axis coupling, defined as off-axis response norm over absolute main-axis response.
Metric
Value
Individual load-cell manufacturer specifications
Rated axial capacity
5kgf ( ≈49N )
Rated output
2.0±10%mV/V
Zero balance
±2% F.S.
Nonlinearity
0.15% F.S.
Hysteresis
0.05% F.S.
TABLE III: Fingertip force sensing: manufacturer specifications and assembled-module characterization.
Item
Value
Policy backbone
ACT [ 7 ]
Camera views
3 RGB views
Image resolution (height × width)
480×640
Vision encoder
ResNet-18
Vision pretraining
ImageNet
Non-visual encoder
Two-layer MLP
TABLE IV: Policy architecture and training hyperparameters.
Fig. 7: Physical objects used in each manipulation task (A1–C2). Each object is photographed alongside a standard ruler to indicate real-world scale. Objects are labeled as seen or unseen according to whether they appear in policy training. Detailed dimensions and preparation parameters are summarized in Table VI and Appendix E .
TABLE V: Task set summary with training-time variations and held-out test conditions. Representative task images from the second camera view are shown in the rightmost column.
ID
Object / nominal specification
Material / medium
Preparation protocol
A1
Barnacle model, 50×40×40mm
PLA/resin target on turtle-shell substrate; hot-melt adhesive
Apply adhesive at contact root and wait 60s before trial.
A5
Blu-Tack residue, 20×20×5mm
Blu-Tack on fingertip shell
Attach residue to fingertip and randomize its shape and pose.
B1
Marker stain, 120×120mm target region
Purple marker on ceramic plate; sponge tip
Draw stain in predefined region and wet sponge with 2ml water.
TABLE VI: Task objects, nominal specifications, materials, and preparation procedures.
Fig. 8: Task-wise success rates across input configurations on the six core benchmark tasks. All six matched configurations use multi-view RGB observations and joint proprioception; additional inputs include local-frame fingertip forces R , fingertip poses P , and wrist-frame fingertip forces W . Points show mean success rates over three random seeds, with 30 real-robot rollouts per seed; error bars denote 95% confidence intervals computed across seed-level success rates. The full P+W configuration achieves the strongest overall performance across A1–A4, B1, and C1.
Fig. 9: Completion and termination time across input configurations over all core tasks. (a) Termination time over all rollouts, where failed trials are assigned the 120 s timeout. The success rate of each input configuration is reported below the x-axis. (b) Completion time over successful rollouts only. Boxes show the interquartile range, horizontal lines denote medians, whiskers show the non-outlier range, and dots denote individual real-robot rollouts. The full P+W configuration achieves the highest overall success rate and substantially reduces timeout-dominated failures, while maintaining competitive successful-rollout completion time.
Fig. 10: Task-specific efficiency–force/contact trade-off across input configurations. Columns correspond to the three core task groups: adhesive removal (A1–A4), buried retrieval (C1), and surface cleaning (B1). Each point summarizes one input configuration over the same three training seeds and matched rollout budgets used in Fig. 8 , with rollouts pooled across seeds before the class-level scores are computed; the point markers therefore carry no seed-level dispersion. The top row relates completion efficiency, Eq. ( 23 ), to the class-specific force/contact score; the bottom row relates success rate to the same score. The vertical axis is the task-specific score Qc,m of Eq. ( 27 ): for A- and C-class tasks Qc,m=1−Iˉc,m , so a higher score indicates lower cumulative contact effort, whereas for B-class cleaning Qc,m=Iˉc,m , so a higher score indicates stronger sustained contact. Vertical scores should therefore be compared only within a task class. The full ( P+W ) policy shows the most favorable class-specific trade-offs.
Fig. 11: Blu-Tack removal (A3): force feedback enables observation-driven switching in an ACT-based policy. (a) P+W (fingertip pose + wrist-frame forces) and (b) P (fingertip pose only). Left: fingertip trajectories for right/middle/left fingers. Right: wrist-frame force components over time (color indicates finger; line style indicates Fx , Fy , and Fz ). Vertical dashed lines mark the timestamps of snapshots t1–t4. With P+W, changes in force patterns during side-finger boundary interaction provide an observable cue for switching to a distinct action chunk dominated by the middle finger, which ramps up force and achieves detachment. With P only, the policy exhibits prolonged oscillatory contact and fails to satisfy the success criterion within the time limit.
Fig. 12: Failure-mode distribution across tasks and input configurations. Stacked bars show the number of failed rollouts assigned to each dominant failure mode for the six core ablation tasks A1–A4, B1, and C1. Counts are pooled over the three training seeds for each task–input configuration and are not normalized by total rollouts. As each input configuration uses the same evaluation budget within a task, lower total bar height indicates fewer failures.
ID
Scenario
Held-out factors
Success rate
Completion time
A5
Fingertip self-cleaning
Residue shape, pose, and color
8/10 ; 80.0% [ 49.0,94.3 ]
68.5±20.4s
B2
Ink cleaning on silicone skin
Stain location, geometry, and color
9/10 ; 90.0% [ 59.6,98.2 ]
57.9±15.3s
C2
Cat litter clump exposure
Clump pose and color-matching condition
7/10 ; 70.0% [ 39.7,89.2 ]
86.7±19.6s
TABLE VII: Extension-scenario evaluation under held-out generalization. A5, B2, and C2 are evaluated only under the full P+W configuration, each with a single trained policy and 10 held-out rollouts. They are reported as extension evaluations and are not included in the matched input-configuration ablation averages or seed-level intervals. Success rates are reported with 95% binomial confidence intervals. Completion time is averaged over successful rollouts.
Task
ACT
Diffusion Policy
A2
28/30
6/30
A3
25/30
9/30
B1
21/30
7/30
TABLE VIII: Diagnostic comparison of ACT and Diffusion Policy under the same full multimodal P+W input configuration. Each entry reports successful rollouts out of 30 real-robot trials.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Sensor
Axis
MSE ( N2 )
R2
Fit
Mid
X
0.4924
0.9865
y=1.193x−0.406
Mid
Y
0.2788
0.9924
y=0.924x+0.894
Mid
Z
0.0656
0.9982
y=0.960x+0.484
Left
X
0.0511
0.9986
y=1.171x−0.373
Left
Y
0.0220
0.9994
y=1.077x+0.296
Left
Z
0.0314
0.9991
y=0.862x−0.003
Appendix
TABLE IX: Calibration of in-house force sensors (X/Y/Z): mean squared error (MSE), R2 , and linear fit y=ax+b .
Fig. 13: Teleoperation interface used during demonstration collection. The demonstrator observes synchronized camera views together with a real-time visualization of the follower hand configuration and fingertip force vectors, which provides augmented visual feedback about contact direction and magnitude.
ID
Nominal
Held-out
Success
Adhesive detachment and self-cleaning
A1
tglue≥3 mm; size 50×40×40 mm
sizes {40×30×30,60×50×50} ; unseen shell
detach; lift ≥hlift for τ ; no damage
A2
pad 15×15 mm; part 65×30×30 mm
triangular/quad prism; poses ±30∘ yaw
detach; clearance for τ ; no violent ejection
A3
patch 150×100×1 mm; mass ≈20 g
blue; ring/strip shapes; placement ±50 mm
remove; residue ≤5% by area differencing
A4
log 200 mm long, 100 mm dia.; specimen 50×40 mm
log rotated ±45∘ ; specimen ±30 mm
detach; lift ≥hlift for τ ; no damage
A5
Blu-Tack residue 20×20×5 mm
clay; thin-sheet and thick-blob shapes
fully removed by visual inspection
Appendix
TABLE X: Additional task details and evaluation criteria. Object identities and constraint types for each task are given in Table V and Table VI .
Contact-rich robotic manipulation studies how robots plan and control physical interactions such as grasping, insertion, pushing, and dexterous object handling.
Teleoperation is widely used to collect high-quality demonstrations for robot learning. Dexterous manipulation, however, requires coordinated control of motion and contact forces. Operators can specify motion, but struggle to reproduce the subtle force adjustments of human manipulation without feeling the robot's contacts. We introduce Force-Modulated Dexterity (FOMO-Dex), a unified compliance formulation for contact-rich manipulation. FOMO-Dex expresses nominal actions in coordinates defined by task geometry and contact structure. It applies force-driven motion corrections along selected directions while leaving the remaining command components unchanged. Across card picking, paper cutting, screwdriving, and nut-and-bolt assembly, FOMO-Dex reduces excessive contact forces compared with stiff control. A user study shows that operators collect more successful demonstration data per hour after brief practice. For policy learning, imitating the corrected actions improves card-picking success over imitating raw operator commands. Policies trained on FOMO-Dex demonstrations also achieve more rotation in nut-and-bolt manipulation and screwdriving than those trained on demonstrations collected with the stiff controller. Videos are shown on https://yingyuan0414.github.io/fomo-dex/ .
Human demonstrations offer a scalable data source for dexterous manipulation, but transferring them to robot actions remains challenging due to the embodiment gap. Today's retargeting is mostly kinematic, yet manipulation is decided by force, which governs how the hand interacts with the object and how the object moves. In this paper, we present ReForce, a Force-aware Retargeting method that turns human motion and forces into robot actions that reproduce the intended contact. ReForce predicts a residual on the kinematically retargeted action to reach the desired force, using a general force tracker trained on large-scale simulation interactions. It supports both online force-aware teleoperation and offline data translation. In simulation and on real hardware, ReForce achieves lower force-tracking error and stronger multi-finger contact engagement on contact-rich tasks such as paper-cup grasping and tongs manipulation.
Learning dexterous manipulation from demonstrations is bottlenecked by data: the contact forces that determine whether a grasp succeeds are absent from every scalable source of human demonstrations. This paper builds on two observations. First, what survives the change from a human hand to a robot hand is the contact structure of a demonstration - which finger regions touch which object locations, and in what order - rather than its joint motion. Second, physical consistency need not be engineered per task: a single residual reinforcement learning (RL) policy, trained once across diverse demonstrations, can repair kinematic recordings into physically consistent, contact-annotated trajectories, and the same residual formulation restores dynamic feasibility after retargeting. These observations yield a three-stage pipeline that converts human motion-capture recordings into dexterous robot policies with no real-robot training data: physics refinement with a simulated MANO hand recovers contacts and forces, contact-anchored retargeting transfers the demonstrated contact structure through an objective independent of hand morphology, and residual policy learning adapts the result to robot actuation. The pipeline reconstructs 25,454 single-hand trajectories (success 7.3% -> 59.3%) and 25 dual-hand tasks (16.0% -> 62.4%) with one shared policy per setting, transfers one human dataset to four morphologically distinct robot hands (+62.4 pp), and executes four contact-rich bimanual tasks on physical hardware with zero real-robot training data.
Zihao Yang, Chengyuan Liu, Yu Zhou +6
DexGEM Lab · Shanghai Jiao Tong University · Tongji University +1