ReForce: Learning Force-aware Retargeting for Dexterous Manipulation

Yuhang Wu; Lingqi Zeng; Changwei Jing; Jianglong Ye; Xiaolong Wang

cs.ROAug 16, 2026

ReForce: Learning Force-aware Retargeting for Dexterous Manipulation

Authors: Yuhang Wu, Lingqi Zeng, Changwei Jing, Jianglong Ye, Xiaolong Wang

Organizations: UC San Diego

Abstract

Human demonstrations offer a scalable data source for dexterous manipulation, but transferring them to robot actions remains challenging due to the embodiment gap. Today's retargeting is mostly kinematic, yet manipulation is decided by force, which governs how the hand interacts with the object and how the object moves. In this paper, we present ReForce, a Force-aware Retargeting method that turns human motion and forces into robot actions that reproduce the intended contact. ReForce predicts a residual on the kinematically retargeted action to reach the desired force, using a general force tracker trained on large-scale simulation interactions. It supports both online force-aware teleoperation and offline data translation. In simulation and on real hardware, ReForce achieves lower force-tracking error and stronger multi-finger contact engagement on contact-rich tasks such as paper-cup grasping and tongs manipulation.

Figures & tables

Figure 1: Force-aware retargeting overview. Human-guided motion and contact references specify the intended interaction, while ReForce uses robot-side fingertip force feedback to adapt the reference into dexterous-hand commands during contact-rich execution.

Figure 2: ReForce training and inference pipeline. During training, randomized and augmented simulation interactions provide joint and force trajectories from which future-window motion and contact targets are constructed. During deployment, ReForce adapts references using online fingertip force feedback.

Figure 3: Human demonstration and robot execution setup. Teleoperation uses a Quest controller for wrist pose, a Manus glove for hand pose, and calibrated fingertip FSR sensors for human contact force. For ACT demonstration collection, a RealSense L515 camera tracks an AprilTag attached near the wrist, while the glove and FSR setup remains unchanged. A UFACTORY xArm equipped with an XHand executes the robot-side motion and provides fingertip tactile feedback.

Grasp	Replay	Admittance	ReForce
Side	$0.309\pm 0.006$	$0.280\pm 0.013$	$\mathbf{0.247\pm 0.036}$
Top	$0.736\pm 0.000$	$0.619\pm 0.048$	$\mathbf{0.474\pm 0.049}$

Table 1: Real-world paper cup grasping replay. Force-tracking error in Newtons, reported as mean

\pm

standard deviation over five trials. Lower is better.

Figure 4: Qualitative comparison of execution behavior. Direct replay does not react to contact error, while admittance control uses a fixed force-to-motion mapping. ReForce learns contact-dependent command corrections from interaction data.

Task	Method	Force-safe success $\uparrow$	Tracking error $\downarrow$	Over-force $\downarrow$	Severe missing contact $\downarrow$	Active fingers $\uparrow$
Paper cup	Reference policy	$30.0\%$	$0.812$	$0/10$	$7/10$	$0.17$
Paper cup	Reference policy + Admittance	$60.0\%$	$0.683$	$4/10$	$1/10$	$1.26$
Paper cup	Reference policy + ReForce	$\mathbf{70\%}$	$\mathbf{0.124}$	$2/10$	$\mathbf{0/10}$	$\mathbf{2.61}$
Tongs	Reference policy	$50.0\%$	$0.454$	$0/10$	$5/10$	$0.94$
Tongs	Reference policy + Admittance	$\mathbf{90.0\%}$	$\mathbf{0.419}$	$0/10$	$1/10$	$1.34$
Tongs	Reference policy + ReForce	$54.5\%$	$0.441$	$4/11$	$1/11$	$\mathbf{1.62}$

Table 2: Real-world execution with learned references. Force-safe success requires task completion, forces below the task-specific threshold, and no severe missing-contact failure. Tracking error is the mean absolute force error over time and fingertips, reported in Newtons.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Item	Value
Future-target horizon	Up to $16$ future frames; joint and force targets are averaged over the available future window
Policy architecture	Per-finger MLP policy; the four non-thumb fingers use a shared $[128,128]$ trunk with $[64]$ finger-specific heads, while the thumb uses a separate $[128,128]$ trunk and $[64]$ head; ReLU activations
Joint-update limit	$0.088$ rad per controlled joint, approximately $5^{\circ}$
Observation perturbations	Gaussian joint noise, Gaussian force noise with standard deviation $0.1$ N, and per-finger force dropout with probability $0.30$
Optimization	AdamW with learning rate $10^{-3}$ , weight decay $10^{-5}$ , batch size $1024$ , $20$ epochs, and $500$ training steps per epoch

Appendix

Table 4: ReForce training configuration. Main hyperparameters used for the reported experiments.

Item	Value
Training data	Separate policy per task; approximately $30$ – $40$ human demonstrations per task
Temporal context	History length $H=9$ and prediction chunk length $C=32$
Sampling frequency	Raw demonstrations at $83$ Hz; training anchors sampled at $10$ Hz
Phase representation	$D_{\phi}=3$ , with $\phi_{t}=[\tau_{t},\sin(2\pi\tau_{t}),\cos(2\pi\tau_{t})]$ , where $\tau_{t}=(t-1)/(T-1)$
Force-loss weight	$\lambda_{F}=1.0$
Network architecture	Context MLP with widths $[128,128,256]$ , followed by separate $2$ -layer motion and force Transformer decoders with width $256$ , $4$ attention heads, feed-forward width $1024$ , and dropout $0.1$

Appendix

Table 5: ACT reference-policy training configuration. Main parameters used for the learned motion-and-force reference policies.

Figure 6: Force tracking under replayed references. Real-robot force trajectories for side and top paper-cup grasps using direct replay, admittance control, and ReForce . These curves correspond to the results in Table 4.1.1 .

Explore similar work

A Minimalist Retargeting-Guided Reinforcement Learning Recipe for Dexterous Manipulation
Jul 13, 2026Yunhai Feng, Natalie Leung, Jiaxuan Wang +3Robot Skill Learning Contact-Rich Robotic Manipulation
Cornell University · Amazon FAR
Generative Neural Retargeting for Human-to-Robot Dexterous Manipulation
Oct 8, 2026Dechen Gao, Yue Yang, Ben Abbatematteo +8Motion Retargeting Robotic Manipulation
Meta Reality Labs Research · UC Davis · UNC Chapel Hill +3
TopoRetarget: Interaction-Preserving Retargeting for Dexterous Manipulation
Jun 15, 2026Jielin Wu, Shenzhe Yao, Guanqi He +6Robot Skill Learning Contact-Rich Robotic Manipulation
IIIS, Tsinghua University

Jul 13, 2026J/K move · Enter open · S save

A Minimalist Retargeting-Guided Reinforcement Learning Recipe for Dexterous Manipulation

Yunhai Feng, Natalie Leung, Jiaxuan Wang, Lujie Yang+2

Cornell University · Amazon FAR

Robot Skill Learning Contact-Rich Robotic Manipulation Sim-to-Real Transfer Motion Retargeting Robotic Manipulation Robot Imitation Learning+1

Open paper AskPDF arXiv

Recent work in humanoid whole-body control has found success with a simple recipe: retarget human motion to robot kinematic references, then train policies via reinforcement learning (RL) to track them. But how does this recipe transfer to dexterous manipulation? The answer is not obvious, as manipulation involves complex, contact-rich dynamics and requires delicate regulation of contact modes and forces. We present REGRIND, a minimalist retargeting-guided RL pipeline that learns dexterous manipulation policies from a single human demonstration. REGRIND retargets human hand-object motion to a robot reference that preserves hand-object spatial and contact relationships, trains a residual RL policy in simulation to track object-centric keypoints along that reference, and transfers the resulting policy zero-shot to hardware with careful system identification. The resulting policies produce fluid, human-like behavior on two different multi-fingered hands across contact-rich tool-use tasks, including operating a pair of scissors and turning a screwdriver. Through systematic hardware experiments, we identify and analyze the key factors that govern sim-to-real transfer in dexterous manipulation, offering practical guidance for retargeting-based learning in contact-rich settings. Videos and code are available at https://yunhaifeng.com/REGRIND.

Abstract

Explore similar work

A Minimalist Retargeting-Guided Reinforcement Learning Recipe for Dexterous Manipulation

Generative Neural Retargeting for Human-to-Robot Dexterous Manipulation

TopoRetarget: Interaction-Preserving Retargeting for Dexterous Manipulation