FlashDexRetarget: Accelerating Dexterous Manipulation Data Generation through Multi-Motion Retargeting

Kyungmin Lee; Sibeen Kim; Dongyoon Hwang; Yoonsang Oh; Donghu Kim; Youngdo Lee; I Made Aswin Nahrendra; Jaegul Choo

cs.ROOct 1, 2026

FlashDexRetarget: Accelerating Dexterous Manipulation Data Generation through Multi-Motion Retargeting

Authors: Kyungmin Lee, Sibeen Kim, Dongyoon Hwang, Yoonsang Oh, Donghu Kim, Youngdo Lee, I Made Aswin Nahrendra, Jaegul Choo, +1 more

Organizations: KAIST AI · Holiday Robotics

Abstract

Human hand-object demonstrations provide a scalable source of data for dexterous robot learning, but transferring them across embodiments requires physically feasible retargeting. Existing physics-based methods typically optimize each demonstration independently, leading to either limited success under finite simulation budgets or training costs that grow with dataset size. We introduce FlashDexRetarget, an RL framework for multi-reference dexterous retargeting. We formulate retargeting as multi-reference tracking, jointly learning a single policy across many demonstrations with off-policy RL and geometric supervision of the demonstrated interactions. This shared training formulation amortizes optimization across references while enabling the policy to track diverse hand-object interactions. On a 50-motion benchmark from TACO, OakInk2, and HOT3D using XHand and Sharpa Wave Hand as target embodiments, FlashDexRetarget retargets 90% of demonstrations using about 30 GPU-hours, compared with about 46% at about 3,000 GPU-hours for CHORD. This corresponds to about 100 times lower training compute and a 44-percentage-point improvement in retargeting success. Ablations examine the key design choices, while experiments with up to 1,000 motions and real-world replay further demonstrate the scalability and practical applicability of our method.

Figures & tables

Fig. 2: Qualitative retargeting results. Representative snapshots of dexterous robot manipulation generated by FlashDexRetarget in simulation, spanning tool use, container handling, and everyday object manipulation across diverse object geometries and hand-object configurations.

Method	$\mathrm{Motion}$	$\mathrm{SR}_{\mathrm{SPIDER}}$ $\uparrow$	$\mathrm{SR}_{\mathrm{MT}\text{-}\mathrm{Obj}}$ $\uparrow$	$\mathrm{SR}_{\mathrm{MT}}$ $\uparrow$	MOPE (mm) $\downarrow$	MORE (rad) $\downarrow$	GPU-h $\downarrow$
(a) XHand
Do as I Do [ 5 ]	Single	0.36	0.04	0.00	47.32	0.28	66
DexMachina [ 9 ]	Single	0.48	0.32	0.12	37.49	0.29	447
CHORD [ 7 ]	Single	0.46	0.02	0.00	55.70	0.16	2,847
FlashDexRetarget	Multi	0.90	0.86	0.86	10.95	0.25	29
(b) Sharpa Wave Hand

TABLE I: Quantitative comparison of dexterous motion retargeting methods

Fig. 3: Ablation of the components of FlashDexRetarget. Curves show the cumulative number of successfully converted reference motions from the 50-motion benchmark over 300 million environment steps. Each reference is counted once a rollout satisfies $\mathrm{SR}_{\mathrm{MT\text{-}Obj}}$ . We examine (a) future reference encoding; (b) point-cloud (PCD) and interaction (Int) reward components, with the corresponding observations and rewards ablated together; (c) a unified actor-critic versus left- and right-hand actor-critic pairs; and (d) PPO, default FlashSAC, and FlashSAC with a larger replay buffer, a wider critic, or both.

Fig. 4: Real-world validation of retargeted trajectories. The robot successfully executes three retargeted manipulation tasks in the real world: wiping a board, pouring into a pan, and closing a lid.

Fig. 5: Scaling to larger reference collections. Cumulative successful conversions for shared policies on 200, 500, and 1,000 reference motions. Larger collections yield more conversions for the same GPU-hour budget.

Explore similar work

Jul 4, 2026cs.RO

ObjRetarget: An Object-Aware Motion Retargeting Framework with Anthropomorphic Arm Constraints and Polyhedral Hand Modeling

Learning robot dexterous manipulation from human manipulation videos requires reliably retargeting human intent to executable robot actions while maintaining stable hand-object contact, which remains a key challenge in embodied intelligence. Existing retargeting methods often ignore explicit contact modeling or rely on reinforcement learning, resulting in limited accuracy and generalization. To address this, we propose ObjRetarget, a human-to-robot motion retargeting framework for learning robot dexterous manipulation from human videos, which integrates anthropomorphic arm trajectory constraints with structured hand-object geometric modeling. For arm motion, reference trajectories extracted from human videos are used for initialization, followed by anthropomorphic constraints and redundancy-aware optimization to generate natural and accurate movements. For hand manipulation, ObjRetarget represents multi-finger contacts using polytope clusters and preserves contact structure through geometric invariants to improve stability. Experiments on real robots show that ObjRetarget improves manipulation success rates and contact stability across multiple dexterous tasks, and generalizes well to different demonstrations, object poses, and task settings.

Yuanchuan Lai, Qing Gao, Ziyan Liang +2

School of Electronics and Communication Engineering, Sun Yat-sen University, Shenzhen 518107, China. · School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen, Shenzhen 518172, China. · School of Computing, University of Portsmouth, Portsmouth PO1 3HE, UK.

Cross-Embodiment Retargeting Dexterous Manipulation Robotic Systems

Ask PDF

Jul 9, 2026cs.RO

AnyDexRT: Calibration-Free Dexterous Hand Retargeting with Few-Shot Human Guidance

Teleoperation is a key interface for controlling dexterous robotic hands and collecting demonstrations for imitation learning. Its effectiveness largely depends on kinematic retargeting, which maps operator hand motions to feasible and intuitive robot hand motions. Existing methods often require hand-crafted objectives, precise calibration, or global shape matching between human and robot hand spaces, making them sensitive to hand-specific tuning and less reliable across different dexterous hands. We propose AnyDexRT, a calibration-free retargeting method for intuitive dexterous teleoperation across human-like dexterous hands. AnyDexRT combines self-supervised fingertip correspondence learning with few-shot human guidance to anchor the mapping in task-relevant regions, and further refines pinch-related poses using a contact classifier. Experiments on diverse dexterous hands and real-world teleoperation tasks show that AnyDexRT improves retargeting quality, reduces manual tuning, and provides more intuitive and efficient control than prior retargeting methods. Project website: https://chenxi-wang.github.io/projects/anydexrt

Chenxi Wang, Ying Feng, Hongjie Fang +4

1Noematrix · 2Shanghai Jiao Tong University · 3Shanghai Innovation Institute

Cross-Embodiment Retargeting Dexterous Manipulation Teleoperation Imitation Learning Ground-Truth Correspondence

Ask PDF

Jun 15, 2026cs.RO

TopoRetarget: Interaction-Preserving Retargeting for Dexterous Manipulation

Human hand-object demonstrations provide dense reference motions for training dexterous manipulation reinforcement learning (RL) policies through reference tracking. However, to use such demonstrations for RL policy learning, retargeting must preserve hand pose and task-relevant hand-object contact structure. Otherwise, contact and feasibility artifacts can degrade downstream RL policy performance. We introduce TopoRetarget, an interaction-preserving retargeting framework that uses a single set of parameters across diverse retargeting conditions while maintaining task-relevant hand-object interaction and adapting human demonstrations to dexterous robot hands. The method constructs a sparse interaction graph over hand and object keypoints and optimizes distance-weighted Laplacian deformation with directional consistency, kinematic constraints, and penetration handling. Evaluations show that the generated references improve both interaction fidelity and policy learning: TopoRetarget achieves the best contact precision and alignment over all baselines on the ContactPose Dataset, improves Pen-Spin training success by 40.6 percentage points over the existing baseline methods, and enables zero-shot transfer to Wuji Hand hardware on cube reorientation and pen spinning.

Jielin Wu, Shenzhe Yao, Guanqi He +6

IIIS, Tsinghua University

Cross-Embodiment Retargeting Dexterous Manipulation

Ask PDF