Reinforcement learning (RL) in simulation can train dexterous manipulation policies without robot demonstrations, but training a single generalist policy with task-agnostic rewards faces a severe exploration problem: approaching, grasping, and reorienting diverse objects with many degrees of freedom is difficult to discover from scratch. Prior works make exploration tractable with high-quality robot demonstrations, per-task reward shaping, or by restricting policies to narrow modes of behavior. We propose X-Reset, a framework that instead resolves exploration with human hand-object demonstrations. Rather than imitating or tracking retargeted human motion, X-Reset kinematically retargets hand-object states to noisy robot states, filters out states that are unstable in simulation, and samples the remainder as resets during RL training with general-purpose object-centric rewards. The resulting policy depends only on object state and goal, with demonstrations entering training through the reset distribution. We show that X-Reset trains generalist policies on 20 objects across three embodiments---a 22-DoF hand on two different arms and a parallel-jaw gripper---and resolves the exploration challenges of RL from scratch. X-Reset scales with the number of training objects, generalizes to unseen objects, can learn from imperfect hand-pose estimates, and transfers behaviors zero-shot from sim-to-real.
Figures & tables
Figure 1 . Overview of X-Reset . We introduce X-Reset , a simple recipe to pair generalist object-centric RL in simulation with kinematically retargeted human hand-object states as resets. Learning to approach, grasp, and reorient objects from scratch results in a challenging exploration problem, while X-Reset directly exposes policies to high-value states from which exploration is easy. With a common reward formulation, we train generalist policies for diverse robot embodiments and demonstrate sim-to-real transfer.
Figure 2 . X-Reset Pipeline. (Top) First, we kinematically retarget human hand-object states to noisy robot states. Then, we spawn states into simulation and retain a bank of stable states. (Bottom) During RL training, we randomly sample diverse resets leveraging the retargeted states, and train an RL policy which consumes object keypoints at inference time and can be deployed in the real world.
Figure 3 . X-Reset Supplements RL Training. We report reorientation success at ϵ= 2 cm on 20 DexYCB objects (7 Easy, 7 Medium, 6 Hard). Evaluation uses 20 demonstrated start/goal pairs per object. Pale bars show RL baselines trained from scratch; saturated caps add the X-Reset gain ( p=0.9 ), with gain labels in percentage points. Avg. weights all objects equally. X-Reset improves both reward formulations across embodiments and object groups, with the best recipe being low-bias object-only rewards seeded with X-Reset for exploration on average. Error bars show 3-seed RL training SE.
Figure 4 . Scaling and Generalization. We report reorientation success at ϵ= 2 cm on 12 EBench objects. Scratch baseline trains on the EBench roster for its full 10k epochs. Blue connected markers show zero-shot transfer after 10k epochs of X-Reset pretraining on 1–20 DexYCB objects, outperforming training from scratch on EBench with as little as 10 DexYCB training objects. Orange diamonds (FT) show 5k epochs of 20-object X-Reset pretraining followed by 5k epochs of EBench fine-tuning, showing X-Reset is a strong pre-training foundation. Error bars show 3-seed RL training SE.
Figure 5 . Impact of Demo-Reset Probability. We measure Avg. Episode Return from ρdefault at train-time for UR7e+Sharpa with Obj-Only Reward. Higher demo-reset probability accelerates early learning, with all X-Reset policies converging in a similar band. Curves show 3-seed means and shaded bands show ±1 SE.
Figure 6 . Learning from Noisy Hand Poses. (Top) We visualize retargeted Sharpa hand-object states under varying levels of noise. (Bottom) Avg. episode return from ρdefault of X-Reset trained with noisy hand poses.
Figure 7 . Sim-to-Real Evaluation. (Top) UR7e+Sharpa filmstrips show each object manipulated to goal poses in the air. (Bottom) Grasp and reorientation successes out of 10 physical trials per object for Obj-Only Reward+ X-Reset .
Recent work in humanoid whole-body control has found success with a simple recipe: retarget human motion to robot kinematic references, then train policies via reinforcement learning (RL) to track them. But how does this recipe transfer to dexterous manipulation? The answer is not obvious, as manipulation involves complex, contact-rich dynamics and requires delicate regulation of contact modes and forces. We present REGRIND, a minimalist retargeting-guided RL pipeline that learns dexterous manipulation policies from a single human demonstration. REGRIND retargets human hand-object motion to a robot reference that preserves hand-object spatial and contact relationships, trains a residual RL policy in simulation to track object-centric keypoints along that reference, and transfers the resulting policy zero-shot to hardware with careful system identification. The resulting policies produce fluid, human-like behavior on two different multi-fingered hands across contact-rich tool-use tasks, including operating a pair of scissors and turning a screwdriver. Through systematic hardware experiments, we identify and analyze the key factors that govern sim-to-real transfer in dexterous manipulation, offering practical guidance for retargeting-based learning in contact-rich settings. Videos and code are available at https://yunhaifeng.com/REGRIND.
The ability to manipulate tools significantly expands the set of tasks a robot can perform. Yet, tool manipulation represents a challenging class of dexterity, requiring grasping thin objects, in-hand object rotations, and forceful interactions. Since collecting teleoperation data for these behaviors is challenging, sim-to-real reinforcement learning (RL) is a promising alternative. However, prior approaches typically require substantial engineering effort to model objects and tune reward functions for each task. In this work, we propose SimToolReal, taking a step towards generalizing sim-to-real RL policies for tool manipulation. Instead of focusing on a single object and task, we procedurally generate a large variety of tool-like object primitives in simulation and train a single RL policy with the universal goal of manipulating each object to random goal poses. This approach enables SimToolReal to perform general dexterous tool manipulation at test-time without any object or task-specific training. We demonstrate that SimToolReal outperforms prior retargeting and fixed-grasp methods by 37% while matching the performance of specialist RL policies trained on specific target objects and tasks. Finally, we show that SimToolReal generalizes across a diverse set of everyday tools, achieving strong zero-shot performance over 120 real-world rollouts spanning 24 tasks, 12 object instances, and 6 tool categories.
Learning dexterous manipulation from demonstrations is bottlenecked by data: the contact forces that determine whether a grasp succeeds are absent from every scalable source of human demonstrations. This paper builds on two observations. First, what survives the change from a human hand to a robot hand is the contact structure of a demonstration - which finger regions touch which object locations, and in what order - rather than its joint motion. Second, physical consistency need not be engineered per task: a single residual reinforcement learning (RL) policy, trained once across diverse demonstrations, can repair kinematic recordings into physically consistent, contact-annotated trajectories, and the same residual formulation restores dynamic feasibility after retargeting. These observations yield a three-stage pipeline that converts human motion-capture recordings into dexterous robot policies with no real-robot training data: physics refinement with a simulated MANO hand recovers contacts and forces, contact-anchored retargeting transfers the demonstrated contact structure through an objective independent of hand morphology, and residual policy learning adapts the result to robot actuation. The pipeline reconstructs 25,454 single-hand trajectories (success 7.3% -> 59.3%) and 25 dual-hand tasks (16.0% -> 62.4%) with one shared policy per setting, transfers one human dataset to four morphologically distinct robot hands (+62.4 pp), and executes four contact-rich bimanual tasks on physical hardware with zero real-robot training data.
Zihao Yang, Chengyuan Liu, Yu Zhou +6
DexGEM Lab · Shanghai Jiao Tong University · Tongji University +1