ADEPT: Accelerating Dexterity via Pre-Training and Post-Training using Reinforcement Learning

Jayjun Lee; Jessica Yin; Asif Rana; Nicholas Blauch; Sam Mady; Mohak Bhardwaj; Nima Fazeli; Nathan Ratliff

cs.ROAug 19, 2026

ADEPT: Accelerating Dexterity via Pre-Training and Post-Training using Reinforcement Learning

Authors: Jayjun Lee, Jessica Yin, Asif Rana, Nicholas Blauch, Sam Mady, Mohak Bhardwaj, Nima Fazeli, Nathan Ratliff, +2 more

Organizations: NVIDIA Corporation · Robotics Department, University of Michigan

Abstract

We introduce Accelerating Dexterity via Pre-Training (ADEPT), a large-scale reinforcement learning (RL) framework for learning sim-to-real transferable dexterity across high degree-of-freedom (DoF) robot embodiments that can solve long-horizon tasks directly from raw visuo-tactile perception. ADEPT pretrains a dexterous policy on a generic object reposing task, then post-trains downstream policies with this pretrained behavior as a prior. ADEPT enables learning new behaviors that are otherwise difficult to discover from scratch on multi-fingered robots and avoids learning the same set of skills over again for every new downstream task. The pretrained policy zero-shots the reposing phase of downstream tasks, but naïve RL fine-tuning rapidly degrades this capability during transfer. We address this with a stable post-training recipe combining behavior-cloning distillation, critic warm-up, and conservative on-policy updates. To safely exploit the full kinematic dexterity, we introduce a joint-space Geometric Fabric that mediates between the RL policy and the robot. We distill post-trained teachers into perceptive students that zero-shot sim-to-real transfer on two embodiments: a 23 DoF Kuka-Allegro with two RGB cameras, and a 29 DoF Flexiv-Sharpa with two RGB cameras and five vision-based tactile sensors, and can solve long-horizon tasks from challenging initial states with dexterity at human-level speed.

Figures & tables

Figure 1: ADEPT enables sim-to-real RL of high DoF arm-hand robots to reach, grasp, lift, reorient, transport, align, and insert. We demonstrate on a 23 DoF Kuka-Allegro from two RGB cameras and on a 29 DoF Flexiv-Sharpa from two RGB cameras and five fingertip tactile sensors.

Figure 2: ADEPT Overview. (1) Pre-train $(\pi_{\mathrm{pre}},V_{\mathrm{pre}})$ via PPO on a generic reposing task. (2) Post-train into $(\pi_{\mathrm{post}},V_{\mathrm{post}})$ for the downstream contact-rich task via BC distillation, frozen actor critic warm-up, and conservative PPO. (3) Distill $\pi_{\mathrm{post}}$ into a stereo RGB student $\pi_{\text{student}}$ . (4) Deploy $\pi_{\text{student}}$ zero-shot on the real robot.

Metric	Primitive	FMB Peg	VisDex [ 4 ] (Fixed Goal)
Number of objects	16	2	152
Kuka-Allegro SR $\uparrow$	$0.73\pm 0.003$	$0.76\pm 0.003$	$0.77\pm 0.011$
Flexiv-Sharpa SR $\uparrow$	$0.64\pm 0.007$	$0.58\pm 0.011$	$0.61\pm 0.015$

Table 1: Episodic success rates (SR) of pre-trained reposing teacher on unseen objects. Mean

\pm

std across seeds at the final ADR level.

	Zero-shot Success Rate (%) at ADR Level across 1024 Episodes
Task (Embodiment)	0	5	10	15	20	25	30	35	40	45	50
FMB Peg (Kuka-Allegro)	$98.6$	$94.1$	$84.7$	$78.6$	$71.9$	$67.9$	$58.7$	$52.5$	$39.6$	$18.0$	$0.00$
FMB Peg (Flexiv-Sharpa)	$94.5$	$87.1$	$80.3$	$76.8$	$74.9$	$70.4$	$66.8$	$59.2$	$52.3$	$22.9$	$0.70$

Table 2: Zero-shot success rate of the pretrained reposing teacher on FMB peg reposing across ADR.

Figure 3: Training curves on FMB peg insertion comparing (1) training from scratch, (2) direct PPO finetuning of the pretrained reposing teacher, and (3) ADEPT post-training. Blue inset: success-rate collapse for (2) during transfer.

	Variant	BC	WU	LR	Clip	ADR50	Train SR (%)	Time to ADR 50
(a)	ADEPT (full)	✓	✓	$10^{-5}$	$.05$	$5/5$	$46.0\pm 1.1$	$19.9\pm 1.0$ h
(b)	No BC	✗	✓	$10^{-5}$	$.05$	$4/5$	$38.5\pm 1.2$	$35.2\pm 4.9$ h
(c)	No warm-up	✓	✗	$10^{-5}$	$.05$	$4/5$	$28.4\pm 3.7$	$20.2\pm 1.5$ h
(d)	Standard PPO	✓	✓	$10^{-3}$	$.20$	$0/5$	$0.0\pm 0.0$	Collapse (ADR 20)
(e)	High LR only	✓	✓	$10^{-3}$	$.05$	$0/5$	$0.0\pm 0.0$	Collapse (ADR 20)
(f)	Loose clip	✓	✓	$10^{-5}$	$.20$	$5/5$	$46.8\pm 2.0$	$17.6\pm 1.8$ h

Table 3: Post-training component ablations on KUKA-Allegro FMB peg insertion;

5

independent non-PBT seeds per row, all from one pretrained checkpoint. ADR50 counts seeds reaching the final ADR level; time is averaged over seeds that reached it. Train SR is the instantaneous training success rate over parallel environments.

Figure 4: Real-world setup. (A) The 23 DoF Kuka-Allegro and 29 DoF Flexiv-Sharpa platforms, each with two RealSense RGB cameras (left and center). (B) FMB pegs and boards, plates, and dish rack. (C) Diverse initial states and lighting used in real-world testing.

Modality	Robot	Task	Reach	Grasp	Lifting	Reorient	Align	Insert (SR)
Vision	Kuka-Allegro	FMB Star	10/10	9/10	8/10	8/10	7/10	5/10
Vision	Kuka-Allegro	FMB Square/Round	10/10	8/10	6/10	4/10	3/10	3/10
Vision	Flexiv-Sharpa	FMB Square/Round	10/10	7/10	5/10	3/10	3/10	3/10
Visuo-Tactile	Flexiv-Sharpa	FMB Square/Round	10/10	10/10	10/10	9/10	8/10	8/10
Vision	Kuka-Allegro	Dish	10/10	10/10	8/10	7/10	6/10	6/10

Table 4: Per-stage success rates on real-world long-horizon manipulation, trained with ADEPT. Each stage is cumulative: success at stage

k

requires success at all earlier stages.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Reposing Task. Visualizes the out-of-bound range in red, the goal sampling range in green, and the object spawn range in orange. (a) visualizes a frame of the initial grasp of the object and (b) visualizes a frame where the robot has completed the task by reaching the target pose frame. The object point clouds can be seen in green spheres, which are on the object.

Figure 6: 16 primitive objects used during pre-training reposing task. The scale and physical properties of the objects are randomized during pre-training.

Figure 7: Downstream FMB Task. Visualizes the peg (orange) and board (sky blue) spawn ranges, object keypoints and goal pose keypoints. Grey keypoints represent the path set by the ADR level (denoted goal path annealing). At ADR 0, the goal is set to where the light blue sphere is placed and at ADR 50, the goal is set to the insertion goal where the light green sphere is placed. (a) shows sample initialization of the scene where the peg is dropped on to the table. In (b), we show how the reposing teacher reaches the goal at ADR 20, which is above the board. In (c), we visualize a sample policy rollout that inserts the peg into the board successfully where the keypoints align by the matching colors. The object point clouds in green spheres can be seen on the peg.

Hyperparameter	Pretraining (reposing)	Post-training (FMB)
PPO Actor learning rate	$1\times 10^{-3}$	$1\times 10^{-5}$
Actor LR schedule	KL-adaptive	linear decay over Max epochs
BC actor learning rate	-	$1\times 10^{-4}$
Max epochs	$750{,}000$	$200{,}000$
Critic learning rate	$5\times 10^{-5}$	$5\times 10^{-5}$
PPO clip $\epsilon$	$0.20$	$0.05$

Appendix

Table 5: PPO hyperparameter comparison between reposing pretraining and FMB post-training. The actor learning rate is reduced by

100\times

and the PPO clip is tightened by

4\times

during post-training, while the critic learning rate is held constant. The actor learning-rate schedule changes from KL-adaptive (pretraining, with target KL = Actor KL threshold) to linear decay over Max epochs (post-training); the Actor KL threshold is logged but not used during post-training under the linear schedule.

	Stage 1 : reposing		Stage 2 : FMB		Stage 3 : Distillation
Observation field	Actor	Critic	Actor	Critic	Teacher (actor)	Student (actor)
Proprioceptive / robot state
robot_dof_pos (noisy)	23	23	23	23	23	23
robot_dof_vel (noisy)	23	23	23	23	23	23
hand_pos (5 bodies $\times$ 3)	15	15	15	15	15	15
hand_vel (5 bodies $\times$ 3)	15	15	15	15	15	15

Appendix

Table 6: Observation fields consumed at each pipeline stage in the current Cspace-mode codebase. Stage 1 = reposing pretraining (PPO, actor + asymmetric critic); Stage 2 = FMB post-training (PPO, actor + asymmetric critic); Stage 3 = two-camera RGB vision distillation, where a student network is supervised by the Stage 2 actor (treated as the teacher) via behavior cloning. Actor and student rows read the noisy variant of each proprioceptive field; the critic reads the corresponding clean variant. Each cell shows the dimensionality of the field when it is consumed at that stage; ✗ means the field is not consumed at that stage, and “image” denotes an image-tensor input. Rows newly introduced at Stage 2 (FMB extras / nominal delta) and Stage 3 (two-camera RGB) are highlighted.

	Stage 1 : reposing		Stage 2 : FMB		Stage 3 : Distillation
Observation field	Actor	Critic	Actor	Critic	Teacher (actor)	Student (actor)
Proprioceptive / robot state
proprio (29 joints + 6 FK bodies)	94	94	94	94	94	94
last_actions	29	29	29	29	29	✗
fabric_q , qd , qdd ( $3\times 29$ )	87	87	87	87	87	87
proprio_nominal_delta (29 joint + 18 FK)	✗	✗	✗	✗	✗	✗

Appendix

Table 7: Flexiv–Sharpa (29 DoF). Observation fields consumed by the Rizon–Sharpa Cspace pipeline. Stage 1 uses the reposing PBT actor and asymmetric critic; Stage 2 uses the FMB bootstrap teacher; Stage 3 distills that state-based teacher into the stereo RGB + five-fingertip TacMap student. The entries and dimensions follow the checked-in agent configurations. “Image” denotes a tensor encoded outside the flat low-dimensional input.

Figure 8: Training curves on FMB peg insertion comparing (1) training from scratch, (2) direct PPO finetuning of the pretrained reposing teacher, and (3) ADEPT post-training. Blue inset: success-rate collapse for (2) during transfer.

Figure 9: Training curves across different seeds for training from scratch. ADEPT post-training reliably converges to a solution while most seeds for training from scratch either plateau or take a very long time to train for a fixed compute budget.

Figure 10: Real-world camera views for (a) Kuka-Allegro and (b) Flexiv-Sharpa setups.

Figure 11: A sample 16x16 grid of visual randomizations on the left camera of the FMB training scene for distillation.

Parameter	Range / Distribution
Physics (per-env at reset, ADR-annealed)
Object mass scale	$(0.5,\,3.0)\times$ nominal
Object static friction	$(0.5,\,1.2)$
Object dynamic friction	$(0.3,\,1.0)$
Object restitution	$(0.8,\,1.0)$
Robot joint stiffness scale	$(0.5,\,2.0)\times$ default

Appendix

Table 8: Domain randomization parameters used during teacher–student distillation. Per-env ranges are sampled uniformly at episode reset; ADR-annealed ranges advance from the lower bound to the upper bound over training as the running success rate exceeds the ADR threshold; per-step noise is added at every control step; per-frame randomization is applied at each camera render.

Figure 12: Comparison of grasp behaviors. (a, b, c, d) Downstream-task teacher trained from scratch at ADR level 50, with grasps that lack the natural finger gaits emerging from reposing pretraining. (e, f) Pretrained reposing teacher applied zero-shot to FMB peg insertion at ADR level 20, occasionally producing undesirable grasps (e.g., bottom-end, upside-down) that solve the reposing segment but fail at insertion. (g, h) Downstream-task teacher trained with ADEPT at ADR level 50, with natural grasps where the fingers wrap around the peg and caging.

Explore similar work

Pre-training Visual Dexterity in Simulation
Aug 16, 2026Sarthak Kamat, Adam Rashid, Satvik Sharma +4Robotic Manipulation Dexterous Manipulation
Stanford University · MIT · Scale AI
DexSim2Real: Foundation Model-Guided Sim-to-Real Transfer for Generalizable Dexterous Manipulation
May 3, 2026Zijian Zeng, Fei Ding, Huiming Yang +2Domain Randomization Dexterous Manipulation
Tsinghua University · Alibaba Group · Bengbu University +1
SkillWeave: Weaving Heterogeneous Demonstrations into Long-Horizon Manipulation Skills
Oct 8, 2026Ryosei Tamura, Xiaoxiang Dong, Uksang Yoo +4Robotic Manipulation Long-Horizon Robotic Manipulation
Robotics Institute, Carnegie Mellon University, USA. · Department of Computer Science, Keio University, Japan. · College of Engineering, University of California, Berkeley, USA. +1

Aug 16, 2026, cs.ROJ/K move · Enter open · S save

Pre-training Visual Dexterity in Simulation

Sarthak Kamat, Adam Rashid, Satvik Sharma, Aseem Doriwala+3

Stanford University · MIT · Scale AI

Robotic Manipulation Dexterous Manipulation Data-Efficient Robot Learning Bimanual Robotic Manipulation

Open paper AskPDF arXiv

Large-scale pre-training has made robot policy fine-tuning increasingly data-efficient, but this progress has largely been driven by datasets and embodiments built around simple parallel-jaw grippers. Dexterous, multi-fingered hands remain comparatively data-starved because real teleoperation is costly to scale, while human hand video is off-embodiment and requires lossy pose estimation and retargeting. We introduce Simulation Pre-training for Dexterity (SPD), a pre-training framework for dexterous manipulation that uses data entirely collected in simulation. In SPD, humans manipulate virtual objects inside a VR headset, enabling on-embodiment trajectories and robot-free collection. With the help of five operators, we collect 75 hours of multi-task dexterous manipulation over one week, and use it to pre-train a causal transformer on a sequence modeling objective. We study the benefits of simulation pre-training on real-world tasks by fine-tuning on 1-2 hours of physical demonstrations on a 56-DoF bimanual dexterous setup. We find that our approach outperforms training behavior cloning policies from scratch, showing that simulation teleoperation is a viable pre-training source for real-world dexterous manipulation. We perform ablation studies, measuring the benefits of history conditioning and short action chunks for reactive control.

Abstract

Explore similar work

Pre-training Visual Dexterity in Simulation

DexSim2Real: Foundation Model-Guided Sim-to-Real Transfer for Generalizable Dexterous Manipulation

SkillWeave: Weaving Heterogeneous Demonstrations into Long-Horizon Manipulation Skills