Sim-to-real RL has achieved impressive results in dexterous manipulation but remains limited to tasks with accurate rigid-body simulation. However, these methods cannot be applied to skills that involve deformable and particles, with which the simulation speed will become too slow for large-scale RL learning. We present DAPPT, a sim-to-real framework for end-to-end dexterous tool-use manipulation skills, e.g., sand scooping, play-dough rod-rolling, without simulating non-rigid materials. Our method learns a domain-aware RL policy by tracking procedural trajectories parameterized by domain parameters, and distills it into an end-to-end domain-aware sensorimotor policy. Furthermore, explicit domain represen-tation enables fast selection of domain parameters that can maximize downstream task performance. We evaluate DAPPT end-to-end on six dexterous manipulation tasks, including sand scooping, play-dough rod-rolling, sand scraping, mouse-wheel scrolling, dispenser pressing, and spray triggering. Our method achieves 1.5-3.8x in task metrics through the best domain parameter and transfers zero-shot to the real world.
Figures & tables
Fig. 1 : DAPPT learns dexterous skills on the 19-dof hand-arm system in rigid-body simulation and zero-shot transfer to the real-world. We evaluate on 6 dexterous manipulation tasks. The left 3 skills are scraping sand, rod-rolling play-dough, and scooping sand. The right 3 are manipulating articulated tools: mouse-wheel rolling, dispenser pressing, and spray triggering.
Fig. 2 : Illustration of domain-aware policy π(p;o;c) and an example of the domain parameters c={α,d,vi,vl} of the sand scooping task. We visualize the geometric meanings to the procedural trajectory of the scoop.
Scrape ↓
Rod-Roll ↑
Scoop ↑
Genesis Best (Ours)
17%
2.85 cm
13.9 g
SimToolReal [ 12 ]
95%
0 cm
9.2 g
Mouse-Wheel ↑
Dispenser ↑
Spray ↑
Genesis Best (Ours)
0.37 rad
# 2.26k
# 9.12k
SimToolReal [ 12 ]
0.19 rad
# 0
# 0
TABLE I : Comparison of the task metrics between Genesis Best of our domain-aware policy and the policy from SimToolReal [ 12 ] .
Fig. 3 : The percentage of difference of Genesis Best over Genesis Avg , in terms of the task metric and the average per-step tracking error (poses / joint values).
Mouse-Wheel
Dispenser
Scoop
Domain-Conditioned
0.37 rad
# 2.26k
13.9 g
Fixed-Domain
0.19 rad
# 0.96k
0.32 g
Performance Drop
48.6%
57.5%
97.7%
TABLE II : We compare the Genesis Best with our domain aware policy v.s. a policy trained on the fixed Genesis Best domain parameters. There is a significant drop in task metric.
Fig. 4 : Ablation study on without using the tele-op demo (w/o demo) or without using the contact trajectory as the referenced learning rewards (w/o contact). We show the average pose / joint tracking error during the tracking stage at the end of the RL training, clamped at a maximal of 0.2 (m/rad).
Motion Success
Task Success
Task Metric
Scooping
20/20
14/20
7.7 g
Rod-Rolling
16/20
11/20
2.7 cm
Scraping
19/20
19/20
29%
Mouse-Wheel
19/20
19/20
0.33 rad
Dispenser
18/20
14/20
N/A
Spray
16/20
10/20
N/A
TABLE III : Real-world experiment of all 6 dexterous tasks. We report motion success, task success and task metrics over task success trials.
Fig. 5 : Qualitative examples on the different between rollouts with the Genesis Best domain parameter and some other parameter (Section V-C ).
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Task
Genesis Avg
Genesis Best
Tracking Err
Task Metric
Tracking Err
Task Metric
Scooping
5.1 cm
3.64 g
4.1 cm
13.9 g
Rod-Rolling
5.6 cm
1.55 cm
4.3 cm
2.85 cm
Scraping
3.8 cm
33%
3.2 cm
17%
Mouse-wheel
0.85 rad
0.23 rad
0.85 rad
0.37 rad
Dispenser
6.7 mm
# 0.97k
7.2 mm
# 2.26k
Appendix
TABLE IV : We show the Tracking Error of average pose distance (Scooping, Rod-Rolling, Scraping) / joint values (Dispenser, Mouse-Wheel, Spray) with the target procedural trajectory and the Task Metric in simulation evaluation
Fig. 6 : Testing our arm controller calibration. We compare the trajectory with calibrated Kd s, original parameters in simulation and with the real-robot trajectory, under the same open-loop command sequence.