cs.ROOct 24, 2026
SaveLearning Dexterous Manipulation: Hands, Simulators, Training, and Evaluation
Organizations: Independent Researcher
Abstract
A dexterous hand can change an object's pose without setting it down. This survey covers 113 learning-based methods that attempt it, the hands they run on, the simulators they train in, and how they report results. The field has largely converged on one recipe: reinforcement learning on privileged state in a GPU simulator, distilled into a vision-only student for transfer to hardware. Almost half the corpus trains that way, nearly all of it in one simulator, whose own successors are barely used. A second and faster-growing branch learns from teleoperated demonstration instead, through vision-language-action models, diffusion policies, and systems whose contribution is the collection rig itself. Published work has tripled since 2022. Grasping remains the largest task family and bimanual coordination the fastest-rising, and almost every experiment runs on one of four hands. What the survey adds comes from reading released code and vendor specifications beside the papers. Eight hands that can be bought or built from published designs appear in no method paper here. Where the depth a hand sinks into the object it holds is reported at all, it is not measured in the physics that produced the motion. Section VII proposes a protocol for the unreported quantities; the body gives the counts behind every claim above.
Figures & tables
Fig. 1 : The field on one page, and the map of this survey: nodes sized by corpus papers, edges drawn for the routes that carry at least two fifths of the smaller endpoint, so the dominant path reads as a path, each column head naming the section that covers it.
| task family | what is held | what moves | what gravity does | what counts as failure | shortest success criterion as stated |
|---|---|---|---|---|---|
| In-hand reorientation | one rigid object, held by the fingers for the whole episode | the object’s orientation in the palm frame, while contacts break and remake | a disturbance whose direction is set by hand orientation, and a load to counteract with the palm down, [ 16 ] | the object leaves the hand, e.g. the fall reset at goal distance 0.24 m in [ 14 ] | orientation error below a stated tolerance under a stated stopping rule, 0.4 rad in [ 2 ] and [ 14 ] , 0.1 rad in [ 15 ] |
| Grasping | nothing at the start, the object rests on a support | the hand closes, then the object is lifted clear | the test itself, since the grasp exists to resist it | no lift, or a drop inside the hold window | a stated lift height held for a stated duration, 10 cm for 20 s in [ 10 ] , 10 cm to episode end in [ 47 ] , 0.5 s in [ 11 ] |
| Functional and tool use | a tool, held in the pose its function requires rather than the most stable one | the tool’s own joint, or the tool acting on a second object | secondary, the function dominates | a stable grasp in the wrong place, [ 13 ] | no agreed criterion. Criteria range from never dropping the object, [ 13 ] , to a four-point rubric, [ 48 ] . The one mechanical threshold is holding the tool joint past a commanded angle for the episode, [ 49 ] |
| Tracking a human reference | whatever the human held in the captured clip | hand and object follow a recorded trajectory under physics | full, and the reference never obeyed it, so the reference is not dynamically feasible | per-frame error leaves the band, or the object drops | all stated per-frame errors below threshold for the whole clip, 30 deg and 3 cm and 8 cm joint and 6 cm fingertip in [ 17 ] . Thresholds are not comparable across papers, and DexMachina [ 18 ] calls the results “highly sensitive to the threshold values” |
| Bimanual coordination | one object by two hands, or one object per hand | both hands, the object, and the relative pose of the hands | can be carried by one hand while the other acts, [ 40 ] | the object drops, the arms collide, or the hands work against each other | no agreed criterion. Task-specific thresholds dominate, e.g. 0.035 m block-to-cup distance in [ 40 ] . The one criterion with a physical-plausibility gate is 2.0 s of hold with total penetration under 1.5 mm, [ 41 ] |
| Handover and in-hand transfer | by the giver at the start, by the receiver at the end | the object crosses between hands, released only after it is regained | unopposed during the exchange, and total in a thrown transfer, [ 19 ] | a drop at the moment of release, and the catcher’s grasp phase, [ 19 ] Sec. 5.4 | the receiver holds the object and moves it clear, more than 10 cm from the first hand in [ 50 ] , or holds it to episode end in [ 42 ] |
TABLE I : Task families against the properties that define success.
Fig. 2 : Three answers to where the motor goes, and what each one costs. A coupling bar saves an actuator by tying two joints, so the fingertip reaches a curve through the reachable region rather than the region itself.
Fig. 3 : Which hands the literature runs on, banded by how the hand can be got, with every tabulated hand shown and those no paper uses as named slots at zero. A paper naming two hands counts under both.
Fig. 4 : The geometry the solver uses against the geometry the renderer draws. A single convex hull fills the concavity with material that exists only in the collision model, and holds the fingertip out of a contact the visual mesh allows.
Fig. 5 : One simulation step, and where engines differ: the stages every rigid-body engine runs, with the choices that separate the engines used for hands under each; dashed arrows mark the three stages that overlap.
Fig. 6 : A compliant tactile sensor reads the overlap depth. The tactile image, the distributed force and the binary contact flag are all functions of the one quantity every other engine in this section works to drive to zero.
Fig. 7 : What supervises a dexterous policy, subdivided to where the 113 method papers actually differ; each leaf names two of its corpus keys and counts the rest, bars scaled by paper count, dashed curves marking overlap.
Fig. 8 : A retargeted pose can be scored good and still be inside the object. The joint-angle error is a sum over joints with no term for the object; a few degrees at each joint compose along the finger into a fingertip displacement larger than a thin object is thick.
Fig. 9 : Papers whose released code was readable, and the 8 where a value in it differs from the value the paper prints. Table II names them, with the file and both values.
| method | conf. | repository | file, linked at the revision read | what the paper prints | what that file contains |
|---|---|---|---|---|---|
| PDDM [ 108 ] | high | google-research/pddm | cube_env.py _get_obs | observation dimension 46 for in-hand reorientation, Table 2 | the six fields _get_obs concatenates sum to 39 on the code’s own inline comments |
| DeXtreme [ 14 ] | high | isaac-sim/IsaacGymEnvs | AllegroHandDextremeADR.yaml actionDeltaPenaltyScale | action-delta penalty weight -0.25, Table 2 | actionDeltaPenaltyScale: -0.2 , and -0.01 in the ManualDR yaml beside it |
| Visual Dexterity [ 16 ] | high | Improbable-AI/dexenv | dclaw.yaml alg.num_envs | 32,000 teacher training environments, Table S1 | num_envs: 8000 , and 16384 in the parent hand_default.yaml |
| DexPBT [ 110 ] | high | NVIDIA-Omniverse/IsaacGymEnvs | allegro_kuka_base.py compute_kuka_reward | four staged reward terms, Sec. III-C | eight summed components, hand_delta_penalty multiplied by 0 with the comment currently disabled |
| PhysHOI [ 143 ] | high | wyhuai/PhysHOI | physhoi.py compute_humanoid_reward | object-rotation weights 0.1 and 0.01 for GRAB, Table 4 | eor and eorv set to torch.zeros_like(ep) , the computation commented out on the same lines |
| UniDexGrasp [ 151 ] | high | PKU-EPIC/UniDexGrasp | shadow_hand_grasp.py compute_hand_reward, goal_cond branch | seven named weights from 0.1 to 10, Table 7 | a torch.where cascade on literals -0.5, -1.0, 0.9, 0.1, 0.2, none of the seven present |
TABLE II : The 8 papers whose repository states a different value from the paper. Each file links to the revision read.
Fig. 10 : Four mechanisms for controlling two dexterous hands, counted over the 28 papers in this corpus with a learned closed-loop controller on two multi-fingered hands, each read from its note. The panels are not a partition: the fourth is the same single paper as the third, and the two papers whose notes never say which architecture they use appear in neither.
Fig. 11 : Where a capture set’s contact annotation comes from. Nothing in the chain is measured at a surface, and a vertex that has gone through the surface is written down as a contact rather than as an error.
Fig. 12 : Two failures of synthesised two-hand grasps, each enlarged at the contact: (A) a finger inside the object, (C) a finger inside the other hand. Both look plausible whole [ 41 ] .
Fig. 13 : What gets reported, and what does not. Each bar is a count of the 113 corpus method papers; the pale track behind it is the denominator that statistic is quoted against, printed beside the count, because four of the six are not 113. A faint tail past a track is the papers whose notes left that question unsettled. The two rarest reporting practices are marked in color.
Fig. 14 : Where a denominator is stated it is small: the 39 per-cell real-trial counts, binned by fives, median 15 marked. The 24 grand totals are a separate quantity, counted beneath the axis and not pooled in.
| axis | what is measured | minimum trials | reported with it |
|---|---|---|---|
| Task success | fraction of episodes meeting a criterion written before the run, and a graded rubric score in | 100 real and 200 sim per policy, task and condition for an absolute rate at a 10-point interval; a comparison alone needs 57 matched pairs per arm | raw counts, the criterion verbatim, the rubric, the initial-condition protocol, and an interval whose kind is named |
| Robustness | success under each perturbation axis, and the ratio to the unperturbed anchor | 40 per axis per policy to rank the axes; 101 per axis and 101 at the anchor before any claim that a named axis hurt | both absolute rates with their Wilson intervals, the axis ranges, and the ratio without an interval unless the certifying count was run |
| Unseen objects | mean over held-out objects of per-object success | 20 objects at 5 trials to screen; 93 objects for a 10-point claim | the object list, the per-object rates, a bootstrap interval over objects, and the within-object trial count |
| Physical plausibility | maximum and mean hand-object penetration depth per rollout, and the fraction of frames above 2 mm | 100 rollouts, simulation only | an interval over rollouts, the sample density, the threshold, the solver’s depenetration settings, and at least one rendered rollout |
| Sample and wall-clock cost | environment steps and wall-clock to the checkpoint that produced the headline number | 3 training seeds | the range over seeds, and the statement that 3 seeds is a range and not an interval |
| Real-robot transfer | paired real and simulated outcomes on matched initial conditions, their correlation, and the rectifier variance against the real variance | the 100 real trials paired to 100 sim; about 457 pairs to separate two correlations 0.1 apart | the correlation with its Fisher-z interval, the variance ratio with its bootstrap interval, and which decision the count supports |
TABLE III : The proposed evaluation protocol, and this survey’s own contribution. No cell is empty: every cell is a proposal.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
| hand | maker | DoF | act. DoF | actuation | mass g | force N | force kind | tactile | price USD | open HW | status | uses |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Shadow Dexterous Hand [ 51 ] | Shadow Robot Co. | 24 | 20 | tendon | 4 300 | – | – | optional | – | no | sold | 21 |
| Proception ProHand 1.0 [ 87 ] | Proception Inc | 22 | – | tendon | – | – | – | unstated | – | no | sold | 0 |
| SharpaWave 22-DoF [ 80 ] | Sharpa Robotics | 22 | 22 | – | 1 300 | 20 / 12 | grip | unstated | – | no | sold | 6 |
| BiDexHand [ 59 ] | Northwestern Univ. | 21 | 16 | tendon | – | 2.14 | fingertip normal | none | – | yes | open-source | 0 |
| Ruka-v2 [ 57 ] | NYU and NYU Shanghai | 20 | 16 | tendon | – | – | – | optional | 1 500 | yes | open-source | 0 |
| TESOLLO DG-5F [ 86 ] | Tesollo Inc. | 20 | 20 | direct drive | 1 763 | – | – | none | – | no | sold | 0 |
TABLE IV : Hands that can be obtained, sorted by joint count. 68 of 160 specification cells are empty.
| hand | maker | DoF | act. DoF | actuation | mass g | force N | force kind | tactile | price USD | open HW | status | uses | claim date | evidence |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Clone Robotics [ 78 ] | Clone Robotics | 27 | – | fluidic muscle | <910 | – | – | array | – | no | internal-only | 0 | 2025-11-24 | video |
| 1X NEO [ 76 ] | 1X Tech. | 25 | 25 | tendon | – | 45 | vendor, no protocol | unstated | – | no | announced | 0 | 2026-07-09 | datasheet |
| LEAP Hand V2 [ 187 ] | CMU | 21 | 17 | tendon | – | – | – | none | 3 000 | yes | announced | 2 | undated | datasheet |
| Sanctuary AI Phoenix [ 188 ] | Sanctuary AI | 21 | – | fluidic muscle | – | – | – | unstated | – | no | internal-only | 0 | 2024-12-17 | press |
| ILDA hand [ 55 ] | Korea Univ. | 20 | 15 | linkage | 1 100 | 34 | fingertip normal | array | – | no | prototype | 0 | 2021-11-05 | datasheet |
| Pisa/IIT SoftHand [ 54 ] | Centro E. Piaggio, Univ. Pisa and IIT | 19 | 1 | tendon | – | – | – | none | – | no | prototype | 0 | 2014 | datasheet |
TABLE V : Hands announced, prototyped or held internally, which cannot be obtained. 60 of 104 specification cells are empty.
| engine | contact model | solver | diff. | GPU | dt (s) | penetration exposed | hands |
|---|---|---|---|---|---|---|---|
| Brax [ 192 ] | convex relaxation | explicit/spring | yes | yes | – | no | 1 |
| ComFree-Sim [ 96 ] | complementarity-free | closed form | – | yes | 0.002 | yes | – |
| Dojo [ 95 ] | hard NCP | interior point | yes | no | – | yes | – |
| Genesis [ 99 ] | soft | Newton | yes | yes | – | – | 1 |
| Isaac Gym [ 92 ] | convex relaxation | PGS | no | yes | 0.008333 | no | 3 |
| Isaac Lab [ 97 ] | rigid, model unstated | Gauss-Seidel | no | yes | – | no | 4 |
TABLE VI : 15 simulators and physics engines, by contact formulation and penetration exposure. 40 of 105 property cells are empty.
| survey | yr | dexterous scope | taxonomy | bimanual | hardware | evaluation | gaps named |
|---|---|---|---|---|---|---|---|
| Surveys of the field | |||||||
| Zhao et al. [ 202 ] | 2026 | ||||||
| An et al. [ 198 ] | 2025 | ||||||
| Bai et al. [ 36 ] | 2025 | ||||||
| Welte et al. [ 199 ] | 2025 | ||||||
| Firoozi et al. [ 201 ] | 2023 | ||||||
TABLE VII : What 14 predecessor surveys cover: covered, partly, not. 24 of 84 coverage cells are empty.
More in this topic
Robotic Manipulation
Robotic manipulation studies how robots perceive, plan, and control physical interactions to grasp, move, assemble, and use objects.
See all 1,671 papers