One Hand to Rule Them All: Canonical Representations for Unified Dexterous Manipulation

Zhenyu Wei; Yunchao Yao; Mingyu Ding

cs.ROFeb 18, 2026

One Hand to Rule Them All: Canonical Representations for Unified Dexterous Manipulation

Authors: Zhenyu Wei, Yunchao Yao, Mingyu Ding

Organizations: University of North Carolina at Chapel Hill

Abstract

Dexterous manipulation policies today largely assume fixed hand designs, severely restricting their generalization to new embodiments with varied kinematic and structural layouts. To overcome this limitation, we introduce a parameterized canonical representation that unifies a broad spectrum of dexterous hand architectures. It comprises a unified parameter space and a canonical URDF format, offering three key advantages. 1) The parameter space captures essential morphological and kinematic variations for effective conditioning in learning algorithms. 2) A structured latent manifold can be learned over our space, where interpolations between embodiments yield smooth and physically meaningful morphology transitions. 3) The canonical URDF standardizes the action space while preserving dynamic and functional properties of the original URDFs, enabling efficient and reliable cross-embodiment policy learning. We validate these advantages through extensive analysis and experiments, including grasp policy replay, VAE latent encoding, and cross-embodiment zero-shot transfer. Specifically, we train a VAE on the unified representation to obtain a compact, semantically rich latent embedding, and develop a grasping policy conditioned on the canonical representation that generalizes across dexterous hands. We demonstrate, through simulation and real-world tasks on unseen morphologies (e.g., 81.9% zero-shot success rate on 3-finger LEAP Hand), that our framework unifies both the representational and action spaces of structurally diverse hands, providing a scalable foundation for cross-hand learning toward universal dexterous manipulation. Project Page: https://zhenyuwei2003.github.io/OHRA/

Figures & tables

Fig. 2 : Comparison of canonical and original URDFs across seven dexterous hands with different finger numbers and handedness. For each hand (from left to right): canonical URDF, original URDF, overlay of initial poses, and overlay of grasp poses, showing close morphological and kinematic consistency between the canonical and original models.

Name	DoF	Joint Types
Name	DoF	Thumb	Index	Middle	Ring	Little
Shadow Hand	22	RAAFF	AFFF	AFFF	AFFF	FAFFF
Sharpa Wave	22	AFAFF	AFFF	AFFF	AFFF	FAFFF
WUJI Hand	20	FAFF	AFFF	AFFF	AFFF	AFFF
DexHand 021	12(19)	AFFF	AFFF	FFF	AFFF	AFFF
Orca Hand	16	FAFF	AFF	AFF	AFF	AFF

TABLE I : Comparison of representative dexterous robotic hands and their kinematic configurations. F, A, and R denote flexion, abduction/adduction, and axial rotation. DoF values in parentheses include passive or mechanically coupled joints.

Panel of Figure 3 — Fig. 3 : Structure of the canonical URDF. A right-hand configuration is shown for clarity, but the representation is applicable to both left- and right-handed hands.

Panel of Figure 4 — Fig. 4 : Coordinate frame inconsistencies in URDFs. (a) Global orientations vary across sources, (b) local joint frames use inconsistent axis definitions, leading to kinematic ambiguity.

Fig. 5 : Visualization of latent-space interpolation between two dexterous hands. Canonical URDFs are shown at the ends, with decoded reconstructions and interpolated morphologies in between, demonstrating smooth transitions in DoF, finger arrangement, and overall geometry.

Fig. 6 : Two-stage cross-embodiment grasp generation pipeline.

Fig. 7 : Smoothed log of training reward over simulation steps during training. The solid lines mark the mean value, and the shaded area consists of the standard deviation.

Policy	Steps-to-Fall $\uparrow$	Cumulative Rotation $\uparrow$
Shadow (Original)	369.66	9.09
Shadow (Canonical)	390.62	10.92
LEAP (Original)	397.62	5.63
LEAP (Canonical)	326.98	6.31

TABLE II : Comparison of in-hand reorientation policies trained with the canonical URDF vs. the original URDF.

Method	Success Rate (%)
Method	Allegro	Barrett	ShadowHand
Ours (Canonical)	84.20	88.10	62.90
Ours (Original)	71.60 (-12.60)	88.70 (+0.60)	62.60 (-0.30)
D(R,O) (Original)	92.30	87.30	83.00
D(R,O) (Canonical)	92.38 (+0.08)	87.34 (+0.04)	78.63 (-4.37)

TABLE III : Comparison of grasp success rates when transferring across canonical and original URDFs.

Method	Success Rate (%) $\uparrow$			Time (sec.) $\downarrow$
Method	Allegro	Barrett	ShadowHand	Time (sec.) $\downarrow$
DFC	76.2	86.3	58.8	$>$ 1800
GenDexGrasp	51.0	67.0	54.2	19.71
D(R,O) Grasp	92.3	87.3	83.0	0.65
Ours	84.2	88.1	62.9	0.13

TABLE IV : Grasp performance comparison.

Method	Success Rate (%)
Method	Allegro	Barrett	ShadowHand
Specific	82.1	87.6	55.4
Unified	84.2	88.1	62.9

TABLE V : Comparison of grasp success rates. “Specific” indicates that each embodiment is trained independently, while “Unified” denotes joint training across all embodiments.

Model	Success Rate
Model	Apple	Band Aid	Coke	Cube	Football	Mayo	Orange	Pear	Sheep	Soccer	Average
leap_3333 (trained)	8/10	7/10	9/10	7/10	10/10	6/10	8/10	9/10	10/10	9/10	83/100
leap_3033 (trained)	8/10	8/10	2/10	6/10	9/10	6/10	7/10	9/10	10/10	10/10	75/100
leap_3033 (zero-shot)	8/10	10/10	5/10	5/10	7/10	2/10	9/10	7/10	9/10	9/10	71/100
leap_3303 (trained)	7/10	8/10	5/10	3/10	9/10	4/10	9/10	7/10	9/10	9/10	70/100
leap_3303 (zero-shot)	9/10	6/10	4/10	5/10	9/10	5/10	8/10	6/10	9/10	10/10	71/100

TABLE VI : Real-World grasp success rates.

Panel of Figure 13 — Fig. 8 : Real-world grasping objects and results.

Model	Success Rate (%)
Model	leap_3033	leap_3303	leap_3330
All Data	76.1	85.4	43.3
No leap_3033 Data	67.8	83.4	31.5
No leap_3303 Data	81.5	81.9	46.9
No leap_3330 Data	74.7	81.6	36.3

TABLE VII : Comparison of grasp success rates. Underlined values indicate zero-shot evaluation.

Table 15

Panel of Figure 16 — Fig. 9 : Gradient magnitude visualization.

Param Name	Num	Param Meaning
palm_radius	1	radius of palm cylinder
finger_radius	1	radius of finger capsule
finger_lengths	6	thumb link lengths and finger link lengths
finger_xyz	15	knuckle origin translations of 5 fingers
little_extra_origin	6	joint origin (xyz+rpy) of little finger extra origin
thumb_rpy	3	knuckle origin rotation of thumb

TABLE X : Canonical URDF parameter definition.

Param Name	Num	Param Meaning
palm_radius	1	radius of palm cylinder
finger_radii	5	radii of 5 fingers
finger_lengths	15	link lengths of 5 fingers
joint_origins	72	origins of 12 joints
joint_axes	36	axes of 12 joints
joint_lowers	22	lower ranges of all joints

TABLE XI : Extended canonical URDF parameter definition.

Observation	Description
$\tilde{\mathbf{q}}\in\mathbb{R}^{n_{\text{dof}}}$	Normalized hand joint positions
$\mathbf{q}^{\text{tar}}\in\mathbb{R}^{n_{\text{dof}}}$	Normalized target joint (action)
$\mathbf{p}_{\text{cube}}\in\mathbb{R}^{3}$	Cube position
$\mathbf{r}_{\text{cube}}\in\mathbb{R}^{3}$	Cube orientation Euler angles.
$\dot{\mathbf{q}}\in\mathbb{R}^{n_{\text{dof}}}$	Hand joint velocities.
$\mathbf{v}_{\text{cube}}\in\mathbb{R}^{3}$	Cube linear velocity.

TABLE XIII : Observation states used for policy training.

Fig. 13 : Visualization of canonical LEAP Hand variants.

Hand	# Grasps	Hand	# Grasps	Hand	# Grasps	Hand	# Grasps	Hand	# Grasps	Hand	# Grasps	Hand	# Grasps	Hand	# Grasps
leap_0000	0	leap_0001	0	leap_0002	0	leap_0003	0	leap_0010	0	leap_0011	0	leap_0012	0	leap_0013	0
leap_0020	0	leap_0021	157	leap_0022	134	leap_0023	304	leap_0030	0	leap_0031	452	leap_0032	226	leap_0033	25599
leap_0100	0	leap_0101	0	leap_0102	0	leap_0103	65	leap_0110	0	leap_0111	0	leap_0112	1	leap_0113	4
leap_0120	0	leap_0121	0	leap_0122	34	leap_0123	173	leap_0130	640	leap_0131	118	leap_0132	13	leap_0133	3801
leap_0200	0	leap_0201	562	leap_0202	2103	leap_0203	10114	leap_0210	9	leap_0211	63	leap_0212	212	leap_0213	592
leap_0220	199	leap_0221	61	leap_0222	1567	leap_0223	3293	leap_0230	5972	leap_0231	1010	leap_0232	656	leap_0233	11681

TABLE XIV : Number of valid grasps generated for different LEAP Hand variants after filtering.

Fig. 14 : Visualization of real-world experiment (I).

Fig. 15 : Visualization of real-world experiment (II).

Cross-Embodiment Robot Learning

Cross-embodiment robot learning transfers skills and policies across robots with different bodies, sensors, dynamics, and action spaces.

See all 346 papers

Cross-Embodiment Robot Manipulation via a Unified Hand Action Space
Jul 3, 2026Luis Felipe Casas, Robert Teal, Keval Shah +3Dexterous Manipulation In-Hand Manipulation
Intelligent Robotics and Vision Lab, University of Texas at Dallas · Intelligent Robotics and Interactive Systems Lab, Arizona State University
DexJoCo-X: Benchmarking Action Representations for Multi-Hand Dexterous Manipulation
Oct 2, 2026Xiangwei Jiang, Yao Mu, Lixin Duan +1Action Representation Learning Cross-Embodiment Robot Learning
University of Electronic Science and Technology of China, China. · Shanghai Jiao Tong University, China.
UniDex: A Robot Foundation Suite for Universal Dexterous Hand Control from Egocentric Human Videos
Mar 23, 2026Gu Zhang, Qicheng Xu, Haozhe Zhang +16Robotic Manipulation Cross-Embodiment Robot Learning
Tsinghua University · Shanghai Qizhi Institute · Sun Yat-sen University +1

Jul 3, 2026J/K move · Enter open · S save

Cross-Embodiment Robot Manipulation via a Unified Hand Action Space

Luis Felipe Casas, Robert Teal, Keval Shah, Abhijit Tadepalli+2

Intelligent Robotics and Vision Lab, University of Texas at Dallas · Intelligent Robotics and Interactive Systems Lab, Arizona State University

Dexterous Manipulation In-Hand Manipulation Generalization in Robotic Manipulation Cross-Embodiment Robot Learning Robotic Manipulation

Open paper AskPDF arXiv

Robot manipulation policies are typically tied to specific robotic hand embodiments, limiting the transfer of learned behaviors across platforms with different kinematic structures. In this work, we propose the Unified Hand Action Space (UHAS), a sphere-based unified action representation for cross-embodiment dexterous manipulation. UHAS represents robotic hand actions as geometric deformations of a canonical sphere and uses a Cascade Inverse Kinematics (CIK) algorithm to map the shared representation to embodiment-specific joint configurations. Using reinforcement learning, we train dexterous manipulation policies directly in the proposed action space for in-hand cube reorientation tasks. We evaluate our method in both simulation and real-world experiments across multiple robotic hands, including the Allegro Hand, LEAP Hand, Shadow Hand, and MANO Human Hand. Experimental results demonstrate effective dexterous manipulation, zero-shot transfer to unseen hands, rapid finetuning across embodiments, and successful real-world deployment. Our experiments show that the proposed UHAS representation enables stable dexterous control and cross-embodiment policy transfer across robotic hands.

Abstract

Cross-Embodiment Robot Learning

Explore similar work

Cross-Embodiment Robot Manipulation via a Unified Hand Action Space

DexJoCo-X: Benchmarking Action Representations for Multi-Hand Dexterous Manipulation

UniDex: A Robot Foundation Suite for Universal Dexterous Hand Control from Egocentric Human Videos