GOTT: Object-centric Dexterous Manipulation with a Reusable Cross-Embodiment Primitive

Yulin Liu; Lai Wei; Yen-Jen Wang; Akash Sharma; Pieter Abbeel; Henrik I. Christensen; Haozhi Qi

cs.ROOct 2, 2026

GOTT: Object-centric Dexterous Manipulation with a Reusable Cross-Embodiment Primitive

Authors: Yulin Liu, Lai Wei, Yen-Jen Wang, Akash Sharma, Pieter Abbeel, Henrik I. Christensen, Haozhi Qi

Organizations: Amazon FAR · UC San Diego · University of Chicago · UC Berkeley

Abstract

Foundation models and large-scale human data provide rich sources of manipulation intent, but translating this intent into multi-fingered robot behavior remains difficult. Dexterous hands still lack a reusable low-level primitive that reliably establishes contact across tasks and embodiments. We propose GOTT, a reach-acquire-move framework built around a single cross-embodiment contact-acquisition primitive. Given a robot-agnostic object trajectory and a reach specification, GOTT first brings the hand near a task-relevant contact region. The shared closed-loop primitive then establishes stable contact from this approximate initialization, and a pose-conditioned controller tracks the desired object motion. Reach specifications may come from future-aware planning, external models, or human demonstrations, while the primitive and tracking backend remain unchanged. Simulation and real-world experiments show that GOTT is able to establish robust contact across diverse objects, arm-hand platforms, and seen and unseen hand morphologies. It also consistently improves end-to-end task success over open-loop grasp execution.

Figures & tables

Figure 1: Contact-acquisition training and cross-embodiment policy. (A) The RL policy starts from randomized initial configurations and learns to establish stable contact using local hand-object observations. (B) Padding and binary masks map different hands into a shared observation and action space, allowing one policy to operate across hand embodiments.

Figure 2: Future-aware planning. We roll out the hand motion induced by each candidate initial configuration over the full object trajectory and solve IK along this motion. Red segments indicate joint jumps from discontinuous IK solutions. We reject candidates that are initially reachable but become infeasible later and select the remaining candidate with the least cumulative joint motion.

Figure 3: Alternative sources of reach specifications. Given a desired object trajectory, GOTT can construct a reach specification through future-aware planning or use additional guidance from reference grasps or human approach trajectories. The shared contact-acquisition primitive refines the resulting approximate initializations into stable contact.

Figure 4: Real-world evaluation setup. (a) in-distribution (blue) and out-of-distribution (purple) test objects; (b) three robot embodiments used for evaluation.

Method	Setting	XHand	Sharpa	Allegro	Avg.
BODex [ 7 ]	-	45.4	27.2	43.6	38.7
DRO [ 53 ]	-	31.3	47.0	43.2	40.5
CrossDex [ 61 ]	Specialist	30.4 $\pm$ 10.9	74.9 $\pm$ 1.5	34.5 $\pm$ 3.3	46.6 $\pm$ 4.1
CrossDex [ 61 ]	Unified	44.7 $\pm$ 2.9	52.8 $\pm$ 3.7	57.6 $\pm$ 1.0	51.7 $\pm$ 1.5
Ours	Specialist	64.3 $\pm$ 0.7	81.4 $\pm$ 1.2	80.4 $\pm$ 1.8	75.4 $\pm$ 1.0
Ours	Unified	74.1 $\pm$ 1.2	75.0 $\pm$ 1.1	87.1 $\pm$ 4.4	78.7 $\pm$ 1.1

Table 1: Simulation lifting success on three left hands (%). BODex and DRO are executed as open-loop grasp baselines. CrossDex and GOTT are evaluated as specialist policies trained separately for each hand and unified policies trained jointly across all three hands. Learned-policy results report mean

\pm

standard deviation across three seeds. Best results per column are bolded.

Variant	4433	4333	4324	Avg.
Oracle (per-variant)	69.0 $\pm$ 6.8	71.6 $\pm$ 1.1	76.0 $\pm$ 4.0	72.2 $\pm$ 1.2
Allegro-only (zero-shot)	29.4 $\pm$ 19.7	12.9 $\pm$ 8.6	9.4 $\pm$ 8.1	17.2 $\pm$ 12.1
Three-hand (zero-shot)	74.4 $\pm$ 6.5	72.2 $\pm$ 4.8	69.6 $\pm$ 9.6	72.1 $\pm$ 6.9

Table 2: Zero-shot generalization to unseen Allegro variants. Lifting success rate (%), reported as mean

\pm

standard deviation over three seeds. xyzw denotes per-finger link counts ( 4444 = original Allegro). Oracle trains one policy per variant; Allegro-only trains on the original Allegro; Three-hand is our unified policy over XHand+Sharpa+original Allegro, evaluated zero-shot.

Figure 5: Qualitative grasps on training and unseen embodiments. We compare the full GOTT (top) with the variant without $r_{\mathrm{dist}}$ (bottom) on three training hands and three unseen Allegro variants.

Figure 6: Qualitative illustration of four real-world tasks.

	Trash Cleanup		Mustard Pouring		Whiteboard Erasing		Drill Use		Avg. (%)
Method	gSR	tSR	gSR	tSR	gSR	tSR	gSR	tSR	gSR	tSR
DRO	6/10	0/10	5/10	5/10	5/10	3/10	7/10	6/10	57.5	35.0
BODex	5/10	2/10	4/10	2/10	7/10	3/10	7/10	7/10	57.5	35.0
Ours	8/10	7/10	8/10	7/10	10/10	9/10	9/10	9/10	87.5	80.0

Table 3: Real-world task results. Per-task grasp success rate (gSR) and task success rate (tSR) across four manipulation tasks on Franka-Sharpa platform, 10 trials each. Best per column in bold.

Figure 7: Success rate versus table height offset

z

on the SimToolReal eraser object.

Figure 8: Visualization of three sources of object trajectories.

Figure 9: Execution of unseen DexYCB demonstrations. Transparent hands indicate motion over time. (a) Annotated human hand-object motion. (b) Open-loop kinematic replay fails to maintain contact in physics simulation. (c) Our reactive primitive acquires stable contact and executes the demonstrated object trajectory.

Variant	XHand	Sharpa	Allegro	Avg.
Full GOTT	74.1 $\pm$ 1.2	75.0 $\pm$ 1.1	87.1 $\pm$ 4.4	78.7 $\pm$ 1.1
w/o link position obs.	49.4 $\pm$ 6.4	57.9 $\pm$ 10.3	75.7 $\pm$ 10.6	61.0 $\pm$ 4.4
proprio history length $=1$	56.5 $\pm$ 5.1	65.0 $\pm$ 2.1	67.5 $\pm$ 3.0	63.0 $\pm$ 3.3
OHRA-style [ 54 ] obs.	24.9 $\pm$ 22.8	31.2 $\pm$ 25.2	20.9 $\pm$ 15.6	25.7 $\pm$ 18.3
w/o $r_{\mathrm{contact}}$	0.0 $\pm$ 0.0	0.0 $\pm$ 0.0	0.0 $\pm$ 0.0	0.0 $\pm$ 0.0
w/o $r_{\mathrm{lift}}$	0.8 $\pm$ 0.1	2.7 $\pm$ 1.3	1.0 $\pm$ 1.2	1.5 $\pm$ 0.8

Table 4: Ablation results. Success rate (%) on three left hands.

Selection	Affordance	Future-aware	gSR (%)	tjSR (%)	func. (%)	tSR (%)
Random	✗	✗	100	80	35	20
IK-only	✗	✓	95	90	30	30
VLM (gpt-5.6-sol)	✓	✓	90	70	80	55
VLM (gemini-flash-latest)	✓	✓	70	50	65	50
VLM (claude-opus-4-7)	✓	✓	80	70	85	65
VLM (claude-opus-5)	✓	✓	95	75	85	60

Table 5: Affordance- and Future-aware selection on Drill Use (20 scenes). Random and IK-only use affordance-agnostic candidates and select by random sampling and trajectory feasibility, respectively. VLM variants first generate candidates from a task-relevant contact region and then apply future-aware IK selection.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Term	Value
$w_{d}$	distance	$16.0$
$w_{c}$	contact	$4.0$
$w_{o}^{p}$	object position stability	$10.0$ (init. $2.0$ )
$w_{o}^{r}$	object rotation stability	$10.0$ (init. $2.0$ )
$w_{l}$	lifting	$1.0$
$T_{\text{cur}}$	annealing steps	$10^{5}$

Appendix

Table 6: Reward weights.

Hand	DoF	Thumb (6)	Index (5)	Middle (5)	Ring (5)	Pinky (5)
Allegro	16	RAF[ ]F[ ]	R[ ]FFF	R[ ]FFF	R[ ]FFF	[ ][ ][ ][ ][ ]
Sharpa Wave	22	[ ]AFAFF	[ ]AFFF	[ ]AFFF	[ ]AFFF	FAFFF
XHand	12	[ ]AF[ ]F[ ]	[ ]AFF[ ]	[ ][ ]FF[ ]	[ ][ ]FF[ ]	[ ][ ]FF[ ]

Appendix

Table 7: Canonical 26-DoF joint layout and per-hand instantiation. Each finger is allocated a fixed number of slots in the canonical layout (6 for the thumb, 5 for each non-thumb finger); slots are filled by a hand’s native joint when present, and masked ( [] ) otherwise. F, A, and R denote flexion, abduction/adduction, and axial rotation respectively. Reading order within each finger is base

\to

tip.

Component	Composition	Dim
Joint positions	$26$ canonical $\times\,K$ frames	$26K$
Joint targets	$26$ canonical $\times\,K$ frames	$26K$
Link positions	$21$ links $\times\,3\times\,K$ frames	$63K$
Hand-object vectors	$21$ links $\times\,1$ nearest pt $\times\,3$	$63$
Joint mask	active canonical joints	$26$
Link mask	active canonical links	$21$

Appendix

Table 8: Observation components.

K\!=\!5

Group	Parameter	Range	Type
Object	Scale	$[0.96,1.04]$	$\times$ nominal
	Mass	$[0.5,3.0]$	$\times$ nominal (from $\rho\cdot V\cdot\text{scale}^{3}$ )
	Static/dynamic friction	$[0.5,1.5]$	absolute
	Restitution	$[0.0,0.1]$	absolute
Robot links	Static/dynamic friction	$[0.5,2.0]$	absolute
Robot links	Restitution	$[0.1,0.5]$	absolute

Appendix

Table 9: Domain randomization ranges. All distributions are uniform. Multiplicative ranges (

\times

) scale the per-object or per-robot nominal value; absolute ranges directly set the parameter value.

Parameter	Value
Parallel environments	$4096$
Horizon (rollout) length	$60$
Minibatch size	$8192$
Mini-epochs per update	$5$
Discount $\gamma$	$0.95$
GAE $\lambda$ ( tau )	$0.90$

Appendix

Table 10: PPO hyperparameters.

Figure 11: Task-aware region selection for Drill Use . (a) Four rendered views provided to the VLM. (b) The VLM identifies functional parts and selects the pistol grip as the task-relevant holding region, with multiple grasp points spanning the grip. (c) Back-projected grasp points define the contact region (yellow), from which candidate initializations are sampled.

Embodiment	O1	O2	O3	O4	O5	O6	O7	O8	O9	O10	Total (%)
Franka-Sharpa	4	4	5	5	5	5	4	3	5	5	90
Franka-XHand	5	5	5	4	5	3	5	4	5	4	90
Tianji-XHand	5	5	4	4	5	5	5	5	5	5	96

Appendix

Table 11: Per-object grasp success across embodiments (out of 5 trials). Objects are abbreviated as O1-O10: O1 mustard-bottle , O2 bleach-cleanser , O3 drill-red , O4 tomato-soup , O5 meat-can , O6 red-bottle , O7 drill-green , O8 apple-tea , O9 eraser , O10 maize . Objects O1–O5 are seen during training; O6–O10 are unseen.

Figure 12: Qualitative comparison on real-world tasks. Subfigures (a)-(b) are from BODex, and (c)-(d) are from DRO.

Scale	@006	@008	@010	@012	avg.
Floating	69.79	73.79	72.09	67.26	70.73
Tabletop	53.29	60.90	60.05	54.28	57.13

Appendix

Table 12: Effect of generation-time table constraints on BODex grasp success rate (%). Allegro right hand, 4 object scales, evaluated under BODex’s original protocol. Only the generation phase differs between the two columns. Objects are taken from DexGraspNet [ 51 ] , normalised so that each object’s bounding-box diagonal measures

1\,

m, and then rescaled to four target diagonals ( @006 =

0.06\,

m, @008 =

0.08\,

m, @010 =

0.10\,

m, @012 =

0.12\,

m).

Figure 13: Evaluation protocol comparison. The same BODex grasps evaluated under BODex’s original protocol and our tabletop protocol.

DRO variant	Allegro	Sharpa	XHand	Av.
Released ckpt	43.2	47.0	31.3	40.5
Fine-tune 1 ep.	21.6	52.2	31.1	35.1
Fine-tune 5 ep.	20.0	37.9	32.0	30.0
Retrain 5 ep.	19.8	40.6	32.6	31.0
Retrain 40 ep.	18.6	35.5	31.3	28.5

Appendix

Table 13: DRO variants. Lifting success rate (%) under our tabletop protocol; best per column in bold.

Stage	Time
4-view rendering	0.11 $\pm$ 0.01 s
VLM-based affordance reasoning	14.9 $\pm$ 0.9 s
Candidate generation	0.76 $\pm$ 0.01 s
IK filtering	3.95 $\pm$ 0.06 s
cuRobo planning ( $N{=}10$ )	9.6 $\pm$ 0.5 s
FoundationPose	45 Hz

Appendix

Table 14: Per-stage runtime, averaged over 10 rollouts.

Figure 14: Contact-predictor accuracy under observation noise. Per-step accuracy of the debounced switch signal, averaged over the three hands, as the predictor input is perturbed along one axis at a time (solid: object position, 0–5 cm; dashed: joint angle, 0–0.5 rad). Our proprioceptive predictor is unaffected by object-pose noise and degrades least under joint noise.

Explore similar work

MANGO-Grasp: Mahalanobis Fields over Geometry-Oriented 3D Gaussians for Cross-Embodiment Dexterous Grasping
Aug 3, 2026Heng Zhang, Kevin Yuchen Ma, Mike Zheng Shou +2Robotic Grasping Cross-Embodiment Robot Learning
Robotics & Autonomous Systems Division, Institute for Infocomm Research, Agency for Science, Technology and Research (A*STAR-I2R), Singapore · College of Computing and Data Science, Nanyang Technological University, Singapore · Show Lab, National University of Singapore, Singapore
Touch2Robot: Robot Touch in the Human Demonstration Loop
Sep 21, 2026Shengcheng Luo, Xiaoyang Cheng, Hong Ying +6Robotic Data Collection Motion Retargeting
ShanghaiTech University · Beijing Institute for General Artificial Intelligence (BIGAI) · Shanghai Jiao Tong University +1
ContactExplorer: Contact Coverage-Guided Exploration for General-Purpose Dexterous Manipulation
Mar 11, 2026Zixuan Liu, Ruoyi Qiao, Chenrui Tie +5Contact-Rich Robotic Manipulation RL Exploration
School of Computing, National University of Singapore · RoboScience

Aug 3, 2026J/K move · Enter open · S save

MANGO-Grasp: Mahalanobis Fields over Geometry-Oriented 3D Gaussians for Cross-Embodiment Dexterous Grasping

Heng Zhang, Kevin Yuchen Ma, Mike Zheng Shou, Weisi Lin+1

Robotics & Autonomous Systems Division, Institute for Infocomm Research, Agency for Science, Technology and Research (A*STAR-I2R), Singapore · College of Computing and Data Science, Nanyang Technological University, Singapore · Show Lab, National University of Singapore, Singapore

Robotic Grasping Cross-Embodiment Robot Learning

Open paper AskPDF arXiv

Cross-embodiment dexterous grasping aims to synthesize stable grasps across heterogeneous multi-fingered hands with little or no embodiment-specific tuning. Existing interaction-centric methods achieve promising results, but their object representations often underrepresent local surface geometry, while their robot descriptors do not explicitly encode both robot morphology and kinematics. We propose MANGO-Grasp, an anisotropic interaction framework that represents objects as geometry-oriented 3D Gaussian primitives and robot hands as surface keypoints encoded into morpho-kinematic descriptors. The object primitives are adaptively allocated by geometric complexity and shaped as surface-aligned plates with outward normals, encoding local geometry. Mahalanobis fields over keypoint--primitive pairs serve as interaction prediction targets during training and as optimization guidance for grasp realization at inference. These fields rise sharply for displacement along the surface normal but only gently within the tangent plane, matching the directional structure of contact. Grasps are realized with one shared optimization formulation and hyperparameter setting across all embodiments. On the CMAP and MultiGripperGrasp benchmarks, MANGO-Grasp outperforms the strongest seen-hand baseline by up to 8.24 percentage points in simulation. It also transfers zero-shot to the unseen SharpaWave hand, improving over the strongest zero-shot baseline by up to 16.57 percentage points, and achieves 86% success in real-world experiments. The code and additional materials will be made available upon publication at https://connor-zh.github.io/MANGO-Grasp/.

Abstract

Explore similar work

MANGO-Grasp: Mahalanobis Fields over Geometry-Oriented 3D Gaussians for Cross-Embodiment Dexterous Grasping

Touch2Robot: Robot Touch in the Human Demonstration Loop

ContactExplorer: Contact Coverage-Guided Exploration for General-Purpose Dexterous Manipulation