SHARE: Sharing Human Actions across Robot Embodiments through Unified Retargeting and Tracking

Junzhe Xu; Haoyu Zhang; Yunheng Wang; Taowen Wang; Lusong Li; Liang Lin; Xiaodong He; Zecui Zeng

cs.ROJan 12, 2026

SHARE: Sharing Human Actions across Robot Embodiments through Unified Retargeting and Tracking

Authors: Junzhe Xu, Haoyu Zhang, Yunheng Wang, Taowen Wang, Lusong Li, Liang Lin, Xiaodong He, Zecui Zeng, +1 more

Organizations: JD Explore Academy Beijing, China

Abstract

The central challenge in reusing human actions across robots is the need to realize shared intent through incompatible kinematics, dynamics, and control interfaces. We present SHARE, which decomposes action sharing into two connected stages---reference generation and feedback execution---and uses native robot references as their interface to achieve shared structure, native realization. In the reference-generation stage, AdaMorph extracts human-motion intent with a shared temporal backbone and generates embodiment-compatible references through robot-conditioned decoding. In the feedback-execution stage, AdaTrack combines these references with proprioception, action history, and morphology, then uses variable-length dynamic representations, morphology-conditioned attention, expert distillation, and closed-loop reinforcement learning to emit native actions from one shared Actor. On G1, HI, AD, and T1, AdaMorph attains seen/unseen root-speed Pearson correlation coefficients (PCCs) of 0.951/0.894; AdaTrack reaches 94.00%/91.96% closed-loop Success Rate with 21.15% of the total parameters in four specialist Actors, while cross-embodiment pretraining improves seen/unseen adaptation by 9.67/13.62 points with only 1% target data.

Figures & tables

Fig. 1: SHARE computation graph from Human Motion through AdaMorph references to AdaTrack closed-loop Robot Execution. AdaMorph combines a Morphology-Agnostic Intent Encoder, Learnable Static Robot Prompts, and Embodiment-Specific Output Adapters; AdaTrack combines Dynamic and Morphology Sequences, Dynamic–Morphology Cross-Attention, Morphology-Conditioned Self-Attention, and a Morphology-Aware Action Head. The “ $+$ ” junctions are schematic: the first denotes decoder Cross-Attention and AdaLN conditioning, while the second denotes a dynamic residual stream under morphology conditioning.

Fig. 2: Source-motion distributions by duration (outer rings) and composite-dynamics percentile (inner rings); percentages are within-partition.

Fig. 3: AdaMorph semantic consistency (1,048 pairs). Blue/left denotes seen and orange/right unseen motions; boxes show IQRs, white lines medians, and whiskers the 10th–90th percentiles.

Robot	Seen semantic consistency		Unseen semantic consistency		Seen label fidelity
Robot	Root PCC	Activity PCC	Root PCC	Activity PCC	Root vel. (m/s)	Root ori. ( ∘ )	Joint ( ∘ )
G1	0.965	0.864	0.913	0.779	0.269	8.13	0.426
HI	0.900	0.866	0.856	0.770	0.310	9.28	0.388
AD	0.969	0.855	0.895	0.761	0.575	9.68	0.383
T1	0.943	0.854	0.882	0.753	0.382	9.45	0.387
All	0.951	0.860	0.894	0.763	0.347	9.11	0.399

TABLE I: AdaMorph retargeting results.

Fig. 4: AdaMorph realization of one unseen action at nine temporally and heading-aligned phases. Rows show the human source and generated G1, HI, AD, and T1 references at fixed per-row camera height and scale.

Robot	Seen		Unseen
Robot	Success	MPKPE	Success	MPKPE
G1	90.67	50.5	89.29	55.7
HI	92.67	37.3	90.18	35.5
AD	96.00	65.2	91.96	69.3
T1	96.67	43.3	96.43	43.0
All	94.00	49.1	91.96	50.9

TABLE II: Per-embodiment AdaTrack results.

Controller	Seen				Unseen
Controller	Success	Completion	MPKPE	R-MPKPE	Success	Completion	MPKPE	R-MPKPE
Robot-specific	97.67	99.21	34.3	29.6	91.29	96.08	42.5	39.3
AdaTrack BC-only	56.17	70.87	51.6	44.9	56.25	72.69	53.3	48.1
AdaTrack PPO-only	46.83	64.42	72.4	69.8	53.13	69.27	69.3	64.8
AdaTrack BC+PPO	94.00	97.19	49.1	43.5	91.96	96.22	50.9	46.4

TABLE III: AdaTrack training-method ablation and specialist comparison. Success and completion are percentages; position errors are millimeters.

Held-out embodiment	1%	5%	10%	25%	100%
G1	$+6.67/+8.93$	$+9.33/+6.25$	$+10.00/+1.79$	$+10.00/+14.29$	$+3.33/+3.57$
HI	$+14.00/+11.61$	$+7.33/+17.86$	$+14.00/+9.82$	$+8.67/+6.25$	$+2.00/+0.00$
AD	$+6.67/+13.39$	$+4.67/-0.89$	$+7.33/+1.79$	$+0.67/+6.25$	$+0.67/+1.79$
T1	$+11.33/+20.54$	$+14.67/+16.07$	$+2.67/+0.89$	$+1.33/+5.36$	$+2.00/+2.68$
Aggregate	$\mathbf{+9.67/+13.62}$	$\mathbf{+9.00/+9.82}$	$\mathbf{+8.50/+3.57}$	$\mathbf{+5.17/+8.04}$	$\mathbf{+2.00/+2.01}$

TABLE IV: Per-embodiment transfer gains over scratch.

Fig. 5: Held-out-embodiment adaptation versus target-data fraction (%). Cross-embodiment pretraining improves scratch, while same-embodiment pretraining leads at the lowest budgets.

Explore similar work

KITE: Decoupling Kinematics and Interaction for Zero-Shot Cross-Embodiment Manipulation
Jun 20, 2026Qianxu Wang, Kuan FangZero-Shot Robotic Manipulation Generalization in Robotic Manipulation
Cornell University
LaST-HD: Learning Latent Physical Reasoning from Scalable Human Data for Robot Manipulation
Jun 22, 2026Jiaming Liu, Yinxi Wang, Chenyang Gu +15World Models for Robotics Generalization in Robotic Manipulation
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · The Chinese University of Hong Kong · Aether Tech +1
The Embodiment Gap in Robot Foundation Models
Aug 19, 2026Yukiyasu Domae, Keisuke Shirai, Hanbit Oh +7Robotics Robot Foundation Models
National Institute of Advanced Industrial Science and Technology (AIST), Japan

Jun 20, 2026J/K move · Enter open · S save

KITE: Decoupling Kinematics and Interaction for Zero-Shot Cross-Embodiment Manipulation

Qianxu Wang, Kuan Fang

Cornell University

Zero-Shot Robotic Manipulation Generalization in Robotic Manipulation Cross-Embodiment Robot Learning Robotic Manipulation Robot Imitation Learning Cross-Embodiment Transfer

Open paper AskPDF arXiv

Generalizing manipulation policies across robot embodiments remains difficult because standard policies entangle task reasoning with embodiment-specific motor control. We study zero-shot cross-embodiment manipulation, where a policy trained on source embodiments must be deployed on a structurally different target embodiment without additional task demonstrations. We introduce Kinematic Interaction Transfer across Embodiments (KITE), which decouples manipulation into embodiment-agnostic task reasoning and embodiment-specific motor control, connected through a learned latent representation of interaction intent based on contact patterns. Task reasoning is performed by a shared policy that predicts latent intents from source demonstrations, while motor control is performed by an intent-conditioned action decoder learned from each embodiment's kinematic model. With KITE, adaptation to a new embodiment requires only training a new action decoder using its kinematic model, without recollecting demonstration data. We evaluate KITE on three manipulation tasks spanning transfer between parallel grippers, dexterous hands, and composite embodiments. KITE consistently achieves zero-shot transfer to structurally different target embodiments, outperforming state-of-the-art baselines in transfer success and task-embodiment scope.

Abstract

Explore similar work

KITE: Decoupling Kinematics and Interaction for Zero-Shot Cross-Embodiment Manipulation

LaST-HD: Learning Latent Physical Reasoning from Scalable Human Data for Robot Manipulation

The Embodiment Gap in Robot Foundation Models