Dexterous manipulation is essential for real-world robot autonomy, mirroring the central role of human hand coordination in daily activity. Humans rely on rich multimodal perception--vision, sound, and language-guided intent--to perform dexterous actions, motivating vision-based, language-conditioned manipulation systems for robots. However, training reliable vision-language-action (VLA) models for dexterous manipulation requires large-scale demonstrations across many robotic hands. In addition, as new dexterous embodiments appear rapidly, collecting data for each becomes costly and impractical, creating a need for scalable cross-embodiment learning. We introduce XL-VLA, a vision-language-action framework integrated with a unified latent action space shared across diverse dexterous hands. This embodiment-invariant latent space is directly pluggable into standard VLA architectures, enabling seamless cross-embodiment training and efficient reuse of both existing and newly collected data. Experimental results demonstrate that XL-VLA consistently outperforms baseline VLA models operating in raw joint spaces, establishing it as an effective solution for scalable cross-embodiment dexterous manipulation.
Figures & tables
Paper
Data
Deployment
Input
Output
Type(s)
Embodiment
EEF ↔ EEF
Vision
Lang
Prop
Decoder
ZS
UniVLA [ 8 ]
human video; teleoperation
1 arm+1 gripper
gripper ↔ gripper
✓
✓
✗
✓
✗
ATE [ 71 ]
teleoperation; simulation
2 arms+2 grippers
—
✓
✓
✗
✗
✗
LAD [ 3 ]
teleoperation
1 arm+1 hand/gripper
hand ↔ hand/gripper
✓
✗
✗
✓
✗
EgoBridge [ 45 ]
human video; teleoperation
2 arms+2 pushers
—
✓
✗
✓
✗
✗
CoMo [ 65 ]
internet video; teleoperation
1 arm+1 gripper
—
✓
✗
✗
✗
✗
Table 1: Related Work Summary. Summary of related work comparing data sources, deployment settings, and input/output capabilities for latent-based cross-embodiment methods. Data indicates the training modalities used in each work. Deployment specifies the robot embodiments evaluated and whether cross–end-effector transfer is supported. Input denotes which modalities (vision, language, proprioception) are used for training. Output reports whether a method includes a cross-embodiment decoder and whether it enables zero-shot transfer to unseen embodiments.
Figure 2: Model Pipeline. XL-VLA builds on π0 [ 6 ] with vision and language encoders paired with an action expert that operates in a shared latent action space for cross-embodiment control. During VLA training, the action expert is finetuned while the pretrained latent encoders and decoders remain frozen.
Method
Hand
PF
SC
SoC
HB
RL
PS
RB
PuS
PoS
PC
Mean
π0 [ 6 ]
Ability
0.10
0.10
0.00
0.70
0.20
0.80
0.60
0.30
0.30
0.60
0.37
Inspire
0.10
0.20
0.00
0.30
0.10
0.50
0.30
0.20
0.20
0.80
0.27
Paxini
0.40
0.40
0.30
0.20
0.00
0.80
0.60
0.30
0.10
0.40
0.35
XHand
0.20
0.40
0.00
0.40
0.10
0.60
0.30
0.20
0.30
0.40
0.29
Mean
0.20
0.28
0.08
0.40
0.10
0.68
0.45
0.25
0.23
0.55
0.32
XL-VLA
Ability
0.80
0.80
0.40
1.00
0.70
1.00
0.70
0.30
0.90
0.70
0.73
Table 2: Vision-Language-Action Modeling. We compare XL-VLA with π0 under cross-embodiment training. Although π0 can handle different embodiments by adjusting sequence length, our method achieves consistently higher success rates across all hands and tasks. The first row PF, SC, SoC, etc. denote each task introduced in Task & Dataset, and each task is executed for 10 times, and we compute the success rate for each task cross hand.
Figure 3: Latent space pretraining pipeline. For each hand type, joint positions qh are mapped through an encoder MLP into a shared latent space and reconstructed by a decoder MLP. The diagram also indicates the placement of the reconstruction loss L1 , retargeting loss L2 via differentiable forward kinematics, and latent regularization loss L3 .
Figure 4: Zero-shot Unseen Tasks Generalization . For each hand, we randomly select some tasks as unseen tasks, whose data are held out from the training dataset. Then we test the unseen tasks with model trained on other data. Results show that by training with an aligned latent action space, XL-VLA gets the ability to generalize to novel hand-task combination in a zero-shot manner. PSR stands for “Partial Success Rate”, where policy is rewarded with half success if only one arm finishes its task.
Ability
Inspire
X-Hand1
Paxini DexH13
#Fingers
5
5
5
4
#DoF(mimic)
12(6)
12(6)
12
16(3)
Table 3: Dexterous Hand Comparison.
Figure 5: G1 Cross-Robot Performance. Co-training with latent xArm and humanoid data outperforms using raw actions.
Model
Combination
PF
SC
SoC
HB
RL
PS
RB
PuS
PoS
PC
Mean
LAD [ 3 ]
Ability+Inspire
0.8
0.4
0.5
0.4
0.6
0.6
0.6
0.5
0.7
0.9
0.60
Paxini+XHand
0.7
0.5
0.6
0.7
0.5
0.8
0.7
0.4
0.6
0.6
0.61
XL-VLA
Ability+Inspire
0.9
0.7
0.8
0.7
0.8
0.7
0.8
0.8
1.0
1.0
0.82
Paxini+XHand
0.8
0.7
0.9
0.8
0.6
0.9
0.9
0.6
1.0
0.9
0.81
Table 4: Latent replay comparison . We compare our latent space with Latent Action Diffusion (LAD) [ 3 ] . For each hand combination, teleoperated trajectories collected on one source hand are encoded into the latent space, decoded onto the target hand, and replayed on real hardware. A replay is counted as successful if the encoded–decoded sequence can be executed without breaking contact or causing self-collisions. Higher replay success indicates better cross-embodiment consistency of the latent representation.
Reconstruction ↓
Cross Embodiment ↓
Latent Continuity ↓
Interpolation ↓
Exp
Joint
Tip
PT dir
PT dist
RT dir
RT dist
Joint
Tip
Accel.
Jerk
Ours
5.476
3.703
11.857
1.872
10.492
6.295
4.492
8.534
8.683
9.659
−L1
61.672
39.400
11.741
1.857
10.398
6.375
24.784
58.858
12.028
16.852
−L2dist
5.195
3.580
3.972
4.413
6.788
24.488
4.073
8.168
8.525
9.240
−L2dir
4.966
3.378
46.217
2.251
53.546
5.518
4.451
9.217
8.742
9.551
−L2 (both)
3.781
2.602
62.733
8.080
71.765
62.809
2.823
6.757
8.602
9.426
Table 5: Ablations. Ablation results comparing reconstruction accuracy, cross-embodiment retargeting, latent-space continuity, and interpolation smoothness. Exp denotes model variants: removing losses ( −L2 , −L2dist , −L2dir , or both), changing hidden sizes ( Hab ), or changing latent dimension ( Ld ). Metrics include joint and tip RMSE for reconstruction; pinch- and random-motion direction/distance errors for retargeting; joint/tip latent continuity; and mean acceleration/jerk for interpolation.
Figure 7: More Latent Visualizations. Latent decoding results cross embodiment.
Figure 8: xArm Camera Setup. We use a single RealSense L515 camera with the front view. Note that the D435 camera here is not used for XL-VLA.
Figure 9: Dexterous Hands. We use 4 kinds of hands, with various shapes, scales, degrees of freedom, and actuated joints.
Figure 10: xArm Camera View. This is what our camera sees and also the input for XL-VLA and all the baseline methods.
Figure 11: G1 Scene. We mount an L515 camera near the neck of G1 to have an egocentric view.
Figure 12: G1 Egocentric Camera View.
Figure 13: Objects. We use various everyday objects from existing datasets. They vary in scale, shape, texture, weight, etc., thus requiring the manipulation policy to be robust.
Figure 14: Apple Vision Por for Data Collection. Apple Vision Pro is used to track the human teleoperator’s hands and wrists. Then the tracked data is processed with retargeting and Inverse Kinematics to control the real-world robots.
Figure 15: G1 Teleoperation System. We build the G1 upper-body teleoperation system from HOMIE [ 4 ] . We use a pair of MANUS Mocap glove to track the human hand pose.
Figure 16: Task Visualizations. We design 10 various tasks to test all the models. The tasks require varied manipulation skills and have different difficulties.
Figure 17: Latent Visualization of a Grasping Trajectory. A trajectory is shown here with all the robot hands.
Vision-Language-Action (VLA) models have recently become a central direction in embodied AI, but current systems are restricted to either dual-gripper control or single-arm dexterous hand manipulation. While low-dimensional gripper control can often be handled with simpler methods, high-dimensional dexterous hand control benefits greatly from full end-to-end VLA learning. In this work, we introduce Dexora, the first open-source VLA system that natively targets dual-arm, dual-hand high-DoF manipulation. We design a hybrid teleoperation pipeline that decouples gross arm kinematics (captured with a custom exoskeleton backpack) from fine finger motion (markerless hand tracking via Apple Vision Pro), and that drives both a physical dual-arm dual-hand platform and an identical MuJoCo digital twin. Using that interface, we assemble a large training corpus: an embodiment-matched synthetic corpus (100K simulated trajectories, 6.5M frames) and a real-world dataset of 10K teleoperated episodes (2.92M frames). To mitigate noisy teleoperation demonstrations, we propose a data-quality-aware training recipe: an offline discriminator provides clip-level weights for diffusion-transformer policy training, down-weighting low-quality demonstrations. Empirically, Dexora outperforms competitive VLA baselines on both basic and dexterous benchmarks (e.g., average dexterous success 66.7% vs. 51.7%), attains 90% success on basic tasks, and shows robust out-of-distribution and cross-embodiment generalization. Ablations confirm the importance of real data and the discriminator for dexterity.
Zongzheng Zhang, Jingrui Pang, Zhuo Yang +22
Tsinghua University · Beijing Academy of Artificial Intelligence · The University of Hong Kong +3
Vision-Language-Action (VLA) models have become a powerful paradigm for robot manipulation, but training a single generalist policy for heterogeneous robot embodiments remains an open problem. Existing methods have two main limitations. First, they underuse dynamics priors shared across diverse visual and interaction data, limiting cross-embodiment transfer. Second, they require extensive manual preprocessing to convert embodiment-specific actions into a common format. To overcome these limitations, we propose DyPES-VLA, a cross-embodiment VLA that learns shared Dynamics Priors and Embodiment-Specific control. First, we learn shared dynamics priors by training the vision-language model (VLM) with a future-prediction objective on cross-embodiment data, driving the shared query representation to capture object motion, contact, and interaction-induced scene changes. Second, an embodiment-specific Mixture-of-Experts (MoE) action head translates these shared dynamics priors into executable controls directly in each embodiment's native action space, without manually pre-aligning heterogeneous actions into a common format. This head shares attention layers to capture common temporal action structures, while its embodiment-specific feed-forward experts resolve the unique kinematic constraints and control semantics of distinct embodiments. As a generalist policy, our \ourmethod achieves state-of-the-art performance across simulation and real-world evaluations, reaching 98.0% success on LIBERO, 59.25% on RoboCasa-GR1, and 89.02% on RoboTwin~2.0.
Junfeng Li, Junjie He, Zhide Zhong +12
1The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · 2COCO Matrix, Shanghai, China
Building a generalist robot that can perceive, reason, and act across diverse tasks remains an open challenge, especially for dexterous manipulation. A major bottleneck lies in the scarcity of large-scale, action-annotated data for dexterous skills, as teleoperation is difficult and costly. Human data, with its vast scale and diverse manipulation behaviors, provides rich priors for learning robotic actions. While prior works have explored leveraging human demonstrations, they are often constrained by limited scenarios and a large visual gap between human and robots. To eliminate these limitations, we propose METIS, a vision-language-action (VLA) model for dexterous manipulation pretrained on multi-source egocentric datasets. We first construct EgoAtlas, which integrates large-scale human and robotic data from multiple sources, all unified under a consistent action space. We further extract motion-aware dynamics, a compact and discretized motion representation, which provides efficient and expressive supervision for VLA training. Built upon them, METIS integrates reasoning and acting into a unified framework, enabling effective deployment to downstream dexterous manipulation tasks. Our method demonstrates exceptional dexterous manipulation capabilities, achieving highest average success rate in six real-world tasks. Experimental results also highlight the superior generalization and robustness to out-of-distribution scenarios. These findings emphasize METIS as a promising step toward a generalist model for dexterous manipulation.
Yankai Fu, Ning Chen, Junkai Zhao +5
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Beijing Academy of Artificial Intelligence