One Policy, Many Embodiments: Unified Camera-Centric Action Geometry Pre-training for Heterogeneous Embodied Manipulation

Xiaomi Embodied Intelligence Team; University of Macau; :; Shaoqing Xu; Fang Li; Guozhi Zhan; Zhixiang Duan; Yuhan Wang

cs.ROAug 26, 2026

One Policy, Many Embodiments: Unified Camera-Centric Action Geometry Pre-training for Heterogeneous Embodied Manipulation

Authors: Xiaomi Embodied Intelligence Team, University of Macau, :, Shaoqing Xu, Fang Li, Guozhi Zhan, Zhixiang Duan, Yuhan Wang, +16 more

Organizations: Xiaomi Embodied Intelligence Team · University of Macau

Abstract

Scaling generalist vision-language-action (VLA) policies is severely bottlenecked by the inherent heterogeneity of embodied data, which spans diverse robot morphologies, camera configurations, and low-level action spaces. Existing paradigms typically address this mismatch through explicit action retargeting, human-to-robot video synthesis, or dataset-specific adaptation branches, fundamentally hindering the joint learning of a unified policy. We introduce UCAG-P, a camera-centric unified action formulation that structurally aligns heterogeneous embodied datasets into a shared geometric action space. Rather than treating robot-specific commands as the shared policy target, UCAG-P represents manipulation through camera-observable anchor motion in image and camera-frame coordinates, treating robot arms, humanoids, and human hands as different embodiments of a common action schema. A geometry-conditioned action translator combines predicted motion with target-embodiment kinematics to produce executable controls. The resulting decoupled architecture allows a shared VLA policy to learn transferable manipulation geometry while retaining embodiment-specific controllability. UCAG-P is trained on 4.03K hours of robot and simulation data and 2.34K hours of human demonstrations. A single checkpoint reaches 98.3% on LIBERO, 88.7% and 89.2% on RoboTwin Easy and Hard, 82.0% zero-shot on LIBERO-Plus, and 62.0% on RoboCasa GR-1, without benchmark-specific fine-tuning.

Figures & tables

Figure 1: Cross-embodiment benchmark performance. Success-rate comparison across LIBERO, RoboTwin Easy, RoboTwin Hard, and RoboCasa GR-1. UCAG-P uses a single unified checkpoint without benchmark-specific fine-tuning and remains competitive across single-arm, bimanual, and humanoid settings.

Figure 2: Paradigms for heterogeneous manipulation learning. (a) Embodiment-specific policies isolate data; (b) unified camera-frame end-effector actions retain robot-specific kinematic targets; (c) UCAG-P shares camera-observable anchor motion before translating it into embodiment-specific commands.

Figure 3: Overview of UCAG-P for unified learning from heterogeneous manipulation data, spanning single-arm robots, bimanual robots, dexterous robotic hands, and human hands. UCAG-P maps their incompatible native action spaces into a shared camera-centric motion space defined by 3D camera-frame trajectories of the wrist or end-effector and grasp center. A geometry-conditioned translator then combines the unified motion prediction with camera-to-base transforms, Jacobians, and robot state to produce embodiment-specific executable actions. This unified interface enables joint robot–human training and improves performance across LIBERO, LIBERO-Plus, RoboTwin, and RoboCasa GR-1.

Category	Dataset	Episodes	Hours	Ratio	Supervision
Real robot	RoboChallenge-HDF5 [ 64 ]	5,080	33.212	0.52%	Robot action
	RoboCoin-Piper-LeRobot [ 62 ]	21,854	114.057	1.79%	Robot action
	DROID-HDF5 [ 18 ]	26,835	119.079	1.87%	Robot action
	Real-robot subtotal	53,769	266.348	4.18%	Robot action
Simulation	RoboCasa GR-1 [ 38 ]	24,000	83.612	1.31%	Robot action
	LIBERO-HDF5 [ 26 ]	1,692	2.531	0.04%	Robot action

Table 1: Composition of the embodied pre-training corpus. Episode counts follow dataset metadata when available, and proportions are computed from the total number of hours.

Data family	Representative sources	Embodiment	Supervision and training use
Vision-language	ShareRobot [ 17 ] ; RefSpatial-v2 [ 74 ] ; RoboVQA [ 49 ] ; RoboAfford [ 56 ]	image–text	Stage 1. Instruction following; spatial and affordance grounding; trajectory understanding; text/JSON $p_{0}/p_{1}$ coordinates.
Human	VITRA [ 25 ] ; EgoDex [ 15 ] ; EgoVerse [ 44 ]	egocentric human hand	Stages 1–3. RGB and language; wrist and grasp-center anchors; 3D camera-centric trajectories.
Simulation	LIBERO [ 26 ] ; RoboTwin 2.0 [ 36 ] ; RoboCasa GR-1 [ 38 ] ; InternData-A1 [ 57 ]	single arm, dual arm, humanoid	Stages 1–3. RGB and language; robot state; calibrated $p_{0}/p_{1}$ geometry; camera-frame trajectories; executable commands.
Real robot	DROID [ 18 ] ; RoboChallenge [ 64 ] ; RoboCoin [ 62 ]	single and dual arm	Stages 1–3. RGB and language; robot state and commands; calibration when available; derived $p_{0}/p_{1}$ supervision.

Table 2: Training data, supervision, and stage-wise use. Missing labels are masked, so each sample contributes only the targets available from its source.

Figure 4: Data organization and three-stage training pipeline . Vision-language and human demonstrations establish shared visual-geometric prediction, while simulation and real-robot data additionally supervise geometry-conditioned action translation and executable control.

Figure 5: Overview of shared camera-centric action space and definition of semantic anchor pair.

Figure 6: Overview of UCAG-P. We train a unified VLA policy on heterogeneous embodied data spanning single-arm robots, bimanual robots, humanoids, and human-hand datasets. UCAG-P converts these data sources into a shared camera-centric $p_{0}/p_{1}$ action schema and learns to predict anchor motion in camera-frame coordinates. A geometry-conditioned translator maps the shared motion prediction into executable commands for the target embodiment, allowing one policy to support heterogeneous robot control and human-to-robot transfer without explicit action retargeting or robot-video synthesis.

Full results are provided in Section 9 .		$\lhd$ Single-arm	Bimanual		Humanoid
Method	Type	LIBERO	RoboTwin Easy	RoboTwin Hard	RoboCasa GR-1
$\pi_{0}$ [ 4 ]	Specialist	94.4	65.9	58.4	–
StarVLA-OFT [ 53 ]	Specialist	96.6	50.4	–	48.8
GR00T N1.6 [ 39 ]	Specialist	97.2	47.6	–	49.9
$\pi_{0.5}$ [ 43 ]	Specialist	97.6	82.7	76.8	37.0
ABot-M0 [ 66 ]	Specialist	98.6	86.0	85.0	58.3

Table 3: Unified cross-embodiment manipulation success rates (%). "Specialist" denotes methods post-trained on a specific benchmark. Bold and underline indicate the best and second-best results in each column, respectively.

Method	Setting	Camera	Robot	Lang.	Light	Bkg.	Noise	Layout	Avg.
OpenVLA [ 20 ]	Zero-shot*	0.8	3.5	23.0	8.1	34.8	15.2	28.5	15.6
OpenVLA-OFT	Zero-shot*	56.4	31.9	79.5	88.7	93.3	75.8	74.2	69.6
OpenVLA-OFT-w	Zero-shot*	10.4	38.7	70.5	76.8	93.6	49.9	69.9	55.8
OpenVLA-OFT-m	Zero-shot*	55.6	21.7	81.0	92.7	91.0	78.6	68.7	67.9
NORA [ 16 ]	Zero-shot*	2.2	37.0	65.1	45.7	58.6	12.8	62.1	39.0
WorldVLA [ 7 ]	Zero-shot*	0.1	27.9	41.6	43.7	17.1	10.9	38.0	25.0

Table 4: LIBERO-Plus success rates (%) across seven perturbation categories. Baselines results are from ABot-M0 [ 66 ] , "*" denote the methods are post-trained on single standard LIBERO dataset but without fine-tuning on LIBERO-Plus.

Figure 7: Real-world success rates on Piper robots. Bread pickup evaluates human-to-robot transfer, while drawer opening and bowl stacking use robot demonstrations; bowl stacking additionally requires bimanual hand detection.

Figure 8: Representative start-to-end task executions in the real world (left) and simulation (right). The real-world tasks include bread grasping, drawer opening, and bimanual bowl stacking; the simulation tasks include hammering, bowl placement with drawer closing, and waist-unlocked humanoid cup placement.

Stage	Objective	GPUs	Steps	Image size	Batch / GPU	Global batch
1	Camera-centric specialization	128 H20	200K	$224\times 224$	12	1,536
2	Geometry-conditioned action translation	8 H20	10K	N/A	128	1,024
3	Joint robot-human training	64 H20	10K	$224\times 224$	12	768

Table 8.1: Stage-wise UCAG-P training schedule. Batch size denotes the per-device batch size; global batch size is computed before any gradient accumulation.

Figure 8.1 : Architecture of the motion head. Two residual MLP blocks map VLM action-token features to a 30-step camera-centric motion prediction.

Figure 8.2 : Architecture of the geometry-conditioned action head. VLM-conditioned features, camera pose, Jacobian, and ground-truth or predicted motion are fused by a Transformer encoder to produce a 30-step qpos command chunk.

Block	Slots	Semantic content
Left manipulator	$0$ – $9$	$p_{0}$ XYZ, $p_{1}$ XYZ, $\sin/\cos$ rotation, gripper, padding
Right manipulator	$10$ – $19$	$p_{0}$ XYZ, $p_{1}$ XYZ, $\sin/\cos$ rotation, gripper, padding
Camera motion	$20$ – $29$	camera XYZ translation, 6D continuous rotation, padding

Table 8.2: Canonical 30-dimensional camera-centric motion layout. Slot ranges are inclusive. The final padding slot in each block is structural and is excluded by the validity mask when unused.

Component	Slots	Block content
Left arm	$0$ – $9$	joint state or command, up to 10 DoF
Left end effector	$10$ – $19$	XYZ position (3), 6D continuous rotation (6), padding (1)
Left hand/gripper	$20$ – $29$	hand or gripper commands
Right arm	$30$ – $39$	joint state or command, up to 10 DoF
Right end effector	$40$ – $49$	XYZ position (3), 6D continuous rotation (6), padding (1)
Right hand/gripper	$50$ – $59$	hand or gripper commands

Table 8.3: Canonical 80-dimensional action layout. Slot ranges are inclusive; each row reserves a 10-dimensional semantic block regardless of the active degrees of freedom of a particular embodiment.

Dataset	Active blocks	Canonical slots
LIBERO	Left arm and gripper	$0$ – $9$ , $20$ – $29$
RoboTwin	Left/right arms and grippers	$0$ – $9$ , $20$ – $39$ , $50$ – $59$
RoboCasa GR-1	Left/right arms and hands, waist	$0$ – $9$ , $20$ – $39$ , $50$ – $69$

Table 8.4: Dataset-specific activation of the canonical 80-dimensional action blocks. Listed blocks are active; unused blocks and padded dimensions are masked.

Figure 8.3 : Question-answer examples for the three training stages. Stage 1 uses visual-language queries for affordance, anchor, and trajectory prediction. Stage 2 trains action translation from camera-centric motion and robot context. Stage 3 jointly supervises anchor motion and executable actions with available loss masks.

Evaluation setting	Robot	Success (%)
Standard Easy	ALOHA	88.66
Standard Hard	ALOHA	89.20
Cross-embodiment	ARX	35.0

Table 9.1 : RoboTwin performance under the source ALOHA embodiment and the ARX cross-embodiment setting.

Action-head conditioning	RoboCasa GR-1 success rate (%)
W/o VLM features	58.3
Attend-pooled VLM features	62.0

Table 9.2: Effect of adding pooled VLM features to the action head on RoboCasa GR-1.

Method	Spatial	Object	Goal	Long	Average
Diffusion Policy [ 11 ]	78.5	87.5	73.5	64.8	76.1
OpenVLA [ 20 ]	84.7	88.4	79.2	53.7	76.5
SpatialVLA [ 47 ]	88.2	89.9	78.6	55.5	78.1
CoT-VLA [ 71 ]	87.5	91.6	87.6	69.0	83.9
$\pi_{0}$ -FAST [ 42 ]	96.4	96.8	88.6	60.2	85.5
$\pi_{0}$ [ 4 ]	96.8	98.8	95.8	85.2	94.2

Table 9.3 : Success rates (%) on the four standard LIBERO suites.

Task	GR00T-N1.6 [-1pt] [ 39 ]	VP-VLA [-1pt] [ 60 ]	ABot-M0 [-1pt] [ 66 ]	JoyAI-RA [-1pt] [ 70 ]	UCAG-P
BottleToCabinetClose	51.5	54.0	86.0	84.0	82.0
CanToDrawerClose	13.0	72.0	74.0	90.0	68.0
CupToDrawerClose	8.5	44.0	48.0	48.0	52.0
MilkToMicrowaveClose	14.0	74.0	46.0	84.0	44.0
PotatoToMicrowaveClose	41.5	34.0	50.0	70.0	60.0
WineToCabinetClose	16.5	48.0	66.0	54.0	62.0

Table 9.4 : Task-level success rates (%) on RoboCasa GR-1.

	$\pi_{0}$ [ 4 ]		$\pi_{0.5}$ [ 43 ]		X-VLA [ 73 ]		Motus [ 36 ]		LingBot-VLA [ 63 ]		ZR-0 [ 22 ]		UCAG-P
Task	Clean	Rand.	Clean	Rand.	Clean	Rand.	Clean	Rand.	Clean	Rand.	Clean	Rand.	Clean	Rand.
AdjustBottle	99	95	100	99	100	99	89	93	100	100	100	99	100	100
BeatBlockHammer	79	84	96	93	92	88	95	88	92	89	85	88	98	96
BlocksRankingRGB	80	63	92	85	83	83	99	97	92	91	92	91	81	91
BlocksRankingSize	14	5	49	26	67	74	75	63	76	70	70	81	67	69
ClickAlarmclock	77	68	98	89	99	99	100	100	97	43	96	82	78	88

Table 9.5 : RoboTwin 2.0 task-level evaluation results. We report success rate under Clean and Randomized settings.

Figure 10.1 : Mapping human-hand demonstrations to robot supervision. Human videos provide camera-centric anchor motion used to train the robot policy.

Figure 10.2 : Human-to-robot conversion for the bread-pickup task using MediaPipe [ 29 ] hand keypoints.

Panel of Figure 10 — Figure 10.3 : Qualitative manipulation examples from (a) simulated benchmarks, including LIBERO, RoboTwin, and RoboCasa GR-1, and (b) real-world Piper robot tasks.

Figure 11.1 : Representative failure cases caused by errors in anchor detection, trajectory prediction, and action translation.

Explore similar work

Aug 6, 2026

DyPES-VLA: Learning Shared Dynamics Priors and Embodiment-Specific Control for Cross-Embodiment Manipulation

Vision-Language-Action (VLA) models have become a powerful paradigm for robot manipulation, but training a single generalist policy for heterogeneous robot embodiments remains an open problem. Existing methods have two main limitations. First, they underuse dynamics priors shared across diverse visual and interaction data, limiting cross-embodiment transfer. Second, they require extensive manual preprocessing to convert embodiment-specific actions into a common format. To overcome these limitations, we propose DyPES-VLA, a cross-embodiment VLA that learns shared Dynamics Priors and Embodiment-Specific control. First, we learn shared dynamics priors by training the vision-language model (VLM) with a future-prediction objective on cross-embodiment data, driving the shared query representation to capture object motion, contact, and interaction-induced scene changes. Second, an embodiment-specific Mixture-of-Experts (MoE) action head translates these shared dynamics priors into executable controls directly in each embodiment's native action space, without manually pre-aligning heterogeneous actions into a common format. This head shares attention layers to capture common temporal action structures, while its embodiment-specific feed-forward experts resolve the unique kinematic constraints and control semantics of distinct embodiments. As a generalist policy, our \ourmethod achieves state-of-the-art performance across simulation and real-world evaluations, reaching 98.0% success on LIBERO, 59.25% on RoboCasa-GR1, and 89.02% on RoboTwin~2.0.

Junfeng Li, Junjie He, Zhide Zhong +12

1The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · 2COCO Matrix, Shanghai, China

Cross-Embodiment Robot Learning Robotics Robotic Manipulation Vision-Language-Action Models

Ask PDF

May 28, 2026

Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments

Embodied intelligence is often studied through specialized models for individual tasks such as manipulation or navigation, resulting in fragmented capabilities and limited generalization across tasks, environments, and robot embodiments. In this work, we study whether heterogeneous embodied decision-making problems can be unified within a single vision-language-action model. We present Qwen-VLA, a unified embodied foundation model that extends Qwen's vision-language modeling stack from perception, understanding, and reasoning to continuous action and trajectory generation through a DiT-based action decoder. Qwen-VLA is trained with a large-scale joint pretraining recipe over diverse data sources, including robotics manipulation trajectories, human egocentric demonstrations, synthetic simulation data, vision-and-language navigation data, trajectory-centric supervision, and auxiliary vision-language data. To support multiple robot platforms, we introduce embodiment-aware prompt conditioning, where robot-specific textual descriptions specify the current embodiment and control convention. We further cast manipulation, navigation, and trajectory prediction into a unified action-and-trajectory prediction framework, enabling transferable visual grounding, spatial reasoning, and continuous action generation across robot morphologies, task families, and environments. Experiments on manipulation, navigation, and trajectory-centric benchmarks show consistent multi-task performance and out-of-distribution generalization under variations in scene layout, background, lighting, object configuration, and robot embodiment. Qwen-VLA-Instruct achieves 97.9% on LIBERO, 73.7% on Simpler-WidowX, 86.1%/87.2% on RoboTwin-Easy/Hard, 69.0% OSR on R2R, 59.6% SR on RxR, 76.9% average OOD success in real-world ALOHA experiments, and 26.6% zero-shot success on DOMINO dynamic manipulation.

Qiuyue Wang, Mingsheng Li, Jian Guan +37

Qwen Team

Language-Conditioned Robot Manipulation Robot Foundation Models Generalization in Robotic Manipulation Cross-Embodiment Robot Learning Vision-Language Navigation Vision-Language-Action Models

Ask PDF

Jun 7, 2026

GEAR-VLA: Learning Geometry-Aware Action Representations for Generalizable Robotic Manipulation

Vision-Language-Action (VLA) models achieve strong benchmark performance but still struggle in real-world deployment with unseen objects, background shifts, and different robot embodiments. We argue that this stems from the lack of a unified geometry-aware manipulation representation, leaving existing VLAs vulnerable to low-level trajectory supervision, misaligned 3D features, and embodiment differences. To address this, we propose GEAR-VLA, a VLA framework for learning unified geometry-aware action representations for generalizable robotic manipulation. GEAR-VLA adopts coarse-to-fine action learning, where multi-source embodied pretraining equips the VLM with embodied reasoning and discrete action understanding before latent action tokens connect action semantics to a gradient-decoupled DiT continuous action expert. It further performs semantic-aligned 3D integration by aligning a trainable 3D spatial backbone with the VLA representation while freezing the original VLM-aligned visual pathway. To share this representation across robots, GEAR-VLA uses embodiment canonicalization, where embodiment-aware states and embodiment-invariant actions confine robot differences to the low-level interface. Extensive simulation and real-world experiments demonstrate strong generalization: GEAR-VLA achieves state-of-the-art performance on LIBERO, zero-shot LIBERO-Plus, and RoboTwin 2.0, reaches 85.9% success on AgileX and 81.0% on the pretraining-unseen LDT-01 embodiment, and obtains 90.1% success on a 6,360-trial universal grasping benchmark with 212 unseen objects. Code and models will be released at https://github.com/babynabeauty/GEAR-VLA.

Yuan Zhang, Shiqi Zhang, Yedong Shen +11

Anhui University · University of Science and Technology of China · iFLYTEK

Generalization in Robotic Manipulation Cross-Embodiment Robot Learning Geometric Representation Learning Robotic Manipulation Vision-Language-Action Models Robot Learning

Ask PDF