Vision-Language-Action (VLA) models have achieved promising performance for generalized robotic manipulation, yet their reliance on expensive real-robot data remains a fundamental bottleneck. A scalable alternative is to transfer knowledge from off-the-shelf human-centric videos. However, existing methods extract latent action representations that are entangled with scene-specific noise, diluting the transferred knowledge. Motivated by this limitation, we propose Action-Intent Transfer (AIT), a two-stage method that transfers knowledge through a clean action representation shared across humans and robots. In the first stage, AIT extracts explicit action intents and disentangles the implicit action intent from manipulation videos. In the second stage, AIT aligns the human and robot domains within this shared intent space and decodes the intent into executable robot actions. Extensive experiments in both simulation and real-world environments demonstrate that AIT transfers human knowledge more effectively and consistently outperforms prior baselines.
Figures & tables
Figure 1: Evolution of human-to-robot learning paradigms. (a) MoCap-based methods depend on specialized capture devices, so data remains costly to collect, and the large embodiment gap limits generalization. (b) Latent action learning is trained directly from abundant raw human videos, but the learned representations are contaminated by scene noise. (c) Our method also learns from easy-to-acquire raw videos, yet extracts a clean Action Intent shared across humans and robots, enabling noise-free knowledge transfer to the robot.
Figure 2: Action-Intent-Guided Latent Disentanglement stage. It comprises: 1) VLM-based and 2) Rule-based Generation, which synthesize instructions l and explicit intents lintent for human videos and robot data, respectively; and 3) Intent-Guided Disentanglement, which extracts the implicit intent Zic from these manipulation videos under intent guidance.
Figure 3: Action Intent Policy Alignment stage. The explicit ( lintent , yellow) and implicit ( Zic , brown) intents form a shared space that bridges the embodiment gap, allowing our Prismatic-7B backbone to transfer manipulation knowledge from human videos to robots. At deployment, an Action Decoder maps these shared tokens to low-level robot actions.
methods
Spatial
Object
Goal
Long
Avg.
LAPA[2024]
73.8
74.6
58.8
55.4
65.7
Octo[2024]
78.9
85.7
84.6
51.1
75.1
MDT[2024]
78.5
87.5
73.5
64.8
76.1
Openvla[2024]
84.7
88.4
79.2
53.7
76.5
MaIL[2024]
74.3
90.1
81.8
78.6
83.5
Univla[2025]
90.1
91.3
88.3
82.7
88.1
Table 1: Average Success Rate of our method and state-of-the-art baselines on the LIBERO benchmark across four evaluation suites.
Figure 4: Success rates on the real-world benchmark across 12 tasks (T1–T12): UniVLA vs. Ours.
methods
Dir. Acc
Grip. Acc
Ours (Robot)
70.1
90.7
Ours (Robot+Human)
78.6
92.8
Table 2: Action Intent Prediction Analysis. We report the Direction Accuracy (Dir. Acc) and Gripper State Accuracy (Grip. Acc) of our model variants on the unseen Bridge test set.
methods
T9
T10
T11
T12
Avg.
Univla (Robot)
70
75
45
50
60.0
Univla (Robot+Human)
70
80
50
60
65.0
Ours (Robot)
75
85
70
65
73.8
Ours (Robot+Human)
95
90
80
75
85.0
Table 3: Success Rates on Real-World Tasks. Comparison of UniVLA and our method across different pre-training data configurations (Robot vs. Robot+Human).
methods
Spatial
Object
Goal
Long
Avg.
Ours (w/o Intent)
90.1
91.3
88.3
82.7
88.1
Ours (Direct)
92.3
92.9
90.2
83.1
90.8
Ours (Full)
93.4
94.2
91.2
86.4
91.3
Table 4: Ablation Study of the Action Intent Contribution.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Category
ID
Language Instruction
Standard Object Manipulation
T1
Pick up the yellow lemon.
T2
Pick up the white steamed bun.
T3
Pick up the purple toy eggplant.
T4
Pick up the orange toy orange.
T5
Pick up the egg tart.
T6
Pick up the white garlic.
Appendix
Table 5: Summary of the 12 tasks defined in the Astribot Dataset, categorized by required manipulation skills.
Label Type
# Samples
Accuracy
Language Instruction (human)
300
96.7%
Explicit Intent – Grasp (human)
500
96.4%
Explicit Intent – Move (human)
500
82.4%
Explicit Intent (robot, rule-based)
–
100%
Appendix
Table 6: Accuracy of the VLM pseudo-labels prior to human correction. Human-video labels are audited on random samples, whereas robot labels are derived from precise kinematic coordinates.
Evaluation Protocol
Move Acc.
Grasp Acc.
Strict coordinate matching (10,000)
72.6%
91.3%
Human semantic evaluation (200)
85.5%
94.0%
Appendix
Table 7: Intent prediction accuracy of the trained model on the unseen Bridge test set. The gap on “move” reflects multi-axis motion ambiguity under a strict single-axis metric, as confirmed by human semantic evaluation.
Method
Task completed in a row (%) ↑
Avg.
1
2
3
4
5
Len.
RT-1 ( Brohan et al. 2022 )
53.3
22.2
9.4
3.8
1.3
0.90
RoboFlamingo ( Li et al. 2024b )
82.4
61.9
46.6
33.1
23.5
2.48
SuSIE ( Black et al. 2024b )
87.0
69.0
49.0
38.0
26.0
2.69
GR-1 ( Wu et al. 2024 )
85.4
71.2
59.6
49.7
40.1
3.06
OpenVLA ∗ ( Kim et al. 2024 )
91.3
77.8
62.0
52.1
43.5
3.27
Appendix
Table 8: Language-conditioned visuomotor control on CALVIN ABC → D. We report the success rate of completing 1 – 5 tasks in a row and the average number of completed tasks (Avg. Len.) per instruction chain. All methods use only third-view RGB and language inputs. ∗ Reproduced with action-chunk prediction.
Method
OpenVLA
UniVLA
Ours
Control Freq. (Hz)
1.5
10
8
Appendix
Table 9: Real-world control frequency on a single RTX 4090.
Figure 5: Qualitative Results in real-world environments.
Figure 6: Qualitative Results in simulation environments.
Embodied foundation models have significantly propelled the advancement of artificial general intelligence, yet they rely heavily on large-scale data. Recent studies identify human behavior as one of the most scalable sources for fostering robotic physical intelligence. However, the inherent embodiment gap remains a fundamental challenge for effective human-to-robot learning. In this work, we model human intention to enable efficient learning from human demonstrations. Among human behavioral cues, gaze simultaneously encodes high-level task semantics and exhibits a synergistic coupling with low-level action. By modeling intention through gaze, we derive a mid-level representation that facilitates cross-embodiment knowledge transfer. We demonstrate that robots can internalize human intentions to enhance manipulation, even without robotic intention annotations. To capture the synergy between intention and action, we adopt a Chain-of-Thought reasoning paradigm during inference. Our approach enables the execution of long-horizon and fine-grained tasks across diverse embodiments. Extensive experiments in both simulation and real-world demonstrate that our method consistently outperforms baseline approaches and generalizes to unseen settings. Project page: https://gazevla.github.io .
Chengyang Li, Kaiyi Xiong, Yuan Xu +3
Shanghai Jiao Tong University · Eastern Institute of Technology, Ningbo · Peking University +1
Recent progress in generalizable embodied control has been driven by large-scale pretraining of Vision-Language-Action (VLA) models. However, most existing approaches rely on large collections of robot demonstrations, which are costly to obtain and tightly coupled to specific embodiments. Human videos, by contrast, are abundant and capture rich interactions, providing diverse semantic and physical cues for real-world manipulation. Yet, embodiment differences and the frequent absence of task-aligned annotations make their direct use in VLA models challenging. This survey provides a unified view of how human videos are transformed into effective knowledge for VLA models. We categorize existing approaches into four classes based on the action-related information they derive: (i) latent action representations that encode inter-frame changes; (ii) predictive world models that forecast future frames; (iii) explicit 2D supervision that extracts image-plane cues; and (iv) explicit 3D reconstruction that recovers geometry or motion. Beyond this taxonomy, we highlight three key open challenges in this area: structuring unstructured videos into training-ready episodes, grounding video-derived supervision into robot-executable actions under embodiment and viewpoint heterogeneity, and designing evaluation protocols that better predict real-world deployment performance and transfer efficiency, thereby informing future research directions. A curated list of papers and resources is available at https://github.com/AaronFengZY/HumanCentricToVLA-Survey.
Zhiyuan Feng, Qixiu Li, Huizhi Liang +12
1Tsinghua University · 5Microsoft Research Asia · 2HKUST +4
Vision-Language-Action (VLA) models are bottlenecked by the scarcity of robot data. Egocentric human manipulation videos offer an abundant, diverse, and semantically-rich alternative that inherently captures manipulation intent. However, the embodiment gap makes it difficult to extract action-relevant representations from these videos. From the perspective of action execution, human hands and robots share analogous manipulation primitives---reaching, grasping, placing---yet exhibit distinct kinematics tied to their embodiments. This might motivate our key approach: egocentric video pretraining can yield shared action representations capturing transferable manipulation skills, while human-specific motion patterns should be disentangled to avoid interfering with robot policy learning. To this end, we propose EXE, an Egocentric Cross(X)-Embodiment pretraining framework that learns a factorized latent action representation from egocentric human videos, separating shared manipulation priors from embodiment-specific signals. By leveraging the shared action space, the robot policy acquires generalizable manipulation semantics from diverse human demonstrations without being biased by human-specific kinematics. This shared knowledge transfers across embodiments with different kinematic structures, enabling the same pretraining to benefit both parallel grippers and dexterous hands. EXE yields consistent gains over baseline without pretraining on RoboTwin 2.0, RoboCasa GR1, and real-world manipulation tasks.
Qingyuan Cai, Bo Zhang, Yongzhen Huang +1
ROBOTERA. · Beijing Normal University. · Tsinghua University.