EgoHumanoid-V2: Human-to-Humanoid Transfer of Coordinated Whole-Body Skills for Loco-Manipulation

Jin Chen; Yiming Jiang; Chongyang Xu; Modi Shi; Shijia Peng; Li Chen; Tianyu Li; Mu Xu

cs.ROSep 29, 2026

EgoHumanoid-V2: Human-to-Humanoid Transfer of Coordinated Whole-Body Skills for Loco-Manipulation

Authors: Jin Chen, Yiming Jiang, Chongyang Xu, Modi Shi, Shijia Peng, Li Chen, Tianyu Li, Mu Xu, +3 more

Organizations: OpenDriveLab at The University of Hong Kong · Alibaba Group · Shanghai Innovation Institute · Fudan University · Beihang University · Sichuan University · Archon Robotics

Abstract

Human demonstrations capture diverse scenes and rich whole-body skills without requiring robot teleoperation. Prior work on egocentric transfer has emphasized scene generalization in loco-manipulation under decoupled control, leaving direct transfer of coordinated whole-body skills less explored. We present EgoHumanoid-V2, the first egocentric human-to-humanoid skill transfer framework for coordinated whole-body loco-manipulation. At its core, coarse-to-fine action alignment combines kinematic reference correction with dynamics-aware refinement. It improves end-effector pose accuracy while preserving whole-body coordination. We also use robot-arm rendering and training-time image augmentation to reduce the visual embodiment gap and improve viewpoint robustness. On four real-world tasks, vision-language-action (VLA) policies trained on aligned human data show zero-shot skill transfer without target-task robot demonstrations. Task scores are comparable to those of policies trained on teleoperation data at a lower collection cost. These results support human data as direct skill supervision.

Figures & tables

Figure 2: Hardware for human demonstration capture and humanoid deployment.

Figure 3: Action and visual alignment pipelines. (a) Human motion initializes a robot trajectory through a SONIC rollout ( $H$ ). Offline IK corrects the reference geometry ( $K$ ), and dynamics-aware refinement ( $D$ ) uses an MLP response model and iterative execution feedback to produce aligned robot motion. (b) Human forearms and wrist trackers are masked and inpainted, then replaced with rendered robot arms. Image augmentation produces varied training views.

Figure 4: Q1: Skill transfer on four real-world tasks. Each task column shows human demonstrations (top), humanoid motion illustrations (middle), and task scores with 25, 50, 100, or 200 human demonstrations (bottom). Policies use no target-task robot demonstrations.

Figure 5: Task scores and collection labor with human or robot demonstrations. Each point represents an independently trained policy evaluated over 20 trials. Human-only results reuse Figure 4 . Cost strips include capture, setup, calibration, troubleshooting, and resets.

End-effector Tracking Error ( $\downarrow$ )
Alignment	Close Laptop	Open Fridge	Pull Curtain	Close Drawer	Mean
Alignment	Pos. (cm) / Ori. ( ∘ )	Pos. (cm) / Ori. ( ∘ )	Pos. (cm) / Ori. ( ∘ )	Pos. (cm)	Pos. (cm) / Ori. ( ∘ )
$H$	14.08 / 29.43	13.27 / 19.37	10.31 / 14.86	7.17	11.21 / 21.22
$H+K$	4.22 / 13.84	4.24 / 14.72	3.13 / 12.74	6.22	4.45 / 13.77
$H+K+D_{s}$	3.04 / 9.50	2.76 / 5.77	2.02 / 4.85	5.80	3.41 / 6.71
Real-World Task Score (%, $\uparrow$ )

Table 1: Q3: Effect of action alignment. Top: position (cm) / orientation ( ∘ ) errors in simulation (

\downarrow

). Bottom: real-world task scores (%,

\uparrow

) with 200 demonstrations per task and identical visual preprocessing. Means are computed across tasks; Close Drawer uses sole position only and is excluded from orientation averages. Bold/underline mark best/second best. Selection follows Appendix C.2 .

	Endpoint Error		Whole-Body Motion Quality
Motion initialization	Pos. (cm)	Ori. ( ∘ )	Pelvis drop (cm)	Tilt ( ∘ )	Joint step ( ∘ )	Extra yaw ( ∘ )
SONIC Rollout	3.41	6.71	3.28	11.14	5.94	7.18
Retargeting	3.42	5.32	3.30	11.87	6.51	11.45
Whole-Body IK	3.82	6.35	5.25	12.77	7.75	23.67

Table 2: Effect of motion initialization. Task-averaged endpoint error at

D_{s}

and whole-body motion quality. Pelvis drop measures maximum height loss; tilt, maximum base inclination; joint step, maximum frame-to-frame joint-angle change; extra yaw, heading deviation from the human reference. All variants share

K

, eight

D

rounds, and selection. Drawer orientation is excluded. Lower is better; bold/underline mark best/second best. Appendix C.3 defines the metrics; Tables 6 and 7 report results for each task and initialization.

Figure 6: Complementarity and depth of $K$ and $D$ . Task-macro averages of raw stage outputs for $D^{k}$ , repeated $K^{m}$ , $KD^{k}$ , and $K^{m}D^{8}$ . Position averages four tasks; orientation averages the three hand tasks. Protocol and processing costs appear in Appendix C.4 .

Response	Close Laptop	Open Fridge	Pull Curtain	Close Drawer	Mean
	Pos. (cm) / Ori. ( ∘ )	Pos. (cm) / Ori. ( ∘ )	Pos. (cm) / Ori. ( ∘ )	Pos. (cm)	Pos. (cm) / Ori. ( ∘ )
Unit response	3.18 / 9.74	2.89 / 5.88	2.19 / 5.06	9.18	4.36 / 6.89
MLP response	3.04 / 9.50	2.76 / 5.77	2.02 / 4.85	5.80	3.41 / 6.71

Table 3: MLP response versus unit response. Position (cm) / orientation ( ∘ ) errors of historically selected outputs within eight refinement rounds, with shared inputs and evaluation trajectories. Means weight tasks equally; orientation excludes drawer. Bold marks lower errors (Appendix C.5 ).

Visual alignment		Task score (%)				Mean
Image augmentation	Robot-arm rendering	Close Laptop	Open Fridge	Pull Curtain	Close Drawer
$\times$	$\times$	31.67	16.67	35.00	50.00	33.33
$\times$	$\checkmark$	51.67	23.33	41.67	—	38.89
$\checkmark$	$\times$	61.67	33.33	33.33	61.67	47.50
$\checkmark$	$\checkmark$	53.33	38.33	53.33	—	48.33

Table 4: Q4: Visual-alignment ablation. Task scores with 200 human demonstrations per task. Means average the available tasks; robot-arm rendering is not evaluated for foot-operated Close Drawer. Bold/underline mark best/second best; shading indicates the full pipeline.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Stage 1	Stage 2	Stage 3
Close Laptop	Right hand reaches a suitable position above the lid.	Right hand makes an effective downward press on the lid.	Lid is fully closed.
Open Fridge	Left hand approaches the door gap.	Left hand hooks into the gap.	Refrigerator door opens.
Pull Curtain	Right hand grasps the curtain.	Hand pulls the curtain.	Curtain is pulled open by one meter.
Close Drawer	Right foot lifts, with the toe in front of the lowest drawer’s outer face.	Right foot pushes the drawer inward.	Drawer is fully closed.

Appendix

Table 5: Ordered binary scoring criteria for physical policy evaluation.

Task	SONIC Rollout	Retargeting	Whole-Body IK
Close Laptop	3.04 / 9.50	2.67 / 6.33	2.75 / 6.19
Open Fridge	2.76 / 5.77	3.15 / 5.74	3.68 / 7.25
Pull Curtain	2.02 / 4.85	2.05 / 3.88	2.73 / 5.61
Close Drawer	5.80 / —	5.80 / —	6.11 / —

Appendix

Table 6: Task-wise endpoint error for motion initialization. Position/orientation errors at

D_{s}

are in cm/ ∘ , averaged over 25 episodes per task. Drawer orientation is not evaluated. Lower is better; bold and underline denote the best and second-best values for each metric.

Task	Method	Pelvis drop (cm)	Tilt ( ∘ )	Joint step ( ∘ )	Extra yaw ( ∘ )
Close Laptop	SONIC Rollout	0.28	9.62	2.87	3.94
	Retargeting	0.50	10.86	3.95	3.67
	Whole-Body IK	2.06	11.32	6.07	22.60
Open Fridge	SONIC Rollout	3.51	10.39	7.79	7.39
	Retargeting	3.35	11.62	8.28	13.96
	Whole-Body IK	8.08	13.04	8.42	16.70

Appendix

Table 7: Task-wise whole-body motion quality for motion initialization. Metrics are averaged over the same fixed random 25-trajectory cohort per task as Table 2 , using the definitions in Appendix C.3 . Lower is better. Bold and underline mark the best and second-best values per task.

Configuration	Time (s/frame)	Pos. (cm)	Ori. ( ∘ )
$K^{1}$	0.65	4.86	16.46
$K^{8}$	2.55	4.77	16.51
$D^{8}$	1.27	8.40	16.40
$D^{16}$	2.11	7.16	12.36
$K^{2}D^{8}$	1.80	4.04	7.90
$K^{4}D^{8}$	2.33	4.11	7.94

Appendix

Table 8: Processing cost and endpoint accuracy. Representative configurations from Figure 6 . Time includes preparation, optimization, and replay, with reused

H/K

costs included. It excludes queueing, discarded failed attempts, and subsequent selection and validation. Values are equal-task means; orientation averages the three hand tasks. Errors use raw outputs at each depth (Figure 6 ).

Stage	Pos. (cm)	Ori. ( ∘ )
$H$	17.03	36.06
$H+K$	5.22	21.81
$H+K+D_{s}$	3.83	10.85

Appendix

Table 9: Stage comparison on the expanded benchmark. Task-equal endpoint errors over 112 paired trajectories across 50 tasks; lower is better.

H+K+D_{s}

uses historical candidate selection.

Figure 7: Refinement depth on the expanded benchmark. Task-equal averages of raw stage outputs for $D^{k}$ , repeated $K^{m}$ , $KD^{k}$ , and $K^{m}D^{8}$ on paired trajectories.

Response	Pos. (cm)	Ori. ( ∘ )
Unit response	4.25	9.39
MLP response	4.10	9.21

Appendix

Table 10: Response-model comparison on the expanded benchmark. Task-equal errors over 112 trajectories across 50 tasks after

KD^{8}

, without historical selection. Lower is better.

Variant	Three hand tasks	Foot-operated Close Drawer
A	IK controls arms and waist; SONIC controls legs.	IK controls both legs; SONIC controls remaining joints.
B	Same as A, with SONIC’s balance contribution added.	Same as A, with SONIC’s balance contribution added.
C	IK controls arms; SONIC controls waist and legs.	IK controls the right operating leg; SONIC controls remaining joints.

Appendix

Table 11: Joint-command assignments for the three online-IK variants.

Variant	Task	Pos. (cm)	Ori. ( ∘ )	Tilt ( ∘ )	Init. falls	Events
A	Close Laptop	4.45	10.43	16.97	0/5	0
	Open Fridge	2.33	15.34	18.97	0/5	0
	Pull Curtain	1.58	10.17	7.03	0/5	0
	Close Drawer	—	—	—	5/5	—
B	Close Laptop	2.43	8.28	13.30	0/5	0
	Open Fridge	1.55	14.45	17.26	0/5	0

Appendix

Table 12: Online-IK stability diagnostics on five episodes per task. Errors use measured endpoints, not IK solutions. Init. falls counts failed runs; Events counts logged falls, including startup. Dashes indicate no formal replay. Bold/underline rank values within each task from lowest to second lowest; they do not indicate that execution is stable.

Panel of Figure 8 — Figure 8: Online-IK instability: variant A (IK without balance overlay). Each row follows one continuous segment from left to right, selected by peak base tilt as described in Appendix D.3 . A fixed crop within each sequence preserves relative motion and keeps robot scale comparable across tasks. Close Drawer ends in an initialization fall. Task objects are omitted.

Panel of Figure 9 — Figure 9: Online-IK instability: variant B (IK with SONIC balance overlay). Each row follows one continuous segment from left to right, selected by peak base tilt as described in Appendix D.3 . A fixed crop within each sequence preserves relative motion and keeps robot scale comparable across tasks. Close Drawer ends in an initialization fall. Task objects are omitted.

Panel of Figure 10 — Figure 10: Online-IK instability: variant C (restricted IK joint control). Each row follows one continuous segment from left to right, selected by peak base tilt as described in Appendix D.3 . A fixed crop within each sequence preserves relative motion and keeps robot scale comparable across tasks. Close Drawer ends in an initialization fall. Task objects are omitted.

Explore similar work

Towards a General Humanoid Loco-Manipulation Model via Egocentric Whole-Body Human Data Pretraining
Sep 30, 2026Chongyang Xu, Zhao Wu, Jin Chen +9Humanoid Loco-Manipulation Egocentric Dataset
Alibaba Group · Sichuan University · Shanghai Innovation Institute +2
HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos
May 24, 2026Zhi Wang, Botao He, Kelin Yu +4Human-To-Robot Transfer Egocentric Video
University of Maryland
EgoAlign: Bridging the Human-Humanoid Gap for Long-Range Loco-Manipulation
Sep 29, 2026Yiming Jiang, Jin Chen, Chongyang Xu +3Egocentric Dataset Humanoid Loco-Manipulation
Beihang University · Alibaba Group · Shanghai Innovation Institute +1

Sep 30, 2026, cs.ROJ/K move · Enter open · S save

Towards a General Humanoid Loco-Manipulation Model via Egocentric Whole-Body Human Data Pretraining

Chongyang Xu, Zhao Wu, Jin Chen, Yiming Jiang+8

Alibaba Group · Sichuan University · Shanghai Innovation Institute+2

Humanoid Loco-Manipulation Egocentric Dataset Articulated Objects Flow-Matching Vision-Language-Action Human Agency

Open paper AskPDF arXiv

Humanoid whole-body manipulation has advanced rapidly, enabling policies to coordinate locomotion, posture, bimanual interaction, and dexterous hand movements. Meanwhile, egocentric human videos provide diverse examples of everyday interactions across objects and scenes, offering scalable supervision without robot operation. However, existing supervision from these videos provides limited coverage of whole-body movement and coordination with hand-object interaction, while obtaining such supervision through humanoid teleoperation is also costly and difficult to scale. We therefore explore how human experience can support scalable learning of humanoid loco-manipulation. To support this study, we introduce HumanVerse-500, a 500-hour dataset of diverse human loco-manipulation behaviors in open-world environments, collected with a lightweight wearable system that synchronizes egocentric video with body and hand motion. Building on this dataset, we develop

λ_0

, a whole-body humanoid vision-language-action policy, through three-stage training that first learns interaction from diverse egocentric datasets, then coordinates body and hand motion using HumanVerse-500, and finally adapts the policy to downstream tasks and robot embodiments. Across these stages,

λ_0

learns a shared representation space for human experience transfer, while domain-specific interfaces handle differences between human and robot states and actions. We evaluate

λ_0

on SIMPLE and 4 real-world loco-manipulation tasks, achieving state-of-the-art performance, and further analyze its scaling behavior, generalization, and training-stage contributions to understand how human data support downstream whole-body humanoid control. We will release our code, models, and data to support further research.

Abstract

Explore similar work

Towards a General Humanoid Loco-Manipulation Model via Egocentric Whole-Body Human Data Pretraining

HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos

EgoAlign: Bridging the Human-Humanoid Gap for Long-Range Loco-Manipulation