cs.ROOct 13, 2026
Save: Scaling Open-World Egocentric Human Data for Humanoid Loco-Manipulation
Organizations: Alibaba Token Hub and Amap, Alibaba Group
Abstract
Humanoid loco-manipulation requires coordinated whole-body interaction with the physical world, yet collecting robot demonstrations at scale remains costly. Human demonstrations offer a scalable alternative, but existing egocentric human supervision often omits locomotion and posture, while embodiment differences and controller dynamics hinder accurate end-effector execution. We introduce HumanVerse, a 500-hour dataset of open-world human loco-manipulation pairing egocentric video with synchronized body and hand motion. Our human-to-humanoid pipeline aligns camera viewpoints and arm appearance, and refines motion references and controller inputs offline to reduce end-effector errors while preserving whole-body coordination. We validate the pipeline through zero-shot transfer to Unitree G1 on representative whole-body manipulation and long-range loco-manipulation tasks, without target-task robot demonstrations. Building on this pipeline, we develop , a humanoid foundation model for whole-body loco-manipulation, with robot-aligned human demonstrations providing supervision for mid-training. Increasing whole-body human data consistently improves downstream task progress. achieves 90.0% success on SIMPLE and 62.5% across real-world loco-manipulation tasks. We expect this study to guide scalable whole-body human data collection toward humanoid foundation models for richer interaction with the physical world. Homepage: https://lambdahumanoid.github.io/
Figures & tables
Figure 1: Introducing with HumanVerse . Efficient robot-free capture and human-to-humanoid alignment enable scalable whole-body human supervision, supporting three-stage training and improved humanoid loco-manipulation as human data scales.
Figure 2: HumanVerse . The wearable capture setup, task and motion distributions, and paired egocentric video and whole-body motions illustrate the dataset’s scale and diversity.
Figure 3: Robot-free capture, curation, and robot alignment. We select open-world scenes and tasks and record egocentric video, body motion, and speech with a robot-matched camera viewpoint. Curation includes synchronization, language/task annotation, body and hand motion reconstruction, observation alignment, action alignment, and quality inspection. Modality thumbnails are schematic.
Figure 4: Representative examples from HumanVerse . Sixty first-person frames are arranged in a gallery to illustrate diversity in tasks, scenes, objects, and hand–object interactions.
Figure 5: Synchronized human motion and egocentric observations. Top: egocentric RGB frames. Bottom: corresponding SMPL body meshes with MANO hand motion and illustrative camera frusta. Camera frusta schematically illustrate viewing direction.
Figure 6: Human-to-humanoid action alignment. The first two stages modify the human and robot references. The third accounts for execution dynamics and directly optimizes motion tokens using gradient-based updates. End-effector targets remain fixed throughout. Rendered poses illustrate the procedure and are not experimental rollouts.
Figure 7: Three-stage training of . The policy learns interaction from egocentric data, whole-body coordination from HumanVerse , and downstream control from real-robot data.
Figure 8: Task progress from aligned human demonstrations. Scores include partial completion; long-range results average seen and unseen target locations. N/R indicates missing before-alignment scores.
Figure 9: Task performance and collection labor with human or robot supervision. Equal-weight means over Close Laptop and Pull Curtain. Left: task scores versus demonstrations per task, evaluated over 20 trials per policy. Right: person-seconds per demonstration, including capture, amortized setup, calibration, troubleshooting, and resets, with one person for human capture and two for teleoperation.
Figure 10: Real-world data collection and evaluation. (a) Egocentric teleoperation collects whole-body demonstrations. (b) Four loco-manipulation tasks evaluate task success. (c) Robot-seen and robot-unseen objects are used for standard task and generalization evaluation, respectively.
| Method | Bottle Disposal | Toy Storage | Box Transport | Chair Placement | Average | |||||
| ( Black et al., 2025 ) | 2/10 | 28.0 | 5/10 | 64.5 | 5/10 | 58.0 | 4/10 | 65.5 | 16/40 | 54.0 |
| GR00T N1.6 ( GEAR Team et al., 2025 ) | 0/10 | 10.0 | 1/10 | 19.0 | 0/10 | 10.0 | 2/10 | 46.0 | 3/40 | 21.3 |
| ( Wei et al., 2026b ) | 1/10 | 26.5 | 2/10 | 30.0 | 1/10 | 37.5 | 0/10 | 33.0 | 4/40 | 31.8 |
| StarVLA ( StarVLA Community, 2026 ) | 0/10 | 10.0 | 5/10 | 55.0 | 3/10 | 71.0 | 3/10 | 82.5 | 11/40 | 54.6 |
| (Ours) | 5/10 | 79.0 | 7/10 | 81.0 | 7/10 | 84.5 | 6/10 | 77.0 | 25/40 | 80.4 |
Table 1: Real-world task performance. We report success rates over ten trials and weighted progress (%) for each task. Averages weight the four tasks equally.
| Model | XMovePick | BendPick | Handover | Mobile P&P | Grasp | XMoveBendPick | Average | Overall |
| H-RDT ( Bi et al., 2026 ) | 0/0/20 | 0/0/10 | 0/10/0 | 0/0/0 | 0/0/0 | 0/0/0 | 0.0/1.7/5.0 | 2.2 |
| InternVLA-M1 ( Chen et al., 2025 ) | 0/0/0 | 50/50/0 | 0/0/0 | 0/0/0 | 0/0/0 | 30/50/70 | 13.3/16.7/11.7 | 13.9 |
| EgoVLA ( Yang et al., 2026b ) | 0/10/20 | 70/50/80 | 0/40/30 | 0/0/0 | 100/100/70 | 30/50/40 | 33.3/41.7/40.0 | 38.3 |
| Diffusion Policy ( Chi et al., 2025 ) | 30/30/20 | 100/80/60 | 30/20/40 | 40/0/0 | 80/90/80 | 0/0/0 | 46.7/36.7/33.3 | 38.9 |
| GR00T N1.6 ( GEAR Team et al., 2025 ) | 100/100/70 | 70/70/60 | 10/30/30 | 0/0/0 | 90/90/70 | 40/40/10 | 51.7/55.0/40.0 | 48.9 |
| ( Black et al., 2025 ) | 70/50/10 | 100/100/80 | 50/40/50 | 30/30/30 | 100/100/80 | 0/0/0 | 58.3/53.3/41.7 | 51.1 |
Table 2: SIMPLE task performance. Success rates (%) at Levels 0–2 with ten trials per task–level pair. Average is computed per level. Overall averages all 18 task–level pairs.
Figure 11: Human-data scaling of validation loss. (a) Training curves on representative and clean holdouts at five data fractions. (b) Minimum validation loss versus data scale with descriptive log-linear fits over the measured range. Minima are selected independently on each holdout.
Figure 12: Average real-world task progress versus Stage-II human-data fraction.
| Variant | S1 | S2 | Robot | SIMPLE (%) | Real-world (%) | ||||
| L0 | L1 | L2 | Overall | Success | Progress | ||||
| w/o human-data pretraining | 83.3 | 83.3 | 71.7 | 79.4 | 15.0 | 54.0 | |||
| w/o Stage II mid-training | 83.6 | 86.7 | 76.7 | 82.3 | 22.5 | 59.9 | |||
| w/o Stage I pretraining | 88.0 | 91.7 | 83.7 | 87.8 | 50.0 | 75.9 | |||
| (Full Recipe) | 90.0 | 93.3 | 86.7 | 90.0 | 62.5 | 80.4 | |||
Table 3: Training-stage ablations with fixed robot supervision. Filled/open circles indicate included/removed data. Metrics report SIMPLE success and real-world averages across four tasks. SIMPLE configuration-selection limitations are detailed in Appendix D.2 .
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
| Resource | Hours | Body / hands | SMPL | Humanoid interface | Capture setup |
| EgoDex ( Hoque et al., 2026 ) | 829 | Upper body and articulated hands | NR | NR in dataset release | Apple Vision Pro |
| HOT3D ( Banerjee et al., 2025 ) | 13.9 | Hands and objects without full-body coverage | NR | NR | Aria / Quest 3 with external tracking |
| NymeriaPlus ( DeTone et al., 2026 ) | NR a | Full body, finger motion NR | Provided | NR | Aria, wristbands, XSens |
| EgoHumanoid ( Shi et al., 2026 ) | NR | Full body and articulated hands | NR | G1: wrist targets + locomotion commands | PICO + trackers, ZED X Mini |
| EgoScale ( Zheng et al., 2026c ) | 20,854 | Wrist and hand action supervision | NR | G1 transfer via wrist / hand interface | Multiple human-data sources |
| Index ( Figure, 2026 ) | NR | Pose schema NR | NR | Figure / Helix, G1 NR | Distributed creator recordings |
Table 4: Egocentric human-data resources and documented training interfaces. Body and hand coverage denotes motion supervision. NR denotes an unreported interface, not an incompatible one.
| Method | Human signal | Role in training | Controlled scale | Primary question |
| WholeBodyVLA ( Jiang et al., 2026a ) | 300 h action-free egocentric video | latent locomotion pre-training, robot manipulation latents, and teleop grounding | 0–100% video teleop trajectories | Can visual latent actions reduce loco-manipulation teleoperation? |
| EgoHumanoid ( Shi et al., 2026 ) | same-task RGB and wearable motion | direct human–robot co-training after view/action alignment | 0–300 human demonstrations and sampling ratio | Can task-matched human data improve scene generalization? |
| ( Wei et al., 2026b ) | 829 h wrist/fingertip-labeled EgoDex video | task-space VLM pre-training before robot joint-action learning | full EgoDex versus 10% | Which staged data recipe yields an effective humanoid foundation model? |
| OpenHLM ( Hu et al., 2026 ) | task-targeted stationary HuMI motion | heterogeneous co-training after whole-body VLA adaptation | whole-body teleop count and data-source choice | Which interfaces and data sources yield a whole-body-native VLA? |
| -0 ( Li et al., 2026b ) | public human video and whole-body motion (ARCTIC, Xperience-10M, Motion-X) | action-token VLM pre-training and SONIC replay for predictive action-latent learning | human-data volume scaling not reported; data-source and model ablations | Can latent visual foresight improve concurrent whole-body loco-manipulation? |
| 500 h synchronized RGB, full-body motion, and bilateral hands | human-data mid-training followed by robot post-training | 0–100% Stage-II human data with a fixed robot budget | What scales with explicit whole-body human experience under fixed grounding? |
Table 5: Problem-level comparison with the closest humanoid VLA and world-action studies. “General” denotes human data not collected for the downstream evaluation tasks. “Task-targeted” denotes demonstrations whose task or object coverage is selected for the downstream study. This table compares experimental questions rather than reported scores.
| Stage | Optimizer updates | Updated modules |
| I | 200k | Vision–language backbone |
| II | 100k | Backbone, action expert, human-data projections |
| III | 50k | Backbone, action expert, robot interfaces and conditioning |
Table 6: Default training schedule for the 2B model. Stages II and III predict 50-step action chunks.
| Task | Scored task stages (points) | Terminal success predicate |
| Bottle disposal | 1. Approach the bottle (10) 2. Grasp stably (15) 3. Lift clear of the table (15) 4. Arrive at the bin (15) 5. Press the pedal to open the lid (20) 6. Release the bottle into the bin (20) 7. Remain stable after disposal (5) | Bottle fully inside the trash can and robot standing without contact assistance |
| Single-hand toy storage | 1. Approach the toy (10) 2. Establish contact (15) 3. Grasp stably (20) 4. Lift clear of the table (15) 5. Transport above the box (15) 6. Place inside the box (20) 7. Release with the toy remaining inside (5) | Toy fully inside the box, released, and not displaced during hand withdrawal |
| Bimanual parts-box transport | 1. Approach the box (10) 2. Establish bilateral contact (15) 3. Establish a stable grasp (20) 4. Lift clear of the table (15) 5. Transport above the target surface (20) 6. Place on the target surface (15) 7. Release with the box remaining stable (5) | Box inside the target region, both hands released, and no contained parts spilled |
| Chair placement | 1. Approach the chair (10) 2. Turn and align with the chair (15) 3. Establish bilateral contact (10) 4. Grasp stably and initiate pushing (20) 5. Push continuously toward the table (20) 6. Complete insertion under the table (20) 7. Release and stop safely (5) | Chair footprint, position, and yaw satisfy pre-registered under-table tolerances, with the robot released and stable |
Table 7: Operational definitions for the real-world evaluation. Progress is the sum of point values for completed stages (shown in parentheses, totaling 100 per task). Terminal success additionally requires the task-specific predicate to remain satisfied for three seconds.
More in this topic
Humanoid Loco-Manipulation
Humanoid loco-manipulation studies how robots coordinate walking, balance, and whole-body interaction to move and handle objects.
See all 86 papers