cs.RONov 4, 2026

Perceptive Humanoid Motion from Simulation

Authors: Siyuan Hu, Abulikemu Abuduweili, Yue Meng, Shipeng Liu, Quentin Leboutet, Chen Chen, Matthias Müller, Vladlen Koltun

Organizations: Apple

Abstract

Humanoid robots have the potential to automate a wide range of physical tasks. Progress toward generalist humanoid autonomy, however, remains constrained by a data bottleneck: mapping pixels and free-form language to low-level joint commands requires paired supervision, and no corpus of such data exists at sufficiently large scale. Some approaches rely on human teleoperation and wearable capture devices, which require at least an hour of human time to produce an hour of data. We present HALO, a humanoid autonomy system powered by a pipeline that synthesizes vision-language-motion data across a thousand procedurally generated homes. The corpus scales with compute rather than human labor. HALO acquires a repertoire of physical skills, including sitting on chairs, navigating to named objects, stepping sideways through half-open doors, crouching and crawling beneath tables, and pushing boxes. It orchestrates these skills to solve composite, long-horizon tasks. Without ever seeing a real home during training, HALO transfers zero-shot to the physical world, achieving a 91% success rate across 360 trials in six unmapped real-world environments. See https://apple-aiml-research.github.io/ml-halo/ for code and videos.

Figures & tables

Explore similar work

CardsList
  1. Zero-HLM: Generalizable Humanoid Loco-Manipulation without Real Robot Data

    Oct 26, 2026Dianyi Yang, Mengya Liu, Zhumei Wang +9Humanoid Loco-ManipulationSim-to-Real Transfer

  2. HALO: Human-Aligned Data Collection for WhoLe-body MObile Manipulation

    Nov 12, 2026Shuai Yang, Fangyi Xu, Yishu Shen +2Robotic Data CollectionWhole-Body Robotic Manipulation