Organizations: Institute of Artificial Intelligence (TeleAI), China Telecom · University of Science and Technology of China · The University of Hong Kong · Zhejiang University · Shanghai Jiao Tong University · ShanghaiTech University · Gamma Robotics
Humanoid robots must coordinate locomotion and manipulation in everyday environments, yet progress is difficult to assess across differing policies, controllers, and task settings. LocoManiLab provides an integrated testbed spanning data collection and generation, policy training, evaluation, and physical deployment. Its task suite covers 234 atomic and compositional tasks across five household scene categories, supported by 51.5 hours of simulated teleoperation. The evaluation covers twelve policies with a shared controller across task and scene splits and perturbation levels, two policies across five controllers, and studies of demonstration scale and data-source mixtures. Success rates fall on unseen tasks and scenes and generally decline as visual perturbations increase, with differences in generalization and robustness across policies. For data generation, a humanoid motion generation model is fine-tuned for loco-manipulation to study the alignment of its outputs with task execution requirements and its integration into the data generation pipeline. Three simulated loco-manipulation tasks closely match real-world setups, enabling evaluation of three policies in both domains. Similar policy rankings support the reliability of simulation-based evaluation.
Figures & tables
Figure 1: Overview of LocoManiLab . Scene categories span kitchens, living rooms, bedrooms, bathrooms, and offices, with affordances including articulated appliances, drawers, object placement, and desk interactions. Data collection combines teleoperation with controller-executed motion generation, while dedicated interfaces connect policies to compatible controllers. Evaluation measures task completion and execution quality on seen tasks, unseen tasks and scenes, and under domain randomization, with a shared workflow.
Figure 2: Teleoperation data statistics. Panels (a)–(c) describe the SONIC corpus by scene category, task-chain duration, and body motion. Duration is shown by median, interquartile range, and 5th–95th percentile. Panel (d) compares recorded execution time for 50 demonstrations per task on ten shared bathroom tasks, totaling 500 episodes per controller. Stacked segments separate atomic and compositional tasks, and bar labels give total hours.
Figure 3: Controller comparison at 50 demonstrations per task. Each controller panel shows GR00T N1.7 and π0.5 on atomic profiles with solid lines and filled circles, and compositional profiles with dashed lines and hollow diamonds. SR uses all episodes and other metrics use successful episodes only. Radial ticks show normalized values, with outward values better. Shared raw ranges, rounded to three decimals, are SR 5–79%, power 13.795–317.734 W, slip 0.093–20.911 cm/s, action rate 3.048–45.392 rad/s, falls 0–22.785%, and relative time 0.873–1.859. SR and falls use linear scales and other metrics use logarithmic scales.
Figure 5: Task-execution sequences in real-world (a–c) and simulated (d–f) settings, arranged as real-world sequences in the left column and simulated sequences in the right column. Rows show conveyor box handling, cup transport, and fridge opening. Selected frames progress from left to right within each row. Real and simulated sequences are not temporally aligned, and success is measured separately in each environment. Corresponding rates appear in Table 6 .
Figure 6: Representative task demonstrations across five scene categories. Upper rows show atomic objectives: (a) open a fridge, (b) place a book in a basket, (c) lift a basket from a stool, (d) close a cabinet, and (e) approach a desk. Lower rows show compositional objectives: (f) open a drawer, place an apple inside, and close it; (g) pack a candle and statue into a basket; (h) approach and open a safe; (i) open a drawer, retrieve a paper roll, and close it; and (j) approach a trash can and lift a box. Each row contains six chronological frames from a recorded SONIC demonstration, rendered through action replay; nonuniform frame intervals highlight task stages.
Figure 7: Task environments and representative interaction objects. (a) Thirteen household scenes from Holodeck and iTHOR, labeled by room category and scene identifier, followed by two Lab layouts. (b) Thirty-two representative task assets, grouped into articulated objects, containers and surfaces, and movable objects. The groups illustrate affordances for opening and closing, object placement, and manipulation within atomic and compositional tasks. Rooms and objects are individually scaled for visibility and are not shown to a common physical scale.
Figure 7
Figure 10: Head camera used in the real-robot experiments. (a) Front and (b) side views of the ZED Mini stereo camera mounted on the G1 head; policies use its left RGB image.
Humanoid foundation models are advancing faster than we can evaluate them. While real-world testing is expensive and difficult to reproduce, existing simulation benchmarks focus primarily on table-top or wheeled robots. A scalable and reproducible benchmark for whole-body humanoid loco-manipulation remains an open problem. To this end, we present SIMPLE, a unified simulation testbed for humanoid policy learning and evaluation. SIMPLE couples the accurate contact-rich dynamics of MuJoCo with the photorealistic rendering of IsaacSim. It provides a large-scale environment comprising 16 diverse whole-body tasks, 50 indoor scenes, and over 1,000 object assets. To facilitate scalable data collection, the framework integrates two data generation pipelines: automated trajectory generation via motion planning and a low-latency VR teleoperation interface. We further integrate and benchmark mainstream humanoid policies at scale in SIMPLE, including lightweight imitation networks, large vision-language-action (VLA) models, and recent world action models (WAMs). Our experiments reveal a strong correlation between policy performance in simulation and the real world. Furthermore, we demonstrate that policies trained on data collected in SIMPLE can be transferred zero-shot to physical humanoid robots under similar settings, providing a robust and reproducible foundation for humanoid robotics research.
Visual loco-manipulation policies that can generalize to novel scenes and objects have long been a goal of robotics research. However, today's data-hungry algorithms make collecting sufficient demonstrations a struggle for tabletop manipulation, and even more so for humanoids that must also walk and balance. Learning from simulated data and transferring that behavior to the real world, as is commonly done in locomotion, sidesteps this struggle, so we replicate that recipe for loco-manipulation. In doing so, we find that cloning synthetic demonstrations results in a low performance ceiling no matter the amount of training data. Reinforcement learning breaks through it, and refining the cloned policy with Flow-GRPO on a single sparse reward yields performance that synthetic behavior cloning cannot match. Together, these stages form our end-to-end sim-to-real pipeline spanning more than 150,000 scenes, which we use to train FetchMan. We evaluate it on FetchMan-Bench, a simulation benchmark we release, and deploy it zero-shot on a real Unitree G1, where our single-object reach-and-pick policy walks to and grasps a target across unseen scenes at 73.3% success. Finally, we extend this recipe to multi-object training, a first step toward loco-manipulation generalist policies at this data scale.
Omar Rayyan, Zhi Li, Max Argus +4
University of California, Los Angeles · Allen Institute for AI · University of Washington
Imitation learning is a promising approach for training humanoid robots to both walk and manipulate, but it requires a large number of demonstrations, which are time-intensive and difficult to collect via teleoperation. Existing data-generation algorithms can automatically synthesize demonstrations for manipulators, but they are ineffective on humanoids because their high-dimensional composite action spaces involve arms, legs, and torsos. We present HumanoidMimicGen, a method for generating humanoid legged loco-manipulation data. Our method adapts contact-rich whole-body skills from a handful of source demonstrations to new states, generalizing across changes in object pose. By interleaving these single- and dual-arm skills with whole-body locomotion and manipulation planning, the method generates stable, collision-free data across diverse scenes and layouts. To evaluate our approach, we introduce a new simulated loco-manipulation benchmark containing nine diverse tasks that test humanoid loco-manipulation capabilities. There, we demonstrate that HumanoidMimicGen automatically generates large datasets for imitation learning and enables a systematic study of how data generation and policy learning decisions impact model performance. We show that whole-body visuomotor policies co-trained with data generated by HumanoidMimicGen outperform those trained only on real-world data by 20%.