Organizations: Institute of Artificial Intelligence (TeleAI), China Telecom · University of Science and Technology of China · The University of Hong Kong · Zhejiang University · Shanghai Jiao Tong University · ShanghaiTech University · Gamma Robotics
Humanoid robots must coordinate locomotion and manipulation in everyday environments, yet progress is difficult to assess across differing policies, controllers, and task settings. LocoManiLab provides an integrated testbed spanning data collection and generation, policy training, evaluation, and physical deployment. Its task suite covers 234 atomic and compositional tasks across five household scene categories, supported by 51.5 hours of simulated teleoperation. The evaluation covers twelve policies with a shared controller across task and scene splits and perturbation levels, two policies across five controllers, and studies of demonstration scale and data-source mixtures. Success rates fall on unseen tasks and scenes and generally decline as visual perturbations increase, with differences in generalization and robustness across policies. For data generation, a humanoid motion generation model is fine-tuned for loco-manipulation to study the alignment of its outputs with task execution requirements and its integration into the data generation pipeline. Three simulated loco-manipulation tasks closely match real-world setups, enabling evaluation of three policies in both domains. Similar policy rankings support the reliability of simulation-based evaluation.
Figures & tables
Figure 1: Overview of LocoManiLab . Scene categories span kitchens, living rooms, bedrooms, bathrooms, and offices, with affordances including articulated appliances, drawers, object placement, and desk interactions. Data collection combines teleoperation with controller-executed motion generation, while dedicated interfaces connect policies to compatible controllers. Evaluation measures task completion and execution quality on seen tasks, unseen tasks and scenes, and under domain randomization, with a shared workflow.
Figure 2: Teleoperation data statistics. Panels (a)–(c) describe the SONIC corpus by scene category, task-chain duration, and body motion. Duration is shown by median, interquartile range, and 5th–95th percentile. Panel (d) compares recorded execution time for 50 demonstrations per task on ten shared bathroom tasks, totaling 500 episodes per controller. Stacked segments separate atomic and compositional tasks, and bar labels give total hours.
Figure 3: Controller comparison at 50 demonstrations per task. Each controller panel shows GR00T N1.7 and π0.5 on atomic profiles with solid lines and filled circles, and compositional profiles with dashed lines and hollow diamonds. SR uses all episodes and other metrics use successful episodes only. Radial ticks show normalized values, with outward values better. Shared raw ranges, rounded to three decimals, are SR 5–79%, power 13.795–317.734 W, slip 0.093–20.911 cm/s, action rate 3.048–45.392 rad/s, falls 0–22.785%, and relative time 0.873–1.859. SR and falls use linear scales and other metrics use logarithmic scales.
Figure 5: Task-execution sequences in real-world (a–c) and simulated (d–f) settings, arranged as real-world sequences in the left column and simulated sequences in the right column. Rows show conveyor box handling, cup transport, and fridge opening. Selected frames progress from left to right within each row. Real and simulated sequences are not temporally aligned, and success is measured separately in each environment. Corresponding rates appear in Table 6 .
Figure 6: Representative task demonstrations across five scene categories. Upper rows show atomic objectives: (a) open a fridge, (b) place a book in a basket, (c) lift a basket from a stool, (d) close a cabinet, and (e) approach a desk. Lower rows show compositional objectives: (f) open a drawer, place an apple inside, and close it; (g) pack a candle and statue into a basket; (h) approach and open a safe; (i) open a drawer, retrieve a paper roll, and close it; and (j) approach a trash can and lift a box. Each row contains six chronological frames from a recorded SONIC demonstration, rendered through action replay; nonuniform frame intervals highlight task stages.
Figure 7: Task environments and representative interaction objects. (a) Thirteen household scenes from Holodeck and iTHOR, labeled by room category and scene identifier, followed by two Lab layouts. (b) Thirty-two representative task assets, grouped into articulated objects, containers and surfaces, and movable objects. The groups illustrate affordances for opening and closing, object placement, and manipulation within atomic and compositional tasks. Rooms and objects are individually scaled for visibility and are not shown to a common physical scale.
Figure 7
Figure 10: Head camera used in the real-robot experiments. (a) Front and (b) side views of the ZED Mini stereo camera mounted on the G1 head; policies use its left RGB image.