Humanoid Robotics

Ask about this topic

Overview

Humanoid robotics studies how to build and control human-shaped machines for tasks that combine walking, balance, manipulation, and interaction in spaces designed for people. Common approaches include model-based whole-body control and model-predictive control, reinforcement learning and motion imitation, and perception systems that use cameras, depth, touch, or proprioception. Human demonstrations, video, and simulation are increasingly used to train skills, while benchmarks test locomotion, object handling, collaboration, and social behavior. A central challenge is making skills learned in simulation or from demonstrations reliable under real-world contact, uncertainty, and disturbances, while evaluating safety and task success consistently.

Latest papers

CardsList

152 papers32 papers in the last four weeks, up 39% on the four weeks before. 0.2% of all new papers.

May 12, 2026

OmniHumanoid: Streaming Cross-Embodiment Video Generation with Paired-Free Adaptation

Cross-embodiment video generation aims to transfer motions across different humanoid embodiments, such as human-to-robot and robot-to-robot, enabling scalable data generation for embodied intelligence. A major challenge in this setting is that motion dynamics are partly transferable across embodiments, whereas appearance and morphology remain embodiment-specific. Existing approaches often entangle these factors, and many require paired data for every target embodiment, which limits scalability to new robots. We present OmniHumanoid, a framework that factorizes transferable motion learning and embodiment-specific adaptation. Our method learns a shared motion transfer model from motion-aligned paired videos spanning multiple embodiments, while adapting to a new embodiment using only unpaired videos through lightweight embodiment-specific adapters. To reduce interference between motion transfer and embodiment adaptation, we further introduce a branch-isolated attention design that separates motion conditioning from embodiment-specific modulation. In addition, we construct a synthetic cross-embodiment dataset with motion-aligned paired videos rendered across diverse humanoid assets, scenes, and viewpoints. Experiments on both synthetic and real-world benchmarks show that OmniHumanoid achieves strong motion fidelity and embodiment consistency, while enabling scalable adaptation to unseen humanoid embodiments without retraining the shared motion model.

Yiren Song, Xiyao Deng, Pei Yang +2

Show Lab, National University of Singapore

Unsupervised Domain AdaptationStreaming Video GenerationSynthetic Data Generation for RoboticsVideo GenerationCross-Embodiment TransferHumanoid Robotics
AskPDF
May 12, 2026

Mapping Embodied Affective Touch Strategies on a Humanoid Robot

Affective touch in human-robot interaction is shaped not only by emotional intent, but also by robot embodiment, including touch location, physical constraints, and perceived agency or social role. Existing HRI studies typically focus on one or two isolated body parts, limiting understanding of how affective touch generalises across the full humanoid body. We present a study with 32 participants interacting with the iCub robot, which is equipped with full-body distributed tactile sensors. Participants expressed eight emotions under three conditions: free touch, arm-only touch, and torso-only touch. Results show that body region and spatial constraints jointly shaped both touch location and dynamics. In free touch, participants preferred socially accessible upper-body regions, while less frequently touched areas showed stronger emotion-specific selectivity. Emotion-related variation was more evident in motion features for arm-only touch and pressure features for torso-only touch. Touch strategies also did not transfer directly between free and constrained conditions, even within the same coarse body region. Participants reported increased closeness to the robot after interaction, with around 30 percent reporting a change in perceived social relationship. Together, these findings show that affective touch expression is strongly body-region dependent and shaped by embodiment constraints.

Qiaoqiao Ren, Omar Eldardeer, Francesca Cocchella +3

Faculty of Engineering and Architecture, 1IDLab-AIRO, Ghent University – imec, Technologiepark 126, 9052 Gent, Belgium · School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology · COgNiTive Architecture for Collaborative Technologies Unit of the Italian Institute of Technology

Human-Robot InteractionHumanoid Robotics
AskPDF
May 5, 2026

BifrostUMI: Bridging Robot-Free Demonstrations and Humanoid Whole-Body Manipulation

High-quality demonstration data are essential for humanoid robot skill learning, especially for whole-body behaviors that require coordinated perception, locomotion, and manipulation. Existing data-collection methods largely rely on robot teleoperation, which is constrained by hardware accessibility, operator expertise, and limited efficiency. Inspired by the Universal Manipulation Interface (UMI), we propose BifrostUMI, a portable and robot-free framework for humanoid whole-body data collection. BifrostUMI uses lightweight VR devices and UMI-inspired grippers to collect sparse human keypoint trajectories, wrist-view observations, and gripper actions. These demonstrations train a high-level policy to predict future keypoints, which are retargeted to robot-native whole-body references and executed by a whole-body controller. Experiments in five real-world scenarios demonstrate the effectiveness of the proposed framework and validate the collected demonstrations for transferable humanoid whole-body skill learning.

Hongwu Wang, Chenhao Yu, Youhao Hu +3

Beijing Academy of Artificial Intelligence

Robotic Data CollectionHumanoid Robot ControlRobot Imitation LearningHumanoid Robotics
AskPDF
May 2, 2026

SixthSense: Task-Agnostic Proprioception-Only Whole-Body Wrench Estimation for Humanoids

Humanoid robots are entering our physical world at scale, yet as oversized toys--good at singing and dancing, but short on force-interaction capabilities for practical tasks. Bridging this gap necessitates prioritizing reliable contact perception as a fundamental requirement. Estimating external wrenches in humanoids is complicated by floating-base dynamics and indeterminate contact locations. Existing analytical frameworks require idealistic assumptions and hard-to-obtain measurements, which are often unavailable in practice. To bridge this gap, we propose SixthSense, a task-agnostic approach that infers whole-body contact timing, location, and wrenches from proprioception and IMU data alone. To capture the multi-modal dynamics between unstructured contact inputs and the uncertain motion outputs, we employ conditional flow matching to tokenize proprioceptive histories and estimate a spatiotemporally sparse contact-event flow. SixthSense serves as a plug-and-play perception module for applications including collision detection, physical human-robot interaction, and force-feedback teleoperation. Experiments across standing, walking, and whole-body motion-tracking policies showcased unprecedented performance in diverse behaviors.

Xingzhou Chen, Xiayan Xu, Yan Ning +8

The Hong Kong University of Science and Technology, Hong Kong SAR, China · Zhejiang University, Hangzhou, China · Tencent Robotics X, Shenzhen, China

Robotic PerceptionHumanoid RoboticsContact Force Estimation
AskPDF
Apr 30, 2026

Simulating Infant First-Person Sensorimotor Experience via Motion Retargeting from Babies to Humanoids

Motion retargeting from humans to human-like artificial agents is becoming increasingly important as humanoid robots grow more capable. However, most existing approaches focus only on reproducing kinematics and ignore the rich sensorimotor experience associated with human movement. In this work, we present a framework for simulating the multimodal sensorimotor experiences of infants using physical and virtual humanoids. From a single video, our method reconstructs the infant's body configuration by extracting its skeletal structure and estimating the full 3D pose from each frame. Then we map the reconstructed motion onto several developmental platforms: the physical iCub robot and the virtual simulators pyCub, EMFANT and MIMo. Replaying the retargeted motions on these embodiments produces simulated multisensory streams including proprioception (joints and muscles), touch, and vision. For the best-matching embodiment, the retargeting achieves sub-centimeter accuracy and enables a rich multimodal analysis of infant development as well as enhanced automated annotation of behaviors. This framework provides a unique window into the infant's sensorimotor experience, offering new tools for robotics, developmental science, and early detection of neurodevelopmental disorders. The code is available at https://github.com/ctu-vras/motion-retargeting/.

Francisco M. López, Hoshinori Kanazawa, Ondrej Fiala +8

Frankfurt Institute for Advanced Studies, Germany · Graduate School of Information Science and Technology, The University of Tokyo, Japan · Department of Cybernetics, Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic +1

Robotics SimulationSynthetic Data Generation for RoboticsMotion RetargetingHumanoid Robotics
AskPDF
Apr 28, 2026

Egocentric Tactile and Proximity Sensors as Observation Priors for Humanoid Collision Avoidance

Collision-free motion is often aided by tactile and proximity sensors distributed on the body of the robot due to their resistance to occlusion as opposed to external cameras. However, how to shape the sensor's properties, such as sensing coverage; type; and range, to enable avoidant behavior remains unclear. In this work, we present a reinforcement learning framework for whole-body collision avoidance on a humanoid H1-2 robot and use it to characterize how sensor properties shape learned avoidance behavior. Using dodgeball as a benchmark task, we ablate the properties of sensors distributed across the upper body of the robot and find that raw proximity measurements can substitute for explicit object localization provided the sensing range is sufficient and that sparse non-directional proximity signals outpace dense directional alternatives in sample efficiency.

Carson Kohlbrenner, Niraj Pudasaini, William Xie +3

University of Colorado Boulder, 1111 Engineering Drive, Boulder, CO USA.

RL for RoboticsRobotic Tactile SensingRobotic PerceptionHumanoid RoboticsRobot Collision Avoidance
AskPDF
Apr 24, 2026

RecoverFormer: End-to-End Contact-Aware Recovery for Humanoid Robots

Humanoid robots operating in unstructured environments must recover from unexpected disturbances-a capability that remains challenging for end-to-end control policies. We present RECOVERFORMER, a fully end-to-end humanoid recovery policy that learns when and how to switch among recovery behaviors-including compensatory stepping, hand-environment contact, and center-of-mass reshaping-while maintaining robust performance under model mismatch. The architecture combines a causal transformer over a 50-step observation history with two novel heads: a latent recovery mode that enables smooth transitions among distinct recovery strategies, and a contact affordance head that predicts which environmental surfaces (walls, railings, table edges) are beneficial for stabilization. We evaluate RECOVERFORMER on the Unitree G1 humanoid in MuJoCo. Trained only on open floor, RECOVERFORMER transfers zero shot to walled environments, achieving 100% recovery success across 100-300 N pushes and across wall distances from 0.25-1.4m. Under zero-shot dynamics mismatch, RECOVERFORMER reaches 75.5% at plus +25% mass, 89% under 30 ms latency, 91.5% at low friction, and 99% under compound friction, latency and mass perturbation. The learned latent modes specialize across force regimes without mode-level supervision, validated by t-SNE analysis of 300 episodes. Taken together, these results show that a single end-to-end policy can deliver multi-modal, contact aware humanoid recovery that generalizes across perturbation magnitude, contact geometry, and dynamics shift.

Zihui Liu

Stanford University

Humanoid RoboticsHumanoid Robot ControlRobot Failure RecoveryHumanoid Robot LocomotionRobot Policy GeneralizationRobust RL+1
AskPDF
Apr 20, 2026

SynAgent: Generalizable Cooperative Humanoid Manipulation via Solo-to-Cooperative Agent Synergy

Controllable cooperative humanoid manipulation is a fundamental yet challenging problem for embodied intelligence, due to severe data scarcity, complexities in multi-agent coordination, and limited generalization across objects. In this paper, we present SynAgent, a unified framework that enables scalable and physically plausible cooperative manipulation by leveraging Solo-to-Cooperative Agent Synergy to transfer skills from single-agent human-object interaction to multi-agent human-object-human scenarios. To maintain semantic integrity during motion transfer, we introduce an interaction-preserving retargeting method based on an Interact Mesh constructed via Delaunay tetrahedralization, which faithfully maintains spatial relationships among humans and objects. Building upon this refined data, we propose a single-agent pretraining and adaptation paradigm that bootstraps synergistic collaborative behaviors from abundant single-human data through decentralized training and multi-agent PPO. Finally, we develop a trajectory-conditioned generative policy using a conditional VAE, trained via multi-teacher distillation from motion imitation priors to achieve stable and controllable object-level trajectory execution. Extensive experiments demonstrate that SynAgent significantly outperforms existing baselines in both cooperative imitation and trajectory-conditioned control, while generalizing across diverse object geometries. Codes and data will be available after publication. Project Page: http://yw0208.github.io/synagent

Wei Yao, Haohan Ma, Hongwen Zhang +6

School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China · School of Artificial Intelligence, Beijing Normal University, Beijing 100875, China +2

Robotic ControlHOI GenerationGeneralization in Robotic ManipulationRobot Imitation LearningCooperative MARLHumanoid Robotics+1
AskPDF
Apr 19, 2026

A Rapid Deployment Pipeline for Autonomous Humanoid Grasping Based on Foundation Models

Deploying a humanoid robot to manipulate a new object has traditionally required one to two days of effort: data collection, manual annotation, 3D model acquisition, and model training. This paper presents an end-to-end rapid deployment pipeline that integrates three foundation-model components to shorten the onboarding cycle for a new object to approximately 30 minutes: (i) Roboflow-based automatic annotation to assist in training a YOLOv8 object detector; (ii) 3D reconstruction based on Meta SAM 3D, which eliminates the need for a dedicated laser scanner; and (iii) zero-shot 6-DoF pose tracking based on FoundationPose, using the SAM3D-generated mesh directly as the template. The estimated pose drives a Unity-based inverse kinematics planner, whose joint commands are streamed via UDP to a UnitreeG1 humanoid and executed through the Unitree SDK. We demonstrate detection accuracy of [email protected] = 0.995, pose tracking precision of σ<1.05σ< 1.05 mm, and successful grasping on a real robot at five positions within the workspace. We further verify the generality of the pipeline on an automobile-window glue-application task. The results show that combining foundation models for perception with everyday imaging devices (e.g., smartphones) can substantially lower the deployment barrier for humanoid manipulation tasks.

Yifei Yan, Yankai Liao, Linqi Ye

School of Future Technology, Shanghai University, Shanghai, 200444, China · School of Mechanical Engineering, Shanghai Institute of Technology, Shanghai, 201418, China

Robotic GraspingHumanoid Robotics
AskPDF
Apr 13, 2026

CLAW: Composable Language-Annotated Whole-body Motion Generation

Training language-conditioned whole-body controllers for humanoid robots demands large-scale motion-language datasets. Existing approaches based on motion capture are costly and limited in diversity, while text-to-motion generative models produce purely kinematic outputs that are not guaranteed to be physically feasible. We present CLAW, a pipeline for scalable generation of language-annotated whole-body motion data for the Unitree G1 humanoid robot. CLAW composes motion primitives from a kinematic planner, parameterized by movement, heading, speed, pelvis height, and duration, and provides two browser-based interfaces--a real-time keyboard mode and a timeline-based sequence editor--for exploratory and batch data collection. A low-level controller tracks these references in MuJoCo simulation, yielding physically grounded trajectories. In parallel, a template-based engine generates diverse natural-language annotations at both segment and trajectory levels. To support scalable generation of motion-language paired data for humanoid robot learning, we make our system publicly available at: https://github.com/JianuoCao/CLAW

Jianuo Cao, Yuxin Chen, Masayoshi Tomizuka

Nanjing University · University of California, Berkeley

Robot Motion GenerationSynthetic Data Generation for RoboticsHumanoid Robotics
AskPDF
Apr 9, 2026

LEGO: Latent-space Exploration for Geometry-aware Optimization of Humanoid Kinematic Design

Designing robot morphologies and kinematics has traditionally relied on human intuition, with little systematic foundation. Motion-design co-optimization offers a promising path toward automation, but two major challenges remain: (i) the vast, unstructured design space and (ii) the difficulty of constructing task-specific loss functions. We propose a new paradigm that minimizes human involvement by (i) learning the design search space from existing mechanical designs, rather than hand-crafting it, and (ii) defining the loss directly from human motion data via motion retargeting and Procrustes analysis. Using screw-theory-based joint axis representation and isometric manifold learning, we construct a compact, geometry-preserving latent space of humanoid upper body designs in which optimization is tractable. We then solve design optimization in this latent space using gradient-free optimization. Our approach establishes a principled framework for data-driven robot design and demonstrates that leveraging existing designs and human motion can effectively guide the automated discovery of novel robot design.

Jihwan Yoon, Taemoon Jeong, Jeongeun Park +5

Department of Artificial Intelligence, Korea University, Seoul 02841, Republic of Korea · Contoro Robotics, Austin, TX 78758, USA · Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA +1

Robot Morphology-Control Co-DesignHumanoid RoboticsLatent Space Optimization
AskPDF
Apr 5, 2026

Dynamic Whole-Body Dancing with Humanoid Robots -- A Model-Based Control Approach

This paper presents an integrated model-based framework for generating and executing dynamic whole-body dance motions on humanoid robots. The framework operates in two stages: offline motion generation and online motion execution, both leveraging future state prediction to enable robust and dynamic dance motions in real-world environments. In the offline motion generation stage, human dance demonstrations are captured via a motion capture (MoCap) system, retargeted to the robot by solving a Quadratic Programming (QP) problem, and further refined using Trajectory Optimization (TO) to ensure dynamic feasibility. In the online motion execution stage, a centroidal dynamics-based Model Predictive Control (MPC) framework tracks the planned motions in real time and proactively adjusts swing foot placement to adapt to real world disturbances. We validate our framework on the full-size humanoid robot Kuavo 4Pro, demonstrating the dynamic dance motions both in simulation and in a four-minute live public performance with a team of four robots. Experimental results show that longer prediction horizons improve both motion expressiveness in planning and stability in execution.

Shibowen Zhang, Jiayang Wu, Guannan Liu +12

Department of Automation, University of Science and Technology of China, Hefei 230027, China. · State Key Laboratory of General Artificial Intelligence, Beijing Institute for General Artificial Intelligence (BIGAI), Beijing 100080, China · Department of Automation, Tsinghua University, Beijing 100084, China. +2

Robot Motion GenerationHumanoid Robot ControlHumanoid Robotics
AskPDF
Apr 3, 2026

A Rapid Instrument Exchange System for Humanoid Robots in Minimally Invasive Surgery

Humanoid robot technologies have demonstrated immense potential for minimally invasive surgery (MIS). Unlike dedicated multi-arm surgical platforms, the inherent dual-arm configuration of humanoid robots necessitates an efficient instrument exchange capability to perform complex procedures, mimicking the natural workflow where surgeons manually switch instruments. To address this, this paper proposes an immersive teleoperated rapid instrument exchange system. The system utilizes a low-latency mechanism based on single-axis compliant docking and environmental constraint release. Integrated with real-time first-person view (FPV) perception via a head-mounted display (HMD), this framework significantly reduces operational complexity and cognitive load during the docking process. Comparative evaluations between experts and novices demonstrate high operational robustness and a rapidly converging learning curve; novice performance in instrument attachment and detachment improved substantially after brief training. While long-distance spatial alignment still presents challenges in time cost and collaborative stability, this study successfully validates the technical feasibility of humanoid robots executing stable instrument exchanges within constrained clinical environments.

Bingcong Zhang, Yihang Lyv, Lianbo Ma +6

Tianjin University, Tianjin, China · Institute of Medical Robotics and Intelligent Systems, Tianjin University, Tianjin, China

Surgical RoboticsHumanoid RoboticsRobot Teleoperation
AskPDF
Mar 26, 2026

Towards Embodied AI with MuscleMimic: Unlocking full-body musculoskeletal motor learning at scale

Learning motor control for muscle-driven musculoskeletal models is hindered by the computational cost of biomechanically accurate simulation and the scarcity of validated, open full-body models. Here we present MuscleMimic, an open-source framework for scalable motion imitation learning with physiologically realistic, muscle-actuated humanoids. MuscleMimic provides two validated musculoskeletal embodiments - a fixed-root upper-body model (126 muscles) for bimanual manipulation and a full-body model (416 muscles) for locomotion - together with a retargeting pipeline that maps SMPL-format motion capture data onto musculoskeletal structures while preserving kinematic and dynamic consistency. Leveraging massively parallel GPU simulation, the framework achieves order-of-magnitude training speedups over prior CPU-based approaches while maintaining comprehensive collision handling, enabling a single generalist policy to be trained on hundreds of diverse motions within days. The resulting policy faithfully reproduces a broad repertoire of human movements under full muscular control and can be fine-tuned to novel motions within hours. Biomechanical validation against experimental walking and running data demonstrates strong agreement in joint kinematics (mean correlation r = 0.90), while muscle activation analysis reveals both the promise and fundamental challenges of achieving physiological fidelity through kinematic imitation alone. By lowering the computational and data barriers to musculoskeletal simulation, MuscleMimic enables systematic model validation across diverse dynamic movements and broader participation in neuromuscular control research. Code, models, checkpoints, and retargeted datasets are available at: https://github.com/amathislab/musclemimic

Chengkun Li, Cheryl Wang, Bianca Ziliotto +4

EPFL, Switzerland · McGill University, Canada · Feil & Oberfeld CRIR-Jewish Rehabilitation Hospital Research Center, Canada +1

Musculoskeletal ModelingHuman-to-Robot Motion ImitationRobotics SimulationRobot Imitation LearningHumanoid Robotics
AskPDF
Mar 26, 2026

SafeGuard ASF: SR Agentic Humanoid Robot System for Autonomous Industrial Safety

The rise of unmanned ``dark factories'' operating without human presence demands autonomous safety systems capable of detecting and responding to multiple hazard types. We present SafeGuard ASF (Agentic Security Fleet), a comprehensive framework deploying humanoid robots for autonomous hazard detection in industrial environments. Our system integrates multi-modal perception (RGB-D imaging), a ReAct-based agentic reasoning framework, and learned locomotion policies on the Unitree G1 humanoid platform. We address three critical hazard scenarios: fire and smoke detection, abnormal temperature monitoring in pipelines, and intruder detection in restricted zones. Our perception pipeline achieves 94.2% mAP for fire or smoke detection with 127ms latency. We train multiple locomotion policies, including dance motion tracking and velocity control, using Unitree RL Lab with PPO, demonstrating stable convergence within 80,000 training iterations. We validate our system in both simulation and real-world environments, demonstrating autonomous patrol, human detection with visual perception, and obstacle avoidance capabilities. The proposed ToolOrchestra action framework enables structured decision-making through perception, reasoning, and actuation tools.

Thanh Nguyen Canh, Thang Tran Viet, Thanh Tuan Tran +1

Strike Robotics

Robot SafetyHumanoid RoboticsRobotic InspectionRobotic Perception
AskPDF
Mar 20, 2026

Morphology-Consistent Humanoid Interaction through Robot-Centric Video Synthesis

Equipping humanoid robots with versatile interaction skills typically requires either extensive policy training or explicit human-to-robot motion retargeting. However, learning-based policies face prohibitive data collection costs. Meanwhile, retargeting relies on human-centric pose estimation (e.g., SMPL), introducing a morphology gap. Skeletal scale mismatches result in severe spatial misalignments when mapped to robots, compromising interaction success. In this work, we propose Dream2Act, a robot-centric framework enabling zero-shot interaction through generative video synthesis. Given a third-person image of the robot and target object, our framework leverages video generation models to envision the robot completing the task with morphology-consistent motion. We employ a high-fidelity pose extraction system to recover physically feasible, robot-native joint trajectories from these synthesized dreams, subsequently executed via a general-purpose whole-body controller. Operating strictly within the robot-native coordinate space, Dream2Act avoids retargeting errors and eliminates task-specific policy training. We evaluate Dream2Act on the Unitree G1 across four whole-body mobile interaction tasks: ball kicking, sofa sitting, bag punching, and box hugging. Dream2Act achieves a 37.5% overall success rate, compared to 0% for conventional retargeting. While retargeting fails to establish correct physical contacts due to the morphology gap (with errors compounded during locomotion), Dream2Act maintains robot-consistent spatial alignment, enabling reliable contact formation and substantially higher task completion.

Weisheng Xu, Jian Li, Yi Gu +12

Hong Kong University of Science and Technology (Guangzhou) · Harbin Institute of Technology, Shenzhen · Shenzhen University +1

Robot Motion GenerationHumanoid RoboticsZero-Shot Robotic ManipulationRobot Learning
AskPDF
Mar 12, 2026

HumDex: Humanoid Dexterous Manipulation Made Easy

This paper investigates humanoid whole-body dexterous manipulation, where the efficient collection of high-quality demonstration data remains a central bottleneck. Existing teleoperation systems often suffer from limited portability, occlusion, or insufficient precision, which hinders their applicability to complex whole-body tasks. To address these challenges, we introduce HumDex, a portable teleoperation system designed for humanoid whole-body dexterous manipulation. Our system leverages IMU-based motion tracking to address the portability-precision trade-off, enabling accurate full-body tracking while remaining easy to deploy. For dexterous hand control, we further introduce a learning-based retargeting method that generates smooth and natural hand motions without manual parameter tuning. Beyond teleoperation, HumDex enables efficient collection of human motion data. Building on this capability, we propose a two-stage imitation learning framework that first pre-trains on diverse human motion data to learn generalizable priors, and then fine-tunes on robot data to bridge the embodiment gap for precise execution. We demonstrate that this approach significantly improves generalization to new configurations, objects, and backgrounds with minimal data acquisition costs. The entire system is fully reproducible and open-sourced at https://github.com/physical-superintelligence-lab/humdex.

Liang Heng, Yihe Tang, Jiajun Xu +3

USC Physical Superintelligence (PSI) Lab · WorldEngine AI

Robotic ManipulationWhole-Body Robotic ManipulationHumanoid RoboticsRobot TeleoperationRobotic Data CollectionRobot Imitation Learning
AskPDF
Mar 10, 2026

ZeroWBC: Learning Natural Whole-Body Humanoid Interaction from Human Egocentric Data

Achieving versatile and natural whole-body humanoid interaction control remains challenging due to the high cost of whole-body teleoperation data. We present ZeroWBC, a teleoperation-free framework that learns humanoid whole-body interaction from human egocentric videos paired with synchronized whole-body motion and text annotations. ZeroWBC adopts a generation-then-tracking formulation to tackle the static scene whole-body interaction control problem. Given an initial egocentric image and a language instruction, a fine-tuned Vision-Language Model generates future human whole-body motion tokens, which are decoded into continuous motions and retargeted to the humanoid. The resulting reference motions, together with root and key body-part trajectories, are then executed by a general interactive motion tracking policy. To improve interaction performance, we introduce an interaction-oriented tracking reward that prioritizes global root and key body-part trajectory alignment while preserving natural whole-body motion. Experiments on the Unitree G1 humanoid robot show that ZeroWBC enables diverse scene-aware behaviors without robot teleoperation demonstrations. These results suggest a scalable paradigm for learning natural humanoid whole-body interaction from human egocentric data.

Haoran Yang, Jiacheng Bao, Yucheng Xin +5

University of Science and Technology of China · Shanghai AI Laboratory · Tsinghua University +2

Human Motion GenerationWhole-Body ControlRobot Imitation LearningHumanoid Motion TrackingHumanoid Robotics
AskPDF
Mar 9, 2026

Embedding Classical Balance Control Principles in Reinforcement Learning for Humanoid Recovery

Humanoid robots remain vulnerable to falls and unrecoverable failure states, limiting their practical utility in unstructured environments. While reinforcement learning has demonstrated stand-up behaviors, existing approaches treat recovery as a pure task-reward problem without an explicit representation of the balance state. We present a unified RL policy that addresses this limitation by embedding classical balance metrics: capture point, center-of-mass state, and centroidal momentum, as privileged critic inputs and shaping rewards directly around these quantities during training, while the actor relies solely on proprioception for zero-shot hardware transfer. Without reference trajectories or scripted contacts, a single policy spans the full recovery spectrum: ankle and hip strategies for small disturbances, corrective stepping under large pushes, and compliant falling with multi-contact stand-up using the hands, elbows, and knees. Trained on the Unitree H1-2 in Isaac Lab, the policy achieves a 93.4% recovery rate across randomized initial poses and unscripted fall configurations. An ablation study shows that removing the balance-informed structure causes stand-up learning to fail entirely, confirming that these metrics provide a meaningful learning signal rather than incidental structure. Sim-to-sim transfer to MuJoCo and preliminary hardware experiments further demonstrate cross-environment generalization. These results show that embedding interpretable balance structure into the learning framework substantially reduces time spent in failure states and broadens the envelope of autonomous recovery.

Nehar Poddar, Stephen McCrory, Luigi Penco +3

Florida Institute for Human and Machine Cognition, Pensacola, FL, USA. · University of West Florida, Pensacola, FL, USA.

Humanoid Robot Fall RecoveryHumanoid Robot ControlHumanoid RoboticsReinforcement Learning
AskPDF
Mar 7, 2026

Morphology-Independent Facial Expression Imitation for Human-Face Robots

Accurate facial expression imitation on human-face robots is crucial for achieving natural human-robot interaction. Most existing methods have achieved photorealistic expression imitation through mapping 2D facial landmarks to a robot's actuator commands. Their imitation of landmark trajectories is susceptible to interference from facial morphology, which would lead to a performance drop. In this paper, we propose a morphology-independent expression imitation method that decouples expressions from facial morphology to eliminate morphological influence and produce more realistic expressions for human-face robots. Specifically, we construct an expression decoupling module to learn expression semantics by disentangling the expression representation from the morphology representation in a self-supervised manner. We devise an expression transfer module to map the representations to the robot's actuator commands through a learning objective of perceiving expression errors, producing accurate facial expressions based on the learned expression semantics. To support experimental validation, a custom-designed and highly expressive human-face robot, namely Pengrui, is developed to serve as an experimental platform for realistic expression imitation. Extensive experiments demonstrate that our method enables the human-face robot to reproduce a wide range of human-like expressions effectively. All code and implementation details of the robot will be released.

Xu Chen, Rui Gao, Che Sun +4

Robot Imitation LearningHumanoid RoboticsHuman-Robot Interaction
AskPDF
Mar 6, 2026

Towards Motion Turing Test: Evaluating Human-Likeness in Humanoid Robots

Humanoid robots have achieved significant progress in motion generation and control, exhibiting movements that appear increasingly natural and human-like. Inspired by the Turing Test, we propose the Motion Turing Test, a framework that evaluates whether human observers can discriminate between humanoid robot and human poses using only kinematic information. To facilitate this evaluation, we present the Human-Humanoid Motion (HHMotion) dataset, which consists of 1,000 motion sequences spanning 15 action categories, performed by 11 humanoid models and 10 human subjects. All motion sequences are converted into SMPL-X representations to eliminate the influence of visual appearance. We recruited 30 annotators to rate the human-likeness of each pose on a 0-5 scale, resulting in over 500 hours of annotation. Analysis of the collected data reveals that humanoid motions still exhibit noticeable deviations from human movements, particularly in dynamic actions such as jumping, boxing, and running. Building on HHMotion, we formulate a human-likeness evaluation task that aims to automatically predict human-likeness scores from motion data. Despite recent progress in multimodal large language models, we find that they remain inadequate for assessing motion human-likeness. To address this, we propose a simple baseline model and demonstrate that it outperforms several contemporary LLM-based methods. The dataset, code, and benchmark will be publicly released to support future research in the community.

Mingzhe Li, Mengyin Liu, Zekai Wu +9

Fujian Key Laboratory of Urban Intelligent Sensing and Computing, Xiamen University · Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, School of Informatics, Xiamen University · National Institute for Data Science in Health and Medicine, Xiamen University +2

Humanoid Robotics
AskPDF
Mar 4, 2026

Cognition to Control - Multi-Agent Learning for Human-Humanoid Collaborative Transport

Full-stack human-robot collaboration (HRC) can become brittle when replacing a planner, partner model, coordination policy, or controller changes the physical meaning of cross-layer signals. We introduce C2C, an object-centric cognition-to-control architecture that preserves these meanings through physical contracts. Rather than standardizing individual modules, C2C standardizes the physical semantics exchanged between them: task intent is represented by a geometrically verified payload path, partner information by a source-independent physical state, learned coordination by bounded 11-D task-space commands, and robot-specific feasibility remains inside whole-body control (WBC). The reference system combines a vision-language model with deterministic geometric verification and multi-agent reinforcement learning (MARL) for partner-aware coordination. Across nine transport scenarios, adaptive MARL variants achieve 77.1-82.1% mean success under unchanged interfaces, compared with 56.5% for a scripted-partner reference. The same contracts further support neural and interpretable hard-tree actors, two- and three-carrier teams, and simulated-to-human partner substitution. Physical Unitree G1-human tests achieve 100% success in spatially confined transport and 80% in super-long-object handling, while a three-carrier system with two G1 humanoids and one human validates the same interface structure on hardware. Together, these results show that C2C converts a tightly coupled HRC stack into a plug-compatible system in which key components can change without redefining the physical collaboration task.

Hao Zhang, Yisen Li, Ruize Geng +4

University of Texas at Arlington. · Carnegie Mellon University

Human-Robot CollaborationMulti-Agent CollaborationHumanoid RoboticsCooperative MARL
AskPDF
Feb 27, 2026

Humanoid Robots as First Assistants in Endoscopic Surgery

Humanoid robots have become a focal point of technological ambition, with claims of surgical capability within years in mainstream discourse. These projections are aspirational yet lack empirical grounding. To date, no humanoid has assisted a surgeon through an actual procedure, let alone performed one. The work described here breaks this new ground. Here we report a proof of concept in which a teleoperated Unitree G1 provided endoscopic visualization while an attending otolaryngologist performed a cadaveric sphenoidectomy. The procedure was completed successfully, with stable visualization maintained throughout. Teleoperation allowed assessment of whether the humanoid form factor could meet the physical demands of surgical assistance in terms of sustenance and precision; the cognitive demands were satisfied -- for now -- by the operator. Post-procedure analysis identified engineering targets for clinical translation, alongside near-term opportunities such as autonomous diagnostic scoping. This work establishes form-factor feasibility for humanoid surgical assistance while identifying challenges for continued development.

Sue Min Cho, Jan Emily Mangulabnan, Han Zhang +7

Johns Hopkins University, Baltimore, MD, USA. · NVIDIA Corporation, Bethesda, MD, USA. · Johns Hopkins Medical Institutions, Baltimore, MD, USA.

Healthcare RoboticsSurgical RoboticsHumanoid Robotics
AskPDF
Feb 25, 2026

Humanizing Robot Gaze Shifts: A Framework for Natural Gaze Shifts in Humanoid Robots

Leveraging auditory and visual feedback for attention reorientation is essential for natural gaze shifts in social interaction. However, enabling humanoid robots to perform natural and context-appropriate gaze shifts in unconstrained human--robot interaction (HRI) remains challenging, as it requires the coupling of cognitive attention mechanisms and biomimetic motion generation. In this work, we propose the Robot Gaze-Shift (RGS) framework, which integrates these two components into a unified pipeline. First, RGS employs a vision--language model (VLM)-based gaze reasoning pipeline to infer context-appropriate gaze targets from multimodal interaction cues, ensuring consistency with human gaze-orienting regularities. Second, RGS introduces a conditional Vector Quantized-Variational Autoencoder (VQ-VAE) model for eye--head coordinated gaze-shift motion generation, producing diverse and human-like gaze-shift behaviors. Experiments validate that RGS effectively replicates human-like target selection and generates realistic, diverse gaze-shift motions.

Jingchao Wei, Jingkai Qin, Yuxiao Cao +4

School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan 430074, China

Human-Robot InteractionHumanoid RoboticsRobot Motion Generation
AskPDF
Feb 18, 2026

Unified Perceptive Fall Safety for Humanoids

Reliable fall recovery is critical for humanoids operating in cluttered environments. Unlike quadrupeds or wheeled robots, humanoids undergo high-energy impacts, complex whole-body contact, and large viewpoint changes during falls, making recovery essential. Prior work treats fall safety as separate problems (avoidance, impact mitigation, stand-up) or learns end-to-end policies without vision, typically on flat terrain. More fundamentally, it couples pose, dynamics, and terrain into a single data space, requiring exhaustive coverage and limiting generalization. We present a unified visually grounded fall-safety approach spanning all phases of recovery. It builds on two insights: (1) human fall-recovery poses are structured and transferable across terrains via alignment, and (2) effective recovery depends on interpreting the recovery goal in relation to the surrounding terrain. We train a privileged teacher using sparse human demonstrations and simulated terrains, and distill it into a deployable student that relies on egocentric depth and proprioception. The student predicts a goal-in-context latent representing the recovery target in relation to its surroundings, guiding coordinated whole-body reactions. Experiments in simulation and on a Unitree G1 demonstrate robust zero-shot fall safety across diverse terrains. Project page: https://vigor2026.github.io/.

Osher Azulay, Zhengjie Xu, Andrew Scheffer +1

University of Michigan, Ann Arbor

Humanoid Robot Fall RecoveryHumanoid Robot ControlRobot SafetyHumanoid Robotics
AskPDF
Feb 14, 2026

KungfuAthleteBot: Learning High-Dynamic Motion from Videos with Unified Robust Recovery

Learning highly dynamic motions from video provides a scalable alternative to expensive data collection for humanoid robots, yet it is challenged by height floating, missing kinematic and force information, and limited robustness. We present KungfuAthleteBot (KAB), a unified framework that systematically addresses these issues. We first build a high-dynamic motion dataset in collaboration with professional martial artists, alleviating data scarcity while improving motion quality. To eliminate height drift and global jitter during both aerial and landing phases, we introduce a physics-guided parabolic trajectory correction that enforces physical plausibility. To bridge the discrepancy between human and robot actuation in highly dynamic behaviors, we design a three-stage curriculum that progressively enables stable acquisition of such motions. In addition, we propose a direct, teacher-free training paradigm that jointly learns disturbance rejection and fall recovery within a single policy. To the best of our knowledge, KAB achieves the fastest recovery time reported for a unified policy (≈0.7\approx 0.7s). Experiments on a real humanoid robot demonstrate successful learning of highly dynamic motions from video and strong robustness, including rapid recovery under severe disturbances.

Zhongxiang Lei, Lulu Cao, Xuyang Wang +3

Beijing Institute of Technology · QIYUAN Lab

Humanoid Robot Fall RecoveryHumanoid RoboticsRobot Learning from VideoHuman-to-Robot Motion ImitationRobust RL
AskPDF
Feb 2, 2026

HumanX: Toward Agile and Generalizable Humanoid Interaction Skills from Human Videos

Enabling humanoid robots to perform agile and adaptive interactions has long been a core challenge in robotics. Current approaches are bottlenecked by either the scarcity of realistic interaction data or the need for meticulous, task-specific reward engineering, which limits their scalability. To narrow this gap, we present HumanX, a unified framework that compiles human video into generalizable, real-world interaction skills for humanoids, without per-task design. HumanX integrates two co-designed components: XGen, an anchor-centric data generation pipeline that yields physically plausible robot interaction data from monocular video and augments it to create a diverse, scalable training dataset; and XMimic, an interaction-prioritized learning framework to achieve generalizable interaction skills from seconds of demonstration. Evaluated across five distinct domains--basketball, football, badminton, cargo pickup, and reactive fighting--HumanX successfully acquires 9 different skills and transfers them zero-shot to a physical Unitree G1 humanoid. The learned capabilities include complex maneuvers such as pump-fake turnaround fadeaway jumpshots without any external perception, as well as interactive tasks like sustained human-robot passing sequences over 10 consecutive cycles--learned from a single video demonstration. Our experiments show that HumanX achieves higher generalization success than prior methods, demonstrating a scalable pathway within a class of anchorable interactions for learning versatile, real-world robot interactive skills.

Yinhuai Wang, Qihan Zhao, Yuen Fui Lau +6

The Hong Kong University of Science and Technology · Shanghai AI Laboratory

Robot Skill LearningHumanoid RoboticsRobot Imitation LearningHuman-Robot InteractionSynthetic Data Generation for Robotics
AskPDF
Oct 9, 2025

Humanoid Everyday: A Comprehensive Robotic Dataset for Open-World Humanoid Manipulation

From loco-motion to dextrous manipulation, humanoid robots have made remarkable strides in demonstrating complex full-body capabilities. However, the majority of current robot learning datasets and benchmarks mainly focus on stationary robot arms, and the few existing humanoid datasets are either confined to fixed environments or limited in task diversity, often lacking human-humanoid interaction and lower-body locomotion. Moreover, there are a few standardized evaluation platforms for benchmarking learning-based policies on humanoid data. In this work, we present Humanoid Everyday, a large-scale and diverse humanoid manipulation dataset characterized by extensive task variety involving dextrous object manipulation, human-humanoid interaction, locomotion-integrated actions, and more. Leveraging a highly efficient human-supervised teleoperation pipeline, Humanoid Everyday aggregates high-quality multimodal sensory data, including RGB, depth, LiDAR, and tactile inputs, together with natural language annotations, comprising 10.3k trajectories and over 3 million frames of data across 260 tasks across 7 broad categories. In addition, we conduct an analysis of representative policy learning methods on our dataset, providing insights into their strengths and limitations across different task categories. For standardized evaluation, we introduce a cloud-based evaluation platform that allows researchers to seamlessly deploy their policies in our controlled setting and receive performance feedback. By releasing Humanoid Everyday along with our policy learning analysis and a standardized cloud-based evaluation platform, we intend to advance research in general-purpose humanoid manipulation and lay the groundwork for more capable and embodied robotic agents in real-world scenarios. Our dataset, data collection code, and cloud evaluation website are made publicly available on our project website.

Zhenyu Zhao, Hongyi Jing, Xiawei Liu +7

University of Southern California · Toyota Research Institute

Robot Manipulation DatasetsRobotic Data CollectionWhole-Body Robotic ManipulationRobot Manipulation BenchmarksHumanoid Robotics
AskPDF
Oct 3, 2025

LapSurgie: Humanoid Robots Performing Surgery via Teleoperated Handheld Laparoscopy

Robotic laparoscopic surgery has gained increasing attention in recent years for its potential to deliver more efficient and precise minimally invasive procedures. However, adoption of surgical robotic platforms remains largely confined to high-resource medical centers, exacerbating healthcare disparities in rural and low-resource regions. To close this gap, a range of solutions has been explored, from remote mentorship to fully remote telesurgery. Yet, the practical deployment of surgical robotic systems to underserved communities remains an unsolved challenge. Humanoid systems offer a promising path toward deployability, as they can directly operate in environments designed for humans without extensive infrastructure modifications -- including operating rooms. In this work, we introduce LapSurgie, the first humanoid-robot-based laparoscopic teleoperation framework. The system leverages an inverse-mapping strategy for manual-wristed laparoscopic instruments that abides to remote center-of-motion constraints, enabling precise hand-to-tool control of off-the-shelf surgical laparoscopic tools without additional setup requirements. A control console equipped with a stereo vision system provides real-time visual feedback. Finally, a comprehensive user study across platforms demonstrates the effectiveness of the proposed framework and provides initial evidence for the feasibility of deploying humanoid robots in laparoscopic procedures.

Zekai Liang, Xiao Liang, Soofiyan Atar +7

Department of Electrical and Computer Engineering, University of California San Diego, La Jolla, CA 92093 USA. · School of Electrical and Computer Engineering, Cornell University, Ithaca, NY 14853 USA. · UC San Diego Health, La Jolla, CA 92037 USA.

Robotic ControlRobot TeleoperationSurgical RoboticsHumanoid Robotics
AskPDF
Oct 3, 2025

Toward Physical Recursive Self-Improvement: A Call for Body-Control Co-Design for Humanoid Robot

Humanoid robots, as general-purpose physical agents, must integrate both intelligent control and adaptive morphology to operate effectively in diverse real-world environments. While recent research primarily optimizes control policies for fixed bodies, we advocate body-control co-design as a path toward physical Recursive Self-Improvement (RSI): using experience from a humanoid's current body and policy to guide successive improvements to both. Inspired by biological evolution, this approach enables robots to recursively adapt both their form and behavior to optimize performance within task-specific and resource-constrained contexts. Despite its promise, co-design in humanoid robotics remains a relatively underexplored domain, raising fundamental questions about its feasibility and necessity in achieving true embodied intelligence. To address these challenges, we discuss potential co-design methodologies grounded in strategic exploration, Sim2Real transfer, and meta-policy learning. We further argue for the essential role of co-design by analyzing it from methodological, application-driven, and community-oriented perspectives. Striving to guide and inspire future studies, we present open research questions, spanning from short-term innovations to long-term goals. Ultimately, co-design offers a path toward physical RSI in humanoids that improves their bodies and policies while learning to make this improvement process more effective.

Guiliang Liu, Bo Yue, Yi Jin Kim +1

The Chinese University of Hong Kong (Shenzhen) · DexForce Technology

Robot Morphology-Control Co-DesignHumanoid Robotics
AskPDF