HALO:Closing Sim-to-Real Gap for Heavy-loaded Humanoid Agile Motion Skills via Differentiable Simulation

Xingyi Wang; Chenyun Zhang; Weiji Xie; Chao Yu; Wei Song; Chenjia Bai; Shiqiang Zhu

cs.ROMar 16, 2026

HALO:Closing Sim-to-Real Gap for Heavy-loaded Humanoid Agile Motion Skills via Differentiable Simulation

Authors: Xingyi Wang, Chenyun Zhang, Weiji Xie, Chao Yu, Wei Song, Chenjia Bai, Shiqiang Zhu

Organizations: Zhejiang University · Institute of Artificial Intelligence (TeleAI), China Telecom · Shanghai Jiao Tong University · Lumos Robotics

Abstract

Humanoid robots deployed in real-world scenarios often need to carry unknown payloads, which introduce significant mismatch and degrade the effectiveness of simulation-to-reality reinforcement learning methods. To address this challenge, we propose a two-stage gradient-based system identification framework built on the differentiable simulator MuJoCo XLA. The first stage calibrates the nominal robot model using real-world data to reduce intrinsic sim-to-real discrepancies, while the second stage further identifies the mass distribution of the unknown payload. By explicitly reducing structured model bias prior to policy training, our approach enables zero-shot transfer of reinforcement learning policies to hardware under heavy-load conditions. Extensive simulation and real-world experiments demonstrate more precise parameter identification, improved motion tracking accuracy, and substantially enhanced agility and robustness compared to existing baselines. Project Page: https://mwondering.github.io/halo-humanoid/

Figures & tables

Fig. 1: Performance of HALO in heavy-loaded scenarios. (a) The payload settings used by HALO. (b, c) Compared to DR, HALO significantly improves the accuracy of straight-line bidirectional walking. (d) Data collection under payload conditions with a single-foot constraint. (e) HALO enables challenging humanoid locomotion skills.

Fig. 2: Overview of HALO. (a) Data Collection: Trajectories are collected under both loaded and unloaded conditions using exploration policy trained with wide DR, followed by real-world deployment with a fixed foot constraint. (b) Data Processing: Full-body trajectories reconstruction from joint-state measurements via forward kinematics and foot-height alignment. (c) Two-stage Payload-related Parameter Identification: Stage 1 optimize the full set of model parameters to yield a calibrated base model using trajectories without payload. Based on the calibrated model, stage 2 optimize only the payload-related parameters, using trajectories collected under loaded conditions. (d) Heavy-loaded Motion Skills: The accurate identified model parameters enabling zero-shot sim-to-real transfer of the learned skills to the physical heavy-loaded humanoid.

Fig. 3: Inconsistent left-right foot heights caused by sensor noise. (a) In the original trajectory, the right foot position calculated via forward kinematics remains suspended in the air. (b) In the optimized trajectory, the right foot maintains physically consistent ground contact.

Term	Nominal Range	Wide Range
Base CoM offset (m)	$\mathcal{U}(-0.03,0.03)$	$\mathcal{U}(-0.08,0.08)$
Torso mass offset (kg)	$\mathcal{U}(-0.5,0.5)$	$\mathcal{U}(2.0,8.0)$
Hands mass offset (kg)	$\mathcal{U}(-0.5,0.5)$	$\mathcal{U}(1.0,4.0)$
Hands CoM offset (m)	$\mathcal{U}(-0.02,0.02)$	$\mathcal{U}(-0.05,0.05)$
Encoder bias (rad)	$\mathcal{U}(-0.01,0.01)$	$\mathcal{U}(-0.01,0.01)$
Foot friction	$\mathcal{U}(0.3,1.2)$	$\mathcal{U}(0.3,1.2)$

TABLE I: Domain Randomization Settings.

Method	Torso Mass(kg)	Left Hand Mass(kg)	Right Hand Mass (kg)	Torso CoM pos x (m)	Torso CoM pos y (m)	Torso CoM pos z (m)
Absolute Nominal Values
	$7.8180$	$0.2546$	$0.2546$	$0.0020$	$0.0003$	$0.1845$
Perturbation Setting 1
Groundtruth	$+6.0000$	$+2.4000$	$+2.0000$	$+0.0100$	$+0.0200$	$+0.0500$
CMA-ES	$+6.0366\pm{\scriptstyle 0.0128}$	$+2.4009\pm{\scriptstyle 0.0012}$	$+1.9970\pm{\scriptstyle 0.0012}$	$\mathbf{+0.0099\pm{\scriptstyle 0.0000}}$	$+0.0199\pm{\scriptstyle 0.0001}$	$+0.0505\pm{\scriptstyle 0.0004}$
HALO	$\mathbf{+5.9989\pm{\scriptstyle 0.0000}}$	$\mathbf{+2.4005\pm{\scriptstyle 0.0000}}$	$\mathbf{+1.9973\pm{\scriptstyle 0.0000}}$	$+0.0098\pm{\scriptstyle 0.0000}$	$\mathbf{+0.0199\pm{\scriptstyle 0.0000}}$	$\mathbf{+0.0505\pm{\scriptstyle 0.0000}}$

TABLE II: Convergence comparison between CMA-ES and HALO. Results are reported as mean

\pm

standard deviation. Groundtruth values (e.g.,

+6.000

+2.4000

) denote the incremental offsets added to the absolute nominal values. Bold values indicate performance closer to the ground truth. While CMA-ES converges under light payloads, it exhibits sensitivity to random seeds under heavy payloads.

Method	$E_{\text{g-mpjpe}}$	$E_{\text{mpjpe}}$	$E_{\text{vel}}$
Steady-state tasks
WDR	$94.91\pm{\scriptstyle 3.21}$	$70.54\pm{\scriptstyle 2.13}$	$0.157\pm{\scriptstyle 0.008}$
CM	$85.87\pm{\scriptstyle 2.54}$	$56.33\pm{\scriptstyle 2.27}$	$0.130\pm{\scriptstyle 0.006}$
HALO	$\mathbf{52.47}\pm{\scriptstyle 2.11}$	$\mathbf{44.87}\pm{\scriptstyle 1.82}$	$\mathbf{0.098}\pm{\scriptstyle 0.008}$
High-agility tasks
WDR	$132.89\pm{\scriptstyle 2.89}$	$81.54\pm{\scriptstyle 2.43}$	$0.136\pm{\scriptstyle 0.016}$

TABLE III: Experimental results of motion tracking performance across various tasks.

Fig. 4: Comparison of convergence performance between two-stage and one-stage methods. The proposed Two-Stage method HALO (orange) demonstrates superior accuracy, converging significantly closer to the estimated reference values (red dashed lines) compared to the One-Stage baseline (blue).

Method	Torso Mass (kg)	L-Hand Mass (kg)	R-Hand Mass (kg)
Reference	13.82	2.25	2.25
One-Stage	$12.93\pm 0.32$	$3.22\pm 0.23$	$3.16\pm 0.44$
HALO	$\mathbf{13.83}\pm 0.43$	$\mathbf{2.13}\pm 0.07$	$\mathbf{2.71}\pm 0.29$

TABLE IV: Summary of identified parameters.

Fig. 5: Experiment settings and results of scenario 1. The metrics we defined are shown in the image, HALO outperforms the baselines across all metrics.

Method	$E_{fpos}$	$E_{epos}$	$E_{ang}$
WDR	$0.45\pm{\scriptstyle 0.13}$	$0.89\pm{\scriptstyle 0.15}$	$N/A$
CM	$0.22\pm{\scriptstyle 0.09}$	$0.45\pm{\scriptstyle 0.13}$	$41.8\pm{\scriptstyle 3.6}$
HALO	$\mathbf{0.12}\pm{\scriptstyle 0.03}$	$\mathbf{0.26}\pm{\scriptstyle 0.07}$	$\mathbf{11.3}\pm{\scriptstyle 2.1}$

TABLE V: Performance of Walking Forward and Backward.

Method	Swallow Balancing	Side Kicking	Roundhouse Kicking
WDR	$0/10$	$0/10$	$0/10$
CM	$5/10$	$7/10$	$5/10$
HALO	$\mathbf{10/10}$	$\mathbf{10/10}$	$\mathbf{10/10}$

TABLE VI: Comparison of success rates for performing challenging maneuvers between HALO and baselines.

Sim-to-Real Transfer

Simulation-to-reality transfer adapts models and policies trained in simulation for reliable performance in physical or otherwise real-world settings.

See all 316 papers

D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping
Mar 1, 2026Haozhe Lou, Mingtong Zhang, Haoran Geng +9Robotic Grasping Robotics Simulation
Physical Superintelligence (PSI) Lab, University of Southern California · Viterbi School of Engineering, University of Southern California · Department of EECS, University of California, Berkeley
AGILE: A Comprehensive Workflow for Humanoid Loco-Manipulation Learning
Mar 20, 2026Huihua Zhao, Rafael Cathomen, Lionel Gulich +6RL for Robotics Humanoid Robot Control
An Real-Sim-Real (RSR) Loop Framework for Generalizable Robotic Policy Transfer
Mar 13, 2025Yuxuan Xu, Shiyu Wang, Jinhao Huang +6Robotic Data Collection Sim-to-Real Transfer
Institute for AI Industry Research (AIR), Tsinghua University · Beijing Jiaotong University · The Hong Kong University of Science and Technology (Guangzhou)

Mar 1, 2026J/K move · Enter open · S save

D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping

Haozhe Lou, Mingtong Zhang, Haoran Geng, Hanyang Zhou+8

Physical Superintelligence (PSI) Lab, University of Southern California · Viterbi School of Engineering, University of Southern California · Department of EECS, University of California, Berkeley

Robotic Grasping Robotics Simulation Sim-to-Real Transfer Robot Dynamics Identification Force-Controlled Robotic Manipulation

Open paper AskPDF arXiv

Simulation provides a cost-effective and flexible platform for data generation and policy learning to develop robotic systems. However, bridging the gap between simulation and real-world dynamics remains a significant challenge, especially in physical parameter identification. In this work, we introduce a real-to-sim-to-real engine that leverages the Gaussian Splat representations to build a differentiable engine, enabling object mass identification from real-world visual observations and robot control signals, while enabling grasping policy learning simultaneously. Through optimizing the mass of the manipulated object, our method automatically builds high-fidelity and physically plausible digital twins. Additionally, we propose a novel approach to train force-aware grasping policies from limited data by transferring feasible human demonstrations into simulated robot demonstrations. Through comprehensive experiments, we demonstrate that our engine achieves accurate and robust performance in mass identification across various object geometries and mass values. Those optimized mass values facilitate force-aware policy learning, achieving superior and high performance in object grasping, effectively reducing the sim-to-real gap.

Abstract

Sim-to-Real Transfer

Explore similar work

D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping

AGILE: A Comprehensive Workflow for Humanoid Loco-Manipulation Learning

An Real-Sim-Real (RSR) Loop Framework for Generalizable Robotic Policy Transfer