HybridMimic: Hybrid RL-Centroidal Control for Humanoid Motion Mimicking

Ludwig Chee-Ying Tay; I-Chia Chang; Yan Gu

cs.ROMar 6, 2026

HybridMimic: Hybrid RL-Centroidal Control for Humanoid Motion Mimicking

Authors: Ludwig Chee-Ying Tay, I-Chia Chang, Yan Gu

Organizations: Department of Computer Science, Purdue University, West Lafayette, IN 47907, USA. · School of Mechanical Engineering, Purdue University, West Lafayette, IN 47907, USA.

Abstract

Motion mimicking, i.e., encouraging the control policy to mimic human motion, facilitates the learning of complex tasks via reinforcement learning (RL) for humanoid robots. Although standard RL frameworks demonstrate impressive locomotion agility, they often bypass explicit reasoning about robot dynamics during deployment, which is a design choice that can lead to physically infeasible commands when the robot encounters out-of-distribution environments. By integrating model-based principles, hybrid approaches can improve performance; however, existing methods typically rely on predefined contact timing, limiting their versatility. This paper introduces HybridMimic, a framework in which a learned policy dynamically modulates a centroidal-model-based controller by predicting continuous contact states and desired centroidal velocities. This architecture exploits the physical grounding of centroidal dynamics to generate feedforward torques that remain feasible even under domain shift. Using physics-informed rewards, the policy is trained to efficiently utilize the centroidal controller's optimization by outputting precise control targets and reference torques. Through hardware experiments on the Booster T1 humanoid, HybridMimic reduces the average base position tracking error by 13% compared to a state-of-the-art RL baseline, demonstrating the robustness of dynamics-aware deployment.

Figures & tables

Fig. 1: Snapshots of Booster T1 executing the kicking task using the proposed HybridMimic controller. (a) The robot starts standing. (b) The robot walks forward. (c) The robot kicks with its left foot. (d) The robot recovers and stabilizes from the kick and steps backwards. More motion can be found in the supplementary video. https://youtu.be/1d5vkqNtCOY

Fig. 2: Diagram illustrating HybridMimic controller formulation in both training and deployment. The dashed items represent training specific elements that are unused during deployment.

Term	Observation	Dimension
Baseline Observation
${q}_{\text{ref}}$	Reference joint position	$n$
$\dot{{q}}_{\text{ref}}$	Reference joint velocity	$n$
$\dot{{P}}_{B}$	Base frame linear velocity	3
$\omega_{B}$	Base frame angular velocity	3
${q}$	Current joint position	$n$

TABLE I: Actor and critic observations. Actor has access to the baseline observation while the critic has both the baseline and privileged observation. An observation history of four policy timesteps are stacked together.

Type	Reward	Weight	Sensitivity
Tracking	Global anchor position	0.7	0.3
	Global anchor orientation	0.7	0.4
	Body position	1	0.2
	Body orientation	1	0.3
	Body linear velocity	1	0.5
	Body angular velocity	1.25	2.09

TABLE II: Values of reward terms. Reward terms are split into tracking, regularization, and HybridMimic categories. Tracking terms are taken directly from [ 1 ] with adjusted parameters. The HybridMimic terms are described in Secs. III-B1 , III-B2 , III-B3 , and III-B4 . For terms with sensitivity

\sigma

, the reward is defined as

r=e^{\frac{\epsilon}{\sigma^{2}}}

where

\epsilon

represents the error between the desired and measured values.

Type	Randomization Term	Range
Observations	Base linear velocity noise (m/s)	+(-0.5, 0.5)
	Base angular velocity noise (rad/s)	+(-0.2, 0.2)
	Joint position noise (rad)	+(-0.02, 0.02)
	Joint velocity noise (rad/s)	+(-0.5, 0.5)
HybridMimic input	Mass randomization $M$ (kg)	*(0.98, 1.02)
	Inertia matrix ${I}_{c}$	*(0.98, 1.02)

TABLE III: Settings of domain randomization applied to improve policy robustness. Terms with + have a uniformly distributed random noise added to the true value. Terms with * have a uniformly distributed random factor multiplied to the true value. We add random noise to HybridMimic inputs to prevent the policy from learning behaviors that exploit using the centroidal controller for noise-free observation.

Training Parameters
Term	Value	Term	Value
# of environments	8192	# of iterations	15000
Episode length	10s	Learning rate	$10^{-3}$
Desired KL	0.01	Value loss coeff.	1.0
Entropy coeff.	0.005	Clip ratio	0.2
Actor size	[1024,512,256,256]	$\gamma$ (Discount factor)	0.99

TABLE IV: Training and HybridMimic parameters.

Sim-to-Sim Results
Task	Model	Base Final Position Error (m)	Base Mean Position Error (m)	Base Mean Orientation Error (rad)	Base Velocity Error (m/s)	Base Angular Velocity Error (rad/s)
Walking to Kneeling	BeyondMimic	0.4739 $\pm$ 0.031	0.3344 $\pm$ 0.021	0.1577 $\pm$ 0.012	0.1331 $\pm$ 0.004	0.2881 $\pm$ 0.007
	HybridMimic+FCS	0.6824 $\pm$ 0.021	0.4768 $\pm$ 0.014	0.2356 $\pm$ 0.007	0.1291 $\pm$ 0.002	0.3021 $\pm$ 0.004
	HybridMimic+FCS-RTC	0.3641 $\pm$ 0.015	0.2633 $\pm$ 0.013	0.1513 $\pm$ 0.006	0.1202 $\pm$ 0.004	0.2679 $\pm$ 0.007
	HybridMimic	0.3455 $\pm$ 0.011	0.2366 $\pm$ 0.008	0.1576 $\pm$ 0.003	0.1105 $\pm$ 0.001	0.2285 $\pm$ 0.002
Running in Circle	BeyondMimic	0.3435 $\pm$ 0.008	0.2653 $\pm$ 0.007	0.1597 $\pm$ 0.005	0.1985 $\pm$ 0.002	0.3961 $\pm$ 0.002

TABLE V: Sim-to-sim and sim-to-real tracking errors across motion clips and controller variants. The bolded figures represent the lowest tracking error in the category. The underlined figures represent the second best in the category. The text with background color is the proposed method .

Fig. 3: The left two images illustrate the robot displacement after two sidesteps for (a) BeyondMimic and (b) HybridMimic controllers. (c) The projected base position $\hat{p}$ is the base position projected onto the line from the starting to ending position in the reference motion clip. While the real HybridMimic trajectory oscillates around the trained motion, the real BeyondMimic trajectory consistently undershoots it and induces trunk wobbling as shown in the experiment video.

Fig. 4: The vertical ground reaction force of left foot during walking in simulation for (a) HybridMimic and (b) HybridMimic+FCS. The highlighted regions indicate periods of time during which the contact state indicates the left foot is contacting the ground. For HybridMimic, the highlighting occurs when $w_{\text{left foot}}>0$ and for HybridMimic+FCS , the highlighting occurs when the reference motion has a foot height below 0.007m.

Fig. 5: Feedforward, actual, and reference torque during walking for the left and right knee pitch joints. The highlighted region corresponds to contact on the corresponding leg determined by when $w_{i}>0$ .

Explore similar work

Learning to Assist: Physics-Grounded Human-Human Control via Multi-Agent Reinforcement Learning
Mar 11, 2026Yuto Shibata, Kashu Yamazaki, Lalit Jayanti +3Humanoid Robot Control Assistive Robotics
Carnegie Mellon University · Keio AI Research Center · Keio University
Constrained Whole-Body Tracking for Humanoid Robots
May 29, 2026Daniel Morton, Pranit Mohnot, Marco PavoneRobotic Control Robot Trajectory Tracking
Stanford University · NVIDIA Research
ForgetMimic: Motion Unlearning for Reinforcement Learning Humanoid Control
Sep 23, 2026Xukun Luan, Zhongxiang Lei, Chen Gong +3Humanoid Robot Control RL for Legged Locomotion
Beijing Institute of Technology · University of Virginia · Shandong University +1

Mar 11, 2026J/K move · Enter open · S save

Learning to Assist: Physics-Grounded Human-Human Control via Multi-Agent Reinforcement Learning

Yuto Shibata, Kashu Yamazaki, Lalit Jayanti, Yoshimitsu Aoki+2

Carnegie Mellon University · Keio AI Research Center · Keio University

Humanoid Robot Control Assistive Robotics Robot Imitation Learning Multi-Agent Reinforcement Learning

Open paper AskPDF arXiv

Humanoid robotics has strong potential to transform daily service and caregiving applications. Although recent advances in general motion tracking within physics engines (GMT) have enabled virtual characters and humanoid robots to reproduce a broad range of human motions, these behaviors are primarily limited to contact-less social interactions or isolated movements. Assistive scenarios, by contrast, require continuous awareness of a human partner and rapid adaptation to their evolving posture and dynamics. In this paper, we formulate the imitation of closely interacting, force-exchanging human-human motion sequences as a multi-agent reinforcement learning problem. We jointly train partner-aware policies for both the supporter (assistant) agent and the recipient agent in a physics simulator to track assistive motion references. To make this problem tractable, we introduce a partner policies initialization scheme that transfers priors from single-human motion-tracking controllers, greatly improving exploration. We further propose dynamic reference retargeting and contact-promoting reward, which adapt the assistant's reference motion to the recipient's real-time pose and encourage physically meaningful support. We show that AssistMimic is the first method capable of successfully tracking assistive interaction motions on established benchmarks, demonstrating the benefits of a multi-agent RL formulation for physically grounded and socially aware humanoid control.

Abstract

Explore similar work

Learning to Assist: Physics-Grounded Human-Human Control via Multi-Agent Reinforcement Learning

Constrained Whole-Body Tracking for Humanoid Robots

ForgetMimic: Motion Unlearning for Reinforcement Learning Humanoid Control