Embedding Classical Balance Control Principles in Reinforcement Learning for Humanoid Recovery

Nehar Poddar; Stephen McCrory; Luigi Penco; Geoffrey Clark; Hakki Erhan Svil; Robert Griffin

cs.ROMar 9, 2026

Embedding Classical Balance Control Principles in Reinforcement Learning for Humanoid Recovery

Authors: Nehar Poddar, Stephen McCrory, Luigi Penco, Geoffrey Clark, Hakki Erhan Svil, Robert Griffin

Organizations: Florida Institute for Human and Machine Cognition, Pensacola, FL, USA. · University of West Florida, Pensacola, FL, USA.

Abstract

Humanoid robots remain vulnerable to falls and unrecoverable failure states, limiting their practical utility in unstructured environments. While reinforcement learning has demonstrated stand-up behaviors, existing approaches treat recovery as a pure task-reward problem without an explicit representation of the balance state. We present a unified RL policy that addresses this limitation by embedding classical balance metrics: capture point, center-of-mass state, and centroidal momentum, as privileged critic inputs and shaping rewards directly around these quantities during training, while the actor relies solely on proprioception for zero-shot hardware transfer. Without reference trajectories or scripted contacts, a single policy spans the full recovery spectrum: ankle and hip strategies for small disturbances, corrective stepping under large pushes, and compliant falling with multi-contact stand-up using the hands, elbows, and knees. Trained on the Unitree H1-2 in Isaac Lab, the policy achieves a 93.4% recovery rate across randomized initial poses and unscripted fall configurations. An ablation study shows that removing the balance-informed structure causes stand-up learning to fail entirely, confirming that these metrics provide a meaningful learning signal rather than incidental structure. Sim-to-sim transfer to MuJoCo and preliminary hardware experiments further demonstrate cross-environment generalization. These results show that embedding interpretable balance structure into the learning framework substantially reduces time spent in failure states and broadens the envelope of autonomous recovery.

Figures & tables

Fig. 1: Stand-up sequence on the Unitree H1-2 hardware. Frames 1–8 show recovery from a fallen configuration (1) to upright stance (8).

Method	HoF	Fall	MP	MT	Ref	EB	AC
FRASA [ 8 ]	✗	✓	✗	✗	✗	✗	✗
HumanUP [ 7 ]	✓	✗	✓	✓	✓	✗	✓
HoST [ 5 ]	✓	✗	✓	✓	✗	✗	✗
HiFAR [ 10 ]	✓	✓	✓	✓	✗	✗	✗
Ours	✓	✓	✓	✓	✗	✓	✓

TABLE I: Comparison of representative RL-based humanoid recovery approaches.

Fig. 2: End-to-end pipeline for the Unitree H1-2 stand-up controller: diverse initial pose distribution with upright target (left); Isaac Lab training with PPO, asymmetric critic, and push curriculum; MuJoCo sim-to-sim validation; SCS2 integration with balance visualization; and zero-shot hardware deployment using proprioception only (right).

Hyperparameter	Value
Iterations	50,000
PPO clip $\epsilon$	0.2
Discount $\gamma$	0.99
GAE $\lambda$	0.95
Learning rate	$1\times 10^{-3}$
Target KL	0.01

TABLE II: PPO training hyperparameters.

Category	Symbol	Description
State	$\mathbf{s}_{t}$	Robot state at discrete time $t$
State	$\Delta t$	Control/simulation time step
CoM	$\mathbf{p}_{c}\in\mathbb{R}^{3}$	3-D CoM position
	$\mathbf{p}_{c,xy}\in\mathbb{R}^{2}$	Horizontal CoM projection
	$h_{c}$	Vertical CoM height ( $p_{c,z}$ )
	$h^{*}$	Target upright CoM height

TABLE III: Summary of symbols used in the reward formulation.

Group	Component	Symbol	Objective
$R^{\mathrm{I}}$	Height tracking	$r_{\mathrm{height}}$	Reach $h^{*}$
	Rise shaping	$r_{\mathrm{rise}}$	Encourage upward motion
	Fall penalty	$r_{\mathrm{fall}}$	Suppress downward velocity
	Stability band	$r_{\mathrm{stab}}$	Prevent oscillation
$R^{\mathrm{II}}$	CoM support	$r_{\mathrm{CoM}}$	Static stability
	Capture point	$r_{\mathrm{CP}}$	Dynamic capturability

TABLE IV: Physics-guided reward decomposition.

Fig. 3: Capture-point evolution during push recovery. The blue marker denotes $\boldsymbol{\xi}$ and polygons represent active foot support regions. The policy progressively drives $\boldsymbol{\xi}$ back inside the support hull through ankle, hip, and stepping responses.

Fig. 4: MuJoCo sim-to-sim recovery analysis across five initial poses and forces 0–500 N. Success requires the CoM to exceed 0.85 m and remain stable for $\geq$ 1 s. (A) Baseline recovery without push. (B–C) Recovery rate degrades gracefully with force; standing and squatting are most robust. (D) Recovery time increases with perturbation magnitude. (E–F) Directional and height sensitivity show consistent trends across poses.

Performance
Metric	Full	Reduced
Mean reward	$\mathbf{+379.2}$	$-115.3$
Episode length (steps)	$\mathbf{975}$	$219$
Recovery
CoM height maintenance	$\mathbf{13.23}$	$0.17$
Progressive recovery	$\mathbf{3.78}$	$0.41$

TABLE V: Ablation at convergence: full policy vs. reduced variant (no privileged critic, no capture-point rewards, no curriculum).

Explore similar work

RecoverFormer: End-to-End Contact-Aware Recovery for Humanoid Robots
Apr 24, 2026Zihui LiuHumanoid Robot Locomotion Humanoid Robot Control
Stanford University
Unified Walking, Running, and Recovery for Humanoids via State-Dependent Adversarial Motion Priors
May 18, 2026Yidan Lu, Yichao Zhong, Liu Zhao +2Humanoid Robot Locomotion Robot Failure Recovery
The University of Hong Kong, Hong Kong SAR, China.
Stubborn: A Streamlined and Unified Reinforcement Learning Framework for Robust Motion Tracking and Fall Recovery for Humanoids
Jun 11, 2026Xiao Ren, Yuhui Yang, Zongbiao Weng +2RL for Robotics Reinforcement Learning
Southern University of Science and Technology, Shenzhen 518055, China

Apr 24, 2026J/K move · Enter open · S save

RecoverFormer: End-to-End Contact-Aware Recovery for Humanoid Robots

Zihui Liu

Stanford University

Humanoid Robot Locomotion Humanoid Robot Control Robot Failure Recovery Robot Policy Generalization Robust RL Humanoid Robotics

Open paper AskPDF arXiv

Humanoid robots operating in unstructured environments must recover from unexpected disturbances-a capability that remains challenging for end-to-end control policies. We present RECOVERFORMER, a fully end-to-end humanoid recovery policy that learns when and how to switch among recovery behaviors-including compensatory stepping, hand-environment contact, and center-of-mass reshaping-while maintaining robust performance under model mismatch. The architecture combines a causal transformer over a 50-step observation history with two novel heads: a latent recovery mode that enables smooth transitions among distinct recovery strategies, and a contact affordance head that predicts which environmental surfaces (walls, railings, table edges) are beneficial for stabilization. We evaluate RECOVERFORMER on the Unitree G1 humanoid in MuJoCo. Trained only on open floor, RECOVERFORMER transfers zero shot to walled environments, achieving 100% recovery success across 100-300 N pushes and across wall distances from 0.25-1.4m. Under zero-shot dynamics mismatch, RECOVERFORMER reaches 75.5% at plus +25% mass, 89% under 30 ms latency, 91.5% at low friction, and 99% under compound friction, latency and mass perturbation. The learned latent modes specialize across force regimes without mode-level supervision, validated by t-SNE analysis of 300 episodes. Taken together, these results show that a single end-to-end policy can deliver multi-modal, contact aware humanoid recovery that generalizes across perturbation magnitude, contact geometry, and dynamics shift.

Abstract

Explore similar work

RecoverFormer: End-to-End Contact-Aware Recovery for Humanoid Robots

Unified Walking, Running, and Recovery for Humanoids via State-Dependent Adversarial Motion Priors

Stubborn: A Streamlined and Unified Reinforcement Learning Framework for Robust Motion Tracking and Fall Recovery for Humanoids