HULK: Learning Whole-Body Forceful Loco-Manipulation for Humanoids

An Dang; Arturo Flores Alvarez; Yu-Ming Chen; Conor Mc Gartoll; Helen Sun; Aaron Ames; Nima Fazeli; Manikantan Nambi

cs.ROOct 6, 2026

HULK: Learning Whole-Body Forceful Loco-Manipulation for Humanoids

Authors: An Dang, Arturo Flores Alvarez, Yu-Ming Chen, Conor Mc Gartoll, Helen Sun, Aaron Ames, Nima Fazeli, Manikantan Nambi

Organizations: Amazon · University of Michigan · University of California, Los Angeles · California Institute of Technology

Abstract

Humanoid loco-manipulation of large, heavy objects demands forceful interaction across the entire body. However, such payloads shift a humanoid's center of mass and impose sustained loads across the upper body, challenging balance and command tracking. We present HULK, a whole-body control framework for forceful loco-manipulation. Using model predictive control (MPC) to guide reinforcement learning with predictions of the loaded dynamics, we train two teachers: one tracks arm motions under wrist forces, and the other locomotes while holding large objects against the body. A capture-point control barrier function augments the wrist-force teacher during training to improve balance under load. We distill both teachers into a single policy. Evaluation spans simulation and the Unitree G1. In simulation, the teacher with the barrier function achieves the lowest forward and lateral velocity tracking errors at 10 kg per arm among evaluated controllers and reduces aggregate divergent component of motion (DCM) excursion magnitude by 35.7% relative to MPC-guided reinforcement learning alone. Our wrist-force teacher withstands torso push disturbances of up to 130 N.

Figures & tables

Fig. 2: Training overview. We decompose training into 3 stages. In stage 1, we train a teacher $\boldsymbol{\pi}_{\theta}^{\text{wf}}$ that follows high-level commands $\mathbf{c}_{t}$ while undergoing force perturbations at the wrists. High-level commands include upper body joint targets $\mathbf{q}^{\text{upper},c}_{t}$ , commanded linear velocity $\mathbf{v}_{t}^{c}$ , commanded angular velocity in z direction $\boldsymbol{\omega}_{t}^{c}$ , and height command $h^{c}_{t}$ . In stage 2, the teacher $\boldsymbol{\pi}_{\psi}^{\text{wbg}}$ follows these commands while maintaining whole-body grasps across multiple object geometries. In stage 3, the student policy $\boldsymbol{\pi}_{\phi}^{\text{distill}}$ is trained using DAgger with teacher mode conditioning $m_{t}\in\{0,1\}$ . The student takes in sensor observations $\mathbf{o}_{t}$ only present on a humanoid such as high-level commands, joint position $\mathbf{q}_{t}$ , joint velocity $\dot{\mathbf{q}}_{t}$ , angular velocity $\boldsymbol{\omega}_{t}$ , previous action $\mathbf{a}_{t-1}$ , and projected gravity $\hat{\mathbf{g}}_{t}$ .

Fig. 3: Centroidal MPC Force Reasoning. In (a), we show the G1 about to fall over with forward angular momentum. The MPC generates a plan of the CoM trajectory and ground reaction forces to re-stabilize the angular momentum of the robot and keep an upright posture. The green annotations correspond to left foot while the blue ones correspond to right foot. The light blue ghosts illustrate the G1 following the MPC plan. (b) and (c) showcase the G1 stabilizing from an external pushes in a hands-free and minifridge grasp scenario.

Fig. 4: CP-CBF guidance. The ghost illustrates a recovery step into a separate admissible footprint; the dashed boundary marks $h_{\mathrm{cp}}=0$ . The inset shows command correction and reward feedback during training only.

Fig. 5: Real-world tasks. Cart push/pull, payload carry, fridge carry, and canister carry on the Unitree G1. Together they span the upper body: force acts at the wrists in the first two and against the torso in the last two, and every task is performed while walking.

	Tracking at $10$ kg/arm $\downarrow$			Disturbance rejection
Method	$E_{v_{x}}$ (m/s)	$E_{v_{y}}$ (m/s)	$E_{\omega_{z}}$ (rad/s)	Worst DCM margin (m) $\uparrow$	Peak tilt (deg) $\downarrow$	Max push (N) $\uparrow$
Thor [ 16 ]	$0.263\pm 0.002$	$0.265\pm 0.003$	$0.450\pm 0.006$	$-0.275\pm 0.016$	$23.17\pm 0.13$	$80$
AMO [ 28 ]	$0.276\pm 0.002$	$0.350\pm 0.004$	$\mathbf{0.196\pm 0.003}$	$-0.705\pm 0.030$	$9.94\pm 0.74$	$50$
FALCON [ 15 ]	$0.367\pm 0.012$	$0.369\pm 0.013$	$0.392\pm 0.005$	$-0.329\pm 0.034$	$7.91\pm 0.23$	$80$
Ours	$0.242\pm 0.009$	$0.257\pm 0.004$	$0.234\pm 0.008$	$-0.251\pm 0.011$	$5.90\pm 0.18$	$\mathbf{110}$
Ours + CP-CBF	$\mathbf{0.210\pm 0.007}$	$\mathbf{0.142\pm 0.003}$	$\mathbf{0.197\pm 0.009}$	$\mathbf{-0.162\pm 0.012}$	$\mathbf{5.59\pm 0.24}$	$\mathbf{130}$

TABLE I: Carrying and disturbance rejection. Tracking under

10

kg/arm at mid-forearm and, separately, standing torso pushes. Both Ours rows evaluate wrist-force teachers. DCM margin entries average episode-wise minimum signed margins; negative values indicate excursions outside the benchmark support envelope. Tilt is measured at

50

N. Protocols, aggregation, and boldface are defined in Sec. IV-A .

	$E_{\mathrm{upper}}$ (rad) $\downarrow$			$E_{\mathrm{root}}$ (composite) $\downarrow$
Method	$\alpha_{g}\!=\!0$	$0.5$	$1.0$	$\alpha_{g}\!=\!0$	$0.5$	$1.0$
Thor [ 16 ]	$0.337\pm 0.282$	$0.329\pm 0.282$	$0.337\pm 0.275$	$0.762\pm 0.264$	$0.849\pm 0.250$	$0.913\pm 0.185$
AMO [ 28 ]	$0.402\pm 0.207$	$0.416\pm 0.212$	$0.445\pm 0.222$	$0.680\pm 0.016$	$1.228\pm 0.064$	$2.111\pm 0.104$
FALCON [ 15 ]	$\mathbf{0.144}\pm 0.072$	$\mathbf{0.148}\pm 0.074$	$\mathbf{0.159}\pm 0.079$	$\mathbf{0.444}\pm 0.041$	$\mathbf{0.525}\pm 0.050$	$0.868\pm 0.134$
Ours	$0.161\pm 0.052$	$0.160\pm 0.056$	$0.162\pm 0.059$	$0.620\pm 0.062$	$0.543\pm 0.044$	$\mathbf{0.839}\pm 0.030$
Ours + CP-CBF	$0.172\pm 0.055$	$0.172\pm 0.059$	$0.176\pm 0.061$	$0.620\pm 0.012$	$0.752\pm 0.019$	$0.948\pm 0.033$

TABLE II: Force rejection while walking. Arm and composite root tracking errors; both Ours rows evaluate wrist-force teachers. Force scales

\alpha_{g}\in\{0,0.5,1.0\}

correspond to

0

20

, and

40

N per wrist. Metrics and reporting follow Sec. IV-A .

	Cart push/pull		Payload carry		Fridge carry		Canister carry
Method	SR. $\uparrow$	Max $\uparrow$	SR. $\uparrow$	Max $\uparrow$	SR. $\uparrow$	Max $\uparrow$	SR. $\uparrow$	Max $\uparrow$
Thor [ 16 ]	2/6	130	2/4	7.25	0/4	2.0	0/4	0.6
AMO [ 28 ]	0/6	75	4/4	12.25	2/4	6.0	0/4	2.6
FALCON [ 15 ]	3/6	200	0/4	3.62	0/4	2.5	0/4	0.0
HULK	5/6	300	4/4	12.25	4/4	7.0	4/4	6.1

TABLE III: Real-world evaluation on the Unitree G1. SR. reports successful trials/total trials at a common load per task. Max reports the largest load completed in a separate sweep in kg. The HULK row evaluates the distilled student; see Sec. IV-A for the protocol.

Fig. 6: Example whole-body grasp configurations. Authored torso-and-arm postures and object transforms are restored together at reset. These illustrate initialization geometry, not cross-object policy evaluation.

Fig. 7: Recorded teacher training returns. Both panels show the reward training curve during training for both teachers. The plots use a moving average filter on the actual training curve to produce a cleaner plot, but the original curve is shown in a lighter color. For the grasp teacher, we do not include the pre-training steps.

Observation	Values per sample
Base angular velocity	$3$
Projected gravity	$3$
Joint position / velocity	$29$ / $29$
Previous action	$29$
Commanded planar velocity / yaw rate	$3$
Gait phase (sine and cosine)	$2$

TABLE IV: Shared observations for wrist-force and grasp teacher actors. Observation noise is uniform: angular velocity

\pm 0.2

rad/s, gravity components

\pm 0.05

, joint position

\pm 0.01

rad, and joint velocity

\pm 1.5

rad/s. Privileged critic observations are listed in Tab. V .

Critic-only observation	Values
Base linear velocity	$3$
Foot height / air time / contact flag	$2+2+2$
Foot contact-force vectors	$6$
MPC CoM reference	$3$
MPC angular-momentum reference	$3$
MPC tilt reference	$3$

TABLE V: Privileged observations for teacher critics. Each critic receives

140

current-state values without added noise. Actors instead stack five samples of

115

values, spanning

80

ms at

50

Hz, without direct object-pose or contact-force observations.

Hyperparameter	Value
Rollout length per environment	$24$ steps ( $0.48$ s)
Samples per GPU ( $4096$ environments)	$98304$
Learning epochs / minibatches	$5$ / $4$
Samples per minibatch per GPU	$24576$
Optimizer steps/partition/update	$20$
Discount $\gamma$ / GAE $\lambda$	$0.99$ / $0.95$

TABLE VI: Shared PPO settings. Generalized advantage estimation (GAE) uses

\lambda=0.95

; the adaptive learning-rate schedule targets Kullback–Leibler (KL) divergence

0.01

separately for each body partition. Batch counts are per graphics processing unit (GPU), each simulating

4096

environments, with synchronized training across GPUs. The two body partitions share transitions but use separate rewards and updates.

Term	Formulation	Wrist-force	Grasp
MPC guidance
CoM position	$\exp[-\sum_{\ell=1}^{5}\alpha_{\ell}^{c}\\|\mathbf{e}_{c,\ell}\\|_{2}^{2}]$ .	$2$	$2$
CoM velocity	$\exp[-8\sum_{\ell=1}^{3}\alpha_{\ell}^{v}\\|\mathbf{e}_{v,\ell}\\|_{2}^{2}]$ .	$4$	$4$
Angular momentum	$-\sum_{\ell=1}^{5}\alpha_{\ell}^{c}(\\|\mathbf{e}_{\kappa,\ell}\\|_{2}^{2}+0.01\,\mathbf{e}_{\dot{\kappa},\ell}^{\top}\mathbf{Q}_{\dot{\kappa}}\mathbf{e}_{\dot{\kappa},\ell})$ .	$0.01$	$0.01$
Ground reaction forces	$\exp[-\frac{1}{\|\mathcal{V}\|}\sum_{i\in\mathcal{V}}\\|\mathbf{e}_{F,i}\\|_{2}^{2}/(150\,\mathrm{N})^{2}]$ ; $\mathcal{V}$ : valid loaded stance feet; $0$ if empty.	$0.002$	$0.002$
Swing foothold	$\frac{1}{\|\mathcal{S}\|}\sum_{i\in\mathcal{S}}\exp[-\\|\mathbf{e}_{p,i,xy}\\|_{2}^{2}/(0.08\,\mathrm{m})^{2}]$ ; $\mathcal{S}$ : valid swing feet; $1$ if empty or $s\leq 0.1$ .	$2$	$2$

TABLE VII: Teacher tracking and gait rewards.

\dagger

marks upper-body terms; others enter the lower-body reward. A dash denotes an absent term.

Term	Formulation	Wrist-force	Grasp
Posture and standstill
Pelvis upright	$\exp[-5\\|\hat{\mathbf{g}}^{\,\mathrm{pelvis}}_{xy}\\|_{2}^{2}]$ .	$1$	$1$
Torso upright	$\exp[-5\\|\hat{\mathbf{g}}^{\,\mathrm{torso}}_{xy}\\|_{2}^{2}]$ .	$0$	$1$
Nominal posture	$\exp[-\operatorname{mean}_{j}((q_{j}-q_{j}^{0})^{2}/\sigma_{j}^{2})]$ over legs and waist, with speed-dependent joint tolerances.	$1$	$1$
Base-height error	$(1+4\,\mathbf{1}[s\leq 0.2])(z-z^{\mathrm{cmd}})^{2}$ .	$-10$	$-10$
Leg splay	$\sum_{j\in\mathrm{hip\,roll/yaw}}q_{j}^{2}$ .	$-4$	$-4$

TABLE VIII: Teacher posture, regularization, and grasp rewards.

L

: leg/waist joints;

U

: arm joints;

q_{j}^{0}

: default joint position;

q_{j}^{\min},q_{j}^{\max}

: soft limits. Markers follow Tab. VII .

Explore similar work

Jun 20, 2026cs.RO

OpenHLM: An Empirical Recipe for Whole-Body Humanoid Loco-Manipulation

Whole-body humanoid loco-manipulation requires coordinating the robot's entire kinematic chain. However, most existing systems typically decouple the upper and lower bodies into separate controllers, limiting such coordination and yielding behaviors similar to those of a wheeled dual-arm platform. In this paper, we ask what it takes to build a whole-body native vision-language-action (VLA) model that maps language and pixels directly to all of the humanoid's degrees of freedom. We conduct a systematic empirical study organized as a roadmap of one-variable-at-a-time experiments across three phases: whole-body teleoperation, VLA model design, and heterogeneous co-training. Our study yields several intriguing findings: a joint-based whole-body teleoperation interface outperforms alternatives that only partially expose the humanoid's degrees of freedom; a VLA pretrained on static and wheeled dual-arm platforms transfers surprisingly well to a humanoid's full action space; and co-training with HuMI, the humanoid analog of UMI, extends the policy to new objects and instructions without additional whole-body teleoperation on those targets. Following this roadmap yields OpenHLM, an open-source recipe for whole-body humanoid loco-manipulation. In a challenging long-horizon task that spans a wide vertical range of the humanoid, OpenHLM outperforms two state-of-the-art humanoid VLA baselines (GR00T N1.6 and

Ψ_0

) using less than half the total demonstration time. Our code, training data, and model checkpoints are available at [https://openhlm-project.github.io/].

Yingdong Hu, Haodong Zhu, Boyuan Zheng +6

Tsinghua University · Shanghai Qi Zhi Institute · Spirit AI

Humanoid Loco-Manipulation Generalist Vision--Language--Action Vision-Language-Action Framework Humanoid Teleoperation Roadmap+1

Ask PDF

Sep 16, 2026cs.RO

Learning Holistic Whole-Body Loco-Manipulation with a Bipedal Mobile Manipulator

Bipedal loco-manipulation enables robots to interact with objects beyond the nominal workspace of their arms by coordinating locomotion and manipulation. Realizing this capability requires a low-level whole-body controller that translates task-level manipulation goals into coordinated arm and leg motions while maintaining balance. We present a unified whole-body controller trained with reinforcement learning that directly maps 6-DoF end-effector targets to coordinated actions for the bipedal base and robotic arm. Given only an end-effector target, the learned controller autonomously coordinates reaching, postural adaptation, and stepping without explicit base-velocity or footstep commands. A reward-gating strategy regulates the trade-offs among end-effector tracking, locomotion, and balance during training, while a temporal context estimator combines windowed Transformer encoding, recurrent GRU memory, and auxiliary dynamics prediction to extract dynamics-relevant information from observation history. Real-robot experiments demonstrate that the same controller supports reaching, postural adaptation, and stepping under commands from VR teleoperation, a learned diffusion policy, and scripted trajectories, providing a common end-effector interface for diverse manipulation tasks.

Zhongyu Chen, Yuxuan Nai, Qian Chen +8

ZJU-UIUC Institute. · LimX Dynamics.

Whole-Body Control Humanoid Loco-Manipulation

Ask PDF

Jul 13, 2026cs.RO

PAKE: Learning Whole-Body Loco-Manipulation with Partial Kinematic Embeddings

Loco-manipulation has recently shown promising capabilities; however, achieving high-precision control, managing the high-dimensional action space induced by many degrees of freedom (DoFs), and fully exploiting the inherent redundancy of whole-body systems remain challenging. In this paper, we propose a novel whole-body control framework that effectively addresses these challenges by decomposing the complex loco-manipulation problem into partial reference motion generation and low-level imitation control. We introduce a new Kinematic Normalizing Flow (KNF) model, trained on a large-scale kinematic dataset, that generates diverse yet feasible partial reference motions. A high-level controller is then trained to navigate the KNF's latent space to exploit redundant solutions, while a low-level controller ensures physically feasible and accurate motion execution. We validate our approach on the quadrupedal robot equipped with a six-DoF robotic arm. In simulation, experimental results show that our approach significantly outperforms state-of-the-art methods in terms of tracking accuracy and feasible workspace coverage. For hardware deployment, we evaluate the system over 24 episodes across 8 different mobile loco-manipulation tasks. The system achieves end-effector pose-tracking errors of 4.5 cm and 0.14 rad, while maintaining accurate locomotion tracking with linear and angular velocity errors of 0.1 m/s and 0.01 rad/s, respectively, outperforming competitive baselines. Our method represents a practical and powerful solution for accurate and generalized whole-body loco-manipulation in high-DoF robotic systems, with promising potential for diverse downstream robotic tasks.

Zhengmao He, Moonkyu Jung, Hyeongjun Kim +4

The Hong Kong University of Science and Technology (Guangzhou). · Korea Advanced Institute of Science and Technology. · ETH Zurich. +1

Humanoid Loco-Manipulation Legged Robots Action Space Normalizing Flows

Ask PDF