HULK: Learning Whole-Body Forceful Loco-Manipulation for Humanoids

An Dang; Arturo Flores Alvarez; Yu-Ming Chen; Conor Mc Gartoll; Helen Sun; Aaron Ames; Nima Fazeli; Manikantan Nambi

cs.ROOct 6, 2026

HULK: Learning Whole-Body Forceful Loco-Manipulation for Humanoids

Authors: An Dang, Arturo Flores Alvarez, Yu-Ming Chen, Conor Mc Gartoll, Helen Sun, Aaron Ames, Nima Fazeli, Manikantan Nambi

Organizations: Amazon · University of Michigan · University of California, Los Angeles · California Institute of Technology

Abstract

Humanoid loco-manipulation of large, heavy objects demands forceful interaction across the entire body. However, such payloads shift a humanoid's center of mass and impose sustained loads across the upper body, challenging balance and command tracking. We present HULK, a whole-body control framework for forceful loco-manipulation. Using model predictive control (MPC) to guide reinforcement learning with predictions of the loaded dynamics, we train two teachers: one tracks arm motions under wrist forces, and the other locomotes while holding large objects against the body. A capture-point control barrier function augments the wrist-force teacher during training to improve balance under load. We distill both teachers into a single policy. Evaluation spans simulation and the Unitree G1. In simulation, the teacher with the barrier function achieves the lowest forward and lateral velocity tracking errors at 10 kg per arm among evaluated controllers and reduces aggregate divergent component of motion (DCM) excursion magnitude by 35.7% relative to MPC-guided reinforcement learning alone. Our wrist-force teacher withstands torso push disturbances of up to 130 N.

Figures & tables

Fig. 2: Training overview. We decompose training into 3 stages. In stage 1, we train a teacher $\boldsymbol{\pi}_{\theta}^{\text{wf}}$ that follows high-level commands $\mathbf{c}_{t}$ while undergoing force perturbations at the wrists. High-level commands include upper body joint targets $\mathbf{q}^{\text{upper},c}_{t}$ , commanded linear velocity $\mathbf{v}_{t}^{c}$ , commanded angular velocity in z direction $\boldsymbol{\omega}_{t}^{c}$ , and height command $h^{c}_{t}$ . In stage 2, the teacher $\boldsymbol{\pi}_{\psi}^{\text{wbg}}$ follows these commands while maintaining whole-body grasps across multiple object geometries. In stage 3, the student policy $\boldsymbol{\pi}_{\phi}^{\text{distill}}$ is trained using DAgger with teacher mode conditioning $m_{t}\in\{0,1\}$ . The student takes in sensor observations $\mathbf{o}_{t}$ only present on a humanoid such as high-level commands, joint position $\mathbf{q}_{t}$ , joint velocity $\dot{\mathbf{q}}_{t}$ , angular velocity $\boldsymbol{\omega}_{t}$ , previous action $\mathbf{a}_{t-1}$ , and projected gravity $\hat{\mathbf{g}}_{t}$ .

Fig. 3: Centroidal MPC Force Reasoning. In (a), we show the G1 about to fall over with forward angular momentum. The MPC generates a plan of the CoM trajectory and ground reaction forces to re-stabilize the angular momentum of the robot and keep an upright posture. The green annotations correspond to left foot while the blue ones correspond to right foot. The light blue ghosts illustrate the G1 following the MPC plan. (b) and (c) showcase the G1 stabilizing from an external pushes in a hands-free and minifridge grasp scenario.

Fig. 4: CP-CBF guidance. The ghost illustrates a recovery step into a separate admissible footprint; the dashed boundary marks $h_{\mathrm{cp}}=0$ . The inset shows command correction and reward feedback during training only.

Fig. 5: Real-world tasks. Cart push/pull, payload carry, fridge carry, and canister carry on the Unitree G1. Together they span the upper body: force acts at the wrists in the first two and against the torso in the last two, and every task is performed while walking.

	Tracking at $10$ kg/arm $\downarrow$			Disturbance rejection
Method	$E_{v_{x}}$ (m/s)	$E_{v_{y}}$ (m/s)	$E_{\omega_{z}}$ (rad/s)	Worst DCM margin (m) $\uparrow$	Peak tilt (deg) $\downarrow$	Max push (N) $\uparrow$
Thor [ 16 ]	$0.263\pm 0.002$	$0.265\pm 0.003$	$0.450\pm 0.006$	$-0.275\pm 0.016$	$23.17\pm 0.13$	$80$
AMO [ 28 ]	$0.276\pm 0.002$	$0.350\pm 0.004$	$\mathbf{0.196\pm 0.003}$	$-0.705\pm 0.030$	$9.94\pm 0.74$	$50$
FALCON [ 15 ]	$0.367\pm 0.012$	$0.369\pm 0.013$	$0.392\pm 0.005$	$-0.329\pm 0.034$	$7.91\pm 0.23$	$80$
Ours	$0.242\pm 0.009$	$0.257\pm 0.004$	$0.234\pm 0.008$	$-0.251\pm 0.011$	$5.90\pm 0.18$	$\mathbf{110}$
Ours + CP-CBF	$\mathbf{0.210\pm 0.007}$	$\mathbf{0.142\pm 0.003}$	$\mathbf{0.197\pm 0.009}$	$\mathbf{-0.162\pm 0.012}$	$\mathbf{5.59\pm 0.24}$	$\mathbf{130}$

TABLE I: Carrying and disturbance rejection. Tracking under

10

kg/arm at mid-forearm and, separately, standing torso pushes. Both Ours rows evaluate wrist-force teachers. DCM margin entries average episode-wise minimum signed margins; negative values indicate excursions outside the benchmark support envelope. Tilt is measured at

50

N. Protocols, aggregation, and boldface are defined in Sec. IV-A .

	$E_{\mathrm{upper}}$ (rad) $\downarrow$			$E_{\mathrm{root}}$ (composite) $\downarrow$
Method	$\alpha_{g}\!=\!0$	$0.5$	$1.0$	$\alpha_{g}\!=\!0$	$0.5$	$1.0$
Thor [ 16 ]	$0.337\pm 0.282$	$0.329\pm 0.282$	$0.337\pm 0.275$	$0.762\pm 0.264$	$0.849\pm 0.250$	$0.913\pm 0.185$
AMO [ 28 ]	$0.402\pm 0.207$	$0.416\pm 0.212$	$0.445\pm 0.222$	$0.680\pm 0.016$	$1.228\pm 0.064$	$2.111\pm 0.104$
FALCON [ 15 ]	$\mathbf{0.144}\pm 0.072$	$\mathbf{0.148}\pm 0.074$	$\mathbf{0.159}\pm 0.079$	$\mathbf{0.444}\pm 0.041$	$\mathbf{0.525}\pm 0.050$	$0.868\pm 0.134$
Ours	$0.161\pm 0.052$	$0.160\pm 0.056$	$0.162\pm 0.059$	$0.620\pm 0.062$	$0.543\pm 0.044$	$\mathbf{0.839}\pm 0.030$
Ours + CP-CBF	$0.172\pm 0.055$	$0.172\pm 0.059$	$0.176\pm 0.061$	$0.620\pm 0.012$	$0.752\pm 0.019$	$0.948\pm 0.033$

TABLE II: Force rejection while walking. Arm and composite root tracking errors; both Ours rows evaluate wrist-force teachers. Force scales

\alpha_{g}\in\{0,0.5,1.0\}

correspond to

0

20

, and

40

N per wrist. Metrics and reporting follow Sec. IV-A .

	Cart push/pull		Payload carry		Fridge carry		Canister carry
Method	SR. $\uparrow$	Max $\uparrow$	SR. $\uparrow$	Max $\uparrow$	SR. $\uparrow$	Max $\uparrow$	SR. $\uparrow$	Max $\uparrow$
Thor [ 16 ]	2/6	130	2/4	7.25	0/4	2.0	0/4	0.6
AMO [ 28 ]	0/6	75	4/4	12.25	2/4	6.0	0/4	2.6
FALCON [ 15 ]	3/6	200	0/4	3.62	0/4	2.5	0/4	0.0
HULK	5/6	300	4/4	12.25	4/4	7.0	4/4	6.1

TABLE III: Real-world evaluation on the Unitree G1. SR. reports successful trials/total trials at a common load per task. Max reports the largest load completed in a separate sweep in kg. The HULK row evaluates the distilled student; see Sec. IV-A for the protocol.

Fig. 6: Example whole-body grasp configurations. Authored torso-and-arm postures and object transforms are restored together at reset. These illustrate initialization geometry, not cross-object policy evaluation.

Fig. 7: Recorded teacher training returns. Both panels show the reward training curve during training for both teachers. The plots use a moving average filter on the actual training curve to produce a cleaner plot, but the original curve is shown in a lighter color. For the grasp teacher, we do not include the pre-training steps.

Observation	Values per sample
Base angular velocity	$3$
Projected gravity	$3$
Joint position / velocity	$29$ / $29$
Previous action	$29$
Commanded planar velocity / yaw rate	$3$
Gait phase (sine and cosine)	$2$

TABLE IV: Shared observations for wrist-force and grasp teacher actors. Observation noise is uniform: angular velocity

\pm 0.2

rad/s, gravity components

\pm 0.05

, joint position

\pm 0.01

rad, and joint velocity

\pm 1.5

rad/s. Privileged critic observations are listed in Tab. V .

Critic-only observation	Values
Base linear velocity	$3$
Foot height / air time / contact flag	$2+2+2$
Foot contact-force vectors	$6$
MPC CoM reference	$3$
MPC angular-momentum reference	$3$
MPC tilt reference	$3$

TABLE V: Privileged observations for teacher critics. Each critic receives

140

current-state values without added noise. Actors instead stack five samples of

115

values, spanning

80

ms at

50

Hz, without direct object-pose or contact-force observations.

Hyperparameter	Value
Rollout length per environment	$24$ steps ( $0.48$ s)
Samples per GPU ( $4096$ environments)	$98304$
Learning epochs / minibatches	$5$ / $4$
Samples per minibatch per GPU	$24576$
Optimizer steps/partition/update	$20$
Discount $\gamma$ / GAE $\lambda$	$0.99$ / $0.95$

TABLE VI: Shared PPO settings. Generalized advantage estimation (GAE) uses

\lambda=0.95

; the adaptive learning-rate schedule targets Kullback–Leibler (KL) divergence

0.01

separately for each body partition. Batch counts are per graphics processing unit (GPU), each simulating

4096

environments, with synchronized training across GPUs. The two body partitions share transitions but use separate rewards and updates.

Term	Formulation	Wrist-force	Grasp
MPC guidance
CoM position	$\exp[-\sum_{\ell=1}^{5}\alpha_{\ell}^{c}\\|\mathbf{e}_{c,\ell}\\|_{2}^{2}]$ .	$2$	$2$
CoM velocity	$\exp[-8\sum_{\ell=1}^{3}\alpha_{\ell}^{v}\\|\mathbf{e}_{v,\ell}\\|_{2}^{2}]$ .	$4$	$4$
Angular momentum	$-\sum_{\ell=1}^{5}\alpha_{\ell}^{c}(\\|\mathbf{e}_{\kappa,\ell}\\|_{2}^{2}+0.01\,\mathbf{e}_{\dot{\kappa},\ell}^{\top}\mathbf{Q}_{\dot{\kappa}}\mathbf{e}_{\dot{\kappa},\ell})$ .	$0.01$	$0.01$
Ground reaction forces	$\exp[-\frac{1}{\|\mathcal{V}\|}\sum_{i\in\mathcal{V}}\\|\mathbf{e}_{F,i}\\|_{2}^{2}/(150\,\mathrm{N})^{2}]$ ; $\mathcal{V}$ : valid loaded stance feet; $0$ if empty.	$0.002$	$0.002$
Swing foothold	$\frac{1}{\|\mathcal{S}\|}\sum_{i\in\mathcal{S}}\exp[-\\|\mathbf{e}_{p,i,xy}\\|_{2}^{2}/(0.08\,\mathrm{m})^{2}]$ ; $\mathcal{S}$ : valid swing feet; $1$ if empty or $s\leq 0.1$ .	$2$	$2$

TABLE VII: Teacher tracking and gait rewards.

\dagger

marks upper-body terms; others enter the lower-body reward. A dash denotes an absent term.

Term	Formulation	Wrist-force	Grasp
Posture and standstill
Pelvis upright	$\exp[-5\\|\hat{\mathbf{g}}^{\,\mathrm{pelvis}}_{xy}\\|_{2}^{2}]$ .	$1$	$1$
Torso upright	$\exp[-5\\|\hat{\mathbf{g}}^{\,\mathrm{torso}}_{xy}\\|_{2}^{2}]$ .	$0$	$1$
Nominal posture	$\exp[-\operatorname{mean}_{j}((q_{j}-q_{j}^{0})^{2}/\sigma_{j}^{2})]$ over legs and waist, with speed-dependent joint tolerances.	$1$	$1$
Base-height error	$(1+4\,\mathbf{1}[s\leq 0.2])(z-z^{\mathrm{cmd}})^{2}$ .	$-10$	$-10$
Leg splay	$\sum_{j\in\mathrm{hip\,roll/yaw}}q_{j}^{2}$ .	$-4$	$-4$

TABLE VIII: Teacher posture, regularization, and grasp rewards.

L

: leg/waist joints;

U

: arm joints;

q_{j}^{0}

: default joint position;

q_{j}^{\min},q_{j}^{\max}

: soft limits. Markers follow Tab. VII .

Explore similar work

OpenHLM: An Empirical Recipe for Whole-Body Humanoid Loco-Manipulation
Jun 20, 2026Yingdong Hu, Haodong Zhu, Boyuan Zheng +6Humanoid Loco-Manipulation Generalist Vision--Language--Action
Tsinghua University · Shanghai Qi Zhi Institute · Spirit AI
Learning Holistic Whole-Body Loco-Manipulation with a Bipedal Mobile Manipulator
Sep 16, 2026Zhongyu Chen, Yuxuan Nai, Qian Chen +8Whole-Body Control Humanoid Loco-Manipulation
ZJU-UIUC Institute. · LimX Dynamics.
PAKE: Learning Whole-Body Loco-Manipulation with Partial Kinematic Embeddings
Jul 13, 2026Zhengmao He, Moonkyu Jung, Hyeongjun Kim +4Humanoid Loco-Manipulation Legged Robots
The Hong Kong University of Science and Technology (Guangzhou). · Korea Advanced Institute of Science and Technology. · ETH Zurich. +1

Jun 20, 2026, cs.ROJ/K move · Enter open · S save

OpenHLM: An Empirical Recipe for Whole-Body Humanoid Loco-Manipulation

Yingdong Hu, Haodong Zhu, Boyuan Zheng, Yihang Hu+5

Tsinghua University · Shanghai Qi Zhi Institute · Spirit AI

Humanoid Loco-Manipulation Generalist Vision--Language--Action Vision-Language-Action Framework Humanoid Teleoperation Roadmap+1

Open paper AskPDF arXiv

Whole-body humanoid loco-manipulation requires coordinating the robot's entire kinematic chain. However, most existing systems typically decouple the upper and lower bodies into separate controllers, limiting such coordination and yielding behaviors similar to those of a wheeled dual-arm platform. In this paper, we ask what it takes to build a whole-body native vision-language-action (VLA) model that maps language and pixels directly to all of the humanoid's degrees of freedom. We conduct a systematic empirical study organized as a roadmap of one-variable-at-a-time experiments across three phases: whole-body teleoperation, VLA model design, and heterogeneous co-training. Our study yields several intriguing findings: a joint-based whole-body teleoperation interface outperforms alternatives that only partially expose the humanoid's degrees of freedom; a VLA pretrained on static and wheeled dual-arm platforms transfers surprisingly well to a humanoid's full action space; and co-training with HuMI, the humanoid analog of UMI, extends the policy to new objects and instructions without additional whole-body teleoperation on those targets. Following this roadmap yields OpenHLM, an open-source recipe for whole-body humanoid loco-manipulation. In a challenging long-horizon task that spans a wide vertical range of the humanoid, OpenHLM outperforms two state-of-the-art humanoid VLA baselines (GR00T N1.6 and

Ψ_0

) using less than half the total demonstration time. Our code, training data, and model checkpoints are available at [https://openhlm-project.github.io/].

Abstract

Explore similar work

OpenHLM: An Empirical Recipe for Whole-Body Humanoid Loco-Manipulation

Learning Holistic Whole-Body Loco-Manipulation with a Bipedal Mobile Manipulator

PAKE: Learning Whole-Body Loco-Manipulation with Partial Kinematic Embeddings