FAME: Force-Adaptive RL for Expanding the Manipulation Envelope of a Full-Scale Humanoid

Niraj Pudasaini; Yutong Zhang; Jensen Lavering; Max Conwat; Alessandro Roncone; Nikolaus Correll

cs.ROMar 9, 2026

FAME: Force-Adaptive RL for Expanding the Manipulation Envelope of a Full-Scale Humanoid

Authors: Niraj Pudasaini, Yutong Zhang, Jensen Lavering, Max Conwat, Alessandro Roncone, Nikolaus Correll

Organizations: Department of Computer Science, University of Colorado Boulder, Boulder, CO 80309.

Abstract

Maintaining balance under external hand forces is critical for humanoid bimanual manipulation, where interaction forces propagate through the kinematic chain and constrain the feasible manipulation envelope. We propose FAME, a force-adaptive reinforcement learning framework that conditions a standing policy on a learned latent context encoding upper-body joint configuration and bimanual interaction forces jointly, since the base moment a load induces depends on the arm configuration through which it acts. Training applies isotropically sampled 3D forces at each hand under an upper-body pose curriculum, exposing the policy to manipulation-induced perturbations across continuously varying arm configurations. At deployment the interaction force is not measured but reconstructed online from joint torques and states through rigid-body inverse dynamics, requiring no wrist force/torque sensing. We evaluate over $100$ upper-body configurations under swept hand forces, scoring each trial by a task-level criterion that requires the robot both to remain upright and to hold its hands near where the task placed them; all such results run with the estimated force in the loop. At a $150$ ,mm tolerance FAME reaches $38.9\%$ task success, against $16.6\%$ for a policy given the same force without encoding, $4.3\%$ for a pose-conditioned curriculum policy, and $24.7\%$ for an adversarially trained locomotion policy, which stays upright but recovers by stepping and so relocates the hands. We further demonstrate transfer to task-generated interaction forces in a MuJoCo kitchen environment, and to asymmetric and bimanual loading on a full-scale Unitree H1-2. Code and videos are available on the https://correlllab.github.io/fame_website.

Figures & tables

Fig. 2 : Overview of the proposed standing framework. During training (top), an upper-body dynamics encoder processes upper-body joint states and sampled hand forces to produce a latent context variable that conditions the base standing policy in simulation. During deployment (bottom), the same encoder operates on measured upper-body joints and estimated hand forces to infer the latent context online, enabling rapid adaptation to upper-body-induced disturbances.

Term	Expression	Weight	Remarks
Base Stability & Uprightness
Base height tracking	$\exp(-4\|h_{\text{base}}-h_{\text{cmd}}\|)$	$3.0$	$h_{\text{cmd}}=1.0$
Vertical velocity	$v_{z}^{2}$	$-5.0$	Penalize bouncing
Angular velocity (xy)	$\\|\bm{\omega}_{xy}\\|_{2}^{2}$	$-0.1$	Suppress roll/pitch motion
Orientation	$\\|\bm{g}_{xy}\\|_{2}^{2}$	$-3.0$	Projected gravity error
Stand still (zero command)	$\mathds{1}(\text{motion at zero cmd})$	$-0.8$	Encourage static stability

TABLE I : Standing reward components and weights, adjusted from [ 11 ] for Unitree H1-2.

	Information available				Task success (%)		Deviation
Variant	Pose curr.	Force to policy	Arm cfg. in encoder	Latent encoding	$T{=}150$ mm	$T{=}250$ mm	$\Delta$ EE (mm)
FAME-pose (HOMIE-style [ 11 ] )	✓	—	✓	✓	4.3	5.6	242.4
+Curr+F raw	✓	✓	—	—	16.6	21.7	220.3
ALMI [ 15 ]	adversarial motion imitation; no force input				24.7	39.3	202.3
FAME	✓	✓	✓	✓	38.9	48.5	179.6

TABLE II: Standing under swept hand-force disturbances over

100

upper-body configurations sampled from the training pose distribution (protocol in Sec. IV-A). A trial succeeds if the robot stays upright and peak end-effector deviation remains below

T

; poses a policy cannot hold count as failures rather than exclusions.

\Delta

EE is the force-induced hand displacement at

T{=}250

mm, differenced against the same trial run without force. ALMI is a separately trained locomotion policy, whose lower body is optimized against upper-body motion its own arms generate. All controlled variants receive the full 27-joint proprioceptive history.

Fig. 3 : Real-robot qualitative results. Snapshot sequence from our real-robot evaluation on the Unitree H1-2 under representative load-interaction disturbances (RE1–RE2). For each experiment, we report the joint trajectories and torques of the hip pitch, ankle pitch, and elbow joints. With FAME, the robot remains stable under external loads and the joint positions stay close to their nominal standing configuration (marked in green ). With FAME-pose, the joint positions drift away from the stable configuration, ultimately causing the robot to lose balance and fall (marked in red ).

Fig. 4 : Task-level comparison in the GOLEM kitchen twin. Both policies perform the same single-arm pick of a loaded object, with the upper body driven by a shared inverse-kinematics stack so that the lower body is the only variable. Left: one seed; FAME holds its stance, ALMI steps out from under the load and falls. Right: pelvis drift over $10$ seeds.

Actor observation term (one-step)	Symbol	Dim
Command (scaled)	$\bm{c}_{t}$	3
Commanded base height	$h^{\text{cmd}}_{t}$	1
IMU angular velocity (body frame)	$\bm{\omega}_{t}$	3
Projected gravity (body frame)	$\bm{g}_{t}$	3
Joint position error (scaled)	$\tilde{\bm{q}}_{t}$	$N_{\text{dof}}$
Joint velocity (scaled)	$\tilde{\dot{\bm{q}}}_{t}$	$N_{\text{dof}}$

TABLE III : Actor (policy) observation space.

Critic observation term	Symbol	Dim
All actor one-step terms	$\bm{o}_{t}$	76
Latent context (from encoder)	$\bm{z}_{t}$	8
Base linear velocity (privileged, scaled)	$\bm{v}^{\text{base}}_{t}$	3
One-step critic obs: $d_{v}=d_{o}+8+3=87$
History stacking: $H_{c}=1\Rightarrow\dim(\bm{o}^{V}_{t})=87$

TABLE IV : Critic observation design. The critic is additionally provided latent context

\bm{z}_{t}

and privileged base linear velocity in simulation during training.

Term	Value
External Disturbances
Push robot	$\text{interval}=4\,\mathrm{s}$ , $v_{\mathrm{xy}}\sim\mathcal{U}(0,0.5)\,\mathrm{m/s}$
Upper-body disturbance curriculum	$\rho_{a}\leftarrow 0.0$ initially; increased with standing quality, interval $=1\,\mathrm{s}$
Actuation & Observation Noise
Joint injection noise	$q\leftarrow q+\epsilon$ , $\epsilon\sim\mathcal{U}(-0.05,0.05)$
Actuation offset	$a\leftarrow a+\delta$ , $\delta\sim\mathcal{U}(-0.05,0.05)$

TABLE V : Domain randomizations used for training the standing policy.

\mathcal{U}(\cdot)

denotes a uniform distribution.

Hyperparameter	Value	Hyperparameter	Value
Optimizer	Adam	Discount factor $\gamma$	0.99
Learning rate	$1\times 10^{-3}$	GAE $\lambda$	0.95
Schedule	adaptive	Clip param $\epsilon$	0.2
Learning epochs	5	Value loss coef	1.0
Mini-batches	4	Clipped value loss	True
Max grad norm	1.0	Entropy coef	0.01

TABLE VI : PPO hyperparameters used for training.

Explore similar work

Sep 21, 2026cs.RO

Brace Yourself: Task-Conditioned Environmental Bracing for Forceful Humanoid Manipulation

Forceful manipulation is challenging for humanoid robots because interaction forces can disturb whole-body balance. We introduce the Supporting Hand Strategy (SHS), which enables a humanoid to brace against the environment with one hand while performing forceful manipulation with the other. SHS optimises a task-conditioned support configuration that guides two synchronous reinforcement-learning policies, without human motion data or online whole-body trajectory planning. On a Unitree G1, SHS achieved usable contact forces up to 60 N, compared with a maximum sustained force of 13.5 N without environmental bracing, while substantially improving force tracking over a task-independent support configuration. The same policies generalised to different task regions without retraining. SHS therefore provides a simple mechanism for substantially extending humanoid forceful-manipulation capability.

Zongyuan Zhang, Christopher Lehnert, Will N. Browne +1

School of Electrical Engineering and Robotics, Queensland University of Technology, 2 George St, Brisbane, 4000, Queensland, Australia. · Australian Cobotics Centre, Queensland University of Technology, 2 George St, Brisbane, 4000, Queensland, Australia. · Centre for Robotics, Queensland University of Technology, 2 George St, Brisbane, 4000, Queensland, Australia.

Humanoid Hand

Ask PDF

Oct 30, 2025cs.RO

Thor: Towards Human-Inspired Whole-Body Reactions for Intense Contact-Rich Environments

Maintaining whole-body stability and motion tracking under large interaction forces remains challenging for humanoids. We present Thor, a reinforcement learning framework for forceful humanoid loco-manipulation. Thor jointly trains lower-body, waist, and upper-body policies with shared whole-body observations and body-specific rewards to coordinate locomotion and force adaptation, waist posture regulation, and upper-body motion tracking. We further introduce a force-adaptive torso-tilt (FAT2) objective that derives a load-dependent horizontal center-of-mass offset reference from quasi-static moment balance. Capacity-matched simulation blations show that the three-policy architecture improves tracking under large external force disturbances, while real-world ablations demonstrate that FAT2 increases peak pulling capability. On the Unitree G1, Thor achieves mean peak dual-hand pulling forces of 167.7 N and 145.5 N during backward and forward locomotion, exceeding the best-performing baseline by 68.9% and 74.7%, respectively. Real-world demonstrations include opening a fire door with one hand using approximately 60 N of pulling force and towing a 1.7-ton passenger car.

Gangyang Li, Hongzhe Shi, Qing Shi +5

Intelligent Robotics Institute, School of Mechatronical Engineering, Beijing Institute of Technology · Beijing Academy of Artificial Intelligence

Ground Reaction Forces Humanoid

Ask PDF

Oct 5, 2026cs.RO

I-BFM: Reward-Conditioned Robust Humanoid Interaction via Unsupervised Reinforcement Learning

Behavioral foundation models (BFMs) have recently shown that a single humanoid policy can support diverse whole-body control, but extending such generality to physical interaction remains challenging. We introduce I-BFM, to our knowledge the first BFM for humanoid-object interaction. Rather than relying on task-specific policies or reference tracking, I-BFM learns a shared representation of the coupled dynamics among the humanoid, objects, and their contacts using forward-backward representations and unsupervised reinforcement learning. Given a downstream task reward, the same policy can be directly conditioned on a latent command to execute closed-loop interaction without task-specific policy optimization. To improve interaction control over different time scales, we further train the policy with both short-horizon interaction targets and longer-horizon goal targets. A single I-BFM policy performs carrying, pushing, and kicking, while also supporting goal reaching, motion tracking, stylistic control, and long-horizon task chaining. More importantly, it remains effective after large deviations from nominal execution: on Carry, I-BFM achieves 94.3% nominal success and retains 89.3% success after robot falls, compared with 1.3% for a planning-based baseline. Real-world experiments on a Unitree G1 further demonstrate diverse loco-manipulation behaviors, rapid recovery from interaction failures and external disturbances, and task chaining without task-specific retraining.

Ziqi Han, Yitang Li, Junhan Sun +8

Tongji University · Tsinghua University · Zhejiang University +3

Full-Body Humanoid Control

Ask PDF