Exploiting Hierarchical Controller Structure in Contextual Parameter Learning for Humanoid Loco-Manipulation

Sebastian Hirt; Lukas Theiner; Jan Peters; Rolf Findeisen

cs.ROOct 3, 2026

Exploiting Hierarchical Controller Structure in Contextual Parameter Learning for Humanoid Loco-Manipulation

Authors: Sebastian Hirt, Lukas Theiner, Jan Peters, Rolf Findeisen

Organizations: Control and Cyber-Physical Systems Laboratory (CCPS), TU Darmstadt · Intelligent Autonomous Systems Laboratory, TU Darmstadt · German Research Center for AI (DFKI), Research Department SAIROL · Hessian Centre for Artificial Intelligence

Abstract

Hierarchical control architectures are widely used to decompose complex control problems into interacting control levels and are particularly important in robotics, where planning, whole-body motion, and lower-level control must be coordinated across different levels of abstraction and time scales. Their overall closed-loop performance, however, depends strongly on parameters distributed across the hierarchy, such that tuning controllers on different levels independently may neglect relevant cross-layer interactions. We propose a contextual Bayesian optimization framework for joint parameter learning in hierarchical control systems. Rather than modeling closed-loop performance only as a scalar black-box function, we retain separate observations of task performance, realization quality, and control effort. A correlated multi-output Gaussian process models these performance components, while their known aggregation into the overall closed-loop objective is evaluated analytically. The formulation exploits three complementary consequences of hierarchical control: informative performance quantities exposed by the hierarchy, coupling between parameters of different controller levels, and variations of these relations with operating conditions. We evaluate the approach for humanoid loco-manipulation, jointly tuning a centroidal predictive controller and a whole-body controller for physical box pushing under varying box mass. The proposed method achieves the lowest mean empirical regret during both training and adaptation among the considered baselines.

Figures & tables

Panel of Figure 1 — Fig. 1: Overview of the proposed framework. State-feedback paths within the controller hierarchy are omitted for clarity. Dotted guides associate task ( $J_{\mathrm{T}}$ ), realization ( $J_{\mathrm{R}}$ ), and effort ( $J_{\mathrm{E}}$ ) information with quantities exposed by the control hierarchy, while all components may depend on the full parameter vector. The hierarchy-informed surrogate provides the aggregated objective used by the joint acquisition $\alpha^{(n)}$ , while context dependence $\chi$ enables transfer across operating conditions.

Fig. 2: The simulated humanoid box-pushing task.

Group	Optimized quantities	Dim.
$\theta_{\mathrm{MPPI}}$	MPPI temperature; motion, momentum, effort,	6
	and wrench-variation weights
$\theta_{\mathrm{WBC}}$	CoM, torso, foot, posture, momentum, and	11
	wrench-tracking gains and task weights
$\theta_{\mathrm{ref}}$	Torso pitch, CoM offsets, step duration,	5
	and swing clearance

TABLE I: Optimized parameter groups.

Panel of Figure 4 — Fig. 3: Empirical regret during training (left) and adaptation at the unseen $4\,\mathrm{kg}$ mass (right). Legend labels indicate, in order, context use ( $c$ ), joint cross-layer tuning ( $j$ ), and component-wise performance observations ( $p$ ). $\mathrm{x}$ denotes an absent feature. Curves show the mean, and shaded regions indicate one standard deviation across ten independent runs.

Panel of Figure 5 — Fig. 4: Representative closed-loop trajectories at $2\,\mathrm{kg}$ . The top panel shows the identical box-progress task for both controllers under a shared external reference (dots). The middle and bottom panels show center-of-mass height and torso pitch, respectively, together with each controller’s tunable reference. Solid lines denote realized trajectories, dashed lines denote references.

Robotic Control

Robotic control studies how machines use sensing and computation to produce reliable movement and interaction in changing environments.

See all 583 papers

Oct 6, 2026

HULK: Learning Whole-Body Forceful Loco-Manipulation for Humanoids

Humanoid loco-manipulation of large, heavy objects demands forceful interaction across the entire body. However, such payloads shift a humanoid's center of mass and impose sustained loads across the upper body, challenging balance and command tracking. We present HULK, a whole-body control framework for forceful loco-manipulation. Using model predictive control (MPC) to guide reinforcement learning with predictions of the loaded dynamics, we train two teachers: one tracks arm motions under wrist forces, and the other locomotes while holding large objects against the body. A capture-point control barrier function augments the wrist-force teacher during training to improve balance under load. We distill both teachers into a single policy. Evaluation spans simulation and the Unitree G1. In simulation, the teacher with the barrier function achieves the lowest forward and lateral velocity tracking errors at 10 kg per arm among evaluated controllers and reduces aggregate divergent component of motion (DCM) excursion magnitude by 35.7% relative to MPC-guided reinforcement learning alone. Our wrist-force teacher withstands torso push disturbances of up to 130 N.

An Dang, Arturo Flores Alvarez, Yu-Ming Chen +5

Amazon · University of Michigan · University of California, Los Angeles +1

Humanoid Loco-Manipulation Whole-Body Control Force-Controlled Robotic Manipulation RL Control

Ask PDF

Sep 16, 2026

Learning Holistic Whole-Body Loco-Manipulation with a Bipedal Mobile Manipulator

Bipedal loco-manipulation enables robots to interact with objects beyond the nominal workspace of their arms by coordinating locomotion and manipulation. Realizing this capability requires a low-level whole-body controller that translates task-level manipulation goals into coordinated arm and leg motions while maintaining balance. We present a unified whole-body controller trained with reinforcement learning that directly maps 6-DoF end-effector targets to coordinated actions for the bipedal base and robotic arm. Given only an end-effector target, the learned controller autonomously coordinates reaching, postural adaptation, and stepping without explicit base-velocity or footstep commands. A reward-gating strategy regulates the trade-offs among end-effector tracking, locomotion, and balance during training, while a temporal context estimator combines windowed Transformer encoding, recurrent GRU memory, and auxiliary dynamics prediction to extract dynamics-relevant information from observation history. Real-robot experiments demonstrate that the same controller supports reaching, postural adaptation, and stepping under commands from VR teleoperation, a learned diffusion policy, and scripted trajectories, providing a common end-effector interface for diverse manipulation tasks.

Zhongyu Chen, Yuxuan Nai, Qian Chen +8

ZJU-UIUC Institute. · LimX Dynamics.

Whole-Body Robotic Manipulation Humanoid Loco-Manipulation Robot Locomotion RL for Robotics Whole-Body Control Robotic Manipulation+1

Ask PDF

Oct 17, 2026

Learning Structured Locomotion References for Optimization-Based Whole-Body Control

Quadruped locomotion requires coordinating body motion,contact forces, foothold placement, and gait timing. We present a hierarchical controller in which reinforcement learning adapts these structured locomotion references while a gait scheduler, reactive footstep planner, and whole-body impulse controller retain responsibility for high-frequency execution. The model-based backbone can sustain a nominal trot independently, allowing the policy to focus on when and how to modify an already functional locomotion controller. We train the complete hierarchy in a vectorized controller-in-the-loop environment using a six-direction disturbance curriculum. Under a common simulator and evaluation protocol, the proposed controller achieves 77.29% mean survival across the tested disturbance grid, compared with 50.35% for the model-based reference controller. Training-time channel ablations identify adaptive foothold placement as the largest marginal contributor to grid survival and reveal direction-dependent effects of gait-rate adaptation. Together, these results demonstrate the viability of the proposed hierarchical controller architecture for integrating learned adaptation with optimization-based whole-body control through structured locomotion references.

Ömer Tarık Banuş, Uluç Saranlı, M. Mert Ankaralı

Center for Robotics and Artificial Intelligence (ROMER), Middle East Technical University (METU), 06800 Ankara, Türkiye. · Defense Systems Technologies Division, ASELSAN Inc., 06200 Ankara, Türkiye. · Department of Computer Engineering, METU, 06800 Ankara, Türkiye. +1

Robot Locomotion Quadruped Locomotion RL for Legged Locomotion Hierarchical Robot Control Whole-Body Control

Ask PDF

Abstract

Robotic Control

Explore similar work

HULK: Learning Whole-Body Forceful Loco-Manipulation for Humanoids

Learning Holistic Whole-Body Loco-Manipulation with a Bipedal Mobile Manipulator

Learning Structured Locomotion References for Optimization-Based Whole-Body Control