Learning Structured Locomotion References for Optimization-Based Whole-Body Control

Ömer Tarık Banuş; Uluç Saranlı; M. Mert Ankaralı

cs.ROOct 17, 2026

Learning Structured Locomotion References for Optimization-Based Whole-Body Control

Authors: Ömer Tarık Banuş, Uluç Saranlı, M. Mert Ankaralı

Organizations: Center for Robotics and Artificial Intelligence (ROMER), Middle East Technical University (METU), 06800 Ankara, Türkiye. · Defense Systems Technologies Division, ASELSAN Inc., 06200 Ankara, Türkiye. · Department of Computer Engineering, METU, 06800 Ankara, Türkiye. · Department of Electrical and Electronics Engineering, METU, 06800 Ankara, Türkiye.

Abstract

Quadruped locomotion requires coordinating body motion,contact forces, foothold placement, and gait timing. We present a hierarchical controller in which reinforcement learning adapts these structured locomotion references while a gait scheduler, reactive footstep planner, and whole-body impulse controller retain responsibility for high-frequency execution. The model-based backbone can sustain a nominal trot independently, allowing the policy to focus on when and how to modify an already functional locomotion controller. We train the complete hierarchy in a vectorized controller-in-the-loop environment using a six-direction disturbance curriculum. Under a common simulator and evaluation protocol, the proposed controller achieves 77.29% mean survival across the tested disturbance grid, compared with 50.35% for the model-based reference controller. Training-time channel ablations identify adaptive foothold placement as the largest marginal contributor to grid survival and reveal direction-dependent effects of gait-rate adaptation. Together, these results demonstrate the viability of the proposed hierarchical controller architecture for integrating learned adaptation with optimization-based whole-body control through structured locomotion references.

Figures & tables

Fig. 1: Proposed controller. A 100 Hz policy supplies structured locomotion commands to a gait scheduler, footstep planner, and WBIC running at 500 Hz. Feedback provides the 54-dimensional observation.

Index	Command	Decoded range
0–1	body $v_{x},v_{y}$ residual	$\pm 0.5$ m/s
2	yaw-rate residual	$\pm 0.3$ rad/s
3–14	four forces $(f_{x},f_{y},f_{z})$	$\pm 50,\pm 50,[0,250]$ N
15–22	four foothold offsets $(x,y)$	$\pm 0.5$ m
23	body-height residual	$\pm 0.05$ m
24	phase increment	$[0,0.01]$ per tick

TABLE I: Action components; leg order is front-right (FR), front-left (FL), hind-right (HR), and hind-left (HL).

Signal	Dimension
Commanded forward velocity $v_{x}^{\rm cmd}$	1
Body linear velocity $\mathbf{v}_{b}^{b}$	3
Body angular velocity $\boldsymbol{\omega}_{b}^{b}$	3
Body orientation $\boldsymbol{\phi}_{b}$ (roll, pitch, yaw)	3
Joint positions $\mathbf{q}_{j}$	12
Joint velocities $\dot{\mathbf{q}}_{j}$	12

TABLE II: Observation components.

Term	Penalty $p_{i}$	$w_{i}$	Term	Penalty $p_{i}$	$w_{i}$
$v_{x}$	$3\|v_{x}^{w}-v_{x}^{\rm ref}\|$	0.25	Straightness	$5\|y_{b}^{w}\|+3\|v_{y}^{w}\|$	0.15
$v_{z}$	$3\|v_{z}^{w}\|$	0.10	Linear acceleration	$\\|\mathbf{a}_{b}^{w}\\|_{2}^{2}$	0.025
Height	$20\|z_{b}-z^{\rm ref}\|$	0.35	Angular velocity	$\\|\boldsymbol{\omega}_{b}\\|_{2}^{2}$	0.025
Orientation	$50(1-\langle\mathbf{q}_{b},\mathbf{q}_{0}\rangle)$	0.15	Action variation	$3\\|\mathbf{a}_{t}-\mathbf{a}_{t-1}\\|_{2}^{2}$	0.10
Phase progression	$\max(0,a_{\theta}+1)$	0.05	Action magnitude	$\sum_{j\in\mathcal{A}}a_{t,j}^{2}$	0.05

TABLE III: Reward penalties and weights. Each healthy-state component is

w_{i}\exp(-p_{i})

. Superscript

w

denotes the world frame, and

\mathbf{q}_{0}

is the neutral-orientation quaternion.

Fig. 2: Go2 simulation environment. The arrow illustrates the disturbance direction; evaluation applies instantaneous velocity changes.

Fig. 3: Training episode length and timeout fraction for three seeds. Shading indicates disturbance ceilings of 1, 2, and 3 m/s, with transitions at 8.1 and 24.3 million transitions. Curves are unsmoothed.

Fig. 4: Survival under additive velocity disturbances, with 100 episodes per cell and policy. Curves and bands show the mean and

\pm 1

sample standard deviation across three training seeds for the proposed and source-recipe-based joint-space controllers. MPC+WBIC is one fixed controller.

Controller	Grid survival (%)
MPC+WBIC	50.35
Source-recipe-based joint-space RL	$54.01\pm 5.55$
Proposed	$77.29\pm 1.15$

TABLE IV: Mean survival over the 60 disturbance conditions for the complete configured systems. Learned-controller results report mean

\pm

sample SD across three training seeds; MPC+WBIC is one fixed controller.

Fig. 5: Training-time action-channel ablations: grid-survival change (left) and directional $c_{50}$ change (right; negative is worse). Backward $c_{50}$ uses extensions above 5.0 m/s excluded from the grid score.

Fig. 6: Nominal behavior of the retrained ablation arms over five undisturbed episodes. Small circles denote episodes, diamonds denote means, and crosses in the speed panel denote commanded-speed means.

Robot Locomotion

Robot locomotion studies how robots move efficiently and reliably across varied terrains, environments, and movement modes.

See all 233 papers

Reinforcement Learning Based Multi-Terrain Adaptive Whole-Body Control for a Quadruped Manipulator
Oct 16, 2026Yanzhou Jin, Fei Suo, Chang Liu +5Legged Robotics Whole-Body Control
Graduate School of Engineering, The University of Tokyo, Japan. · Department of Micro-Nano Mechanical Science and Engineering, Nagoya University, Nagoya, 464-8603, Japan. · Department of Mathematics and Informatics, Ecole Centrale de Lyon, France.
LoComposition: Terrain-Adaptive Energy-Efficient Quadruped Locomotion without Gait Priors
Jun 14, 2026Loukas Kordos, Leonard T. Franz, Simon Rappenecker +4Legged Robotics RL for Robotics
Technical University of Munich. · University of Tübingen. · Hertie Institute for Clinical Brain Research & Center for Integrative Neuroscience. +1
PA-LOCO: Learning Perturbation-Adaptive Locomotion for Quadruped Robots
Jul 5, 2024Zhiyuan Xiao, Xinyu Zhang, Xiang Zhou +1RL for Legged Locomotion Quadruped Locomotion
School of Aeronautics and Astronautics, Sun Yat-sen University, Shenzhen 518107, P.R. China.

Oct 16, 2026J/K move · Enter open · S save

Reinforcement Learning Based Multi-Terrain Adaptive Whole-Body Control for a Quadruped Manipulator

Yanzhou Jin, Fei Suo, Chang Liu, Yasuhisa Hasegawa+4

Graduate School of Engineering, The University of Tokyo, Japan. · Department of Micro-Nano Mechanical Science and Engineering, Nagoya University, Nagoya, 464-8603, Japan. · Department of Mathematics and Informatics, Ecole Centrale de Lyon, France.

Legged Robotics Whole-Body Control Whole-Body Robotic Manipulation Terrain-Aware Robot Locomotion RL for Legged Locomotion

Open paper AskPDF arXiv

Legged robots equipped with manipulators offer promising capabilities for loco-manipulation tasks in unstructured environments. However, achieving task-consistent end-effector behavior during locomotion remains challenging due to terrain-induced disturbances and the strong coupling between locomotion and manipulation, particularly for small-scale quadruped platforms. A reinforcement-learning based whole-body control framework is proposed for quadruped loco-manipulation on complex terrain. The proposed framework combines a hybrid task representation with task-prioritized whole-body learning to coordinate locomotion and manipulation while preserving task-consistent end-effector behavior. A lightweight quadruped manipulator platform is further developed to validate the proposed framework on a small-scale quadruped system. Real-world experiments demonstrate successful deployment on complex terrains such as slopes up to 30 degrees, stairs, and deformable terrain. These results demonstrate the feasibility of deploying learned whole-body control policy for quadruped loco-manipulation in real-world environments and highlight the importance of task-consistent whole-body coordination for terrain-adaptive loco-manipulation.

Abstract

Robot Locomotion

Explore similar work

Reinforcement Learning Based Multi-Terrain Adaptive Whole-Body Control for a Quadruped Manipulator

LoComposition: Terrain-Adaptive Energy-Efficient Quadruped Locomotion without Gait Priors

PA-LOCO: Learning Perturbation-Adaptive Locomotion for Quadruped Robots