Energy-Efficient Gait Adaptation via Hierarchical Reinforcement Learning for Quadrupedal Locomotion Across Diverse Terrains

Ammar Issa; Anubhav Singh; Anton Tsaritsin; Sergey Kolyubin

cs.ROOct 7, 2026

Energy-Efficient Gait Adaptation via Hierarchical Reinforcement Learning for Quadrupedal Locomotion Across Diverse Terrains

Authors: Ammar Issa, Anubhav Singh, Anton Tsaritsin, Sergey Kolyubin

Organizations: Faculty of Control Systems and Robotics, ITMO University, and the Biomechatronics and Energy-Efficient Robotics Lab (BE2R), St. Petersburg, Russia

Abstract

While energy efficiency is a critical objective for legged-robot locomotion control, achieving low energy consumption while maintaining robust performance across different velocity ranges and terrain conditions remains a key challenge. This is particularly true for end-to-end RL policies, where gait generation, motion execution, and energy optimization are tightly coupled, leading to high sensitivity to reward design. In this work, we propose a hierarchical reinforcement learning (HRL) framework that separates a high-frequency policy for stable and robust joint-level motion execution from low-frequency gait adaptation that explicitly minimizes the cost of transport (CoT). The three-stage Isaac-based training procedure enables zero-shot sim-to-real transfer with improved tracking accuracy, robustness, and energy efficiency. The learned hierarchy exhibits automatic speed-dependent gait adaptation, transitioning from pacing at low speeds to trotting at higher speeds. We validate the proposed approach in simulation against representative single-policy and hierarchical locomotion baselines, demonstrating reduced CoT over a broad range of commanded velocities, while maintaining robust locomotion across flat, uneven rough, and inclined terrains. We further demonstrate its practical feasibility through zero-shot deployment on a physical Unitree AlienGo quadruped.

Figures & tables

Fig. 1 : Stable locomotion and slope traversal, with speed-dependent gait modulation from pacing at lower speeds to trotting at higher speeds.

Fig. 2 : Overview of the hierarchical locomotion control framework. The high-level policy operates at a lower frequency to modulate gait parameters, while the low-level controller performs high-frequency motion execution.

Fig. 3 : Three-stage training pipeline: LL policy pretraining, energy-aware HL policy training with the LL policy frozen, and LL policy finetuning with the HL policy frozen.

Fig. 4 : Processing of commanded gait parameters into leg-wise phases, clock inputs, and training-time contact targets.

Reward	Primary gait parameter	Formulation
$r_{\mathrm{ori}}$	$\theta_{x},\theta_{y}$	$\left\\|\mathbf{g}^{\mathrm{cmd}}_{\text{proj},\{x,y\}}-\mathbf{g}_{\text{proj},\{x,y\}}\right\\|^{2}$
$r_{\mathrm{clearance}}$	$h_{f,z}$	$\displaystyle\sum_{i=1}^{4}\left(h^{i}_{f,z}-h^{\mathrm{target},i}_{f,z}\right)^{2}$
$r_{\mathrm{jump}}$	$h_{z}$	$\left\|h_{z}^{\mathrm{cmd}}-h^{\text{body}}_{z}+h_{z}^{\mathrm{ref}}\right\|$
$r_{\mathrm{contact\_force}}$	$d$	$\displaystyle\sum_{i=1}^{4}(1-d^{i})\left[1-\exp\left(-\frac{F_{i}^{2}}{\sigma_{F}}\right)\right]$
$r_{\mathrm{contact\_velocity}}$	$d$	$\displaystyle\sum_{i=1}^{4}d^{i}\left[1-\exp\left(-\frac{v_{i}^{2}}{\sigma_{v}}\right)\right]$
$r_{\mathrm{raibert}}$	$\theta_{1},\theta_{2},\theta_{3},f_{\text{gait}},s_{x},s_{y}$	$\displaystyle\sum_{i=1}^{4}\left\\|\mathbf{p}^{\mathrm{des},i}_{\mathrm{foot}}-\mathbf{p}^{i}_{\mathrm{foot}}\right\\|^{2}$

TABLE I : Augmented auxiliary rewards used to enforce the commanded gait parameters in the LL policy.

Fig. 5 : Foot contact state transitions under varying commanded velocities. The high-level policy autonomously and smoothly adapts the gait pattern, exhibiting pacing at low speeds (two-beat walking), a four-beat walking gait at moderate velocities, and trotting at higher speeds.

Panel of Figure 7 — Fig. 6 : CoT evaluation of the proposed HRL framework. (a) Ablation study on the HL training terrain for the Unitree AlienGo, comparing deployment on flat and rough surfaces, including zero-shot deployment of the flat-trained HL policy on rough terrain. (b) Ablation study on the effect of the HL energy reward for the Unitree AlienGo, comparing CoT with and without the energy reward across different deployment terrains. (c) Comparison of the proposed HRL framework with existing locomotion baselines on the Unitree Go1.

Terrain	$[0.0,0.5]$	$(0.5,1.0]$	$(1.0,1.5]$	$(1.5,2.0]$
Flat terrain	0.0%	0.0%	0.0%	0.0%
Uneven rough terrain	0.0%	0.0%	0.0%	0.6%
Inclined slopes	2.4%	0.4%	0.0%	0.2%

TABLE II : Robot falling rate in simulation across terrain types and commanded-velocity intervals. Each entry aggregates 500 runs with friction and restitution coefficients uniformly randomized in

[0,1]

Method / $v_{x}^{\mathrm{cmd}}$ [m/s]	$\mathbf{(0.0,0.5]}$	$\mathbf{(0.5,1.0]}$	$\mathbf{(1.0,1.5]}$	$\mathbf{(1.5,2.0]}$	$\mathbf{(2.0,2.5]}$
HRL (Ours)	$0.762\,{\scriptstyle\pm\,0.198}$	$\mathbf{0.421}\,{\scriptstyle\boldsymbol{\pm}\,\mathbf{0.034}}$	$\mathbf{0.384}\,{\scriptstyle\boldsymbol{\pm}\,\mathbf{0.004}}$	$\mathbf{0.438}\,{\scriptstyle\boldsymbol{\pm}\,\mathbf{0.031}}$	$\mathbf{0.495}\,{\scriptstyle\boldsymbol{\pm}\,\mathbf{0.010}}$
AER ( $\alpha=1$ )	$\mathbf{0.572}\,{\scriptstyle\boldsymbol{\pm}\,\mathbf{0.021}}$	$0.618\,{\scriptstyle\pm\,0.021}$	$0.681\,{\scriptstyle\pm\,0.022}$	$0.757\,{\scriptstyle\pm\,0.022}$	$0.831\,{\scriptstyle\pm\,0.014}$
WTW–Trotting	$0.962\,{\scriptstyle\pm\,0.216}$	$0.688\,{\scriptstyle\pm\,0.020}$	$0.680\,{\scriptstyle\pm\,0.019}$	$0.711\,{\scriptstyle\pm\,0.015}$	$0.719\,{\scriptstyle\pm\,0.017}$
F&E	$3.560\,{\scriptstyle\pm\,1.338}$	$1.663\,{\scriptstyle\pm\,0.240}$	$1.240\,{\scriptstyle\pm\,0.078}$	$1.057\,{\scriptstyle\pm\,0.059}$	$0.966\,{\scriptstyle\pm\,0.006}$

TABLE III : CoT comparison with existing locomotion baselines across commanded-velocity intervals. Values are reported as mean

\pm

standard deviation over evaluation runs within each commanded-velocity interval.

Fig. 7 : Realized geometrical gait parameters during a representative transition from flat terrain to a

20^{\circ}

uphill slope at a fixed commanded forward velocity. The shaded interval marks the terrain-transition phase.

Method	(0,0.5]	(0.5,1.0]	(1.0,1.5]	(1.5,2.0)
HL w/ Energy, Train: Complex, Test: Flat	$1.249\,\mbox{\footnotesize$ \pm,0.486 $}$	$0.734\,\mbox{\footnotesize$ \pm,0.069 $}$	$0.721\,\mbox{\footnotesize$ \pm,0.037 $}$	$0.714\,\mbox{\footnotesize$ \pm,0.024 $}$
HL w/ Energy, Train: Complex, Test: Rough	$1.846\,\mbox{\footnotesize$ \pm,1.333 $}$	$0.766\,\mbox{\footnotesize$ \pm,0.074 $}$	$0.792\,\mbox{\footnotesize$ \pm,0.054 $}$	$0.805\,\mbox{\footnotesize$ \pm,0.034 $}$
HL w/ Energy, Train: Flat, Test: Rough	$5.029\,\mbox{\footnotesize$ \pm,4.149 $}$	$1.036\,\mbox{\footnotesize$ \pm,0.178 $}$	$0.898\,\mbox{\footnotesize$ \pm,0.052 $}$	$0.942\,\mbox{\footnotesize$ \pm,0.182 $}$
HL w/o Energy, Train: Complex, Test: Flat	$17.118\,\mbox{\footnotesize$ \pm,8.179 $}$	$8.539\,\mbox{\footnotesize$ \pm,0.317 $}$	$7.844\,\mbox{\footnotesize$ \pm,0.277 $}$	$7.649\,\mbox{\footnotesize$ \pm,0.405 $}$
HL w/o Energy, Train: Complex, Test: Rough	$37.114\,\mbox{\footnotesize$ \pm,37.478 $}$	$8.906\,\mbox{\footnotesize$ \pm,0.348 $}$	$9.295\,\mbox{\footnotesize$ \pm,1.615 $}$	$14.791\,\mbox{\footnotesize$ \pm,5.193 $}$

TABLE IV : AlienGo CoT ablation across commanded-velocity intervals.

Fig. 8 : Qualitative reward-sensitivity case for an AER-style end-to-end policy on AlienGo with $\alpha=0.5$ . Excessive reduction of joint effort resulted in a crouched and unstable posture.

Fig. 9 : Velocity-tracking ablation for Unitree AlienGo. “w/o finetuning” denotes the LL policy obtained after Stage 2; “with finetuning” denotes the same policy after Stage 3. Both policies are evaluated on flat and rough terrain.

Panel of Figure 14 — Fig. 10 : Representative real-world locomotion trials on flat terrain, inclined slope, and uneven rough terrain.

Explore similar work

LoComposition: Terrain-Adaptive Energy-Efficient Quadruped Locomotion without Gait Priors
Jun 14, 2026Loukas Kordos, Leonard T. Franz, Simon Rappenecker +4Emergence Prior Knowledge Integration
Technical University of Munich. · University of Tübingen. · Hertie Institute for Clinical Brain Research & Center for Integrative Neuroscience. +1
Towards Torque-Driven Reinforcement Learning for Quadruped Locomotion
Jul 20, 2026Jordan Dowdy, Jean Chagas VazLegged Robots Locomotion
Department of Electrical and Computer Engineering at the University of Louisville. · Faculty of Electrical and Computer Engineering, University of Louisville, Louisville, KY, 40208, USA.
CoRe-MoE: Contrastive Reweighted Mixture of Experts for Multi-Terrain Humanoid Locomotion with Gait Adaptation
Jun 3, 2026Kailun Huang, Zikang Xie, Yanzhe Xie +8Agile Locomotion Gait Dynamics
Hong Kong University of Science and Technology · Hong Kong University of Science and Technology (Guangzhou) · Guangdong University of Technology +3

Jun 14, 2026, cs.ROJ/K move · Enter open · S save

LoComposition: Terrain-Adaptive Energy-Efficient Quadruped Locomotion without Gait Priors

Loukas Kordos, Leonard T. Franz, Simon Rappenecker, Oliver Hausdoerfer+3

Technical University of Munich. · University of Tübingen. · Hertie Institute for Clinical Brain Research & Center for Integrative Neuroscience.+1

Emergence Prior Knowledge Integration

Open paper AskPDF arXiv

Learning-based quadrupedal locomotion typically relies on complex reward formulations that entangle task specification, operational limits, gait preference, and terrain adaptation within a single optimization objective. We instead treat these functions through distinct mechanisms: rewards for task specification, constraints for operational limits, energy minimization for gait preference, and exteroceptive perception for adapting energy use to terrain difficulty. We show that these components jointly enable efficient, terrain-adaptive locomotion, and that removing each component exposes a distinct failure mode. Our formulation removes explicit gait priors (including air-time, contact-count, and foot-clearance targets) in favor of emergent behavior. Compared to a conventional complex-reward baseline, our formulation achieves comparable terrain traversal while reducing cost of transport by 56% and operational-limit violations by 96%. The resulting policies transfer zero-shot to a physical Unitree Go2 using LiDAR-based elevation mapping. Project website with videos: https://tinyurl.com/locomposition.

Abstract

Explore similar work

LoComposition: Terrain-Adaptive Energy-Efficient Quadruped Locomotion without Gait Priors

Towards Torque-Driven Reinforcement Learning for Quadruped Locomotion

CoRe-MoE: Contrastive Reweighted Mixture of Experts for Multi-Terrain Humanoid Locomotion with Gait Adaptation