Learning Locomotion on Complex Terrain for Quadrupedal Robots with Foot Position Maps and Stability Rewards

Matthew Hwang; Yubin Liu; Ryo Hakoda; Takeshi Oishi

cs.ROApr 3, 2026

Learning Locomotion on Complex Terrain for Quadrupedal Robots with Foot Position Maps and Stability Rewards

Authors: Matthew Hwang, Yubin Liu, Ryo Hakoda, Takeshi Oishi

Organizations: University of Tokyo, Japan

Abstract

Quadrupedal locomotion over complex terrain has been a long-standing research topic in robotics. While recent reinforcement learning-based locomotion methods improve generalizability and foot-placement precision, they rely on implicit inference of foot positions from joint angles, lacking the explicit precision and stability guarantees of optimization-based approaches. To address this, we introduce a foot position map integrated into the heightmap, and a dynamic locomotion-stability reward within an attention-based framework to achieve locomotion on complex terrain. We validate our method extensively on terrains seen during training as well as out-of-domain (OOD) terrains. Our results demonstrate that the proposed method enables precise and stable movement, resulting in improved locomotion success rates on both in-domain and OOD terrains.

Figures & tables

Fig. 1 : Overview of our method. With the proposed foot position map and stability reward, our policy achieves successful locomotion over complex terrain.

Fig. 2 : We propose a foot position map which is concatenated with the heightmap providing information of the foot positions relative to the terrain. During training, we add a stability reward based on the CoP to guide the policy to prefer safer actions.

Fig. 3 : CoP-based stability rewards. We calculate the minimum distance of the C enter o f P ressure (CoP) to boundary of the support polygon as the stability reward.

Method	No Reward Hacking	Velocity Accuracy	Simple Training
Local Velocity	✗	✓	✓
Position	✓	✗	✗
Global Velocity	✓	✓	✓

TABLE I: Command type comparison. Global velocity tracking maintains velocity tracking accuracy and simplicity of training while preventing reward exploitation.

	Reward Terms	Definition	Weight
(1)	Linear Velocity Tracking	$\phi(\boldsymbol{v}^{local}_{cmd}-\boldsymbol{v}^{local}_{b})$	$1.5$
(1)	Angular Velocity Tracking	$\phi(\omega_{cmd}-\omega_{yaw})$	$0.5$
(2)	Z Linear Velocity Penalty	$-\boldsymbol{v}_{z}^{2}$	$1.0$
	Angular Velocity Penalty	$-\|\|\boldsymbol{\omega}\|\|^{2}$	$0.05$
	Joint Torques	$-\|\|\boldsymbol{\tau}_{j}\|\|^{2}$	$0.0001$
	Joint Acceleration	$-\|\|\dot{\mathbf{q}}_{j}\|\|^{2}$	$2.5e^{-7}$

TABLE II : Definition of rewards

\phi(x):=\exp(-\frac{||x||^{2}}{0.5})

Fig. 4 : The training environment consists of smooth, rough, stairs up, stairs down, discrete and stones terrain. The OOD evaluation terrains include novel combinations of stones (S), rough (R), stairsup (SU) and stairsdown (SD) terrains as well as beams, pallets, circles, small stones, pits and gaps.

Difficulty Level	Stone Size cm	Stone Gap cm	Max. Shift cm	Max. Height cm
0	92	2.5	0.0	1.0
$\cdots$	$\cdots$	$\cdots$	$\cdots$	$\cdots$
9	40	20	10	10

TABLE III : Parameters of the stone terrain

Fig. 5 : The success rate results. (a) Our proposed method produces higher success rates across the board for all terrains. (b) We observe an improvement at higher difficulty levels. (c) We observe a higher success rate especially at lower velocities.

Method	MLP	CNN	Transf. [ 32 ]	Attn [ 20 ]	Ours
Success rate [%] $\uparrow$	58.7	69.8	56.8	64.2	77.0
Survival rate [%] $\uparrow$	62.1	70.8	57.0	68.7	78.5
Tracking error [ $m/s$ ] $\downarrow$	0.268	0.249	0.229	0.243	0.169
Power [W] $\downarrow$	69.3	60.2	78.7	60.2	58.5

TABLE IV : Results of comparison methods. We osbsere a constant improvement of the success rate, the survival rate tracking error and power consumption for our proposed method.

Fig. 6 : Qualitative locomotion results of our proposed method on different terrains. (a) stones + stairs up, (b) stairs up, (c) stones, (d) gaps, (e) pits, (f) pallets.

Fig. 7 : Qualitative locomotion results of our method on stones. (FR: Front Right, FL: Front Left, RR: Rear Right, RL: Rear Left)

Fig. 8 : We implement stability rewards based on the CoM and the Capture Point for comparing with our proposed CoP-based stability reward.

Stability Reward	None	CoM	CP	CoP(Ours)
Success Rate [%] $\uparrow$	74.6	73.7	66.8	77.0
Survival Rate [%] $\uparrow$	75.3	74.6	68.6	78.5

TABLE V : Ablation of stability rewards. We found that the CoP reward (Ours) increases the success rate in contrast to Com and CP-based methods.

Component	Attention	Footmap	Ours
Success Rate [%] $\uparrow$	64.2	74.6	77.0
Survival Rate [%] $\uparrow$	68.7	75.4	78.5

TABLE VI : Ablation of proposed components. The foot position map produces 10% jump and the dynamic stability rewards produce an additional 2.4% increase in success rates.

Panel of Figure 15 — Fig. 9 : When using global velocity tracking the issue of robots avoiding difficult terrain is resolved.

Fig. 10 : Locomotion results on gaps (top) of 20cm and stairs (bottom) with heights of 20cm. Experiments were conducted in the Gazebo simulator.

Parameters	Range	Unit
Ground Friction	[0.5,1.25]	-
Ground Restitution	[0.0,0.8]	-
Link Mass	[0.9,1.1]×nominal value	Kg
Payload Mass	[-1,2]	Kg
CoM	[-0.05,0.05] 3	m
Motor Strength	[0.9,1.1]×motor torque	Nm

TABLE VII : Domain Randomization

Hyperparameter	Value
Clip range	0.2
Entropy coefficient	0.005(stage1), 0.002(stage2)
Discount factor	0.99
GAE discount factor	0.95
Desired KL-divergence	0.01
Learning rate	1e-4

TABLE VIII : PPO Training Parameters

Quadruped Locomotion

Quadruped locomotion studies how four-legged robots move efficiently, safely, and reliably across varied terrain and tasks.

See all 118 papers

LoComposition: Terrain-Adaptive Energy-Efficient Quadruped Locomotion without Gait Priors
Jun 14, 2026Loukas Kordos, Leonard T. Franz, Simon Rappenecker +4Legged Robotics RL for Robotics
Technical University of Munich. · University of Tübingen. · Hertie Institute for Clinical Brain Research & Center for Integrative Neuroscience. +1
PointWalker: Mapless Quadruped Locomotion Directly From Raw LiDAR Point Clouds
Oct 11, 2026Sicen Li, Zhen Chu, Chao Li +3Quadruped Locomotion Legged Robotics
College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China · Yunshenchu Technology Co., Ltd., Hangzhou, China · Hangzhou Public Library, Hangzhou, China +1
Agile and Generalized Legged Locomotion via Attention-Based Neural Map Encoding
Jan 13, 2026Chong Zhang, Victor Klemm, Fan Yang +1Terrain-Aware Robot Locomotion RL for Legged Locomotion
1Robotic Systems Lab, ETH Zurich, Switzerland · 2Secure, Reliable, and Intelligent Systems Lab, ETH Zurich, Switzerland · 3ETH AI Center, Switzerland

Jun 14, 2026J/K move · Enter open · S save

LoComposition: Terrain-Adaptive Energy-Efficient Quadruped Locomotion without Gait Priors

Loukas Kordos, Leonard T. Franz, Simon Rappenecker, Oliver Hausdoerfer+3

Technical University of Munich. · University of Tübingen. · Hertie Institute for Clinical Brain Research & Center for Integrative Neuroscience.+1

Legged Robotics RL for Robotics Energy-Efficient Robot Locomotion Terrain-Aware Robot Locomotion RL for Legged Locomotion Quadruped Locomotion

Open paper AskPDF arXiv

Learning-based quadrupedal locomotion typically relies on complex reward formulations that entangle task specification, operational limits, gait preference, and terrain adaptation within a single optimization objective. We instead treat these functions through distinct mechanisms: rewards for task specification, constraints for operational limits, energy minimization for gait preference, and exteroceptive perception for adapting energy use to terrain difficulty. We show that these components jointly enable efficient, terrain-adaptive locomotion, and that removing each component exposes a distinct failure mode. Our formulation removes explicit gait priors (including air-time, contact-count, and foot-clearance targets) in favor of emergent behavior. Compared to a conventional complex-reward baseline, our formulation achieves comparable terrain traversal while reducing cost of transport by 56% and operational-limit violations by 96%. The resulting policies transfer zero-shot to a physical Unitree Go2 using LiDAR-based elevation mapping. Project website with videos: https://tinyurl.com/locomposition.

Abstract

Quadruped Locomotion

Explore similar work

LoComposition: Terrain-Adaptive Energy-Efficient Quadruped Locomotion without Gait Priors

PointWalker: Mapless Quadruped Locomotion Directly From Raw LiDAR Point Clouds

Agile and Generalized Legged Locomotion via Attention-Based Neural Map Encoding