PA-LOCO: Learning Perturbation-Adaptive Locomotion for Quadruped Robots

Zhiyuan Xiao; Xinyu Zhang; Xiang Zhou; Qingrui Zhang

cs.ROJul 5, 2024

PA-LOCO: Learning Perturbation-Adaptive Locomotion for Quadruped Robots

Authors: Zhiyuan Xiao, Xinyu Zhang, Xiang Zhou, Qingrui Zhang

Organizations: School of Aeronautics and Astronautics, Sun Yat-sen University, Shenzhen 518107, P.R. China.

Abstract

Numerous locomotion controllers have been designed based on Reinforcement Learning (RL) to facilitate blind quadrupedal locomotion traversing challenging terrains. Nevertheless, locomotion control is still a challenging task for quadruped robots traversing diverse terrains amidst unforeseen disturbances. Recently, privileged learning has been employed to learn reliable and robust quadrupedal locomotion over various terrains based on a teacher-student architecture. However, its one-encoder structure is not adequate in addressing external force perturbations. The student policy would experience inevitable performance degradation due to the feature embedding discrepancy between the feature encoder of the teacher policy and the one of the student policy. Hence, this paper presents a privileged learning framework with multiple feature encoders and a residual policy network for robust and reliable quadruped locomotion subject to various external perturbations. The multi-encoder structure can decouple latent features from different privileged information, ultimately leading to enhanced performance of the learned policy in terms of robustness, stability, and reliability. The efficiency of the proposed feature encoding module is analyzed in depth using extensive simulation data. The introduction of the residual policy network helps mitigate the performance degradation experienced by the student policy that attempts to clone the behaviors of a teacher policy. The proposed framework is evaluated on a Unitree GO1 robot, showcasing its performance enhancement over the state-of-the-art privileged learning algorithm through extensive experiments conducted on diverse terrains. Ablation studies are conducted to illustrate the efficiency of the residual policy network.

Figures & tables

Fig. 1: A Unitree Go1 quadruped robot is subject to a kick, while standing on a grass field.

Fig. 2: The proposed PA-LOCO integrates a teacher-student framework with a residual network and multiple feature encoders. The training process involves three phases. In the first phase, the teacher policy is trained with proprioceptive observations $o_{t}$ and privileged information $F_{t},T_{t},S_{t}$ that is unknown for deployment. In the second phase, the student policy is trained using observations from the proprioceptive sensors. The student policy is learned to clone the teacher’s actions and latent features by supervised learning. In the third phase, the residual policy network is trained to further enhance the performance of the student policy against perturbations.

Name	Expression	Weight
Linear velocity tracking	$\exp(-\frac{\|\|v_{b,xy}^{*}-v_{b,xy}\|\|^{2}}{0.25})$	1dt
Angular velocity tracking	$\exp(-\frac{\|\|\omega_{b,z}^{*}-\omega_{b,z}\|\|^{2}}{0.25})$	0.5dt
Linear velocity penalty	$v_{b,z}^{2}$	-2dt
Angular velocity penalty	$\|\|\omega_{b,xy}\|\|^{2}$	-0.05dt
Trunk orientation	$\|\|\hat{g}_{x}\|\|^{2}+\|\|\hat{g}_{y}\|\|^{2}$	-1dt
Trunk height	$\|\|h_{b}-h_{b}^{*}\|\|^{2}$	-5dt

TABLE I: Reward functions and weights

Parameter	Value
Learning rate	Adaptive
Batch size	98304 (4096 $\times$ 24)
Mini-batch size	24576 (4096 $\times$ 6)
Discount factor	0.99
GAE lambda	0.95
Desired KL-divergence	0.01

TABLE II: PPO hyper-parameters

	Parameter	Range
Randomized dynamics	Trunk mass	[-1,1] kg
	COM displacement	[-0.03, 0.03] m
	Ground friction coefficient	[0.25, 1.5]
Trunk impulse	External force magnitude $F_{x},F_{y}$	[-60, 60] N
	External force magnitude $F_{z}$	[-10, 10] N
	External force noise	[-2,2] N

TABLE III: Parameter setting for training

Module (MLP)	Input	Hidden Layers	Output
$\pi^{T}$	$o_{t},\hat{l}_{t}$	[512, 256, 128]	$\hat{a}_{t}$
$V$	$o_{t},F_{t},S_{t},T_{t}$	[512, 256, 128]	$V_{t}$
$E^{F}$	$F_{t}$	[64, 32]	$\hat{l}^{F}_{t}$
$E^{T}$	$T_{t}$	[256, 128]	$\hat{l}^{T}_{t}$
$E^{S}$	$S_{t}$	[64, 32]	$\hat{l}^{S}_{t}$
$\pi^{S}$	$o_{t},l_{t}$	[512, 256, 128]	$a_{t}$

TABLE IV: Network configurations

Fig. 3: The locomotion behavior when subjected to external force impulses from the front. The trunk velocity and height responses are provided in the second image from the top. The feet’s contact patterns with the ground (F/R denotes Front/Rear and R/L denotes Right/Left) are given in the third image from the top. The image at the bottom shows the plots of the force latent variables given by the force encoder.

Weights	Metrics	Algorithms
Weights	Metrics	Robust	SEFA	MEFA	PA-LOCO
2.2 kg	SR (%) LO (m) RT (s) HO (m)	90 (9/10)	0	$\mathbf{100}$ (10/10)	$\mathbf{100}$ (10/10)
		0.33	-	0.17	$\mathbf{0.16}$
		1.06	-	0.50	$\mathbf{0.45}$
		0.00	-	-0.01	$\mathbf{-0.02}$
4.6 kg	SR (%) LO (m) RT (s) HO (m)	43 (3/7)	0	80 (8/10)	$\mathbf{90}$ (9/10)

TABLE V: Metric results for different algorithms

Fig. 5: The t-SNE visualization of learned latent representation. In different trials, the robot is subjected to a constant backward force of different magnitudes when it moves forward. It indicates that the student policy is perturbation-aware due to the force encoder.

Fig. 6: The correlation heatmap between latent features and different force directions.

l^{T}

l^{S}

l^{F}

denote the latent features of the terrain profile

T_{t}

, robot state

S_{t}

, and external forces

F_{t}

respectively. In different trials, the robot experiences a constant external force of

20

N in different directions.

Fig. 7: Outdoor experiments. (a) Grass. (b) Stairs. (c) Slope.

RL for Legged Locomotion

Reinforcement learning for legged locomotion trains walking robots to move across varied terrain, adapt to disturbances, and carry out tasks.

See all 166 papers

Learning Perceptive Platform Adaptive Locomotion Controllers for Quadrupedal Robots
Jun 23, 2026David Rytz, Kim Tien Ly, Ioannis HavoutisRobot Skill Learning Cross-Embodiment Robot Learning
Dynamic Robot Systems, Oxford Robotics Institute, University of Oxford
Neuromorphic Reinforcement Learning for Quadruped Locomotion Control on Uneven Terrain
May 10, 2026Zhuangyu Han, Abhronil SenguptaEquilibrium Propagation Terrain-Aware Robot Locomotion
School of Electrical Engineering and Computer Science Penn State University University Park, PA 16802
LoComposition: Terrain-Adaptive Energy-Efficient Quadruped Locomotion without Gait Priors
Jun 14, 2026Loukas Kordos, Leonard T. Franz, Simon Rappenecker +4Legged Robotics RL for Robotics
Technical University of Munich. · University of Tübingen. · Hertie Institute for Clinical Brain Research & Center for Integrative Neuroscience. +1

Jun 23, 2026J/K move · Enter open · S save

Learning Perceptive Platform Adaptive Locomotion Controllers for Quadrupedal Robots

David Rytz, Kim Tien Ly, Ioannis Havoutis

Dynamic Robot Systems, Oxford Robotics Institute, University of Oxford

Robot Skill Learning Cross-Embodiment Robot Learning Terrain-Aware Robot Locomotion Robot Locomotion RL for Legged Locomotion Quadruped Locomotion

Open paper AskPDF arXiv

Universal quadrupedal locomotion remains limited by the difficulty of integrating perception across diverse robot morphologies. State-of-the-art controllers rely on single-robot training or blind policies that omit real-time perception, leading to poor cross-embodiment generalization. Designing locomotion policies that remain robust across related quadruped morphologies while incorporating perception is challenging. Moreover, fully perceptive policies are often sensitive to noise, whereas blind controllers lack terrain awareness. In this work, we study how perception should be integrated into morphology-aware reinforcement learning architectures for deployable quadrupedal control. Building on MorAL, we train morphology-specialized universal controllers on multiple reference quadrupeds using adaptive terrain curricula. We compare a blind baseline, a critic-perceptive variant (MorAL+), and a fully perceptive actor-critic (PPAL). Policies are evaluated in simulation on flat and rough terrains, and deployed on ANYmal hardware. Results show that critic-only perception improves robustness and tracking consistency over blind baselines while remaining more stable than fully perceptive policies under perception noise. These findings highlight that perception placement and curriculum design are key factors for scalable, morphology-aware locomotion.

Abstract

RL for Legged Locomotion

Explore similar work

Learning Perceptive Platform Adaptive Locomotion Controllers for Quadrupedal Robots

Neuromorphic Reinforcement Learning for Quadruped Locomotion Control on Uneven Terrain

LoComposition: Terrain-Adaptive Energy-Efficient Quadruped Locomotion without Gait Priors