Learning Whole-Body Control for a Salamander Robot

Mengze Tian; Qiyuan Fu; Chuanfang Ning; Javier Jia Jie Pey; Auke Ijspeert

cs.ROMar 17, 2026

Learning Whole-Body Control for a Salamander Robot

Authors: Mengze Tian, Qiyuan Fu, Chuanfang Ning, Javier Jia Jie Pey, Auke Ijspeert

Organizations: Biorobotics Laboratory (BIOROB), École polytechnique fédérale de Lausanne (EPFL), Lausanne 1015, Switzerland

Abstract

Amphibious legged robots inspired by salamanders are promising in applications in complex amphibious environments. However, despite the significant success of training controllers that achieve diverse locomotion behaviors in conventional quadrupedal robots, most salamander robots relied on central-pattern-generator (CPG)-based and model-based coordination strategies for locomotion control. Learning unified joint-level whole-body control that reliably transfers from simulation to highly articulated physical salamander robots remains relatively underexplored. In addition, few legged robots have tried learning-based controllers in amphibious environments. In this work, we employ Reinforcement Learning to map proprioceptive observations and commanded velocities to joint-level actions, allowing coordinated locomotor behaviors to emerge. To deploy these policies on hardware, we adopt a system-level real-to-sim matching and sim-to-real transfer strategy. The learned controller achieves stable and coordinated walking on both flat and uneven terrains in the real world. Beyond terrestrial locomotion, the framework enables transitions between walking and swimming in simulation, highlighting a phenomenon of interest for understanding locomotion across distinct physical modes.

Figures & tables

Fig. 1: Overview of the learned salamander locomotion across diverse terrains. Top row: traversal from cobblestones to grass. Second row: traversal from stepped terrain, descending a height drop to flat ground. Third row: transition from land to water in simulation. Bottom row: locomotion on slippery sand (left) and rough terrain in the lab (right).

Fig. 2: Control architecture of the learned joint-space controller. The policy receives commanded planar velocity $\mathbf{v}_{\mathrm{cmd}}$ and a proprioceptive observation $\mathbf{o}_{t}=(\mathbf{q},\Delta\mathbf{q},\bm{\omega},\mathbf{g},\bm{\phi})$ , together with a short history of past actions. The policy outputs joint angle residuals $\Delta\mathbf{q}_{\mathrm{des}}$ , which are added to a nominal joint angle $\mathbf{q}_{\mathrm{nominal}}$ to form the desired joint angle $\mathbf{q}_{\mathrm{des}}$ . A low-level joint-space PD controller tracks $\mathbf{q}_{\mathrm{des}}$ and produces joint torques $\bm{\tau}$ .

Reward Term	Expression
Linear Velocity Tracking	$r_{v}=\exp\!\left(-\frac{\left\lVert\mathbf{v}_{\mathrm{cmd},xy}-\mathbf{v}_{b,xy}\right\rVert^{2}}{\sigma_{v}}\right)$
Angular Velocity Tracking	$r_{\omega}=\exp\!\left(-\frac{\left(\omega_{\mathrm{cmd},z}-\omega_{b,z}\right)^{2}}{\sigma_{\omega}}\right)$
Energy Cost	$r_{\text{energy}}=-\sum_{i}\big\|\dot{q}_{i}\,\tau_{i}\big\|$
Feet Phase Consistency	$r_{\text{phase}}=\exp\!\left(-\frac{\sum_{j}\left(z^{\text{foot}}_{j}-z^{\text{ref}}_{j}(\phi_{j})\right)^{2}}{\sigma_{\text{phase}}}\right)$

TABLE I: Reward Functions

Parameter	Value
Training steps	$3\times 10^{8}$
Number of environments	8192
Episode length	1000
Discount factor $\gamma$	0.97
Learning rate	$3\times 10^{-4}$
Entropy coefficient	$1\times 10^{-2}$

TABLE II: PPO Training Configuration

Fig. 3: Training terrains used for locomotion learning with the robot having a maximum ground clearance of $10.2\,\mathrm{cm}$ when all tibia links are vertical. (i) Hill terrain with the max slope angle of $15^{\circ}$ . (ii) Rugged terrain with the max height variation of $4\,\mathrm{cm}$ . (iii) Valley terrain with the max slope angle of $15^{\circ}$ .

Fig. 4: Kinematic modeling of structural backlash. In addition to the actuated joint rotation about the $z$ -axis (blue), two auxiliary passive rotational degrees of freedom about the $x$ -axis and $y$ -axis (orange and green) are introduced at the front and back girdle connections to model accumulated mechanical clearances. All other joints in the model remain rigid and do not include additional passive degrees of freedom.

Fig. 5: Training environment for the transition task. Blue regions indicate water areas.

Fig. 6: Real-world omnidirectional locomotion behaviors. (A) Forward walking under a constant forward velocity command, exhibiting coordinated axial–limb propulsion. (B) Turning behavior driven by a nonzero yaw-rate command, where asymmetric limb coordination induces rotational motion. (C) Diagonal walking resulting from simultaneous forward and lateral velocity commands, demonstrating continuous adaptation of whole-body coordination to mixed-direction motion.

Fig. 7: The robot successfully traverses rough-medium terrain. The terrain consists of inclined ramps and fragmented surface patches with varying elevations. The arrow indicates the direction of motion over time.

Terrain	Ruggedness (cm)	Clearance (%)	Slope (°)
Flat	0	0.0	0
Rough–Easy	2	19.6	15
Rough–Medium	4	39.2	15
Rough–Hard	4	39.2	25

TABLE III: Terrain definitions and corresponding real-world forward velocity statistics. Each result includes data aggregated from at least three trials. Clearance (%) is defined as terrain ruggedness normalized by the robot’s ground clearance.

Fig. 8: (A) Joint angle tracking of two representative joints on the front-left (FL) leg: the hip joint (left) and the foot joint (right). The cyan curve denotes the target command (Goal), the purple curve the simulated response (Sim), and the orange curve the real robot measurement (Real). (B) Visualization of modeling backlash.

Fig. 9: Sim-to-real tracking evaluation of the robot forward velocity. The yellow curve (cmd vx) denotes the commanded forward velocity, the blue curve (sim vx) represents the velocity in simulation, and the purple curve (real vx) shows the velocity measured on the real robot.

Fig. 10: Snapshots of the robot during the transition from land locomotion to swimming. The sequence (i–iv) illustrates the robot approaching the water, entering the water region, and progressively shifting from limb-driven walking to body-driven undulatory swimming.

Fig. 11: Transition from terrestrial to aquatic locomotion in simulation, where $x_{i}$ represents the joint angle. (Top) Body joint trajectories for the spine and tail joints. (Middle) Leg joint trajectories for the four limbs (HL, HR, FL, FR). (Bottom) Binary foot contact states.

Robot Locomotion

Robot locomotion studies how robots move efficiently and reliably across varied terrains, environments, and movement modes.

See all 233 papers

Isaac Sim-to-Real: Reinforcement Learning based Locomotion for Quadrupeds
Jul 20, 2026Jordan Dowdy, Jean Chagas VazSim-to-Real Transfer Whole-Body Control
Department of Electrical and Computer Engineering, University of Louisville · Faculty of Electrical and Computer Engineering, University of Louisville, Louisville, KY, 40208, USA
PolyMorph: Learning Locomotion Policy for Cross-Embodiment Adaptation
Oct 14, 2026Hyunsik Oh, Seunghun Jeon, Donghoon Youm +5Robot Policy Learning Cross-Embodiment Robot Learning
Learning Holistic Whole-Body Loco-Manipulation with a Bipedal Mobile Manipulator
Sep 16, 2026Zhongyu Chen, Yuxuan Nai, Qian Chen +8Whole-Body Robotic Manipulation Humanoid Loco-Manipulation
ZJU-UIUC Institute. · LimX Dynamics.

Jul 20, 2026J/K move · Enter open · S save

Isaac Sim-to-Real: Reinforcement Learning based Locomotion for Quadrupeds

Jordan Dowdy, Jean Chagas Vaz

Department of Electrical and Computer Engineering, University of Louisville · Faculty of Electrical and Computer Engineering, University of Louisville, Louisville, KY, 40208, USA

Sim-to-Real Transfer Whole-Body Control RL for Legged Locomotion Quadruped Locomotion

Open paper AskPDF arXiv

Learning-based approaches to locomotion have risen in popularity in recent years, showing the capability for complex legged locomotion and whole-body control. Reinforcement learning (RL), the primary learning-based approach for locomotion, often utilizes a high-performance simulation tool, providing a controlled and efficient training and development environment. However, policies that perform well in simulation frequently encounter unexpected challenges when deployed on a physical system, known as the sim-to-real gap. This work presents a robust RL locomotion framework capable of whole-body control. The proposed RL framework utilizes Nvidia's new set of simulation tools, Isaac Sim, and its companion RL framework, Isaac Lab, for training, achieving a zero-shot sim-to-real policy. The performance of our policy is validated on physical hardware using the Unitree Go1, with experimental results showing similar velocity tracking performance to the quadruped's integrated controller, with a greater ability to recover from large disturbances, and achieve linear velocities of 2.0 m/s and angular velocities of 1.8 rad/s.

Abstract

Robot Locomotion

Explore similar work

Isaac Sim-to-Real: Reinforcement Learning based Locomotion for Quadrupeds

PolyMorph: Learning Locomotion Policy for Cross-Embodiment Adaptation

Learning Holistic Whole-Body Loco-Manipulation with a Bipedal Mobile Manipulator