Learning Thermal-Aware Locomotion Policies for an Electrically-Actuated Quadruped Robot

Letian Qian; Yuhang Wan; Shuhan Wang; Xin Luo

cs.ROMar 2, 2026

Learning Thermal-Aware Locomotion Policies for an Electrically-Actuated Quadruped Robot

Authors: Letian Qian, Yuhang Wan, Shuhan Wang, Xin Luo

Organizations: State Key Laboratory of Intelligent Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan, China

Abstract

Electrically-actuated quadrupedal robots possess high mobility on complex terrains, but their motors tend to accumulate heat under high-torque cyclic loads, potentially triggering overheat protection and limiting long-duration tasks. This work proposes a thermal-aware control method that incorporates motor temperatures into reinforcement learning locomotion policies and introduces thermal-constraint rewards to prevent temperature exceedance. Real-world experiments on the Unitree A1 demonstrate that, under a fixed 3 kg payload, the baseline policy triggers overheat protection and stops within approximately 7 minutes, whereas the proposed method can operate continuously for over 27 minutes without thermal interruptions while maintaining comparable command-tracking performance, thereby enhancing sustainable operational capability.

Figures & tables

Fig. 2: Whole-body thermal model of the quadruped robot.

Fig. 3: Overview of the proposed training framework. During training, all observations are obtained from the simulator. Among them, the output frequency of all networks and the thermal model are both 50 Hz, while the PD controller outputs torques to the actuators at 200 Hz within the simulation.

Parameters	Range [Min, Max]
Payload Mass	$[0,4]$
CoM Displacement	$[-0.1,0.1]\times[-0.1,0.1]\times[-0.1,0.1]$
External Force	$[-30,30]\times[-30,30]\times[-30,30]$
Ground Friction	$[0.2,1.25]$
Initial Joint Positions	$[0.5,1.5]\times\text{nominal value}$
System Delay	$[0,3\Delta t]$

TABLE I: Domain Randomizations and Respective Range

Reward Item	Equation	Weight
Linear velocity tracking	$\exp\!\Big(-\frac{\\|\boldsymbol{v}_{xy}^{\mathrm{cmd}}-\boldsymbol{v}_{xy}\\|_{2}^{2}}{\sigma}\Big)$	1.0
Angular velocity tracking	$\exp\!\Big(-\frac{(\omega_{\mathrm{yaw}}^{\mathrm{cmd}}-\omega_{\mathrm{yaw}})^{2}}{\sigma}\Big)$	0.8
Linear velocity (z)	$v_{z}^{2}$	-2.0
Angular velocity (roll-pitch)	$\boldsymbol{\omega_{xy}}^{2}$	-0.05
Orientation	$\\|\mathbf{g}\\|_{2}^{2}$	-0.2
Joint accelerations	$\\|\ddot{\boldsymbol{\theta}}\\|_{2}^{2}$	-2.5e-7

TABLE II: Rewards for Thermal-Aware Locomotion Policy

Motor Temperature Reward Function		Other Reward Function
$T_{\max}$	$60^{\circ}\mathrm{C}$	$\sigma$	0.25
$[T_{\min}^{\mathrm{clip}},\,T_{\max}^{\mathrm{clip}}]$	$[55^{\circ}\mathrm{C},\,65^{\circ}\mathrm{C}]$	$h^{\mathrm{target}}$	0.3 m
$\gamma_{T}$	0.35	$p_{z}^{\mathrm{target}}$	-0.2 m

TABLE III: Reward Parameters

Fig. 4: Comparison between baseline and proposed policy for continuous locomotion.

Fig. 5: Real-world experimental results. In the upper figure, the dashed line indicates the motor temperature threshold. Due to outdoor factors, it is not possible to ensure the same initial temperature in these experiments. The lower figure shows the robot’s base velocity under joystick control, which was estimated by the encoder network illustrated in Fig. 3 .

Fig. 6: Comparison of joint positions and torques of the robot during locomotion under the baseline and proposed policy. Torques are shown in absolute values, with black circles indicating the root-mean-square (RMS) values.

Fig. 7: Comparison of GRF curves during locomotion under the baseline and proposed policy. GRFs are expressed in a heading-aligned frame $\mathcal{H}$ , obtained by rotating the world frame to align its yaw angle with that of the robot body.

Fig. 8: Comparison of robot postures during locomotion.

Fig. 9: Comparison of the Jacobian matrices at different time instances. The horizontal axis represents the rows of the matrix, and the vertical axis represents the columns. The values are obtained by subtracting the absolute-value Jacobian matrix under the baseline from the proposed policy.

Robot Locomotion

Robot locomotion studies how robots move efficiently and reliably across varied terrains, environments, and movement modes.

See all 233 papers

Learning to Balance Motor Thermal Safety and Quadrupedal Locomotion Performance with Residual Policy
May 26, 2026Yuhang Wan, Weixian Lin, Letian Qian +5RL for Legged Locomotion Residual Policy Learning
School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan, 430074, China
Extending the Speed Limit of Quadrupedal Locomotion via Refined Actuator Modeling and Adaptive Command Scheduling
Sep 9, 2026Yucheng Tao, Shaowen Cheng, Guorong Lan +3Quadruped Locomotion Robotics
Center for X-Mechanics and Institute of Applied Mechanics, Zhejiang University, Hangzhou 310012, China · ZJU-Hangzhou Global Scientific and Technological Innovation Center, Hangzhou 311200, China · MirrorMe Robotics Co., Ltd., Hangzhou 311215, China
Learning Perceptive Platform Adaptive Locomotion Controllers for Quadrupedal Robots
Jun 23, 2026David Rytz, Kim Tien Ly, Ioannis HavoutisRobot Skill Learning Cross-Embodiment Robot Learning
Dynamic Robot Systems, Oxford Robotics Institute, University of Oxford

May 26, 2026J/K move · Enter open · S save

Learning to Balance Motor Thermal Safety and Quadrupedal Locomotion Performance with Residual Policy

Yuhang Wan, Weixian Lin, Letian Qian, Yiqi Zou+4

School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan, 430074, China

RL for Legged Locomotion Residual Policy Learning Quadruped Locomotion

Open paper AskPDF arXiv

Motor thermal management is often overlooked in the context of electrically-actuated robots, particularly legged robots, but motor overheating is a key factor that limits long-duration locomotion especially under payload conditions. This paper integrates a whole-body thermal model of a quadruped robot into the reinforcement learning pipeline to update motor temperatures, and proposes a two-stage training framework for motor thermal management. In this framework, a nominal policy is first pre-trained as a locomotion baseline capable of traversing diverse terrains. A residual policy is then trained on top of the nominal policy to provide corrective actions based on the robot's thermal state, ensuring high performance under low-temperature conditions and preventing motor overheating under high-temperature conditions. Simulation results demonstrate that the proposed policy achieves an effective balance between motor thermal safety and locomotion performance. Real-world experiments on a Unitree A1 quadruped robot further validate the approach: under a 3 kg payload, the robot achieves stable locomotion across multiple terrains for over 13 minutes, while the nominal policy alone leads to motor overheating in about 5 minutes.

Abstract

Robot Locomotion

Explore similar work

Learning to Balance Motor Thermal Safety and Quadrupedal Locomotion Performance with Residual Policy

Extending the Speed Limit of Quadrupedal Locomotion via Refined Actuator Modeling and Adaptive Command Scheduling

Learning Perceptive Platform Adaptive Locomotion Controllers for Quadrupedal Robots