cs.ROApr 1, 2026

Physical Imitation Learning: Distilling Control Policies into Passive Elasticity

Authors: Huyue Ma, Yurui Jin, Helmut Hauser, Rui Wu

Organizations: School of Engineering Mathematics and Technology, University of Bristol, and the Bristol Robotics Laboratory, BS8 1QU, United Kingdom

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Due to brain-body co-evolution, animals' intrinsic body dynamics play a crucial role in their energy-efficient locomotion. Specifically, the control effort is shared between active muscles and passive body dynamics--a principle often referred to as Physical Intelligence. As a result, the body dynamics are part of the solution. In contrast, robot bodies are typically designed to be as simple as possible, but the active control often fights the intrinsic body dynamics, resulting in low energy-efficiency. We introduce Physical Imitation Learning (PIL), a novel approach that brings current robotics control closer to animals. PIL takes learned control policies obtained with Reinforcement Learning (RL) and systematically splits them up into an active and passive control contribution. The passive part can be then directly offloaded to passive Parallel Elastic Joints (PEJs). As a result, the active control contribution is significantly reduced, lowering the overall energy consumption. Furthermore, the policy can be trained via RL to leverage the PEJ assistance by generating gaits that are more readily emulated by the PEJs. This enables co-design of the active and passive control components, shifting a greater share of actuation effort to the PEJs. Here we demonstrate the potential of this approach in simulated quadrupeds. Our results show that the proposed approach can offload up to 95% of mechanical power to passive body dynamics on flat terrain and 13% on rough terrain. PIL thereby provides a generalisable route to task-specific Physical Intelligence applicable to a wide range of joint-based robot morphologies.

Figures & tables

Figure 1: The Physical Imitation Learning pipeline. From left to right: the learned control policy is trained via RL with a Cost-of-Transport reward. Actuator torque-angle history recorded from each joint are distilled into nonlinear Parallel Elastic Joint (PEJ) characteristic profiles. The residual torque after PEJ subtraction defines the active policy for motor commands. The PEJs can be physically realised using cam mechanism.
Figure 2: Distillation procedure. The joint torque and angle time series are recorded from the parallel simulation of 4,096 robots walking under various speed commands. Top: these time series are mapped to a torque–angle scatter plot, and a piecewise-linear PEJ profile is fitted via bin-wise optimisation to minimise the residual motor power consumption, which is physically implementable using a cam mechanism. Bottom: at each time step, the PEJ torque τPEJ(t)\tau_{\text{PEJ}}(t) is subtracted from the total demanded torque τtotal(t)\tau_{\text{total}}(t) , leaving a residual motor torque τresidual(t)\tau_{\text{residual}}(t) .
Figure 3: Terrain-specific power reduction. (a) Power decomposition of each terrain-specific co-design policy. The stacked bars show total power consumption split into the active motor power (red) and the PEJ contribution (blue), the consumption include the hip motors. (b) Offload percentage versus training terrain level. Green circles: co-design policy (with PEJ vs. without PEJ). Yellow triangles: offload percentage obtained by fitting a PEJ profile post-hoc to the reference policy. (c) Visual examples of flat terrain and level 6 terrain, showing the terrain primitives and their proportions, the full terrain composition is shown in Fig. 8 .
Figure 4: Mean absolute velocity tracking error heatmap. Each cell shows the tracking error when deploying a co-design policy trained on the terrain indicated by the row (P xx ) and evaluated on the terrain indicated by the column (T xx ). Bordered diagonal cells correspond to each policy evaluated on its own training terrain.
Figure 5: Distilled PEJ profiles. Joint names follow the Unitree/IsaacLab URDF convention: thigh denotes the hip flexion/extension joint and calf denotes the knee flexion/extension joint. (a)–(c) PEJ profiles (dashed lines) for the four symmetric joint groups (columns), overlaid on motor torque–angle data points (dots) recorded from the learned policy during locomotion, with 50,00050{,}000 data points each. (a) and (b): co-design and reference policies on flat terrain. (c): co-design policy on level 6 terrain. Tighter clustering of the data points around the profile indicates a more periodic gait and greater passive offloading. (d): co-design PEJ profiles overlaid for all seven terrain levels.
Figure 6: Cross-terrain power offload percentage. Each profile represents a co-design policy trained on a specific terrain level and evaluated across all terrain levels. Policies trained on easier terrain achieve higher offload percentages but degrade faster on harder terrain.
Figure 7: Conceptual design of a manufacturable PEJ, shown for the rear-calf joint group (the most demanding of the four). (a) A cam fixed to the joint shaft bears on a roller-tipped, spring-loaded linear follower; as the joint rotates, the cam geometry shapes the deflection of a single preloaded compression spring, producing the prescribed nonlinear elastic profile. Dimensions shown are for the rear-calf design. (b) Target versus realised spring elastic energy as a function of joint angle. (c) Target versus realised joint torque as a function of joint angle. In both (b) and (c) the realised profiles, obtained with a single off-the-shelf steel compression spring, reproduce the distilled targets with negligible error, confirming physical realisability. Full design parameters are given in Appendix D .
Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Reward termWeight
Velocity tracking
track_lin_vel_xy_exp+2.5+2.5
track_ang_vel_z_exp+0.5+0.5
Body stability
lin_vel_z_l2−2.0-2.0
ang_vel_xy_l2−0.05-0.05
Appendix
Table 1: Reward weights for continued training (Stage 2). CoT weights are in Table 2 .
 FlatL1L2L3L4L5L6
Co-design2.01.00.90.50.40.20.1
Reference0.60.60.50.40.30.150.1
Appendix
Table 2: CoT weight α\alpha for each terrain level.
LevelRandom roughBoxesStairs
Flat———
12.0 – 2.9——
22.9 – 3.86.7 – 8.3—
33.8 – 4.78.3 – 10.0—
44.7 – 5.610.0 – 11.711 – 13
55.6 – 6.411.7 – 13.313 – 15
Appendix
Table 3: Terrain specifications per difficulty level. All heights are in centimetres.
Figure 8: Terrain types and difficulty levels used in simulation. Each level is a mixture of five procedurally generated primitives, with the percentage of each type indicated. As the level increases, harder primitives occupy a larger fraction and their dimensions grow.
 FlatL1L2L3L4L5L6
Co-design policy
Before (W)21.719.624.326.831.039.449.5
After (W)1.123.708.6516.622.332.343.0
Offload (%)94.881.264.438.128.018.213.1
Reference policy
Before (W)18.521.322.625.429.838.248.6
Appendix
Table 4: Average motor power consumption (in watts) before and after PEJ assistance, the power consumption includes the hip motors.
More in this topic

Energy-Efficient Robot Locomotion

Energy-efficient robot locomotion studies how robot mechanics and control can reduce power use and cost of transport across terrains and movement modes.

See all 32 papers

Explore similar work

CardsList
Apr 16, 2026

Model-Based Reinforcement Learning Exploits Passive Body Dynamics for High-Performance Biped Robot Locomotion

Embodiment is a significant keyword in recent machine learning fields. This study focused on the passive nature of the body of a biped robot to generate walking and running locomotion using model-based deep reinforcement learning. We constructed two models in a simulator, one with passive elements (e.g., springs) and the other, which is similar to general humanoids, without passive elements. The training of the model with passive elements was highly affected by the attractor of the system. This lead that although the trajectories quickly converged to limit cycles, it took a long time to obtain large rewards. However, thanks to the attractor-driven learning, the acquired locomotion was robust and energy-efficient. The results revealed that robots with passive elements could efficiently acquire high-performance locomotion by utilizing stable limit cycles generated through dynamic interaction between the body and ground. This study demonstrates the importance of implementing passive properties in the body for future embodied AI.

Tomoya Kamimura, Haruka Washiyama, Akihito Sano

Department of Mechanical Science and Bioengineering, Graduate School of Engineering and Science, The University of Osaka, Toyonaka, Japan. · Department of ElectricalTechnology,and MechanicalAichi,Engineering,Japan. Nagoya Institute of

Robot LocomotionRL for Legged LocomotionModel-Based RL
AskPDF
Jul 20, 2026

Isaac Sim-to-Real: Reinforcement Learning based Locomotion for Quadrupeds

Learning-based approaches to locomotion have risen in popularity in recent years, showing the capability for complex legged locomotion and whole-body control. Reinforcement learning (RL), the primary learning-based approach for locomotion, often utilizes a high-performance simulation tool, providing a controlled and efficient training and development environment. However, policies that perform well in simulation frequently encounter unexpected challenges when deployed on a physical system, known as the sim-to-real gap. This work presents a robust RL locomotion framework capable of whole-body control. The proposed RL framework utilizes Nvidia's new set of simulation tools, Isaac Sim, and its companion RL framework, Isaac Lab, for training, achieving a zero-shot sim-to-real policy. The performance of our policy is validated on physical hardware using the Unitree Go1, with experimental results showing similar velocity tracking performance to the quadruped's integrated controller, with a greater ability to recover from large disturbances, and achieve linear velocities of 2.0 m/s and angular velocities of 1.8 rad/s.

Jordan Dowdy, Jean Chagas Vaz

Department of Electrical and Computer Engineering, University of Louisville · Faculty of Electrical and Computer Engineering, University of Louisville, Louisville, KY, 40208, USA

Sim-to-Real TransferWhole-Body ControlRL for Legged LocomotionQuadruped Locomotion
AskPDF
May 2, 2026

Dynamics Aware Quadrupedal Locomotion via Intrinsic Dynamics Head

Quadrupedal locomotion plays a critical role in enabling agile, versatile movement across complex terrains. Understanding and estimating the underlying physical dynamics are essential for achieving efficient and stable quadrupedal locomotion. We propose a novel training framework for quadrupedal locomotion that enables the Control Policy to understand and reason about physical dynamics. In simulation, we concurrently train an Intrinsic Dynamics (ID) Head that learns state-to-torque dynamics alongside the Control Policy, and we define a dynamics reward enabled by the ID Head that encourages the Policy toward more predictable dynamical behavior. We also provide a mechanism to tune the learned dynamics in the resulting Policy by controlling the training coefficients of the ID Head. Our simulation experiments show that this mechanism drives convergence to better optima across a wide range of standard quadrupedal locomotion rewards, yielding more efficient and smoother policies. Our real-robot experiments demonstrate sim-to-real transfer of these improvements, with significant gains in torque efficiency (16.8%), action rate (18.6%), and mechanical power (12.8%), while improving safe torque occupancy by 6.4%.

Aman Arora, Nalini Ratha

Department of Computer Science, University at Buffalo, The State University of New York, Buffalo, USA

Robot Dynamics LearningRL for Legged LocomotionInverse DynamicsQuadruped LocomotionRobot Learning
AskPDF