Simulator Adaptation for Sim-to-Real Learning of Legged Locomotion via Proprioceptive Distribution Matching

Jeremy Dao; Alan Fern

cs.ROApr 13, 2026

Simulator Adaptation for Sim-to-Real Learning of Legged Locomotion via Proprioceptive Distribution Matching

Authors: Jeremy Dao, Alan Fern

Organizations: Collaborative Robotics and Intelligent Systems Institute, Oregon State University, Corvallis, Oregon, 97331, USA.

Abstract

Simulation trained legged locomotion policies often exhibit performance loss on hardware due to dynamics discrepancies between the simulator and the real world, highlighting the need for approaches that adapt the simulator itself to better match hardware behavior. Prior work typically quantify these discrepancies through precise, time-aligned matching of joint and base trajectories. This process requires motion capture, privileged sensing, and carefully controlled initial conditions. We introduce a practical alternative based on proprioceptive distribution matching, which compares hardware and simulation rollouts as distributions of joint observations and actions, eliminating the need for time alignment or external sensing. Using this metric as a black-box objective, we explore adapting simulator dynamics through parameter identification, action-delta models, and residual actuator models. Our approach matches the parameter recovery and policy-performance gains of privileged state-matching baselines across extensive sim-to-sim ablations on the Go2 quadruped. Real-world experiments demonstrate substantial drift reduction using less than five minutes of hardware data, even for a challenging two-legged walking behavior. These results demonstrate that proprioceptive distribution matching provides a practical and effective route to simulator adaptation for sim-to-real transfer of learned legged locomotion.

Figures & tables

	Hip Friction	Thigh Friction	Calf Friction	Hip Armature	Thigh Armature	Calf Armature
True Values	0.2	0.15	0.1	0.1	0.05	0.075
Wass	0.21	0.153	0.116	0.1001	0.0496	0.0741
MatchS(1)	0.2	0.15	0.09999	0.1	0.04999	0.075
MatchS(20)	0.1999	0.15001	0.0999	0.0999	0.04999	0.07499
MatchO( $\infty$ )	0.1999	0.15001	0.09999	0.09998	0.04997	0.07501

TABLE I : Optimized Joint Friction and Armature Values with different cost functions under idealized target environment conditions.

Panel of Figure 2 — Fig. 1 : Finetuned policy performance for the Model Parameter Shift sim-to-sim test scenario. We average values over 10,000 trajectories with random commands. Error bars represent a 95% confidence interval.

Noise Level	Cost Function	Hip Fric	Thigh Fric	Calf Fric	Hip Arm	Thigh Arm	Calf Arm	Average Error
$\sigma=1.0$	$MatchS(\infty)$	0.450%	3.787%	24.820%	7.400%	2.600%	4.504%	7.2602
	$MatchO(\infty)$	17.130%	6.047%	21.220%	5.317%	33.800%	7.665%	15.1965%
	Wass	0.975%	11.673%	6.680%	6.000%	15.028%	7.467%	7.9705%
$\sigma=2.5$	$MatchS(\infty)$	6.160%	10.147%	73.940%	0.830%	1.924%	0.440%	15.5735%
	$MatchO(\infty)$	26.635%	19.307%	101.700%	19.846%	61.294%	15.561%	40.7238%
	Wass	16.380%	1.540%	20.441%	5.972%	1.538%	4.172%	8.3405

TABLE II : Percent error in optimized Joint Friction and Armature values with different cost functions on noisy data.

\sigma

represents the standard deviation (in

N\cdot m

of Gaussian torque noise applied at each simulatino step.

Panel of Figure 4 — Fig. 2 : Finetuned policy performance for the Spring Joint sim-to-sim test scenario. We average values over 10,000 trajectories with random commands. Error bars represent a 95% confidence interval.

Fig. 3 : A Unitree Go2 quadruped used in sim-to-real experiments. We tie a spring between the foot and hip joint of the rear right leg, affecting the dynamics of the calf joint. We use this to emulate our Spring Joint test on physical hardware.

Fig. 4 : A Unitree Go2 quadruped walking on its hind legs. This is the behavior we attempt to transfer to hardware in our Bipedal Walking sim-to-real experiment.

TABLE III : Tracking performance of

\pi_{0}

and the finetuned policy

\pi_{1}

on hardware for the Spring Joint setting. ”N/A” means that the policy failed.

TABLE IV : Tracking performance of

\pi_{0}

and the finetuned policy

\pi_{1}

on hardware for the Bipedal Walking setting.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Ratio Clipping	0.2
Entropy Loss Weighting	0.01
Learning Rate	$1$ x $10^{-3}$
Discount Factor	0.99
GAE $\lambda$	0.95
KL Threshold	0.01
Maximum Gradient Norm	1.0

Appendix

TABLE V : PPO training hyperparameters.

Sim-to-Real Transfer

Simulation-to-reality transfer adapts models and policies trained in simulation for reliable performance in physical or otherwise real-world settings.

See all 316 papers

Isaac Sim-to-Real: Reinforcement Learning based Locomotion for Quadrupeds
Jul 20, 2026Jordan Dowdy, Jean Chagas VazSim-to-Real Transfer Whole-Body Control
Department of Electrical and Computer Engineering, University of Louisville · Faculty of Electrical and Computer Engineering, University of Louisville, Louisville, KY, 40208, USA
Learning Task-Invariant Properties via Dreamer: Enabling Efficient Policy Transfer for Quadruped Robots
Apr 3, 2026Junyang Liang, Yuxuan Liu, Yabin Chang +5Sim-to-Real Transfer Robot Policy Adaptation
Shenzhen University, Shenzhen, China · Peng Cheng Laboratory, Shenzhen, China
SHOES: Sim-to-real Hardware-adaptation via On-policy Expansion of Skills for Legged Robots
Oct 12, 2026Kota Tsuruhara, Tsung-Yuan Chen, Kento KawaharazukaRobot Policy Adaptation Robot Locomotion
Tokyo University of Agriculture and Technology · The University of Tokyo

Jul 20, 2026J/K move · Enter open · S save

Isaac Sim-to-Real: Reinforcement Learning based Locomotion for Quadrupeds

Jordan Dowdy, Jean Chagas Vaz

Department of Electrical and Computer Engineering, University of Louisville · Faculty of Electrical and Computer Engineering, University of Louisville, Louisville, KY, 40208, USA

Sim-to-Real Transfer Whole-Body Control RL for Legged Locomotion Quadruped Locomotion

Open paper AskPDF arXiv

Learning-based approaches to locomotion have risen in popularity in recent years, showing the capability for complex legged locomotion and whole-body control. Reinforcement learning (RL), the primary learning-based approach for locomotion, often utilizes a high-performance simulation tool, providing a controlled and efficient training and development environment. However, policies that perform well in simulation frequently encounter unexpected challenges when deployed on a physical system, known as the sim-to-real gap. This work presents a robust RL locomotion framework capable of whole-body control. The proposed RL framework utilizes Nvidia's new set of simulation tools, Isaac Sim, and its companion RL framework, Isaac Lab, for training, achieving a zero-shot sim-to-real policy. The performance of our policy is validated on physical hardware using the Unitree Go1, with experimental results showing similar velocity tracking performance to the quadruped's integrated controller, with a greater ability to recover from large disturbances, and achieve linear velocities of 2.0 m/s and angular velocities of 1.8 rad/s.

Abstract

Sim-to-Real Transfer

Explore similar work

Isaac Sim-to-Real: Reinforcement Learning based Locomotion for Quadrupeds

Learning Task-Invariant Properties via Dreamer: Enabling Efficient Policy Transfer for Quadruped Robots

SHOES: Sim-to-real Hardware-adaptation via On-policy Expansion of Skills for Legged Robots