SHOES: Sim-to-real Hardware-adaptation via On-policy Expansion of Skills for Legged Robots

Kota Tsuruhara; Tsung-Yuan Chen; Kento Kawaharazuka

cs.ROOct 12, 2026

SHOES: Sim-to-real Hardware-adaptation via On-policy Expansion of Skills for Legged Robots

Authors: Kota Tsuruhara, Tsung-Yuan Chen, Kento Kawaharazuka

Organizations: Tokyo University of Agriculture and Technology · The University of Tokyo

Abstract

Humans expand their locomotion capability by wearing appropriate footwear, such as hiking boots for mountains or skates for ice. Footwear changes the ground-contact interface and enables traversal of otherwise difficult terrain. In contrast, locomotion controllers are typically trained for a fixed morphology, and their performance can degrade when shoes alter foot geometry, mass, and friction. We propose SHOES, a sample-efficient framework for adapting locomotion policies to footwear-induced morphology changes. SHOES freezes a simulation-trained nominal policy and fine-tunes only lightweight, identity-initialized modulation modules on the robot, preserving the initial behavior while enabling efficient hardware adaptation. We also introduce SafeBox, a workspace-aware waypoint sampler that constrains online adaptation within a user-defined workspace. The simulation results with a shoe-equipped Unitree Go2 show that SHOES substantially recovers locomotion performance after footwear-induced morphology changes using limited additional adaptation. Under the same adaptation budget, SHOES outperforms FiLM-only, Linear Adapter-only, and residual-RL baselines. Real-world experiments on a Unitree Go2 fitted with human shoes further evaluate the effectiveness of SHOES for hardware deployment.

Figures & tables

Figure 2: Bare-foot nominal degrades with shoes attached. Commanded-direction velocity at the

0.3

m/s setpoint for the same bare-foot nominal policy, executed on bare feet (blue) and after attaching shoes (green); the dashed ring marks the target. The morphology shift induced by shoes particularly degrades backward-command tracking.

Figure 3: SHOES two-stage training pipeline. Stage 1 trains a nominal PPO actor-critic on the bare-foot Go2 in simulation with $4096$ parallel envs; the resulting $\pi_{\text{nom}}$ becomes the frozen base for Stage 2. Stage 2 wraps the frozen nominal MLP (gray, ${\sim}190$ k params) with a trainable FiLM generator and a Linear Adapter (coral, ${\sim}5.6$ k params); PPO online-fine-tunes the trainable modules on the shoe-equipped Go2 with the SafeBox sampler at $1$ env $\times\,30$ k env-steps.

Figure 4: SHOES recovers bare-foot tracking across speeds and directions. Per-direction tracked velocity at the three command speeds (

0.3,0.6,1.0

m/s). SHOES (red, best seed from Appendix F ) closely matches the bare-foot nominal reference (blue), while the no-adapt nominal on shoes (green) tracks well inside it across all directions and speeds.

FiLM	Adapter	Trainable	BFP $\uparrow$	$\Delta$ BFP $\uparrow$
✗	✗	$0$	$67.0$	$0.0$
✗	✓	$0.2$ k	$65.1\pm 8.9$	$-1.9$
✓	✗	$5.4$ k	$78.7\pm 10.8$	$+11.7$
✓	✓	$\mathbf{5.6}$ k	$\mathbf{93.5\pm 3.9}$	$\mathbf{+26.5}$

Table 1: Component ablation (

30

k-step budget).

\Delta

BFP is the gain over the no-adapt nominal (first row).

Figure 5: Method comparison (flat shoes). Bare-Foot Parity (%) mean

\pm

std over six seeds. Dashed line: no-adapt nominal.

Figure 6: Simulated footwear variants: (a) Curve Foot, (b) Single Geta, (c) Double Geta, and (d) Skates.

Table 8

Panel of Figure 7 — Figure 7: SHOES recovers backward-velocity tracking on hardware. Backward setpoint $v_{x}^{\text{cmd}}=-0.30$ m/s on the physical Unitree Go2 with flat shoes. (a) SHOES (red) substantially improves backward tracking over the no-adapt nominal on shoes (blue); the bare-foot nominal (green) is shown for reference (black dashed line: command). (b) Snapshots of the robot in the three settings overlaid in the left panel.

Figure 8: Real-robot traversal over a slotted surface. With shoes, the robot crosses the openings (top); without shoes, it becomes trapped (bottom).

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Value
Parallel environments	$1$
Steps per environment per rollout	$256$
PPO epochs per update	$3$
Mini-batches per epoch	$4$
Total PPO iterations	$117$
Total environment steps	$30{,}000$

Appendix

Table 4: Stage-2 PPO hyper-parameters.

Reward	Equation $r_{i}$	Weight $w_{i}$
Lin. velocity tracking	$\exp\!\bigl(-[\lVert\mathbf{v}^{\text{cmd}}_{xy}-\mathbf{v}_{xy}\rVert^{2}+2v_{z}^{2}]/\sigma_{\text{lin}}^{2}\bigr)$	$+1.0$
Ang. velocity tracking	$\exp\!\bigl(-[(\omega_{z}^{\text{cmd}}-\omega_{z})^{2}+0.05\lVert\boldsymbol{\omega}_{xy}\rVert^{2}]/\sigma_{\text{ang}}^{2}\bigr)$	$+1.0$
Body orientation	$\lVert\mathbf{g}^{\text{proj}}_{xy}\rVert^{2}$	$-1.0$
Posture	$\exp\!\bigl(-\tfrac{1}{\|\mathcal{J}\|}\sum_{j}(\theta_{j}-\theta_{j}^{\text{default}})^{2}/\sigma_{\text{pose},j}(\mathbf{v}^{\text{cmd}})^{2}\bigr)$	$+1.0$
Body angular velocity	$\lVert\boldsymbol{\omega}_{xy}\rVert^{2}$	$-0.05$
Angular momentum	$\lVert\mathbf{L}_{\text{root}}\rVert^{2}$	$-0.025$

Appendix

Table 5: Stage-1 rewards (bare-foot nominal training).

\mathbf{v}^{\text{cmd}}_{xy},\omega_{z}^{\text{cmd}}

: commanded linear/yaw velocity (body frame);

\mathbf{v}_{xy},v_{z},\boldsymbol{\omega}_{xy}

: base linear and angular velocities;

\mathbf{g}^{\text{proj}}_{xy}

: projected gravity (body horizontal);

\boldsymbol{\theta},\ddot{\boldsymbol{\theta}}

: joint positions and accelerations;

\boldsymbol{\theta}^{\text{default}}

: default joint pose;

\mathbf{L}_{\text{root}}

: root angular momentum;

p_{z}^{i},\mathbf{v}^{i}_{xy},\mathbf{F}^{i}

: foot

i

height, xy velocity (world), contact force;

c_{i}\in\{0,1\}

: stance indicator;

\eta_{i}\in\{0,1\}

: first-contact (landing) indicator;

\theta_{j}^{\text{low}},\theta_{j}^{\text{up}}

: soft joint position limits; “terminated” covers fall (base tilt

>70^{\circ}

) and non-foot contact (

>10

N);

T_{\text{phase}}=0.6

p_{z}^{\text{tgt}}=0.10

\sigma_{\text{lin}}^{2}=0.25

\sigma_{\text{ang}}^{2}=0.5

Reward	Equation $r_{i}$	Weight $w_{i}$
Lin. velocity tracking (adaptive)	$\exp\!\bigl(-[\lVert\mathbf{v}^{\text{cmd}}_{xy}-\mathbf{v}_{xy}\rVert^{2}+2v_{z}^{2}]/\sigma_{\text{lin}}(s)^{2}\bigr)$	$+10.0$
Ang. velocity tracking	$\exp\!\bigl(-[(\omega_{z}^{\text{cmd}}-\omega_{z})^{2}+0.05\lVert\boldsymbol{\omega}_{xy}\rVert^{2}]/\sigma_{\text{ang}}^{2}\bigr)$	$+2.0$

Appendix

Table 6: Stage-2 rewards.

\sigma_{\text{lin}}(s)

from Eq. ( 10 ),

\sigma_{\text{ang}}=0.25

Parameter	Value
Box half-width $a$ (x-axis)	$1.5$ m
Box half-width $b$ (y-axis)	$1.0$ m
Cruise-speed lower bound $v_{\text{cap}}^{\min}$	$0.1$ m/s
Cruise-speed upper bound $v_{\text{cap}}^{\max}(=v_{x}^{\max})$ / body-frame $\|v_{x}\|$ clip $v_{x}^{\max}$	$1.0$ m/s
Body-frame $\|v_{y}\|$ clip $v_{y}^{\max}$	$0.5$ m/s
Max $\|\omega_{z}\|$ ( $\omega_{\text{max}}$ )	$0.5$ rad/s

Appendix

Table 7: SafeBox sampler defaults.

Method	Trainable	BFP $\uparrow$	Best $\uparrow$
nominal (no adapt)	$0$	$67.0$	—
Adapter-only	$0.2$ k	$65.1\pm 8.9$	$76.1$
FiLM-only	$5.4$ k	$78.7\pm 10.8$	$90.3$
Residual SAC	${\sim}38$ k	$66.7\pm 0.2$	$66.8$
Residual PPO	${\sim}38$ k	$63.7\pm 8.3$	$81.8$
SHOES (Ours)	$\mathbf{5.6}$ k	$\mathbf{93.5\pm 3.9}$	$\mathbf{96.9}$

Appendix

Table 8: Method comparison on flat shoes. Mean

\pm

std and best Bare-Foot Parity (BFP) over six seeds.

Robot Policy Adaptation

Robot policy adaptation changes or steers learned robot behaviors to handle new tasks, environments, hardware, and user preferences.

See all 173 papers

Rapid Embodiment Adaptation for Quadrupedal Locomotion
Aug 2, 2026Dichen Li, Bo Ai, Nico Bohlinger +3Robot Dynamics Identification Robot Policy Adaptation
UC San Diego · Stanford University · TU Darmstadt +2
Simulator Adaptation for Sim-to-Real Learning of Legged Locomotion via Proprioceptive Distribution Matching
Apr 13, 2026Jeremy Dao, Alan FernSim-to-Real Transfer Quadruped Locomotion
Collaborative Robotics and Intelligent Systems Institute, Oregon State University, Corvallis, Oregon, 97331, USA.
LatentMimic: Terrain-Adaptive Locomotion via Latent Space Imitation
Apr 7, 2026Zhiquan Wang, Yunyu Liu, Dipam Patel +3Terrain-Aware Robot Locomotion Quadruped Locomotion
Department of Computer Science, Purdue University, USA

Aug 2, 2026J/K move · Enter open · S save

Rapid Embodiment Adaptation for Quadrupedal Locomotion

Dichen Li, Bo Ai, Nico Bohlinger, Jan Peters+2

UC San Diego · Stanford University · TU Darmstadt+2

Robot Dynamics Identification Robot Policy Adaptation RL for Legged Locomotion Quadruped Locomotion

Open paper AskPDF arXiv

Humans readily adapt their movements as their bodies change through aging, injury, or load carrying, but learning-based robot policies often break when hardware properties shift. We introduce an online embodiment adaptation framework for quadrupedal locomotion that infers embodiment parameters from short interaction histories and conditions control on the inferred hardware state. Our method pairs a generalist policy trained under embodiment randomization with a lightweight adaptation module that identifies physical changes within half a second. We evaluate two representative forms of embodiment variation: joint-range constraints and trunk-mass changes, corresponding to joint-level kinematic degradation and body-level dynamic variation. In simulation, the module accurately estimates these changes and enables closed-loop control that substantially outperforms policies conditioned directly on interaction history. On a real Unitree Go2 robot, our system maintains stable locomotion under severe instances of the evaluated changes, including a fully locked leg and a 5 kg payload, where non-adaptive methods fail. These results demonstrate the practicality of explicit online embodiment identification for rapid adaptation to joint-limit and payload-mass changes, and provide a step toward handling broader forms of uncertain, degraded, or changing robot hardware.

Abstract

Robot Policy Adaptation

Explore similar work

Rapid Embodiment Adaptation for Quadrupedal Locomotion

Simulator Adaptation for Sim-to-Real Learning of Legged Locomotion via Proprioceptive Distribution Matching

LatentMimic: Terrain-Adaptive Locomotion via Latent Space Imitation