Reference-Free Humanoid Skateboarding via Termination-Shaped Reinforcement Learning
Authors: Ganeshvar Soenessardien, Michael Przystupa, Kevin Sebastian Luck
Organizations: Department of Artificial Intelligence, Faculty of Science, Vrije Universiteit Amsterdam, De Boelelaan 1105, 1081 HV Amsterdam, The Netherlands.
Humanoid locomotion research has advanced rapidly on walking and running, but \emph {object-mediated} locomotion, in which a robot must coordinate its body with a passive external object, remains difficult. One challenging task is skateboarding, which requires simultaneous interaction with the ground and a passive but movable platform to achieve forward motion. Current methods for tackling skateboarding with humanoid robots either require human demonstrations or utilise analytic steering models that require system identification. In this paper, we introduce a deep reinforcement learning framework that does not rely on prior data such as demonstrations and enables robots such as the K1 humanoid to learn to skateboard using model-free learning. The method combines task rewards with exploit-blocking terminations. We introduce a number of constraints restricting behaviours such as losing the planted board foot, standing with both feet on the board, excessive twisting, separation from the board, and unstable board motion. Their changing prevalence during training produces an implicit progression from stance control towards sustained robot--board locomotion.
Figures & tables
Fig. 1 : This paper investigates the problem of learning to skateboard on mobile platforms (above: front- and back-view) in humanoids (K1 Booster) with model-free reinforcement learning.
Fig. 2 : Initial stance of the robot at s0 .
Observation (dimension)
Noise half-range b
Delay (steps)
Base linear/angular velocity (3+3)
0.5 m/s; 0.2 rad/s
1–2
Projected gravity (3)
0.05
–
Joint position/velocity (22+22)
0.01 rad; 1.5 rad/s
–
Previous action, command (22+3)
–
–
Board relative position (3)
0.02 m
1–3
Board planar velocity, yaw rate (2+1)
0.1 m/s; 0.1 rad/s
1–3
TABLE I : Actor observations. Noise is independently sampled additive uniform noise U[−b,b] . Delays are sampled in discrete 50-Hz control steps (20 ms per step).
Term
Weight
Purpose
Upright
+12
Trunk verticality
Trunk height
+15
Reward height near 0.55m
Board distance
+40
Forward board motion
Body progress
+15
Forward trunk motion in XOR
XOR stance
+5
Maintain push stance
Near ground
+4
Keep off-board foot low
TABLE II : Reward terms used for skateboarding.
Termination
Threshold
Role
board_foot_displaced
displacement >5 cm
Keep board foot planted
board_out_of_bounds
distance >1.4 m
Bound forward travel
both_feet_on_board
both feet on deck
Block standing on board
robot_board_yaw_misaligned
yaw error >0.4 rad
Block twisting
trunk_too_low
trunk z<0.40 m
Block collapsed postures
robot_far_from_board
distance >0.7 m
Block separation
TABLE III : Early terminations. Their changing prevalence produces the implicit curriculum.
Fig. 3 : Learning progress across three experiments (mean ±1 sample std.). Longer episodes indicate progressively more stable rollouts.
Fig. 4 : Emergent skating mechanics across training. Board speed, grounded body progress and trunk speed rise together while the XOR-stance rate saturates near 0.98 , indicating genuine robot–board locomotion rather than board kicking. Bands show ±1 standard deviation.
Fig. 5 : Failure modes driving skill acquisition. Per-iteration termination counts (mean ±1 standard deviation). The dominant failure shifts from board-foot displacement (a stance control problem, early) to board out-of-bounds and time outs (a locomotion problem, late), while both-feet-on-board and yaw exploits stay low– the implicit curriculum made visible.
Run
Reward
Ep. len.
Board
Push
Trunk
XOR
OOB
(steps)
(m/s)
(m/s)
(m/s)
rate
/iter
0
305.96
479.96
0.359
0.394
0.405
0.981
5.83
1
352.45
582.92
0.185
0.210
0.216
0.982
2.63
2
280.71
433.49
0.267
0.287
0.304
0.966
4.83
Mean
313.04
498.79
0.270
0.297
0.308
0.976
4.43
Std.
36.39
76.47
0.087
0.092
0.095
0.009
1.64
TABLE IV : Final scalar outcomes for the three clean seeds.
Fig. 6 : Side view of the propulsive stepping sequence. Six frames show the movement of the propulsive foot from load/contact through early push, toe-off, foot lift and repositioning to re-contact.
Fig. 7 : Motion of the robot–board system through space. Overhead (left) and oblique-left (right) views show snapshots of the robot and board as they move through the simulation. The orange line traces the board-centre trajectory, and the green marker indicates the starting position.
As humanoid robots become increasingly dynamic, coupling them with reinforcement learning offers a promising approach to solving the complex, underactuated mechanics of passive inline skating. Equipping a humanoid robot with passive inline skating wheels presents an opportunity to combine the versatile agility of humanoids with the high-speed, energy-efficient locomotion strategies utilized by human skaters. In this paper, we train and deploy a reinforcement learning control policy that enables novel locomotion strategies for a humanoid robot modified to equip consumer inline skates instead of conventional feet. Unlike previous work limited to quadrupedal robots or actively driven wheels, our system allows for precise 6-DoF control of the skates to execute dynamic, edge-driven propulsion strategies. Our skating strategies emerge entirely from our reward structure, without reliance on human motion data, imitation learning, or kinematic priors. We overcome the inherent instability of passive wheels and simulation contact artifacts by utilizing different geometric wheel models (spherical and ellipsoidal) during training and validation, along with a custom success-based command curriculum and a specialized rolling reward. Consequently, our policy demonstrates up to a 50% reduction in Cost of Transport (CoT) compared to standard walking gaits. The resulting policy successfully transfers zero-shot to the physical Booster T1 hardware. Real-world deployments demonstrate dynamic balance, the ability to reject active physical perturbations, and agile locomotion strategies capable of turning at speed. A video of our results can be found at https://www.youtube.com/watch?v=-_APcOS7uFo.
Ethan Marot, Thomas Bi, Clemens Schwarke +3
Institute for Dynamic Systems and Control, ETH Zurich, Switzerland. · Robotic Systems Lab, ETH Zurich, Switzerland
Humanoid roller-skating is difficult because the robot must coordinate whole-body balance, rolling contacts, and velocity-dependent posture regulation. This paper presents an adversarial motion prior based reinforcement learning framework for two humanoid roller-skating gaits: Pump Glide skating and Push Glide skating. The two gait datasets are collected independently through motion capture and retargeted to the humanoid robot separately. The retargeted data are then smoothed and resampled into reference motion states for AMP training. The two gaits are learned by independent AMP training pipelines with separate reference datasets, separate policies, and independent reward architectures. Simulation experiments are designed to evaluate gait quality, velocity tracking, turning, and gait-specific reward ablations.
Yunkang Cheng, Yutong Wu, Menghan Li +2
Department of Automation, Tsinghua University, Beijing, China · Beijing Key Laboratory of Embodied Intelligence Systems, Beijing, China · Institute for Embodied Intelligence and Robotics, Tsinghua University, Beijing, China
Although recent years have seen significant progress of humanoid robots in walking and running, the frequent foot strikes with ground during these locomotion gaits inevitably generate high instantaneous impact forces, which leads to exacerbated joint wear and poor energy utilization. Roller skating, as a sport with substantial biomechanical value, can achieve fast and continuous sliding through rational utilization of body inertia, featuring minimal kinetic energy loss. Therefore, this study proposes a novel humanoid robot with each foot equipped with a row of four passive wheels for roller skating. A deep reinforcement learning control framework is also developed for the swizzle gait with the reward function design based on the intrinsic characteristics of roller skating. The learned policy is first analyzed in simulation and then deployed on the physical robot to demonstrate the smoothness and efficiency of the swizzle gait over traditional bipedal walking gait in terms of Impact Intensity and Cost of Transport during locomotion. A reduction of 75.86% and 63.34% of these two metrics indicate roller skating as a superior locomotion mode for enhanced energy efficiency and joint longevity.
Junchi Gu, Feiyang Yuan, Weize Shi +6
Institute of Humanoid Robots, Department of Precision Machinery and Precision Instrumentation, University of Science and Technology of China, Hefei, Anhui 230026, China.