Chasing Autonomy: Dynamic Retargeting and Control Guided RL for Performant and Controllable Humanoid Running

Zachary Olkin; William D. Compton; Ryan M. Bena; Aaron D. Ames

cs.ROMar 26, 2026

Chasing Autonomy: Dynamic Retargeting and Control Guided RL for Performant and Controllable Humanoid Running

Authors: Zachary Olkin, William D. Compton, Ryan M. Bena, Aaron D. Ames

Organizations: Department of Control and Dynamical Systems and the Department of Mechanical and Civil Engineering at the California Institute of Technology

Abstract

Humanoid robots have the promise of locomoting like humans, including fast and dynamic running. Recently, reinforcement learning (RL) controllers that can mimic human motions have become popular as they can generate very dynamic behaviors, but they are often restricted to single motion play-back which hinders their deployment in long duration and autonomous locomotion. In this paper, we present a pipeline to dynamically retarget human motions through an optimization routine with hard constraints to generate improved periodic reference libraries from a single human demonstration. We then study the effect of both the reference motion and the reward structure on the reference and commanded velocity tracking, concluding that a goal-conditioned and control-guided reward which tracks dynamically optimized human data results in the best performance. We deploy the policy on hardware, demonstrating its speed and endurance by achieving running speeds of up to 3.3 m/s on a Unitree G1 robot and traversing hundreds of meters in real-world environments. Additionally, to demonstrate the controllability of the locomotion, we use the controller in a full perception and planning autonomy stack for obstacle avoidance while running outdoors.

Figures & tables

Fig. 1: Demonstration of the running controller working inside on a constrained treadmill and in outdoor real world environments. The running appears human-like while still achieving the commanded speed through a combination of optimized retargeted human data and control guided reward shaping.

Fig. 2: Architecture of the proposed pipeline. Human data is retargeted using state constrained dynamic optimization with a hybrid system model. This allows us to build a library of references from a single human reference and then track this in the RL controller. The RL controller uses a control-guided cost function. Then the policy is transferred zero-shot to hardware and can be used in an autonomy stack.

Fig. 3: Visual depiction of the multiple shooting trajectory optimization problem. Two hybrid domains are shown and the node where they meet utilizes the associated reset map. Within each domain, a number of optimization nodes are used. The cost is tracking the human reference motion.

Reward Term	Expression	Weight
Regularization
Torque	$\\|\tau\\|^{2}$	$-1\times 10^{-5}$
Action-rate	$\\|a_{t}-a_{t-1}\\|^{2}$	$-1\times 10^{-3}$
Torque limit	$\\|\|\tau_{t}\|-\tau_{\text{lim}}\\|_{1}^{+}$	$-1.0$
Joint limit	$\\|(\underline{q}-q_{t})^{+}+(q_{t}-\bar{q})^{+}\\|_{1}$	$-1.0$
CLF-RL

TABLE I: Reward terms used for each of the different training types.

CLF-RL

Mimic

Kinematic Retargeted Human Data

Dynamic Opt. Human Data

Kinematic Retargeted Human Data

Speed

Metric

Goal + Adjust

3x Goal + Adjust

Adjust

Goal + Adjust

Goal Only

Goal + Adjust

3x Goal + Adjust

Goal + Adjust

3x Goal + Adjust

1.4 m/s

e_{v_{x}}

(m/s)

0.437

{}\pm{}

0.054

0.228

{}\pm{}

0.063

0.251

{}\pm{}

0.106

0.157

{}\pm{}

0.056

0.139

{}\pm{}

0.045

0.293

{}\pm{}

0.057

0.236

{}\pm{}

0.050

2.281

{}\pm{}

0.032

0.438

{}\pm{}

0.110

e_{v_{y}}

(m/s)

0.223

{}\pm{}

0.321

0.196

{}\pm{}

0.322

0.319

{}\pm{}

0.297

0.221

{}\pm{}

0.313

0.198

{}\pm{}

0.285

0.221

{}\pm{}

0.310

0.214

{}\pm{}

0.310

0.264

{}\pm{}

0.274

0.201

{}\pm{}

0.325

e_{\dot{\psi}}

(rad/s)

0.214

{}\pm{}

0.241

0.188

{}\pm{}

0.248

0.202

{}\pm{}

0.229

0.192

{}\pm{}

0.249

0.216

{}\pm{}

0.258

0.249

{}\pm{}

0.264

0.187

{}\pm{}

0.235

0.259

{}\pm{}

0.295

0.200

{}\pm{}

0.248

2.0 m/s

e_{v_{x}}

(m/s)

0.335

{}\pm{}

0.083

0.237

{}\pm{}

0.062

0.096

{}\pm{}

0.081

0.129

{}\pm{}

0.064

0.104

{}\pm{}

0.058

0.240

{}\pm{}

0.065

0.206

{}\pm{}

0.055

1.672

{}\pm{}

0.033

0.599

{}\pm{}

0.128

TABLE II: Velocity tracking error of the moving average (mean

\pm

std) across policies and speeds. Bold indicates best per-metric value. Shading indicates policy with lowest total error per speed.

Fig. 4: Reference tracking comparison between reference and rewards types. Error bars show one std. dev. The dynamically optimized human data performs the best, and the CLF rewards outperform the Mimic rewards. Optimized retargeted references outperform kinematic human data retargeting.

Fig. 5: Outdoor long range experiments. The robot runs outdoors over distances of more than 150 m. The robot was given yaw targets in the global frame, a target position to track with body lateral motion and a feedforward velocity in the body forward direction. Lidar odometry and a P controller were used for global pose tracking.

Fig. 6: Autonomous collision avoidance while running with CBF-constrained MPC. The robot perceives it environment with a lidar, continuously re-computes a plan to avoid collisions, and passes the corresponding velocity commands to the running controller. With this architecture, the robot can avoid real-world obstacles while running.

Humanoid Robot Locomotion

Humanoid robot locomotion studies how bipedal robots walk, run, recover from disturbances, and traverse varied terrain.

See all 145 papers

Terrain Consistent Reference-Guided RL for Humanoid Navigation Autonomy
May 15, 2026William D. Compton, Zachary Olkin, Aaron D. AmesHumanoid Robot Locomotion Reinforcement Learning
Department of Computing and Mathematical Sciences, California Institute of Technology, Pasadena, CA 91125
HuMBLE: Human Motion-Driven Behavior Learning for Embodied Locomotion
Oct 7, 2026Mike Zhang, Dongho Kang, Kevin Bergamin +19Human-to-Robot Motion Imitation Humanoid Robot Locomotion
RAI Institute, USA · Boston Dynamics, USA · Carnegie Mellon University, USA
Make Tracking Easy: Neural Motion Retargeting for Humanoid Whole-body Control
Mar 23, 2026Qingrui Zhao, Kaiyue Yang, Xiyu Wang +6Motion Retargeting Robot Motion Generation
Nanjing University · Huawei Technologies

May 15, 2026J/K move · Enter open · S save

Terrain Consistent Reference-Guided RL for Humanoid Navigation Autonomy

William D. Compton, Zachary Olkin, Aaron D. Ames

Department of Computing and Mathematical Sciences, California Institute of Technology, Pasadena, CA 91125

Humanoid Robot Locomotion Reinforcement Learning Robot Navigation Terrain-Aware Robot Locomotion Robot Locomotion RL for Legged Locomotion

Open paper AskPDF arXiv

We present a method for training reference-guided, perceptive reinforcement learning locomotion policies for humanoid robots in which reference trajectories are modulated in training to be consistent with terrain geometry. Aiming to deploy our method with standard navigation autonomy infrastructure, we synthesize SE(2)-controllable reference trajectories inside the RL training loop, projecting desired footsteps onto valid footholds and adjusting swing-foot and center-of-mass trajectories to match the terrain. The resulting policy exposes a clean SE(2) velocity interface compatible with standard navigation planners. In simulation, environmentally-conditioned references significantly improve reference tracking performance compared to environment agnostic references. On hardware, we integrate the policy with an MPC + control barrier function planner and demonstrate long-horizon (>70m) closed-loop autonomous navigation on the Unitree G1 through outdoor environments containing rough terrain and consecutive flights of stairs, with all sensing and computation onboard.

Abstract

Humanoid Robot Locomotion

Explore similar work

Terrain Consistent Reference-Guided RL for Humanoid Navigation Autonomy

HuMBLE: Human Motion-Driven Behavior Learning for Embodied Locomotion

Make Tracking Easy: Neural Motion Retargeting for Humanoid Whole-body Control