Perceptive Humanoid Parkour: Chaining Dynamic Human Skills via Motion Matching
Organizations: Amazon FAR · UC Berkeley · CMU · Stanford University
Abstract
While recent advances in humanoid locomotion have achieved stable walking on varied terrains, capturing the agility and adaptivity of highly dynamic human motions remains an open challenge. In particular, agile parkour in complex environments demands not only low-level robustness, but also human-like motion expressiveness, long-horizon skill composition, and perception-driven decision-making. In this paper, we present Perceptive Humanoid Parkour (PHP), a modular framework that enables humanoid robots to autonomously perform long-horizon, vision-based parkour across challenging obstacle courses. Our approach first leverages motion matching, formulated as nearest-neighbor search in a feature space, to compose retargeted atomic human skills into long-horizon kinematic trajectories. This framework enables the flexible composition and smooth transition of complex skill chains while preserving the elegance and fluidity of dynamic human motions. Next, we train motion-tracking reinforcement learning (RL) expert policies for these composed motions, and distill them into a single depth-based, multi-skill student policy, using a combination of DAgger and RL. Crucially, the combination of perception and skill composition enables autonomous, context-aware decision-making: using only onboard depth sensing and a discrete 2D velocity command, the robot selects and executes whether to step over, climb onto, vault or roll off obstacles of varying geometries and heights. We validate our framework with extensive real-world experiments on a Unitree G1 humanoid robot, demonstrating highly dynamic parkour skills such as climbing tall obstacles up to 1.25m (96% robot height), as well as long-horizon multi-obstacle traversal with closed-loop adaptation to real-time obstacle perturbations.
Figures & tables
| Commanded Velocity | 1.0 m/s | 2.0 m/s | ||||
|---|---|---|---|---|---|---|
| 36 cm | 58 cm | 76 cm | 36 cm | 58 cm | 76 cm | |
| Velocity Tracking | 1.00 | 0.00 | 0.00 | 1.00 | 0.00 | 0.00 |
| Uncomposed Data | 0.06 | 0.02 | 0.00 | 0.37 | 0.27 | 0.07 |
| End-to-end Depth | 0.95 | 0.07 | 0.08 | 0.78 | 0.19 | 0.14 |
| Ours | 1.00 | 0.99 | 0.95 | 1.00 | 0.99 | 0.95 |
| Method | 1.0 m/s | 2.0 m/s | ||||
|---|---|---|---|---|---|---|
| 58 cm | 76 cm | 94 cm | 36 cm | 58 cm | 76 cm | |
| Extreme Distances | 0.99 | 0.62 | 0.64 | 0.98 | 0.60 | 0.58 |
| Half Density | 0.95 | 0.32 | 0.57 | 0.99 | 0.85 | 0.81 |
| DAgger Only | 0.16 | 0.03 | 0.12 | 0.63 | 0.09 | 0.10 |
| DAgger & Alive Reward | 1.00 | 0.90 | 0.96 | 0.94 | 0.91 | 0.84 |
| DAgger & Root Tracking | 1.00 | 0.79 | 0.75 | 1.00 | 0.92 | 0.87 |
| Skill | Duration (s) |
|---|---|
| Locomotion | |
| Locomotion | 495.5 |
| Parkour skills @ 1.0 m/s | |
| Step (36 cm) | 2.2 |
| Climb (58 cm) | 12.1 |
| Climb (76 cm) | 8.8 |
| Reward Terms | Equation | Weight |
| Task (Tracking) | ||
| Body Position | ||
| Body Orientation | ||
| Body Linear velocity | ||
| Body Angular velocity | ||
| Anchor Position | ||
| Domain Randomization | Sampling Distribution |
|---|---|
| Physical parameters | |
| Static friction coefficients | |
| Dynamic friction coefficients | |
| Restitution coefficient | |
| Default joint positions (except ankle) [rad] | |
| Default ankle joint positions [rad] | |
| Hyperparameter | Motion Tracking | Distillation |
|---|---|---|
| Architecture | ||
| Actor / Student MLP hidden dims | [512, 256, 128] | [2048, 1024, 512, 256, 128] |
| Critic MLP hidden dims | [512, 256, 128] | [512, 256, 128] |
| Activation function | ELU | ELU |
| Init noise std | 1.0 | 0.01 |
| Depth backbone | – | 3-layer CNN + GAP |
Humanoid Robot Locomotion
Humanoid robot locomotion studies how bipedal robots walk, run, recover from disturbances, and traverse varied terrain.
See all 145 papersHiking in the Wild: A Scalable Perceptive Parkour Framework for Humanoids
Light-Loco-Parkour: Versatile Perceptive Whole-Body Locomotion via Multi-Skill Distillation
Light-Loco-Parkour (LLP), an end-to-end perceptive whole-body locomotion system that closes this gap with a single deployable policy. Conditioned only on onboard depth and a velocity command, the policy decides when to walk, balance, climb, step down, or vault, with no reference input, skill label, hand-coded gate, or runtime motion graph. Compared with prior humanoid systems, LLP makes three contributions. First, it introduces a whole-body perceptive-control pipeline that extends an RL-trained, velocity-tracking locomotion policy with parkour skills learned from object-interacting motions, so the same policy tracks velocity in open terrain, executes whole-body traversal at obstacles, and resumes locomotion afterward. Second, it acquires terrain-conditioned skills from sparse seeds by expanding a single motion into dynamically feasible, terrain-paired references across obstacle geometry, rather than relying on a large motion corpus. Third, it learns autonomous skill transitions from reward, letting the policy decide when and which whole-body skill to invoke from depth and command alone, with no one-hot skill label, hand-coded state machine, or runtime motion generator. Simulation and real-world experiments show high success across both benchmarked terrains and unseen obstacle variations, and the same policy transfers zero-shot to indoor and outdoor hardware experiments. These results demonstrate autonomous perceptive whole-body locomotion on a humanoid in outdoor settings, using only onboard sensing and a single deployable policy.