Watch Your Step: Anticipated Perception for Humanoid Locomotion

Matthias Heyrman; Chenhao Li; Ioannis Dadiotis; Marco Hutter

cs.ROOct 21, 2026

Watch Your Step: Anticipated Perception for Humanoid Locomotion

Authors: Matthias Heyrman, Chenhao Li, Ioannis Dadiotis, Marco Hutter

Organizations: Robotic Systems Laboratory, ETH Zürich, Switzerland. · ETH AI Center, ETH Zürich, Switzerland.

Abstract

Perceptive locomotion policies typically observe a dense robot-centric elevation map, although only the terrain beneath and around the next footsteps informs the control decision. Learned encoders compress the whole visual input, leaving the policy to work out which inputs matter while learning the baseline locomotion task. Learning both at once slows training and, without computationally expensive attention mechanisms, makes generalization to new terrain harder. We instead remove redundant inputs from the elevation map before the policy sees it. A lightweight foothold prediction model (FPM) predicts future footholds from proprioception and the velocity command alone, and we sample the elevation map only in small patches around these predicted footholds. We train the prediction model alongside the locomotion policy, periodically retraining it on new policy rollouts as the robot's gait matures. This allows the perception to anticipate where the policy will step, while the policy learns to leverage terrain information around those steps. With the proposed lightweight preprocessing, the policy reaches a given terrain difficulty in roughly a third of the training iterations required with the complete map. It outperforms policies that observe the complete map where footholds are sparse, including in narrow walled corridors, and on unseen terrain, and it matches an attention-based encoder trained at several times the cost. The policy transfers zero-shot to a Unitree G1 humanoid traversing various obstacles, stairs, gaps, and step-downs beyond the training distribution in both indoor and outdoor environments.

Figures & tables

Fig. 1: Zero-shot multi-terrain locomotion, including stairs, drops, gaps, unstable terrain, and crowded elevation-mapped environments using a Foothold Prediction Model (FPM) filtered elevation map. Predicted future footholds are shown in magenta, and the sampled elevation map points are shown in green and blue.

Fig. 2: Method overview. The policy learns to track velocity commands over varied terrain from an elevation map gated around the FPM’s predicted footholds. Every $n_{\mathrm{r}}=500$ iterations, A. rollouts of the current policy yield ground-truth future footholds and B. the FPM is retrained from scratch to predict them.

Foothold Prediction Model (FPM)
Hidden Layers / Activation	$[128,128]$ / LeakyReLU $(0.1)$
Input Normalization	z-score, train split
Optimizer / Learning Rate	Adam / $5\text{\times}{10}^{-3}$
Loss	Huber ( $\delta=0.01$ )
Batch Size / Epochs per Refit	$1024$ / $25$
Train–Validation Split	$80/20$ , random

TABLE I: Learning parameters for the self-supervised Foothold Prediction Model and parameters and rewards for the RL locomotion policy.

Fig. 3: Mean terrain-curriculum level during training over five seeds; shaded bands show one standard deviation. FPM reaches level

5

7.5

k iterations,

3.5\times

faster than HM and

2.7\times

faster than AME2.

	Success (%)		Sparse terrain (%)			Falls (%)		Corridor (%)
Method	Train	OOD	Stones	Gaps	Pall.	Train	OOD	All	Sparse	Step.	Cont.
FPM (ours)	84.6	83.1	80.2	77.7	81.0	3.3	5.2	89.2	74.2	85.6	99.5
HM	81.6	80.8	69.0	70.5	73.2	4.6	6.2	87.8	42.7	93.8	98.7
HM CNN	81.9	78.5	65.7	58.9	67.9	5.4	13.3	88.7	44.8	94.3	99.6
Wild	80.2	78.6	66.4	66.7	67.0	4.7	9.2	82.5	36.0	86.5	96.3
Naive	77.6	71.8	57.3	47.3	59.2	7.6	13.8	76.0	42.5	73.5	92.4

TABLE II: Command tracking success and fall rate for locomotion and corridor benchmarks. Stones is a training terrain, Gaps and Pallets are OOD. Best result in each column marked in bold , second best in italics .

Fig. 4: Sim-to-real transfer to a Unitree G1 demonstrating locomotion over challenging obstacles, including stairs and rough terrain expected within the training distribution, and a $0.55\text{\,}\mathrm{m}$ gap and $0.6\text{\,}\mathrm{m}$ high drop from outside the training distribution. Predicted footholds are shown in magenta for each step. The commanded x-velocity and the smoothed measured x-velocity are shown for the length of the course.

	Success (%)		Falls (%)		Corridor (%)
Method	Train	OOD	Train	OOD	All	Sparse	Step.	Cont.
FPM	84.6	83.1	3.3	5.2	89.2	74.2	85.6	99.5
FPM1	81.3	78.8	4.8	5.0	84.7	62.2	83.1	95.6
FPM3	79.5	77.3	2.1	4.4	85.0	52.7	86.3	96.4
FPM CNN	82.4	81.1	1.4	4.0	85.6	56.2	85.8	97.1
FPM1 CNN	82.7	82.0	2.3	4.2	86.3	62.7	86.7	95.3

TABLE III: Ablation over prediction horizon and the addition of an exteroceptive CNN encoder. Best result in each column marked in bold , second best in italics .

Fig. 5: Foothold prediction error by prediction step of various FPM horizons for training and OOD terrains and a regression line describing the relationship between prediction step and mean error. Whiskers show one standard deviation across seeds.

Humanoid Robot Locomotion

Humanoid robot locomotion studies how bipedal robots walk, run, recover from disturbances, and traverse varied terrain.

See all 145 papers

Global-Local Attention Decomposition for Terrain Encoding in Humanoid Perceptive Locomotion
May 30, 2026Shengcheng Fu, Yang Zhang, Zhanxiang Cao +4Legged Robotics Humanoid Robot Locomotion
Tongji University, Shanghai, China · Shanghai Jiao Tong University, Shanghai, China · Shanghai Innovation Institute, Shanghai, China +1
FootQuery: Future-Touchdown-Guided Retrieval from Depth History for Perceptive Humanoid Locomotion
Sep 18, 2026Tao Dong, Jia Yu, Yuxuan Fan +5Humanoid Robot Locomotion Terrain-Aware Robot Locomotion
Institute for AI Industry Research (AIR), Tsinghua University, Beijing, China. · University of Science and Technology Beijing, Beijing, China. · Nanyang Technological University, Singapore. +1
Agile and Generalized Legged Locomotion via Attention-Based Neural Map Encoding
Jan 13, 2026Chong Zhang, Victor Klemm, Fan Yang +1Terrain-Aware Robot Locomotion RL for Legged Locomotion
1Robotic Systems Lab, ETH Zurich, Switzerland · 2Secure, Reliable, and Intelligent Systems Lab, ETH Zurich, Switzerland · 3ETH AI Center, Switzerland

May 30, 2026J/K move · Enter open · S save

Global-Local Attention Decomposition for Terrain Encoding in Humanoid Perceptive Locomotion

Shengcheng Fu, Yang Zhang, Zhanxiang Cao, Liyun Yan+3

Tongji University, Shanghai, China · Shanghai Jiao Tong University, Shanghai, China · Shanghai Innovation Institute, Shanghai, China+1

Legged Robotics Humanoid Robot Locomotion Terrain-Aware Robot Locomotion RL for Legged Locomotion Attention Mechanisms

Open paper AskPDF arXiv

Although reinforcement learning has significantly advanced humanoid locomotion, perceptive policies still struggle on sparse-foothold terrain and constrained environments. Success in these scenarios requires both broad terrain awareness and precise foothold selection, two perceptual roles that conventional encoders often entangle. To address this challenge, we propose Global-Local Attention Decomposition (GLAD) for terrain encoding in humanoid locomotion. Realized by a coarse-to-fine encoder over a robot-centric elevation map, GLAD explicitly separates these objectives: a global attention branch utilizes attention pooling to summarize the surrounding terrain context, while a state-conditioned local attention branch sparsifies and encodes precise foothold-relevant geometry. This explicit attention decomposition prevents the dilution of fine-grained spatial cues while reducing training overhead. Experiments demonstrate that GLAD enables reliable locomotion over challenging gaps, stepping stones, and stairs. Furthermore, the learned policy exhibits emergent terrain-responsive behaviors, autonomously following narrow paths and avoiding obstacles under simple velocity commands without explicit navigation planners. In real-world deployment on a Unitree G1 humanoid robot using onboard LiDAR, the proposed method achieves robust zero-shot sim-to-real transfer across diverse sparse-foothold and obstacle-rich domains.

Abstract

Humanoid Robot Locomotion

Explore similar work

Global-Local Attention Decomposition for Terrain Encoding in Humanoid Perceptive Locomotion

FootQuery: Future-Touchdown-Guided Retrieval from Depth History for Perceptive Humanoid Locomotion

Agile and Generalized Legged Locomotion via Attention-Based Neural Map Encoding