Beyond Fixed Visual History: Learning Motion-Adaptive Temporal Receptive Fields for Complex-Terrain Quadruped Locomotion

Hao Zhang; Yuping Yan; Yitian Hong; Luying Feng; Tianhong Wang; Fengqi Dai; Wei Chen; Yaochu Jin

cs.ROOct 10, 2026

Beyond Fixed Visual History: Learning Motion-Adaptive Temporal Receptive Fields for Complex-Terrain Quadruped Locomotion

Authors: Hao Zhang, Yuping Yan, Yitian Hong, Luying Feng, Tianhong Wang, Fengqi Dai, Wei Chen, Yaochu Jin

Organizations: Westlake University, Hangzhou, China. · East China University of Science and Technology, Shanghai, China.

Abstract

Versatile quadruped locomotion depends on a tight coupling between what the robot perceives and what its body can execute. Yet existing visual controllers typically organize depth history using fixed temporal patterns, overlooking that the same temporal span corresponds to different spatial ranges as the robot's speed, posture, and gait phase change. Consequently, the same visual history may be insufficient in one motion state but outdated or misaligned in another. To address this limitation, we propose TempoLoco, a body-aware dynamic visual perception framework that organizes visual memory according to the robot's current locomotion state. TempoLoco learns multiple complementary temporal views from a long depth buffer, encodes them within a shared feature space, and dynamically aggregates them using proprioceptive history and multi-scale terrain evidence. This enables a single policy to align visual anticipation with the robot's current motion capabilities across complex terrains, including stairs, platforms, gaps, discrete obstacles, rough ground, and slopes. From the experiment analysis, TempoLoco achieves a 78.4% success rate, compared with 58.2% for the strongest fixed-history baseline and 62.1% for uniform multi-scale fusion. In separate capability evaluations, it completes 41/45 simulated trials across nine terrain types and 23/25 real-world trials on a Unitree B2 with a fixed arm, demonstrating robust and deployable locomotion through body-aware dynamic visual perception.

Figures & tables

Fig. 1: Overview of TempoLoco. Globally learned temporal strides construct complementary depth histories, which are dynamically aggregated by a body-state-conditioned gate using proprioceptive history and multi-scale terrain evidence. The resulting visual representation drives a single leg-control policy across diverse complex terrains. The strides remain fixed after training, whereas the fusion weights adapt online at every policy step. The mounted arm is fixed and excluded from the locomotion action space.

Fig. 2: Detailed architecture of TempoLoco (actor path shown). At each 50-Hz policy step, a 37-slot depth FIFO and a five-step, 260-dimensional proprioceptive history are provided to the temporal-perception module. Four globally learned continuous strides construct complementary eight-frame histories; the displayed stride values are the initialization, and filled and hollow markers denote integer-slot and interpolated samples, respectively. A scale-shared visual encoder maps each history to a 128-dimensional feature, and a body-state-conditioned temporal gate computes online softmax fusion weights from proprioceptive history and all scale-specific visual features. The fused 128-dimensional visual representation is concatenated with proprioception to form the 388-dimensional actor input, which is processed by the four-expert locomotion MoE and the unchanged action-processing pipeline to produce 12 leg-joint targets. The actor and critic use independent temporal modules, and the critic appends 33 privileged dimensions only after temporal fusion. The lower panel shows deployment on the arm-equipped Unitree B2, with the arm fixed outside the locomotion action space.

Method	Boxes	Stairs	Mesh Boxes	Inv. high	Wave	Overall SR $\uparrow$	Completion $\uparrow$
Zero	54.67	33.78	23.11	21.56	69.33	$40.49\pm 4.16$	$74.04\pm 1.33$
Current	70.44	33.56	23.78	46.22	73.33	$49.47\pm 14.41$	$78.86\pm 8.89$
Dense	75.78	46.67	27.78	59.11	66.44	$\mathbf{55.16}\pm 5.82$	$\mathbf{82.76}\pm 1.53$
Sparse	79.33	42.89	30.89	40.00	69.78	$52.58\pm 10.31$	$76.78\pm 3.29$

TABLE I: Fixed-history diagnosis: 9,000 episodes. Values are percentages; overall results are training-seed mean

\pm

SD.

Fig. 3: Learned temporal supports and actor-gate usage. (a) Final actor strides for seeds 42, 43, and 44, expressed in policy-time FIFO slots; hollow squares mark initialization and error bars show mean $\pm$ SD across seeds. (b) Descriptive mean actor temporal-gate weights for the indicated terrain–speed conditions, obtained by first averaging over policy steps within each episode and then averaging equally over 30 episodes per condition. Branch labels are the original network branch IDs and are not reordered by final learned stride values.

Method	SR $\uparrow$	Completion $\uparrow$	Worst SR $\uparrow$	Termination $\downarrow$
Dense	$58.2\pm 4.2$	$65.2\pm 3.8$	$32.1\pm 5.6$	$21.3\pm 2.1$
Uniform-Multi	$62.1\pm 3.9$	$69.0\pm 3.5$	$38.3\pm 5.1$	$18.8\pm 1.9$
Learned-Multi	$68.4\pm 3.1$	$74.6\pm 2.9$	$46.2\pm 4.3$	$14.4\pm 1.5$
Global-Static	$52.3\pm 4.8$	$60.1\pm 4.1$	$25.8\pm 6.2$	$24.6\pm 2.4$
TempoLoco	$\mathbf{78.4\pm 2.3}$	$\mathbf{83.8\pm 1.8}$	$\mathbf{54.5\pm 3.2}$	$\mathbf{8.2\pm 0.9}$

TABLE II: Main comparison: 40k PPO training iterations, three training seeds, and 100,000 episodes per trained policy. Metrics are percentages (mean

\pm

SD).

Temporal scales	Uniform fusion	State-conditioned fusion
Fixed	$62.1\pm 3.9$	$71.5\pm 3.5$
Learned	$68.4\pm 3.1$	$\mathbf{78.4\pm 2.3}$

TABLE III: Factorial ablation of temporal-scale learning and state-conditioned fusion. SR (%) is reported as mean

\pm

SD over three training seeds under the full-curriculum benchmark.

Course	Geometry	Completed
Gravel	1–2 cm particles	5/5
Rocky terrain	8–30 cm rocks	5/5
Sandy slope	$\approx$ 60 cm high	5/5
Stairs, down–up	$\approx$ 10 cm/step	5/5
Pit, down–up	$\approx$ 47 cm deep	3/5
Total	–	23/25

TABLE IV: Physical trials with the seed-42 policy. Dimensions are approximate.

Quadruped Locomotion

Quadruped locomotion studies how four-legged robots move efficiently, safely, and reliably across varied terrain and tasks.

See all 121 papers

Jun 23, 2026

Learning Perceptive Platform Adaptive Locomotion Controllers for Quadrupedal Robots

Universal quadrupedal locomotion remains limited by the difficulty of integrating perception across diverse robot morphologies. State-of-the-art controllers rely on single-robot training or blind policies that omit real-time perception, leading to poor cross-embodiment generalization. Designing locomotion policies that remain robust across related quadruped morphologies while incorporating perception is challenging. Moreover, fully perceptive policies are often sensitive to noise, whereas blind controllers lack terrain awareness. In this work, we study how perception should be integrated into morphology-aware reinforcement learning architectures for deployable quadrupedal control. Building on MorAL, we train morphology-specialized universal controllers on multiple reference quadrupeds using adaptive terrain curricula. We compare a blind baseline, a critic-perceptive variant (MorAL+), and a fully perceptive actor-critic (PPAL). Policies are evaluated in simulation on flat and rough terrains, and deployed on ANYmal hardware. Results show that critic-only perception improves robustness and tracking consistency over blind baselines while remaining more stable than fully perceptive policies under perception noise. These findings highlight that perception placement and curriculum design are key factors for scalable, morphology-aware locomotion.

David Rytz, Kim Tien Ly, Ioannis Havoutis

Dynamic Robot Systems, Oxford Robotics Institute, University of Oxford

Robot Skill Learning Cross-Embodiment Robot Learning Terrain-Aware Robot Locomotion Robot Locomotion RL for Legged Locomotion Quadruped Locomotion

Ask PDF

Mar 15, 2026

VIP-Loco: A Visually Guided Infinite Horizon Planning Framework for Legged Locomotion

Perceptive locomotion for legged robots requires anticipating and adapting to complex, dynamic environments. Model Predictive Control (MPC) serves as a strong baseline, providing interpretable motion planning with constraint enforcement, but struggles with high-dimensional perceptual inputs and rapidly changing terrain. In contrast, model-free Reinforcement Learning (RL) adapts well across visually challenging scenarios but lacks planning. To bridge this gap, we propose VIP-Loco, a framework that integrates vision-based scene understanding with RL and planning. During training, an internal model maps proprioceptive states and depth images into compact kinodynamic features used by the RL policy. At deployment, the learned models are used within an infinite-horizon MPC formulation, combining adaptability with structured planning. We validate VIP-Loco in simulation on challenging locomotion tasks, including slopes, stairs, crawling, tilting, gap jumping, and climbing, across three robot morphologies: a quadruped (Unitree Go1), a biped (Cassie), and a wheeled-biped (TronA1-W). Through ablations and comparisons with state-of-the-art methods, we show that VIP-Loco unifies planning and perception, enabling robust, interpretable locomotion in diverse environments.

Aditya Shirwatkar, Satyam Gupta, Shishir Kolathaya

Robert Bosch Center for Cyber-Physical Systems, Indian Institute of Science, Bengaluru. · Robert Bosch Center for Cyber-Physical Systems and the Department of Computer Science & Automation, Indian Institute of Science, Bengaluru.

Terrain-Aware Robot Locomotion Vision-Based Robot Control RL for Legged Locomotion Model Predictive Control

Ask PDF

Mar 18, 2026

REAL: Robust Extreme Agility via Spatio-Temporal Policy Learning and Physics-Guided Filtering

Extreme legged parkour demands rapid terrain assessment and precise foot placement under highly dynamic conditions. While recent learning-based systems achieve impressive agility, they remain fundamentally fragile to perceptual degradation, where even brief visual noise or latency can cause catastrophic failure. To overcome this, we propose Robust Extreme Agility Learning (REAL), an end-to-end framework for reliable parkour under sensory corruption. Instead of relying on perfectly clean perception, REAL tightly couples vision, proprioceptive history, and temporal memory. We distill a cross-modal teacher policy into a deployable student equipped with a FiLM-modulated Mamba backbone to actively filter visual noise and build short-term terrain memory actively. Furthermore, a physics-guided Bayesian state estimator enforces rigid-body consistency during high-impact maneuvers. Validated on a Unitree Go2 quadruped, REAL successfully traverses extreme obstacles even with a 1-meter visual blind zone, while strictly satisfying real-time control constraints with a bounded 13.1 ms inference time.

Jialong Liu, Dehan Shen, Yanbo Wen +2

PEAK-Lab, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, 511453, China.

Robot Policy Learning Robot Locomotion Legged Robotics Visuomotor Policy Learning Terrain-Aware Robot Locomotion

Ask PDF

Abstract

Quadruped Locomotion

Explore similar work

Learning Perceptive Platform Adaptive Locomotion Controllers for Quadrupedal Robots

VIP-Loco: A Visually Guided Infinite Horizon Planning Framework for Legged Locomotion

REAL: Robust Extreme Agility via Spatio-Temporal Policy Learning and Physics-Guided Filtering