Beyond Fixed Visual History: Learning Motion-Adaptive Temporal Receptive Fields for Complex-Terrain Quadruped Locomotion

Hao Zhang; Yuping Yan; Yitian Hong; Luying Feng; Tianhong Wang; Fengqi Dai; Wei Chen; Yaochu Jin

cs.ROOct 10, 2026

Beyond Fixed Visual History: Learning Motion-Adaptive Temporal Receptive Fields for Complex-Terrain Quadruped Locomotion

Authors: Hao Zhang, Yuping Yan, Yitian Hong, Luying Feng, Tianhong Wang, Fengqi Dai, Wei Chen, Yaochu Jin

Organizations: Westlake University, Hangzhou, China. · East China University of Science and Technology, Shanghai, China.

Abstract

Versatile quadruped locomotion depends on a tight coupling between what the robot perceives and what its body can execute. Yet existing visual controllers typically organize depth history using fixed temporal patterns, overlooking that the same temporal span corresponds to different spatial ranges as the robot's speed, posture, and gait phase change. Consequently, the same visual history may be insufficient in one motion state but outdated or misaligned in another. To address this limitation, we propose TempoLoco, a body-aware dynamic visual perception framework that organizes visual memory according to the robot's current locomotion state. TempoLoco learns multiple complementary temporal views from a long depth buffer, encodes them within a shared feature space, and dynamically aggregates them using proprioceptive history and multi-scale terrain evidence. This enables a single policy to align visual anticipation with the robot's current motion capabilities across complex terrains, including stairs, platforms, gaps, discrete obstacles, rough ground, and slopes. From the experiment analysis, TempoLoco achieves a 78.4% success rate, compared with 58.2% for the strongest fixed-history baseline and 62.1% for uniform multi-scale fusion. In separate capability evaluations, it completes 41/45 simulated trials across nine terrain types and 23/25 real-world trials on a Unitree B2 with a fixed arm, demonstrating robust and deployable locomotion through body-aware dynamic visual perception.

Figures & tables

Fig. 1: Overview of TempoLoco. Globally learned temporal strides construct complementary depth histories, which are dynamically aggregated by a body-state-conditioned gate using proprioceptive history and multi-scale terrain evidence. The resulting visual representation drives a single leg-control policy across diverse complex terrains. The strides remain fixed after training, whereas the fusion weights adapt online at every policy step. The mounted arm is fixed and excluded from the locomotion action space.

Fig. 2: Detailed architecture of TempoLoco (actor path shown). At each 50-Hz policy step, a 37-slot depth FIFO and a five-step, 260-dimensional proprioceptive history are provided to the temporal-perception module. Four globally learned continuous strides construct complementary eight-frame histories; the displayed stride values are the initialization, and filled and hollow markers denote integer-slot and interpolated samples, respectively. A scale-shared visual encoder maps each history to a 128-dimensional feature, and a body-state-conditioned temporal gate computes online softmax fusion weights from proprioceptive history and all scale-specific visual features. The fused 128-dimensional visual representation is concatenated with proprioception to form the 388-dimensional actor input, which is processed by the four-expert locomotion MoE and the unchanged action-processing pipeline to produce 12 leg-joint targets. The actor and critic use independent temporal modules, and the critic appends 33 privileged dimensions only after temporal fusion. The lower panel shows deployment on the arm-equipped Unitree B2, with the arm fixed outside the locomotion action space.

Method	Boxes	Stairs	Mesh Boxes	Inv. high	Wave	Overall SR $\uparrow$	Completion $\uparrow$
Zero	54.67	33.78	23.11	21.56	69.33	$40.49\pm 4.16$	$74.04\pm 1.33$
Current	70.44	33.56	23.78	46.22	73.33	$49.47\pm 14.41$	$78.86\pm 8.89$
Dense	75.78	46.67	27.78	59.11	66.44	$\mathbf{55.16}\pm 5.82$	$\mathbf{82.76}\pm 1.53$
Sparse	79.33	42.89	30.89	40.00	69.78	$52.58\pm 10.31$	$76.78\pm 3.29$

TABLE I: Fixed-history diagnosis: 9,000 episodes. Values are percentages; overall results are training-seed mean

\pm

SD.

Fig. 3: Learned temporal supports and actor-gate usage. (a) Final actor strides for seeds 42, 43, and 44, expressed in policy-time FIFO slots; hollow squares mark initialization and error bars show mean $\pm$ SD across seeds. (b) Descriptive mean actor temporal-gate weights for the indicated terrain–speed conditions, obtained by first averaging over policy steps within each episode and then averaging equally over 30 episodes per condition. Branch labels are the original network branch IDs and are not reordered by final learned stride values.

Method	SR $\uparrow$	Completion $\uparrow$	Worst SR $\uparrow$	Termination $\downarrow$
Dense	$58.2\pm 4.2$	$65.2\pm 3.8$	$32.1\pm 5.6$	$21.3\pm 2.1$
Uniform-Multi	$62.1\pm 3.9$	$69.0\pm 3.5$	$38.3\pm 5.1$	$18.8\pm 1.9$
Learned-Multi	$68.4\pm 3.1$	$74.6\pm 2.9$	$46.2\pm 4.3$	$14.4\pm 1.5$
Global-Static	$52.3\pm 4.8$	$60.1\pm 4.1$	$25.8\pm 6.2$	$24.6\pm 2.4$
TempoLoco	$\mathbf{78.4\pm 2.3}$	$\mathbf{83.8\pm 1.8}$	$\mathbf{54.5\pm 3.2}$	$\mathbf{8.2\pm 0.9}$

TABLE II: Main comparison: 40k PPO training iterations, three training seeds, and 100,000 episodes per trained policy. Metrics are percentages (mean

\pm

SD).

Temporal scales	Uniform fusion	State-conditioned fusion
Fixed	$62.1\pm 3.9$	$71.5\pm 3.5$
Learned	$68.4\pm 3.1$	$\mathbf{78.4\pm 2.3}$

TABLE III: Factorial ablation of temporal-scale learning and state-conditioned fusion. SR (%) is reported as mean

\pm

SD over three training seeds under the full-curriculum benchmark.

Course	Geometry	Completed
Gravel	1–2 cm particles	5/5
Rocky terrain	8–30 cm rocks	5/5
Sandy slope	$\approx$ 60 cm high	5/5
Stairs, down–up	$\approx$ 10 cm/step	5/5
Pit, down–up	$\approx$ 47 cm deep	3/5
Total	–	23/25

TABLE IV: Physical trials with the seed-42 policy. Dimensions are approximate.

Quadruped Locomotion

Quadruped locomotion studies how four-legged robots move efficiently, safely, and reliably across varied terrain and tasks.

See all 121 papers

Learning Perceptive Platform Adaptive Locomotion Controllers for Quadrupedal Robots
Jun 23, 2026David Rytz, Kim Tien Ly, Ioannis HavoutisRobot Skill Learning Cross-Embodiment Robot Learning
Dynamic Robot Systems, Oxford Robotics Institute, University of Oxford
VIP-Loco: A Visually Guided Infinite Horizon Planning Framework for Legged Locomotion
Mar 15, 2026Aditya Shirwatkar, Satyam Gupta, Shishir KolathayaTerrain-Aware Robot Locomotion Vision-Based Robot Control
Robert Bosch Center for Cyber-Physical Systems, Indian Institute of Science, Bengaluru. · Robert Bosch Center for Cyber-Physical Systems and the Department of Computer Science & Automation, Indian Institute of Science, Bengaluru.
REAL: Robust Extreme Agility via Spatio-Temporal Policy Learning and Physics-Guided Filtering
Mar 18, 2026Jialong Liu, Dehan Shen, Yanbo Wen +2Robot Policy Learning Robot Locomotion
PEAK-Lab, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, 511453, China.

Jun 23, 2026J/K move · Enter open · S save

Learning Perceptive Platform Adaptive Locomotion Controllers for Quadrupedal Robots

David Rytz, Kim Tien Ly, Ioannis Havoutis

Dynamic Robot Systems, Oxford Robotics Institute, University of Oxford

Robot Skill Learning Cross-Embodiment Robot Learning Terrain-Aware Robot Locomotion Robot Locomotion RL for Legged Locomotion Quadruped Locomotion

Open paper AskPDF arXiv

Universal quadrupedal locomotion remains limited by the difficulty of integrating perception across diverse robot morphologies. State-of-the-art controllers rely on single-robot training or blind policies that omit real-time perception, leading to poor cross-embodiment generalization. Designing locomotion policies that remain robust across related quadruped morphologies while incorporating perception is challenging. Moreover, fully perceptive policies are often sensitive to noise, whereas blind controllers lack terrain awareness. In this work, we study how perception should be integrated into morphology-aware reinforcement learning architectures for deployable quadrupedal control. Building on MorAL, we train morphology-specialized universal controllers on multiple reference quadrupeds using adaptive terrain curricula. We compare a blind baseline, a critic-perceptive variant (MorAL+), and a fully perceptive actor-critic (PPAL). Policies are evaluated in simulation on flat and rough terrains, and deployed on ANYmal hardware. Results show that critic-only perception improves robustness and tracking consistency over blind baselines while remaining more stable than fully perceptive policies under perception noise. These findings highlight that perception placement and curriculum design are key factors for scalable, morphology-aware locomotion.

Abstract

Quadruped Locomotion

Explore similar work

Learning Perceptive Platform Adaptive Locomotion Controllers for Quadrupedal Robots

VIP-Loco: A Visually Guided Infinite Horizon Planning Framework for Legged Locomotion

REAL: Robust Extreme Agility via Spatio-Temporal Policy Learning and Physics-Guided Filtering