REAL: Robust Extreme Agility via Spatio-Temporal Policy Learning and Physics-Guided Filtering

Jialong Liu; Dehan Shen; Yanbo Wen; Zeyu Jiang; Changhao Chen

cs.ROMar 18, 2026

REAL: Robust Extreme Agility via Spatio-Temporal Policy Learning and Physics-Guided Filtering

Authors: Jialong Liu, Dehan Shen, Yanbo Wen, Zeyu Jiang, Changhao Chen

Organizations: PEAK-Lab, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, 511453, China.

Abstract

Extreme legged parkour demands rapid terrain assessment and precise foot placement under highly dynamic conditions. While recent learning-based systems achieve impressive agility, they remain fundamentally fragile to perceptual degradation, where even brief visual noise or latency can cause catastrophic failure. To overcome this, we propose Robust Extreme Agility Learning (REAL), an end-to-end framework for reliable parkour under sensory corruption. Instead of relying on perfectly clean perception, REAL tightly couples vision, proprioceptive history, and temporal memory. We distill a cross-modal teacher policy into a deployable student equipped with a FiLM-modulated Mamba backbone to actively filter visual noise and build short-term terrain memory actively. Furthermore, a physics-guided Bayesian state estimator enforces rigid-body consistency during high-impact maneuvers. Validated on a Unitree Go2 quadruped, REAL successfully traverses extreme obstacles even with a 1-meter visual blind zone, while strictly satisfying real-time control constraints with a bounded 13.1 ms inference time.

Figures & tables

Fig. 1: Robust extreme parkour with proposed REAL framework. The robot successfully chains highly dynamic maneuvers across complex terrains with nominal vision (green box), and maintains stable locomotion even under severe visual degradation (red box).

Fig. 2: System architecture of REAL. Stage 1(Privileged Teacher Policy Learning) trains a privileged teacher policy via Proprioception-Terrain Associated Reasoning. Stage 2(Distillation Student Policy Learning) distills a deployable student policy using an onboard Mamba-FiLM spatial-temporal backbone and a physics-guided filtering, stabilized by a consistency-aware loss gating strategy.

Method	Hurdles		Steps		Gaps		Overall Average
Method	SR ( $\uparrow$ )	MXD ( $\uparrow$ )	SR ( $\uparrow$ )	MXD ( $\uparrow$ )	SR ( $\uparrow$ )	MXD ( $\uparrow$ )	SR ( $\uparrow$ )	MXD ( $\uparrow$ )	MEV ( $\downarrow$ )	Time ( $\downarrow$ )	Coll. ( $\downarrow$ )
Extreme Parkour [ 4 ]	0.18	0.15	0.14	0.18	0.10	0.30	0.16	0.21	34.24	0.14	0.27
RPL [ 5 ]	0.05	0.12	0.04	0.09	0.03	0.10	0.04	0.10	1.56	0.02	0.06
SoloParkour [ 20 ]	0.42	0.24	0.49	0.28	0.36	0.34	0.39	0.34	96.93	0.06	0.09
REAL (Ours)	0.82	0.42	0.94	0.53	0.28	0.39	0.78	0.45	18.41	0.02	0.06

TABLE I: Quantitative Performance Breakdown Across Diverse Parkour Terrains

Method	Nominal (No Interference)		Condition A: Frame Drop		Condition B: Gaussian Noise		Condition C: Spatial FoV Occlusion
Method	SR ( $\uparrow$ )	MXD ( $\uparrow$ )	SR ( $\uparrow$ )	MXD ( $\uparrow$ )	SR ( $\uparrow$ )	MXD ( $\uparrow$ )	SR ( $\uparrow$ )	MXD ( $\uparrow$ )
Extreme Parkour [ 4 ]	0.16	0.22	0.16 ( $\downarrow$ 0.00)	0.22 ( $\downarrow$ 0.00)	0.11 ( $\downarrow$ 0.05)	0.21 ( $\downarrow$ 0.01)	0.13 ( $\downarrow$ 0.03)	0.21 ( $\downarrow$ 0.01)
RPL [ 5 ]	0.04	0.10	0.01 ( $\downarrow$ 0.04)	0.04 ( $\downarrow$ 0.06)	0.01 ( $\downarrow$ 0.03)	0.07 ( $\downarrow$ 0.04)	0.01 ( $\downarrow$ 0.03)	0.07 ( $\downarrow$ 0.03)
SoloParkour [ 20 ]	0.39	0.35	0.20 ( $\downarrow$ 0.19 )	0.24 ( $\downarrow$ 0.11)	0.37 ( $\downarrow$ 0.03)	0.35 ( $\downarrow$ 0.00)	0.41 ( $\uparrow$ 0.02)	0.35 ( $\uparrow$ 0.00)
REAL (Ours)	0.78	0.46	0.61 ( $\downarrow$ 0.17)	0.41 ( $\downarrow$ 0.05)	0.51 ( $\downarrow$ 0.27)	0.38 ( $\downarrow$ 0.08)	0.72 ( $\downarrow$ 0.06 )	0.43 ( $\downarrow$ 0.02 )

TABLE II: Robustness Evaluation: Performance Degradation Under Severe Perceptual Interference

Fig. 3: Left. Snapshots of the REAL policy executing dynamic manoeuvres across extreme terrains. Right. Physical hardware setup. The REAL policy has been deployed on a Unitree Go2 quadrupedal robot.

Fig. 4: Simulation results on complex parkour terrains featuring a 1 m vision-masked blind zone (setup detailed in Table IV ). REAL successfully traverses all terrains while maintaining kinematic stability.

Parameter	Range / Value	Distribution
Environment Dynamics
Terrain Friction	$[0.6,2.0]$	Uniform
External Push Interval	$8$ s	Fixed
External Push Velocity (XY)	$[-0.5,0.5]$ m/s	Uniform
Robot Kinematics & Dynamics
Payload Mass (Base)	$[0.0,+3.0]$ kg	Uniform

TABLE III: Domain Randomization Parameters for Go2

Fig. 5: Real-world extreme blind test. Left: Baseline fails immediately upon losing visual input. Right: REAL utilizes proprioceptive history to maintain environmental memory, enabling robust blind traversal across unstructured obstacles.

Method	SR ( $\uparrow$ )	MXD ( $\uparrow$ )	MEV ( $\downarrow$ )	Time ( $\downarrow$ )	Coll. ( $\downarrow$ )
Extreme Parkour	0.11	0.20	44.03	0.15	0.29
RPL	0.00	0.03	0.35	0.00	0.04
SoloParkour	0.36	0.34	103.50	0.06	0.09
REAL (Ours)	0.55	0.39	24.84	0.03	0.08
REAL (w/o Mamba)	0.45	0.51	97.24	0.30	0.06

TABLE IV: Quantitative Performance in Blind-Zone Maneuvers (Vision Masked 1m Before Obstacles)

Fig. 6: Zero-shot sim-to-real transfer of the REAL policy on a physical Unitree Go2 quadruped. Using only onboard perception and computing, the robot completes various real-world obstacle courses: (a) leaping onto a high platform, (b) moving through scattered boxes, and (c) climbing a steep staircase.

Fig. 7: Onboard inference latency measured over 1,000 continuous control steps. The REAL policy maintains a highly predictable execution time of $O(1)$ ( $\sim 13.1$ ms/step), strictly satisfying the high-frequency real-time control budget.

Method	SR ( $\uparrow$ )	MXD ( $\uparrow$ )	MEV ( $\downarrow$ )	Time ( $\downarrow$ )	Coll. ( $\downarrow$ )
REAL (Ours)	0.78	0.45	18.41	0.02	0.06
REAL (w/ MLP Est.)	0.73	0.43	19.34	0.02	0.06
REAL (w/o FiLM)	0.44	0.51	93.43	0.28	0.06
REAL (w/o Mamba)	0.51	0.47	89.96	0.26	0.05

TABLE V: Overall Ablation Analysis of REAL Components

Estimator Architecture	RMSE (Velocity) $\downarrow$
MLP (Baseline)	0.52
MLP + EKF	0.40
1D ResNet (Single frame)	0.33
1D ResNet (10 frames)	0.28
1D ResNet + EKF (10 frames, Ours)	0.23

TABLE VI: Velocity Estimation RMSE Comparison

Fig. 8: Training convergence of the depth actor. Our consistency-aware loss gating accelerates imitation learning and achieves a lower final loss compared to a fixed-weight baseline, improving the stability of the distillation process.

Robot Policy Learning

Robot policy learning develops controllers that map sensory observations, task goals, and interaction history to actions for embodied robots.

See all 372 papers

PUMA: Perception-driven Unified Foothold Prior for Mobility Augmented Quadruped Parkour
Jan 22, 2026Liang Wang, Kanzhong Yao, Yang Liu +4Legged Robotics Terrain-Aware Robot Locomotion
Institute of Cyber-Systems and Control, Zhejiang University, 310027, China · Institute of Artificial Intelligence (TeleAI), China Telecom.
Hiking in the Wild: A Scalable Perceptive Parkour Framework for Humanoids
Jan 12, 2026Shaoting Zhu, Ziwen Zhuang, Mengjie Zhao +2Humanoid Robot Locomotion Robot Locomotion
IIIS, Tsinghua University · Shanghai Qi Zhi Institute · Department of Computer Science and Technology, Tsinghua University
DAWN: Noise-Robust Quadruped Parkour via Depth-Denoising World Models
Sep 24, 2026Yohan Choi, Min-Jun Kim, Jin-Sung Kim +2World Models for Robotics Quadruped Locomotion
Convergence Engineering, Korea University of Technology and Education, Cheonan 31253, South Korea · School of Electrical, Electronics and Communication Engineering, Korea University of Technology and Education, Cheonan, 31253, South Korea

Jan 22, 2026J/K move · Enter open · S save

PUMA: Perception-driven Unified Foothold Prior for Mobility Augmented Quadruped Parkour

Liang Wang, Kanzhong Yao, Yang Liu, Weikai Qin+3

Institute of Cyber-Systems and Control, Zhejiang University, 310027, China · Institute of Artificial Intelligence (TeleAI), China Telecom.

Legged Robotics Terrain-Aware Robot Locomotion RL for Legged Locomotion Robotic Perception Quadruped Locomotion

Open paper AskPDF arXiv

Parkour tasks for quadrupeds have emerged as a promising benchmark for agile locomotion. While human athletes can effectively perceive environmental characteristics to select appropriate footholds for obstacle traversal, endowing legged robots with similar perceptual reasoning remains a significant challenge. Existing methods often rely on hierarchical controllers that follow pre-computed footholds, thereby constraining the robot's real-time adaptability and the exploratory potential of reinforcement learning. To overcome these challenges, we present PUMA, an end-to-end learning framework that integrates visual perception and foothold priors into a single-stage training process. This approach leverages terrain features to estimate egocentric polar foothold priors, composed of relative distance and heading, guiding the robot in active posture adaptation for parkour tasks. Extensive experiments conducted in simulation and real-world environments across various discrete complex terrains, demonstrate PUMA's exceptional agility and robustness in challenging scenarios.

Abstract

Robot Policy Learning

Explore similar work

PUMA: Perception-driven Unified Foothold Prior for Mobility Augmented Quadruped Parkour

Hiking in the Wild: A Scalable Perceptive Parkour Framework for Humanoids

DAWN: Noise-Robust Quadruped Parkour via Depth-Denoising World Models