TRAIN: Terrain-Aware Interactive Navigation for Quadruped Robots via Reinforcement Learning
Authors: Wei Zhu, Irfan Tito Kurniawan, Ye Zhao, Mitsuhiro Hayashibe
Organizations: Department of Robotics, Graduate School of Engineering, Tohoku University, 980-8579, Sendai, Japan. · Laboratory for Intelligent Decision and Autonomous Robots, Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA 30313, USA.
This study introduces TRAIN: TeRrain-Aware Interactive Navigation, a deep reinforcement learning (DRL)-based framework for quadrupedal navigation across unstructured terrain and human-populated environments. Conventional quadrupedal navigation typically separates motion planning from locomotion control, neglecting whole-body constraints and terrain awareness. On the other hand, end-to-end methods are more integrated but require high-frequency sensing, which is often noisy and computationally costly. In addition, most existing approaches assume static environments, limiting their use in human-populated settings. To address these limitations, we propose a two-stage training framework with three DRL pipelines. (1) TRAIN-Loco employs an asymmetric actor-critic (AC) model for quadrupedal locomotion, enabling agile traversal of uneven terrains without explicit terrain or contact observations. (2) TRAIN-Nav applies a symmetric AC framework for navigation with interactive pedestrians, directly mapping transformed LiDAR data to ego-agent actions under differential-drive kinematics. (3) A unified pipeline, TRAIN, integrates TRAIN-Loc and TRAIN-Nav, supporting terrain-aware quadrupedal navigation in uneven and interactive environments. Comprehensive benchmarks against quadrupedal locomotion and robot navigation baselines demonstrate the effectiveness of TRAIN. Hardware experiments further confirm its potential for sim-to-real transfer.
Figures & tables
Fig. 1: Terrain-aware navigation of quadruped robots in interactive environments. The top panel illustrates simulation training scenarios, while the bottom panels show real-world implementations.
Fig. 2: Overall framework with a two-stage training architecture. In the first stage, a quadrupedal locomotion policy and a navigation policy with interaction awareness are trained separately. In the second stage, these two policies are unified into a single quadrupedal navigation policy, which is further retrained in uneven and interactive environments.
Fig. 3: Terrain height map and contact points visualized in IsaacSim.
Reward
Equation ( ri )
Weight ( wi )
Linear vel track
exp{−4(vxycmd−vxy)2}
1.5
Angular vel track
exp{−4(wzcmd−wz)2}
0.75
Linear vel ( z )
vz2
−2.0
Angular vel ( xy )
wxy2
−0.05
Torque
τ2
−2×10−4
Angular acceleration
θ¨2
−2.5×10−7
TABLE I: Reward function for quadrupedal locomotion
Type
Domain
Noise
System dynamics
Friction coefficient
[0.65, 1.25]
Body mass
[-1.0, 3.0] kg
CoM xy
[-0.05, 0.05] m
CoM z
[-0.01, 0.01] m
Observations
Body linear velvt
[-0.1, 0.1] m/s
Body angular velωt
[-0.2, 0.2] rad/s
TABLE II: Locomotion Domain Randomization
Fig. 4: LiDAR scan transformation. The red filled circle represents the ego-agent, blue filled circles denote static obstacles of varying sizes, and blue hollow circles indicate pedestrians with a fixed radius. The left panel shows the LiDAR scan (solid beams) at time t−k , while the right panel illustrates the scan (solid beams) at the current time t . Dashed lines in the right panel correspond to the transformed LiDAR beams from t−k to t . Each point Pi retains the same global position in both panels.
Fig. 5: Reward structure and scenarios. The left panel illustrates the geometric representation used for defining the reward function, while the right panel depicts four possible ego-agent scenarios: collision, goal arrival, discomfort, and open space.
Fig. 6: Progression of terrain difficulty levels over training iterations across five independent random seeds. The solid line indicates the mean, while the shaded region denotes the minimum and maximum bounds.. The Privilege method yields an ideal performance reference (upper bound) rather than a direct baseline.
Fig. 7: Locomotion ablation results evaluated across five random seeds. DreamWaQ-A1 is omitted because all five runs collapsed (terrain difficulty converged to zero) at the early training stage.
Method
Δvx m/s
Δvy m/s
Δwz rad/s
DreamWaQ-Go2
0.12 ± 0.15
0.12 ± 0.15
0.15 ± 0.15
L2T-Go2
0.17 ± 0.18
0.17 ± 0.17
0.28 ± 0.28
SLR-Go2
0.13 ± 0.18
0.13 ± 0.17
0.10 ± 0.11
TAR-Go2
0.45 ± 0.68
0.31 ± 0.60
0.27 ± 0.34
TRAIN-Loco-MLP-Go2
0.13 ± 0.15
0.12 ± 0.14
0.16 ± 0.17
TRAIN-Loco-Go2 (Ours)
0.10 ± 0.12
0.11 ± 0.13
0.13 ± 0.12
TABLE III: Command tracking errors for quadrupedal locomotion on uneven terrains.
Fig. 8: Velocity tracking performance along with simulation scenes. The commanded velocities, [vx,vy,wz] , change from [0.0,0.0,0.0] , to [0.6,0.6,1.0] , to [−0.6,−0.6,−1.0] , and to [0.0,0.0,0.0] . The top left two scenes correspond to the peak tracking errors for our approach, and the other four illustrate the erroneous tracking for DreamWaQ.
Fig. 9: Learning efficiency comparison with five random seeds. The learning curve for HEIGHT is plotted separately in the bottom panel due to its significantly lower learning efficiency (approximately 20 times slower). The remaining learning curves are depicted in the top panel, where DRL-VO achieves an efficiency comparable to our method when utilizing our proposed network architecture.
Method
SR
CR
TR
NT
Reactive
DWA
0.946
0.014
0.040
18.81 ± 3.36
ORCA
0.848
0.004
0.148
20.33 ± 5.59
AVOCADO
0.830
0.044
0.126
18.56 ± 4.61
MPC
N-MPC
0.242
0.002
0.756
19.89 ± 3.85
NeuPAN
0.004
0.020
0.976
40.20 ± 0.85
T-MPC
0.326
0.586
0.106
19.50 ± 7.31
TABLE IV: Navigation evaluations with 500 random tests.
Fig. 10: Trajectories of the ego-agent and pedestrians. The red circle is the ego-agent, the green represents the goal, the cyan circles are static obstacles, and others are pedestrians. As time goes by, the color of the circle changes from light to dark. N-MPC fails due to navigation timeout, while RGL and AVOCADO fail because of collisions.
Method
SR
CR
NT
TRAIN-Nav-CNN
0.892
0.108
18.96 ± 1.40
TRAIN-Nav-Direct
0.956
0.064
20.66 ± 2.83
TRAIN-Nav-Noise
0.960
0.040
18.75 ± 1.48
TRAIN-Nav (Ours)
0.972
0.028
19.26 ± 1.29
TRAIN-Nav-Reverse (Ours)
0.980
0.020
18.81 ± 2.29
TABLE V: TRAIN-Nav: Ablation evaluations with 500 random tests.
Method
SR
CR
TR
NT
Nav-Loco
0.924
0.076
0.000
19.62 ± 2.20
Train-Nav-Loco
0.930
0.066
0.004
19.54 ± 2.26
TRAIN-Direct
0.924
0.076
0.000
19.60 ± 2.15
TRAIN-Warm-Start
0.954
0.042
0.004
20.39 ± 2.81
TRAIN-Clip
0.916
0.076
0.008
21.28 ± 3.38
TRAIN-Loco-DWA
0.894
0.078
0.028
18.90 ± 3.23
TABLE VI: TRAIN: Ablation evaluations and benchmark with 500 random tests.
Fig. 11: Trajectories and simulation environments for representative failure cases. (a) Top row: A head-on collision between the robot and a dynamic pedestrian. (b) Middle row: A scenario where a pedestrian rear-ends the robot due to reactivity differences. (c) Third row: Trajectory analysis of a collision with a static obstacle. (d) Bottom row: Corresponding 3D simulation snapshots for the obstacle collision cases presented in (c).
Fig. 12: Overview of the hardware deployment pipeline. Visual data for pedestrian tracking is acquired via the onboard camera, while localized navigation is achieved through LiDAR-IMU fusion. These data streams are processed on an external PC to execute the real-time perception-to-action mapping.
Fig. 13: Unseen pedestrian environments. The robot starts at (-4, 0) m with a goal position of (4, 0) m. Static obstacles of varying sizes are randomly distributed along the robot’s path. The initial group of pedestrians is distributed in a circular formation, while the upper and lower rectangles define the movement zones for additional pedestrians.
Scenarios
flat
uneven
slopes
stairs
Success/Failure
3/0
4/1
3/0
6/4
TABLE VII: Hardware implementations.
Fig. 14: Real-world terrain configurations. The top panel displays the indoor rough terrain, while the bottom panels illustrate the stair scenarios.
Method
SR
NT
Fall
Collision
Timeout
DWA
0.82
16.40
3/50
2/50
4/50
TRAIN (ours)
0.92
19.15
3 /50
1 /50
0 /50
TABLE VIII: Hardware implementations with simulated pedestrians and obstacles.
Success
Collision
Fall
NT
Descending
9
1
0
17.38
Ascending
8
1
1
22.65
TABLE IX: Hardware implementations with simulated pedestrians and obstacles on stairs.
Networks
Input shape
Hidden layers
Output shape
Actor
(64,)
512×256×128
(12,)
Critic
(286,)
512×256×128
(1,)
Decoder
(19,)
64×128
(45,)
TABLE X: Locomotion networks: actor, critic, and decoder
Quadruped robots have demonstrated impressive agility in parkour locomotion across complex terrains. However, most systems still rely on human intervention for high-level planning, and autonomous parkour navigation remains underexplored. The key challenges include fine-grained velocity regulation, long-horizon anticipatory behaviors, and tight coupling between perception and embodied execution. To address these challenges, we propose a Multimodal Diffusion Policy (MulDP) that integrates visual perception with robot proprioception and goal information to generate temporally coherent and anticipatory navigation velocity commands, tightly coupling perception with embodied control to enable robust autonomous navigation. To support the training of MulDP, we construct the first Quadruped Parkour Navigation Dataset (QPND), a multimodal dataset that encompasses diverse navigation behaviors and complex terrains. Extensive simulation and real-world experiments demonstrate that MulDP enables robust long-horizon autonomous navigation and effective traversal across complex terrains.
Kangmai Hu, Yueqi Zhang, Peng Zhai +5
College of Intelligent Robotics and Advanced Manufacturing, Fudan University, Shanghai 200433, China
Safe and efficient quadruped navigation over unfamiliar terrain requires predicting terrain-robot interaction before contact: geometry and visual appearance alone cannot reveal how the robot will slip, load its feet, or expend energy. This paper presents a continual learning pipeline that uses locomotion experience to learn these interaction outcomes from pre-contact images and continually updates the predictions as new contacts are observed. Pre-contact descriptors, produced by a DINOv3 backbone model frozen during training, are mapped to five proprioceptive indicators weighted according to measurement reliability: planar foot slip, mean normal ground-reaction force, traction index, cost of transport, and touchdown loading rate. A compact evidential regressor allows us to predict these indicators together with aleatoric and epistemic uncertainty from the visual descriptors. Continual adaptation combines bounded experience replay with a validation gate: candidate models replace the deployed predictor only when they improve performance on recent held-out data while keeping degradation on historical held-out data within a prescribed tolerance. Predictions and epistemic uncertainty are projected into a local multilayer map and combined into a conservative traversability score map whose property weights can be adjusted without retraining. The resulting map is used for downstream navigation tests. The ROS2 implementation supports evaluation on a Unitree Go2 in simulation and on hardware, with models trained separately in each domain. On a sequential hardware stream over three previously unseen terrains, gated replay reduces final anchor negative log-likelihood (NLL) degradation by 23.1% relative to replay without the gate while attaining similar new-terrain adaptation.
Luca Bricarello, João Carlos Virgolino Soares, Alberto Sanchez-Delgado +2
Dynamic Legged Systems Lab, Istituto Italiano di Tecnologia, Genoa, Italy. · Universit`a degli Studi di Genova, Genoa, Italy.
Navigating quadruped robots in unstructured 3D environments poses significant challenges, requiring goal-directed motion, effective exploration to escape from local minima, and posture adaptation to traverse narrow, height-constrained spaces. Conventional approaches employ a sequential mapping-planning pipeline but suffer from accumulated perception errors and high computational overhead, restricting their applicability on resource-constrained platforms. To address these challenges, we propose Hierarchical Posture-Adaptive Navigation (HiPAN), a framework that operates directly on onboard depth images at deployment. HiPAN adopts a hierarchical design: a high-level policy generates strategic navigation commands (planar velocity and body posture), which are executed by a low-level, posture-adaptive locomotion controller. To mitigate myopic behaviors and facilitate long-horizon navigation, we introduce Path-Guided Curriculum Learning, which progressively extends the navigation horizon from reactive obstacle avoidance to strategic navigation. In simulation, HiPAN achieves higher navigation success rates and greater path efficiency than classical reactive planners and end-to-end baselines, while real-world experiments further validate its applicability across diverse, unstructured 3D environments.