CART: Context-Aware Terrain Adaptation using Temporal Sequence Selection for Legged Robots

Kartikeya Singh; Youngjin Kim; Yash Turkar; Karthik Dantu

cs.ROApr 15, 2026

CART: Context-Aware Terrain Adaptation using Temporal Sequence Selection for Legged Robots

Authors: Kartikeya Singh, Youngjin Kim, Yash Turkar, Karthik Dantu

Organizations: DRONES LAB, University at Buffalo, NY, USA

Abstract

Animals in nature combine multiple modalities, such as sight and feel, to perceive terrain and develop an understanding of how to walk on uneven terrain in an efficient manner. Similarly, legged robots need to develop their ability to stably walk on complex terrains by developing an understanding of the relationship between vision and proprioception. Most current terrain-adaptation methods remain susceptible to failure on complex off-road terrain because they do not explicitly model the context between exteroceptive terrain appearance and proprioceptive physical interaction. This experience-based learning often creates a Visual-Texture Paradox between what has been seen and how it actually feels. In this work, we introduce CART, a high-level controller built on a context-aware terrain adaptation approach that integrates proprioception and exteroception from onboard sensing to achieve a robust understanding of terrain. We evaluate our method on multiple terrains using the Unitree Go2 and ANYmal-C robot on the IsaacSim simulator and a Boston Dynamics SPOT robot for our real-world experiments. To evaluate whether the learned context improves locomotion behavior under the various paradox circumstances, we measure the robot s stability, traversal success, and task completion time in both simulation and real-world experiments. We compare CART against state-of-the-art locomotion and terrain- adaptation baselines across diverse terrain conditions. CART improves the average success rate by 5% over the baselines in simulation, while improving context-conditioned locomotion behavior, including up to 41% lower base oscillation in simulation and 22% in the real world, without increasing the time required to complete the locomotion tasks.

Figures & tables

Figure 1: Context switching in the real world using CART : Robot traverse through slushy snow overlayed on grass. On the left, we show the actual image projection from the robot’s perspective that describes the raw image and snow-grass segments using Viswanath et al. (2021) . We show a Visual-Texture paradox instance between vision only Ewen et al. (2022) and CART ’s estimated category.

Figure 2: Overview of CART. Multimodal observations from visual terrain cues $S_{v}$ , terrain geometry $S_{m}$ , and proprioception $S_{p}$ are encoded into a context representation $\hat{S}_{t}=[c_{t};z_{p,t}]$ . CART uses this context to adapt the high-level velocity scale and body-height commands. During inference, TSS scores an offline library of short command sequences $\xi_{i}=\{[\alpha,\Delta h]\}_{i:i+\ell}$ and selects the sequence most consistent with the current robot–terrain interaction. The end-to-end framework is supervised using a rollout cost-based loss ( Equation 3 ).

Figure 3: Effect of lateral foot perturbation on base vibration: Larger perturbation magnitude ( $\Delta q$ ) generally increases base vibration across different walking speeds.

Figure 4: Blind-phase context robustness: (A) Terrain height maps and robot trajectories for two representative blind-phase runs. Gray regions indicate intervals where exteroception is disabled. (B) t-SNE visualization of context embeddings with and without CART. Without CART, exteroceptive dropout produces a larger shift between normal and blind samples. With CART, the proprioceptive context remains grouped by terrain, and normal/blind samples stay close within each terrain cluster. (C) CART-selected velocity scale during traversal and normalized blind-phase metrics. Baseline Rudin et al. (2022) +CART reduces body vibration and foot slip while maintaining comparable traversal time, indicating that CART improves blind-phase locomotion behavior without relying on exteroception during masked intervals.

Figure 5: IsaacSim terrain configurations used for training and testing of all baselines along with CART during our experiments. The top and bottom rows represent the difficulty of the same terrain type used for training and testing.

Figure 6: Real-world experiment scenario with multiple rugged terrains and elevations, including Grass, Mud, Concrete, Gravel, and Mulch. The experiments were conducted over various underlying terrains, such as Grass over Mud and Grass over Gravel. The red markers represent the waypoints, resulting in 7 runs per baseline.

Figure 7: Success rates: Success rates during simulation-based point-to-point traversal experiments across terrains of increasing difficulty. Results are reported over 100 independent trials per terrain.

Simulation		Real World
Method	Time $\downarrow$	Method	Dist. $\uparrow$
PPO	35.15	TROT	4.72
Blind	49.45	CRAWL	2.49
ST	31.91	AMBLE	4.44
		VAPOR	4.66
CART	29.28	CART	5.44

Table 1: Performance across methods. Simulation results are averaged over 10 successful trials per terrain, while real-world results are averaged over 7 traversals per method using the experimental layout shown in Figure 6

		Roll		Pitch		Yaw
Scenario	Method	Angle	Rate	Angle	Rate	Angle	Rate
Simulation
Flat	PPO	$0.12{\pm}0.02$	$0.56{\pm}0.59$	$0.06{\pm}0.00$	$0.39{\pm}0.25$	$0.06{\pm}0.01$	$0.15{\pm}0.04$
	BLIND	$0.05{\pm}0.00$	$0.59{\pm}0.57$	$0.15{\pm}0.04$	$0.39{\pm}0.24$	$0.15{\pm}0.04$	$0.18{\pm}0.08$
	ST	$0.09{\pm}0.01$	$0.66{\pm}0.78$	$0.06{\pm}0.01$	$0.44{\pm}0.31$	$0.08{\pm}0.01$	$0.14{\pm}0.04$
	CART	$\mathbf{0.03{\pm}0.00}$	$\mathbf{0.40{\pm}0.28}$	$\mathbf{0.04{\pm}0.00}$	$\mathbf{0.29{\pm}0.14}$	$\mathbf{0.02{\pm}0.00}$	$\mathbf{0.09{\pm}0.01}$

Table 2: Simulation and Real-World Results: Mean

\pm

variance of body angles (rad) and angular rates (rad/s) across flat and elevated terrains. Simulation results are averaged over 10 independent trials per terrain, while real-world results are averaged over 7 traversals per method using the experimental layout shown in Figure 6 .

Terrain-Aware Robot Locomotion

Terrain-aware robot locomotion studies how robots sense, predict, and adapt movement to uneven, deformable, or otherwise challenging ground.

See all 148 papers

Mar 10, 2026

Towards Terrain-Aware Safe Locomotion for Quadrupedal Robots Using Proprioceptive Sensing

Achieving safe quadrupedal locomotion in real-world environments has attracted much attention in recent years. When walking over uneven terrain, achieving reliable estimation and realising safety-critical control based on the obtained information is still an open question. To address this challenge, especially for low-cost robots equipped solely with proprioceptive sensors (e.g., IMUs, joint encoders, and contact force sensors), this work first presents an estimation framework that generates a 2.5-D terrain map and extracts support plane parameters, which are then integrated into contact and state estimation. Then, we integrate this estimation framework into a safety-critical control pipeline by formulating control barrier functions that provide rigorous safety guarantees. Experiments demonstrate that the proposed terrain estimation method provides smooth terrain representations. Moreover, the coupled estimation framework of terrain, state, and contact reduces the mean absolute error of base position estimation by 64.8%, decreases the estimation variance by 47.2%, and improves the robustness of contact estimation compared to a decoupled framework. The terrain-informed CBFs integrate historical terrain information and current proprioceptive measurements to ensure global safety by keeping the robot out of hazardous areas and local safety by preventing body-terrain collision, relying solely on proprioceptive sensing.

Peiyu Yang, Jiatao Ding, Wei Pan +2

Dynamic Legged Systems, Istituto Italiano di Tecnologia (IIT), Italy · Department of Industrial Engineering, University of Trento, Italy · Department of Computer Science, University of Manchester, UK +2

Terrain-Aware Robot Locomotion Quadruped Locomotion Robot State Estimation Control Barrier Functions

Ask PDF

Jun 17, 2026

CTS-MoE: Implicit Terrain Adaptation via Mixture-of-Experts for Perceptive Locomotion

Perceptive legged locomotion over discontinuous terrain (e.g., stairs, gaps, and obstacles) requires adaptive behavior, as a single conservative gait cannot produce the anticipatory maneuvers needed for abrupt topology changes. Cast as multi-task reinforcement learning, this problem introduces a tension between sharing and separation. Tasks use a common locomotion base but have conflicting rewards, so a policy must share behavior while avoiding value interference. Prior work addresses only one side, with monolithic policies sacrificing specialization and hierarchical sub-policies sacrificing generalization across transitions and unseen terrain. We propose CTS-MoE, which combines a dense mixture-of-experts actor with perception-based gating to compose shared behaviors and a multi-critic with task-specific value heads to prevent interference. The model is trained end-to-end in a single-stage concurrent teacher-student setup that handles partial observability and avoids sequential distillation, with task labels used only during training. At deployment, routing depends solely on perception, allowing terrain adaptation without a high-level selector or terrain classifier. Experiments on a Unitree Go1 in simulation and on hardware across seen and unseen terrains show task-aware specialization, with lower tracking error and higher success rates than monolithic baselines. Project Website: https://cts-moe.github.io/ .

Francisco Affonso, Matheus P. Angarola, Ana Luiza Mineiro +3

University of Illinois Urbana-Champaign · University of São Paulo

Robot Policy Adaptation Terrain-Aware Robot Locomotion RL for Legged Locomotion Multi-Task RL Mixture-of-Experts Models

Ask PDF

Apr 3, 2026

Learning Locomotion on Complex Terrain for Quadrupedal Robots with Foot Position Maps and Stability Rewards

Quadrupedal locomotion over complex terrain has been a long-standing research topic in robotics. While recent reinforcement learning-based locomotion methods improve generalizability and foot-placement precision, they rely on implicit inference of foot positions from joint angles, lacking the explicit precision and stability guarantees of optimization-based approaches. To address this, we introduce a foot position map integrated into the heightmap, and a dynamic locomotion-stability reward within an attention-based framework to achieve locomotion on complex terrain. We validate our method extensively on terrains seen during training as well as out-of-domain (OOD) terrains. Our results demonstrate that the proposed method enables precise and stable movement, resulting in improved locomotion success rates on both in-domain and OOD terrains.

Matthew Hwang, Yubin Liu, Ryo Hakoda +1

University of Tokyo, Japan

Quadruped Locomotion Terrain-Aware Robot Locomotion RL for Legged Locomotion Reinforcement Learning

Ask PDF

Abstract

Terrain-Aware Robot Locomotion

Explore similar work

Towards Terrain-Aware Safe Locomotion for Quadrupedal Robots Using Proprioceptive Sensing

CTS-MoE: Implicit Terrain Adaptation via Mixture-of-Experts for Perceptive Locomotion

Learning Locomotion on Complex Terrain for Quadrupedal Robots with Foot Position Maps and Stability Rewards