TACOS: Terrain-Aware COntact Supervision for Visual Footstep Planning

Jean-Michel Fortin; William Larrivée-Hardy; Gabriel Jeanson; Olivier Gamache; Daniele De Martini; François Pomerleau; Philippe Giguère

cs.ROOct 27, 2026

TACOS: Terrain-Aware COntact Supervision for Visual Footstep Planning

Authors: Jean-Michel Fortin, William Larrivée-Hardy, Gabriel Jeanson, Olivier Gamache, Daniele De Martini, François Pomerleau, Philippe Giguère

Organizations: Northern Robotics Laboratory, Université Laval, Canada · Mobile Robotics Group, University of Oxford, United Kingdom

Abstract

Humans navigate their environment based on their vision system, anticipating how the ground will respond before committing to a step. Self-supervised learning could provide legged robots the same ability, but the supervisory signal is typically chosen on intuition, and its effect on what the model learns is rarely examined. We present Terrain-Aware COntact Supervision (TACOS), which proposes a meaningful and consistent footstep instability score, based on the residual of a factor graph in legged state estimation. It projects the continuous score into preceding images to supervise a dense visual predictor. We propose two variants, a hard score that exploits hindsight and a causal soft score available immediately. Both metrics are compared against state-of-the-art alternatives in two independent evaluations that share no data or labels. On 1716 manually annotated contacts, both variants outperform every evaluated alternative at detecting instability and ranking its severity, improving pooled severity ranking by 28 % over the strongest baseline. On three outdoor test runs, our metric also trains the most accurate visual predictor, improving rank correlation by 58 % over the strongest alternative, and by a further 25 % when supplemented with onboard context. In a real-world experiment over a 50 m forest trajectory, the trained predictor handles non-geometric terrain properties such as mud and vegetation, while stepping around geometric obstacles like fallen branches. Code is available for reproducibility at https://github.com/norlab-ulaval/TACOS.

Figures & tables

Fig. 1 : Quadruped robot traversing unstructured forest terrain, where roots, branches, and leaves create candidate footholds whose supportiveness cannot be inferred from geometry alone. Close-ups show, on the right, the GoPro view of the foot getting stuck, wedged among exposed roots, and on the left, the resulting robot fall.

Fig. 2 : Offline label-generation pipeline. (Left) Stances are extracted from the onboard contact detector, where F and H denote front and hind, and L and R left and right. (Center) Contact residuals $d_{i,k}$ from a factor graph leg odometry are aggregated over each stance into the instability score $m_{i,\mathrm{hard}}$ . (Right) Footstep placements are computed in the world frame using exteroceptive odometry and FK ( FK ), then projected into an image acquired before contact and rasterized into a sparse label mask.

Fig. 3 : Visual instability predictor. A frozen DINOv3-B encoder feeds a trainable DPT decoder that outputs a dense instability map. Camera roll and pitch ( $\phi,\theta$ ), and recent instability history ( $\bar{m}_{f},\bar{m}_{h}$ ), enter as context tokens through GCA at each decoder stage. The loss is applied only at projected footstep locations.

Fig. 4 : Precision-Recall curves for unstable-foothold classification on the human-labeled dataset, with each metric’s AP ( AP ) in the legend. Our hard and soft residuals are compared against state-of-the-art alternatives. The dashed line marks the random baseline, set by the proportion of unstable footholds in the dataset.

Fig. 5 : Distribution of the proposed hard TACOS metric for stable footholds grouped by annotated instability severity. The insets show representative events recorded by the feet-facing cameras for each severity. Stable footholds concentrate at low values, and the distributions shift upward at each severity level.

Method	Logs	Ice	Slope	Pooled
ProNav [ 7 ]	0.341	0.323	0.312	0.342
GRS [ 29 ]	0.479	0.339	0.245	0.333
OCELOT [ 12 ]	0.313	0.459	0.440	0.400
[ 31 ]	0.693	0.776	0.773	0.700
[ 32 ]	0.797	0.582	0.777	0.653
[ 17 ]	0.749	0.526	0.700	0.566

TABLE I: VUS ( VUS ) per method, per test run and pooled over all runs. Higher is better; best per column in bold , second best underlined .

Fig. 6 : Network predictions on two unseen outdoor scenes for three supervisory signals, with the input image at the top. WVN is supervised at body level, whereas GRS and ours share the same per-footstep labels, while their metrics differ. Darker colors indicate higher predicted cost, each map being normalized between the 5th and 95th percentiles of its own training labels.

Fig. 7 : Footstep-level Spearman correlation between predicted and true cost labels, per label source, over five-fold cross-validation.

Fig. 8 : Real-world experiment in forest terrain. ( Left ), the geometric costmap ( Center ), and the TACOS costmap ( Right ). Corresponding locations mark \raisebox{-.3pt}{\scriptsizeA}⃝ , terrain associated with foot sinkage; \raisebox{-.3pt}{\scriptsizeB}⃝ , a vegetation barrier; and \raisebox{-.3pt}{\scriptsizeC}⃝ , fallen branches. The smaller photographs illustrate the robot’s interaction with these obstacles. Red and white curves show trajectories. The red cross marks where the geometric costmap incorrectly treats the traversable vegetation as an impassable barrier.

$\phi/\theta$	$\bar{m}_{f,h}$	SBB-2	KAB-2	ALB-3	Pooled
–	–	0.510 $\pm$ .023	0.501 $\pm$ .020	0.373 $\pm$ .026	0.557 $\pm$ .007
$\checkmark$	–	0.509 $\pm$ .027	0.522 $\pm$ .029	0.377 $\pm$ .041	0.580 $\pm$ .015
–	$\checkmark$	0.584 $\pm$ .018	0.633 $\pm$ .016	0.612 $\pm$ .018	0.694 $\pm$ .007
$\checkmark$	$\checkmark$	0.589 $\pm$ .022	0.635 $\pm$ .009	0.613 $\pm$ .017	0.694 $\pm$ .008

TABLE II: Ablation of context conditioning with GCA . All configurations use images, and check marks indicate the added roll/pitch (

\phi,\theta

) and instability history (

\bar{m}_{f,h}

). Spearman correlation between predictions and hard residual labels is reported per test run and pooled, as mean

\pm

standard deviation over 5 folds. Higher is better, with best values in bold and second best underlined .

Terrain-Aware Robot Locomotion

Terrain-aware robot locomotion studies how robots sense, predict, and adapt movement to uneven, deformable, or otherwise challenging ground.

See all 149 papers

TactileStep: Sole Tactile Learning for Regulating Foot-Terrain Interaction in Humanoid Locomotion
Sep 24, 2026Zizhuo Wang, Ming-ju Lee, Shaoting Zhu +3Humanoid Robot Locomotion Terrain-Aware Robot Locomotion
Tsinghua University
Soft Surfaced Vision-Based Tactile Sensing for Bipedal Robot Applications
Feb 20, 2026Jaeeun Kim, Junhee Lim, Yu SheRobotic Tactile Sensing Vision-Based Tactile Sensing
Department of Industrial Engineering, Purdue University, West Lafayette, IN 47906, USA · Department of Computer Science, Purdue University, West Lafayette, IN 47906, USA
Learning Contact Representation for Leg Odometry
Jun 3, 2026Emre Girgin, Cagri KilicLegged Robotics Self-Supervised Learning
Department of Aerospace Engineering, Embry Riddle Aeronautical University, United States

Sep 24, 2026J/K move · Enter open · S save

TactileStep: Sole Tactile Learning for Regulating Foot-Terrain Interaction in Humanoid Locomotion

Zizhuo Wang, Ming-ju Lee, Shaoting Zhu, Haozhe Lou+2

Tsinghua University

Humanoid Robot Locomotion Terrain-Aware Robot Locomotion Robot Locomotion RL for Legged Locomotion Robotic Tactile Sensing

Open paper AskPDF arXiv

Humanoid parkour policies can traverse various terrains, but task completion may mask challenges of harsh landings, edge contacts, and unstable stance contacts. Humans naturally regulate foot-terrain interaction through tactile feedback, modulating contact compliance according to terrain stiffness. This highlights a key domain gap between humans and humanoid robots: the absence of rich tactile sensing in most humanoid systems. We address this problem with TactileStep, a deployable tactile learning framework that brings sole pressure sensing into humanoid locomotion control for softer touchdowns and more stable support. TactileStep aligns tactile simulation with the real pressure insole, allowing the policy to learn from the same contact features available on hardware. During training, we use tactile and motion cues to recognize different foot-contact phases and apply phase-aware rewards that encourage safer landing and more stable stance. Evaluated in simulation and on a Unitree G1 humanoid across diverse terrains, TactileStep reduces peak touchdown force by up to 48.8% and peak A-weighted impact noise by up to 30.1 dB over a strong perceptive baseline, while increasing stance contact area by up to 23.8%.

Abstract

Terrain-Aware Robot Locomotion

Explore similar work

TactileStep: Sole Tactile Learning for Regulating Foot-Terrain Interaction in Humanoid Locomotion

Soft Surfaced Vision-Based Tactile Sensing for Bipedal Robot Applications

Learning Contact Representation for Leg Odometry