Autonomous Trail Racing with a Bicycle Robot
Organizations: Robotics and AI Institute (RAI), Zurich, Switzerland. · Robotic Systems Lab, ETH Zurich, Zurich, Switzerland. · Robotics and AI Institute (RAI), Cambridge, MA, USA.
Abstract
Navigating off-road trails on a mountain bike requires diverse locomotion skills, including aerial maneuvers, cornering, and slip recovery, paired with sophisticated perception for scene understanding. Translating these capabilities to autonomous, two-wheeled robots remains a formidable challenge. We show that autonomous, high-speed trail navigation can be achieved through a single idea: encoding the trail as a semantic layer of the terrain. In simulation, this layer embeds the navigation task into the training environment; during deployment, it is reconstructed onboard from camera and LiDAR data. First, we introduce a unified learning framework that treats local navigation and locomotion as a joint optimization problem, bypassing the twist or pose command interface of classical control hierarchies. With fixed reward weights, the same framework produces both trail racing and agile parkour, shaped only through terrain semantics and episode length. Second, our perception pipeline combines a frozen vision foundation model with a lightweight dual-head decoder that predicts per-pixel terrain semantics and confidence scores, which are fused with LiDAR range measurements into an egocentric semantic elevation map at 30 Hz onboard an NVIDIA AGX Orin. We validate our framework on the Ultra Mobility Vehicle (UMV), a bicycle robot that autonomously raced across all sections of a kilometer-long, unstructured mountain bike trail at speeds exceeding 5 m/s and averaging 3.26 m/s. In simulation, our policy fails up to an order of magnitude less often than hierarchical baselines while driving faster.
Figures & tables
| Agent | Time | Velocity | Success Rate |
| Human (Med.) | unknown | ||
| Human (Rec.) | unknown | ||
| UMV | |||
| ANYmal |
| part | checkpoints | velocity | distance | time |
|---|---|---|---|---|
| upper | 1 | |||
| 2, 3 | ||||
| 4 9 | ||||
| lower | 10 | |||
| 11 | ||||
| 12, 13, 14 |
| Level | Survival Rate | Time to Goal (s) | Time Not on Trail (s) | Avg Speed (m/s) | |
|---|---|---|---|---|---|
| ours | 0 | 0.99 | 8.82 | 0.013 | 3.29 |
| 1 | 0.98 | 8.88 | 0.015 | 3.30 | |
| 2 | 0.98 | 9.00 | 0.011 | 3.30 | |
| 3 | 0.95 | 9.11 | 0.013 | 3.33 | |
| 4 | 0.91 | 9.61 | 0.038 | 3.30 | |
| baseline-1.5 | 0 | 0.85 | 19.7 | 0.767 | 1.45 |
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
| Setting | TL | DT indoor | DT outdoor |
|---|---|---|---|
| Number of environments | |||
| Number of steps | |||
| Episode length in seconds | |||
| Number of epochs | |||
| Entropy coefficient |
| Type | Observations | Dim | Noise |
|---|---|---|---|
| Actor | Fork angle | 1 | |
| Joint body angles | 2 | ||
| Fork velocity | 1 | ||
| Rear-wheel velocity | 1 | ||
| Joint body velocities | 2 | ||
| IMU angular velocity | 6 |
| Joint | Dim | Type | Action Scale | ||
|---|---|---|---|---|---|
| Fork | 1 | pos | |||
| Body | 2 | pos | |||
| Rear wheel | 1 | vel | |||
| Front wheel | 1 | – | – | – | – |
| reward | |||
|---|---|---|---|
| Fork action | |||
| Wheel action | |||
| Body action | |||
| Joint vel | |||
| Joint power |
| Event | Parameter | Range | Mode |
| Randomize rigid body material | static friction | Startup | |
| dynamic friction | Startup | ||
| Reset root state uniform | pose.z | Reset | |
| (in base frame) | pose.roll | Startup | |
| twist.x | Startup | ||
| Randomize rigid body inertia | relative mass | Startup |
| Dataset | Classes | Train / Eval | Purpose |
| RELLIS-3D Jiang et al. (2021) | 20 3 | 3,302 / 983 | Label efficiency |
| Cityscapes Cordts et al. (2016) | 19 3 | 2,975 / – | OOD confidence |
| Lost and Found Pinggera et al. (2016) | – | – / 1,203 | OOD evaluation |
| ORFD Min et al. (2022) | 3 | 8,392 / 2,193 | Inference quality |
| Indoor trail (ours) | 3 | 57 / 15 | Deployment (bicycle) |
| Outdoor trail (ours) | 5 | 28,763 / 3,196 | Deployment (bicycle) |
| Sequence | Scene | mIoU [%] | Acc [%] | Training time [s] |
|---|---|---|---|---|
| SNOW-2 | alpine snow field | 90.7 | 97.4 | 177 |
| ETH-2 | indoor building | 89.0 | 97.1 | 90 |
| ICE-1 | ice cave | 77.3 | 88.8 | 123 |
| mIoU [%] | |
|---|---|
| 20 | |
| 50 | |
| 100 | |
| 200 | |
| 500 | |
| 1,000 |
| Configuration | ms | FPS | mIoU | Acc |
|---|---|---|---|---|
| Backbone only (PyTorch) | 17.74 | 56 | – | – |
| Backbone only (TRT) | 4.38 | 228 | – | – |
| Bilinear + seg 1/4 | 5.34 | 187 | 83.81 | 93.73 |
| Bilinear + seg 1/2 | 8.82 | 113 | 83.85 | 93.75 |
| Bilinear + seg full | 21.42 | 47 | 83.86 | 93.75 |
| JAFAR + seg 1/4 ⋆ | 10.11 | 99 | 87.90 | 95.19 |
| Policy | Level | Survival Rate | Time to Goal | Time Not on Trail | Avg Speed |
|---|---|---|---|---|---|
| (s) | (s) | (m/s) | |||
| policy-1 | 0 | 0.91 | 11.3 | 0.023 | 2.58 |
| 1 | 0.90 | 11.3 | 0.022 | 2.59 | |
| 2 | 0.88 | 11.5 | 0.020 | 2.58 | |
| 3 | 0.82 | 11.8 | 0.021 | 2.58 | |
| 4 | 0.70 | 12.3 | 0.057 | 2.55 |
Off-Road Robot Navigation
Off-road robot navigation studies how ground robots perceive terrain, assess traversability, and plan safe routes through unstructured environments.
See all 59 papers