cs.ROSep 21, 2026

Learning to Drive on Mars: Visual Multimodal Traversability Estimation for Off-World Navigation

Authors: Darren Chiu, Cole Wilson, Andrei Tumbar, Gaurav S. Sukhatme, Steven Myint

Abstract

Autonomous navigation on Mars requires vehicles to distinguish between traversable terrains across diverse and visually challenging environments. However, progress in learning-based navigation for off-world environments has been limited by the lack of large-scale datasets. Since landing in Jezero Crater, the Mars 2020 Perseverance rover has traversed terrain ranging from sandy dunes, rocky patches, and flat bedrocks. As a result, this paper presents a dataset spanning 500 sols and 45km of trajectories driven by both human operators and the onboard planner, ENav. Our dataset contains grayscale stereo image pairs, poses, accelerometer readings, rocker-bogie angles, and estimates of tilt and wheel slip. Building on this dataset, we introduce an uncertainty-aware traversability-estimation framework that learns terrain representations from multimodal driving experience. We compare our proposed method against existing approaches on the Mars 2020 dataset and show that our method achieves an AUROC of 0.874 and an F1 score of 0.758, outperforming the strongest baseline by 0.058 and 0.156, respectively, while also achieving the highest average precision and recall. Finally, we show that the visual representations can be integrated into path planners, such as ENav, on a physical rover test bed. Videos, code, and the M2020 dataset will be available at https://darren-chiu.github.io/learning-to-drive-on-mars.

Explore similar work

CardsList
Date pendingcs.RO

Learning Traversability for Long Horizon Off-Road Navigation

Autonomous navigation across large off-road environments remains a challenging problem. Onboard sensors perceive only the immediate surroundings, yet safe and efficient routes depend on terrain features that extend well beyond the sensor horizon. Geo-spatial data sources such as satellite imagery, aerial LiDAR, and vector maps can close this gap, but learning traversability from them is difficult: dense labels are unavailable at scale, and existing methods rely on short-range sensing. We propose an efficient formulation that learns a continuous traversability map from overhead data, supervised directly by human-driven GPS trajectories and shaped by supervised geometric priors from LiDAR. Alongside the model, we release a dataset, curated from public sources, consisting of 299 scenes spanning ∼ ⁣1,244 km2\sim\!1{,}244\,\mathrm{km}^{2} of diverse terrain, paired with 1,130 km1{,}130\,\mathrm{km} of human driving. In field trials on a Clearpath Warthog across seven routes at two sites,our method achieves trajectories within 5.5%5.5\% of human path length and reduces operator interventions by ∼ ⁣85%\sim\!85\% compared to local-planner-only autonomy.
Kasi Viswanath, Jason M. Gregory, Shaunak Kolhe +1
TraversabilityAutonomous NavigationNavigationLong-Horizon
AskPDF
May 28, 2026cs.CV

MARTIAN: A Rendering Framework for Aerial Mars Imagery from HiRISE Orbital Data

Aerial navigation on Mars requires vision-based pipelines that are robust to the diverse illumination conditions and terrain morphology of the Martian surface. A key bottleneck for training and evaluating such methods is the scarcity of large-scale, annotated aerial datasets. We present MARTIAN, an open-source Blender-based rendering framework that leverages real HiRISE orbital map products to synthesize realistic aerial views of the Martian terrain under controllable lighting conditions and at varying altitudes. MARTIAN generates observations with accurate pose annotations, directly addressing the scarcity of training data for vision-based navigation on Mars. The framework has been validated through its deployment in concurrent work on map-based localization systems for Ingenuity and future Mars rotorcraft, where synthetically trained deep image matchers were successfully evaluated on real Mars imagery. MARTIAN is publicly available at: https://github.com/nasa-jpl/martian.
Dario Pisanti, Georgios Georgakis
Lunar SurfaceVisual-Inertial SystemsPanoramic ImagesVision SensorsTerrainNear-Earth Objects
AskPDF
Jul 20, 2026cs.RO

Distilling Global Traversability Priors for Image-based Affordance Prediction in Off-road Environments

Standard methods for autonomous navigation in unstructured terrain are prone to myopic behaviors in long-horizon scenarios. The use of metric maps built from LiDAR or cameras provides necessary local geometry and semantic information but is strictly limited by depth sensing range. By discarding data beyond the mapping horizon robots suffer from suboptimal, short-sighted decisions. To recover this lost information, we focus on extracting long-range traversability-aware frontiers directly from first-person-view (FPV) images. By leveraging satellite imagery, we compute the set of feasible navigation paths for a dataset of image/pose pairs and use them to supervise our network, reducing the need for extensive human demonstration data. We demonstrate that this approach improves performance in long-range off-road navigation over existing methods by more than 10% in various offline benchmarks and reduces the number of human interventions incurred in a set of real-world experiments. More details can be found at https://theairlab.org/ss_frontiers_iros .
Matthew Sivaprakasam, Samuel Triest, Micah Nye +5
TraversabilityAutonomous NavigationAutonomous Driving PerceptionLight Detection And RangingAffordancesPrior Knowledge+1
AskPDF