cs.RONov 2, 2026

MS-DOTT: A Multi-Modal Dataset for Off-Road Traversability Estimation

Authors: Shivani Subramanyan, Saksham Sharma, Samahith SA, Ahaan Kotian, Suresh Sundaram

Organizations: Dept. of Aerospace Engineering, Indian Institute of Science, Bengaluru, India

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Autonomous robots are gaining traction in off-road environments, driven by their rising demand in agriculture, defense, and search-and-rescue operations. However, these environments are inherently unpredictable and navigation remains a formidable challenge given the complex environmental conditions including challenging terrain and varying illumination. Off-road autonomy requires robust perception across multiple spatial scales and imaging modalities, yet most sensing setups struggle to capture both far-field environmental context and near-field terrain detail simultaneously across various illumination conditions. To facilitate development of robust off-road perception and traversability estimation systems, we propose MS-DOTT (Multi-Sensor Dataset for Off-road Terrain Traversability) a comprehensive multi-modal off-road dataset with long-range 3D sensing and short-range, fine-grained perception to facilitate high-fidelity terrain understanding by combining proprioceptive signals for tracking vehicle dynamics with exteroceptive imaging and pointcloud data. The dataset includes trajectories recorded across nominal lighting, low-light, and no-light conditions with diverse 2D and 3D sensing for both long-range large-scale and short-range high-resolution sensing, as well as 4D RADAR measurements that provide velocity estimates for detected pointclouds. We demonstrate the utility of the dataset by adapting a SOTA multi-sensor fusion method for the purpose of traversability estimation. This dataset will aid the robotics community in advancing research and development of methods for traversability estimation, off-road perception and multi-sensor fusion in unstructured environments.

Figures & tables

Fig. 1 : MS-DOTT dataset overview. MS-DOTT consists of various complementary sensor data which allows for comprehensive terrain traversability estimation in both day and night conditions.
DatasetThermalRGBToFLiDARRadarIMUStereoPlatformIllumination
RUGD [ 13 ]✗✓✗✓✗✓✗UGVDay
RELLIS-3D [ 14 ]✗✓✗✓✗✓✓UGVDay
CaT [ 22 ]✗✓✗✓✗✓✗ATVDay & Night
TartanDrive 2.0 [ 21 ]✗✓✗✓✗✓✓ATVDay
ORFD [ 19 ]✗✓✗✓✗✓✗CarDay & Night
GOOSE [ 15 ]✓✓✗✓✓✓✗CarDay
TABLE I : Comparison between Off-Road Datasets
Fig. 2 : MS-DOTT Data Collection Bench.
Sensor NameOrientationHzRangeFOV (HxV)Mean points per scanPurpose
Livox HAPForward10 Hz150m120° ×\times 25°44.91 ×103\times 10^{3}Long-range environmental awareness, predictive path planning, and place recognition.
Livox MID360Downward10 Hz40m360° ×\times 52°19.83 ×103\times 10^{3}Mid-range mapping, traditional terrain traversability baseline.
ToF: Lucid Helios2+Downward15 Hz5m (8.3m max)67° ×\times 51°233.16 ×103\times 10^{3}Short-range, fine-grained terrain understanding and micro-obstacle detection.
ARS548 RDI RADARForward10 Hz300m120° ×\times 30°3 ×102\times 10^{2}Weather agnostic perception
TABLE II : 3D Sensor Specifications and Functional Roles within the MS-DOTT Dataset Platform
Sensor NameModalityHzResolutionFOVPurpose
Trimer TRA_L6D_ZDNThermal30 Hz640x48030° ×\times 23°Short-range terrain understanding using thermal perception in low/no light.
FLIR Blackfly-SRGB30 Hz1920x108048° ×\times 36°Short-range, baseline RGB data for terrain understanding in daytime.
Lucid Vision Labs TDR054SHDR RGB10 Hz1920x108070° ×\times 50°Short-range, fine-grained terrain understanding robust to low-light and glares.
ZED2 Stereo CameraRGB + Stereo depth10 Hz1280x960110° ×\times 70°Maximum 3D detection range of 20m, auxiliary sensor for short-range terrain understanding.
TABLE III : Vision Sensor Specifications and Functional Roles within the MS-DOTT Dataset Platform
Sensor NameModalityHzSpecsPurpose
Xsens MTi-680GINS400 Hz9-axis INS, L1/L2 RTK GNSS receiver, 0.2° roll/pitch accuracy, 0.5° heading accuracyLocalization and vehicle inertial data
TABLE IV : Proprioceptive Sensor Specifications and Functional Roles within the MS-DOTT Dataset Platform
Varying TerrainIlluminationTrajectoriesDistanceRuntimeCamera FramesPointclouds
Muddy terrain, grassy fields, loose gravel paths, and undulating landscapesDay and Night113000m90 mins432,000243,000
TABLE V : Data Collection
Fig. 3 : Performance comparison of different visual sensors across diverse lighting environments. All costmaps (Bottom) are generated by fusion of Mid360 LiDAR and corresponding camera inputs (Up). (a) shows the effect of glares and shadows while (b) shows performance in nighttime conditions. Red arrow depicts pose of ego robot.
Fig. 4 : Performance comparison of different LiDAR sensors for multi-scale traversability estimation. The image on the left displays the reference FLIR camera frame. The images on the right illustrate the pointclouds and corresponding costmaps arranged vertically from top to bottom for the Helios, MID360, and HAP sensors, respectively. Red arrow depicts pose of ego robot.
More in this topic

Multimodal Dataset

See all 64 papers

Explore similar work

CardsList
Mar 10, 2026

STONE Dataset: A Scalable Multi-Modal Surround-View 3D Traversability Dataset for Off-Road Robot Navigation

Reliable off-road navigation requires accurate estimation of traversable regions and robust perception under diverse terrain and sensing conditions. However, existing datasets lack both scalability and multi-modality, which limits progress in 3D traversability prediction. In this work, we introduce STONE, a large-scale multi-modal dataset for off-road navigation. STONE provides (1) trajectory-guided 3D traversability maps generated by a fully automated, annotation-free pipeline, and (2) comprehensive surround-view sensing with synchronized 128-channel LiDAR, six RGB cameras, and three 4D imaging radars. The dataset covers a wide range of environments and conditions, including day and night, grasslands, farmlands, construction sites, and lakes. Our auto-labeling pipeline reconstructs dense terrain surfaces from LiDAR scans, extracts geometric attributes such as slope, elevation, and roughness, and assigns traversability labels beyond the robot's trajectory using a Mahalanobis-distance-based criterion. This design enables scalable, geometry-aware ground-truth construction without manual annotation. Finally, we establish a benchmark for voxel-level 3D traversability prediction and provide strong baselines under both single-modal and multi-modal settings. STONE is available at: https://konyul.github.io/STONE-dataset/

Konyul Park, Daehun Kim, Jiyong Oh +7

Interdisciplinary Program in Artificial Intelligence, Seoul National University, Seoul, 08826, Korea. · Department of Electrical and Computer Engineering, Seoul National University, Seoul, 08826, Korea.

Off-Road Robot NavigationRobot Traversability Estimation3D Occupancy Prediction
AskPDF
Jan 31, 2025

GO: The Great Outdoors Multimodal Dataset

The Great Outdoors (GO) dataset is a multi-modal annotated data resource aimed at advancing ground robotics research in unstructured environments. Existing off-road datasets often lack sensor diversity and exclude vital modalities like thermal and radar that are critical for operation in degraded conditions (e.g., low visibility or adverse weather). To address these gaps, we introduce a large-scale multimodal off-road dataset with six complementary sensor modalities, along with semantic annotations and GPS traces, to support tasks such as semantic segmentation, object detection, and SLAM. The diverse environmental conditions represented in the dataset present significant real-world challenges, which provide opportunities to develop more robust solutions to support the continued advancement of field robotics, autonomous exploration, and perception systems in natural environments. The dataset can be downloaded at: https://www.unmannedlab.org/the-great-outdoors-dataset/

Peng Jiang, Kasi Viswanath, Akhil Nagariya +8

Mike Walker ’66 Department of Mechanical Engineering, Texas A&M University, College Station, TX, USA. · DEVCOM Army Research Laboratory, Adelphi, MD, USA. · University of Texas at Austin Center for Autonomy, Austin, TX, USA.

Mobile RoboticsRobotic Perception
AskPDF
Sep 21, 2026

Learning to Drive on Mars: Visual Multimodal Traversability Estimation for Off-World Navigation

Autonomous navigation on Mars requires vehicles to distinguish between traversable terrains across diverse and visually challenging environments. However, progress in learning-based navigation for off-world environments has been limited by the lack of large-scale datasets. Since landing in Jezero Crater, the Mars 2020 Perseverance rover has traversed terrain ranging from sandy dunes, rocky patches, and flat bedrocks. As a result, this paper presents a dataset spanning 500 sols and 45km of trajectories driven by both human operators and the onboard planner, ENav. Our dataset contains grayscale stereo image pairs, poses, accelerometer readings, rocker-bogie angles, and estimates of tilt and wheel slip. Building on this dataset, we introduce an uncertainty-aware traversability-estimation framework that learns terrain representations from multimodal driving experience. We compare our proposed method against existing approaches on the Mars 2020 dataset and show that our method achieves an AUROC of 0.874 and an F1 score of 0.758, outperforming the strongest baseline by 0.058 and 0.156, respectively, while also achieving the highest average precision and recall. Finally, we show that the visual representations can be integrated into path planners, such as ENav, on a physical rover test bed. Videos, code, and the M2020 dataset will be available at https://darren-chiu.github.io/learning-to-drive-on-mars.

Darren Chiu, Cole Wilson, Andrei Tumbar +2

University of Southern California, Los Angeles, CA 90089, USA. · Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91109, USA. · Washington State University, Pullman, WA 99164, USA.

Off-Road Robot NavigationSpace RoboticsRobot NavigationSafe Robot NavigationPlanetary Rover Robotics
AskPDF