An Open-Source LiDAR and Monocular Off-Road Autonomous Navigation Stack

Rémi Marsal; Quentin Picard; Adrien Poiré; Sébastien Kerbourc'h; Thibault Toralba; Clément Yver; Alexandre Chapoutot; David Filliat

cs.ROApr 3, 2026

An Open-Source LiDAR and Monocular Off-Road Autonomous Navigation Stack

Authors: Rémi Marsal, Quentin Picard, Adrien Poiré, Sébastien Kerbourc'h, Thibault Toralba, Clément Yver, Alexandre Chapoutot, David Filliat

Organizations: U2IS, ENSTA, Institut Polytechnique de Paris, Palaiseau, France · AMIAD, Pˆole Recherche, France

Abstract

Off-road autonomous navigation demands reliable 3D perception for robust obstacle detection in challenging unstructured terrain. While LiDAR is accurate, it is costly and power-intensive. Monocular depth estimation using foundation models offers a lightweight alternative, but its integration into outdoor navigation stacks remains underexplored. We present an open-source off-road navigation stack supporting both LiDAR and monocular 3D perception without task-specific training. For the monocular setup, we combine zero-shot depth prediction (Depth Anything V2) with metric depth rescaling using sparse SLAM measurements (VINS-Mono). Two key enhancements improve robustness: edge-masking to reduce obstacle hallucination and temporal smoothing to mitigate the impact of SLAM instability. The resulting point cloud is used to generate a robot-centric 2.5D elevation map for costmap-based planning. Evaluated in photorealistic simulations (Isaac Sim) and real-world unstructured environments, the monocular configuration matches high-resolution LiDAR performance in most scenarios, demonstrating that foundation-model-based monocular depth estimation is a viable LiDAR alternative for robust off-road navigation. By open-sourcing the navigation stack and the simulation environment, we provide a complete pipeline for off-road navigation as well as a reproducible benchmark. Code available at https://github.com/LARIAD/Offroad-Nav.

Figures & tables

Panel of Figure 1 — Fig. 1 : Autonomous navigation with the wheeled ground robot in the real and simulation off-road environment.

Fig. 2 : Diagram of our navigation pipeline. It takes as input either a LiDAR point could or monocular camera images with a SLAM sparse point cloud as well as the robot localization. It returns the velocity commands run by the robot.

Panel of Figure 3 — Fig. 3 : Estimated rescaled depth maps and the extracted $3$ D point clouds from the top RGB image. The left side shows results without edge filtering, while the right side shows results with edge filtering applied.

Panel of Figure 4 — Fig. 4 : Top views of the simulation experiments. From left to right, the easy, medium and hard simulated environments, respectively. In each environment, the goal points of the $10$ , $20$ and $30$ meters scenarios is indicated with an orange, blue and green star, respectively. The reference remotely operated trajectories are given in dashed lines and the ones performed by the robot are shown in solid lines. From top to bottom: the robot trajectories are obtained using the LiDAR (sim-tuned), the LiDAR (real-params) and Mono-VINS (STCD only), respectively.

Panel of Figure 5 — Fig. 5 : Top view for the real-world experimental area where obstacles are placed on an off-road terrain. The first row shows the LiDAR experiments and the second one, the runs with the monocular camera. From left to right: the easy, medium and hard scenario. Obstacles are represented in white circles and rectangles. The dashed lines correspond to the remotely operated reference trajectories.

	Easy			Medium			Hard			Average
	SR $\uparrow$	SPL $\uparrow$	DR $\uparrow$	SR $\uparrow$	SPL $\uparrow$	DR $\uparrow$	SR $\uparrow$	SPL $\uparrow$	DR $\uparrow$	SR $\uparrow$	SPL $\uparrow$	DR $\uparrow$
10m goal
LiDAR (sim-tuned)	1.00	0.70	1.00	1.00	0.73	1.00	0.40	0.31	0.76	0.80	0.58	0.92
LiDAR (real-params)	0.90	0.66	0.95	0.80	0.47	0.93	0.40	0.29	0.72	0.70	0.47	0.87
Mono-VINS (STCD only)	1.00	0.73	1.00	0.80	0.59	0.89	0.50	0.39	0.78	0.77	0.57	0.89
20m goal

TABLE I : Navigation performance across simulated scenarios and goal distances.

	Easy			Medium			Hard			Average
	SR $\uparrow$	SPL $\uparrow$	DR $\uparrow$	SR $\uparrow$	SPL $\uparrow$	DR $\uparrow$	SR $\uparrow$	SPL $\uparrow$	DR $\uparrow$	SR $\uparrow$	SPL $\uparrow$	DR $\uparrow$
LiDAR (real-params)	1.00	0.73	1.00	1.00	0.68	1.00	1.00	0.67	1.00	1.00	0.69	1.00
Mono-VINS	1.00	0.51	1.00	1.00	0.63	1.00	1.00	0.63	1.00	1.00	0.59	1.00

TABLE II : Navigation performance across real scenarios with the Barakuda robot.

Module	LiDAR (Hz)	Mono (Hz)
3D perception (CPU/GPU)	20	10
Elevation mapping (GPU)	16	6
Path planning (CPU)	12	12

TABLE III : Frequency of each module with the Jetson ORIN (MAXN profile) for the LiDAR-based and Monocular-based configurations

Edge masking	Smoothing	Easy	Medium	Hard	Avg.
✗	✗	0.50	0.48	0.19	0.39
✗	✓	0.54	0.61	0.30	0.48
✓	✗	0.63	0.74	0.18	0.52
✓	✓	0.54	0.76	0.45	0.58

TABLE IV : Average SPL performances with different configurations for the depth rescaling

Explore similar work

Mar 2, 2026

Learning Vision-Based Omnidirectional Navigation: A Teacher-Student Approach Using Monocular Depth Estimation

Reliable obstacle avoidance in industrial settings demands 3D scene understanding, but widely used 2D LiDAR sensors perceive only a single horizontal slice of the environment, missing critical obstacles above or below the scan plane. We present a teacher-student framework for vision-based mobile robot navigation that eliminates the need for LiDAR sensors. A teacher policy trained via Proximal Policy Optimization (PPO) in NVIDIA Isaac Lab leverages privileged 2D LiDAR observations that account for the full robot footprint to learn robust navigation. The learned behavior is distilled into a student policy that relies solely on monocular depth maps predicted by a fine-tuned Depth Anything V2 model from four RGB cameras. The complete inference pipeline, comprising monocular depth estimation (MDE), policy execution, and motor control, runs entirely onboard an NVIDIA Jetson Orin AGX mounted on a DJI RoboMaster platform, requiring no external computation for inference. In simulation, the student achieves success rates of 82-96.5%, consistently outperforming the standard 2D LiDAR teacher (50-89%). In real-world experiments, the MDE-based student outperforms the 2D LiDAR teacher when navigating around obstacles with complex 3D geometries, such as overhanging structures and low-profile objects, that fall outside the single scan plane of a 2D LiDAR.

Jan Finke, Wayne Paul Martis, Adrian Schmelter +3

Robot Navigation Vision-Based Robot Control

Ask PDF

Mar 10, 2026

STONE Dataset: A Scalable Multi-Modal Surround-View 3D Traversability Dataset for Off-Road Robot Navigation

Reliable off-road navigation requires accurate estimation of traversable regions and robust perception under diverse terrain and sensing conditions. However, existing datasets lack both scalability and multi-modality, which limits progress in 3D traversability prediction. In this work, we introduce STONE, a large-scale multi-modal dataset for off-road navigation. STONE provides (1) trajectory-guided 3D traversability maps generated by a fully automated, annotation-free pipeline, and (2) comprehensive surround-view sensing with synchronized 128-channel LiDAR, six RGB cameras, and three 4D imaging radars. The dataset covers a wide range of environments and conditions, including day and night, grasslands, farmlands, construction sites, and lakes. Our auto-labeling pipeline reconstructs dense terrain surfaces from LiDAR scans, extracts geometric attributes such as slope, elevation, and roughness, and assigns traversability labels beyond the robot's trajectory using a Mahalanobis-distance-based criterion. This design enables scalable, geometry-aware ground-truth construction without manual annotation. Finally, we establish a benchmark for voxel-level 3D traversability prediction and provide strong baselines under both single-modal and multi-modal settings. STONE is available at: https://konyul.github.io/STONE-dataset/

Konyul Park, Daehun Kim, Jiyong Oh +7

Interdisciplinary Program in Artificial Intelligence, Seoul National University, Seoul, 08826, Korea. · Department of Electrical and Computer Engineering, Seoul National University, Seoul, 08826, Korea.

Off-Road Robot Navigation 3D Occupancy Prediction

Ask PDF

Apr 27, 2026

Pushing Radar Odometry Beyond the Pavement: Current Capabilities and Challenges

Radar offers unique advantages for localization in unstructured environments, including robustness to weather, lighting, and airborne particulates. While most prior work has studied radar odometry in urban, largely planar settings, its performance in off-road environments remains less understood. In this paper, we investigate the potential of radar for off-road odometry estimation and identify key challenges that arise from full

SE(3)

vehicle motion, terrain-induced ground returns, and sparse or unstable features. To address these issues, we introduce two simple baselines: Radar-KISSICP, which applies motion compensation to generate 3D-aware radar pointclouds, and Radar-IMU, which leverages IMU preintegration to stabilize scan matching. Experiments on the Great Outdoors (GO) dataset demonstrate that these baselines improve trajectory estimation in challenging routes and provide a reference point for future development of radar odometry in off-road robotics.

Shaunak Kolhe, Peng Jiang, Maggie Wigness +4

Texas A&M University · DEVCOM Army Research Laboratory

Mobile Robotics Radar-Inertial Odometry Point Cloud Registration Robot Localization

Ask PDF