AnyNav: Visual Neuro-Symbolic Friction Learning for Off-road Navigation

Taimeng Fu; Zitong Zhan; Zhipeng Zhao; Yi Du; Shaoshu Su; Xiao Lin; Ehsan Tarkesh Esfahani; Karthik Dantu

Figure 1: We tackle the challenges of off-road navigation by developing AnyNav, a neuro-symbolic framework for predicting friction coefficient and integrating these predictions as physical constraints in path and speed planning for reliable navigation.

Figure 2: AnyNav comprises a neuro-symbolic friction learning module and a physics-informed navigation module. The neuro-symbolic module has a neural network to predict friction coefficients from visual inputs, guided by symbolic physics reasoning for self-supervised training. The inputs are captured by camera and LiDAR before entering a terrain patch, while training occurs afterward using the dynamics observed on that patch. The navigation module builds terrain property maps online and leverages the predicted friction information to generate physically feasible and efficient path and speed profiles for off-road driving.

Figure 3: An example of the Stribeck curve . The plot illustrates the friction coefficient of a terrain material with respect to tire sliding speed. The shape of the curve is governed by four coefficients, which vary across different terrain types.

Figure 4: The configurable vehicle model . The blue dots represent the mass distribution, with their size proportional to the magnitude of the mass at each point.

M

is the total mass,

\mathbf{pt}_{\text{COM}}

is the center of mass,

\mathbf{I}

is the rotational inertia matrix,

\mathbf{c}

are wheel contact points, and

r

is wheel radius.

Figure 5: Architecture of the AnyNav system. Blue boxes represent ROS nodes; each operates in parallel processes. The odometry and mapping stacks run at 10 Hz, while the planning stack runs at 2 Hz for path and speed planning and at 10 Hz for PID control.

Figure 6: Illustration of the real-time mapping process. For each frame, the colorized points are projected onto the map grid. New grid cells are instantiated as needed. The updated cells are then used to update the three terrain property layers.

Figure 7: (a) The eight discretized orientations of the vehicle. (b) Example of graph edges: vertex P (east-facing) connects to

\text{Q}_{1}

,

\text{Q}_{2}

, and

\text{Q}_{3}

via left-turn, straight, and right-turn actions, respectively. (c) Depiction of the turning curvature factor:

\eta=4

for consecutive turns (e.g.,

m_{\text{P}}=m_{\text{Q}}=\text{L}

),

\eta=1

for a single turning action (e.g.,

m_{\text{P}}=\text{D},m_{\text{Q}}=\text{L}

), and

\eta=0

for opposite turns (e.g.,

m_{\text{P}}=\text{R},m_{\text{Q}}=\text{L}

) that cancel after smoothing.

Figure 8: Driving data collection in simulated environments, each featuring diverse terrain types and friction properties.

Figure 9: The two real-world robots, Rover and Racer , are both equipped with LiDAR, camera, IMU, and wheel encoders, but differ in size and steering mechanism. The Rover uses differential steering, whereas Racer has Ackermann steering.

	TartanDrive			PhysORD			UKF ( Vosahlik et al., 2021 )			PF ( Vosahlik et al., 2021 )			AnyNav (Ours)
	ATE	RRE	RTE	ATE	RRE	RTE	ATE	RRE	RTE	ATE	RRE	RTE	ATE	RRE	RTE
Asphalt	1.566	27.248	2.524	1.318	28.334	2.594	2.230	27.405	4.479	2.198	17.191	4.483	0.725	9.700	1.523
Rock	0.988	35.496	1.380	1.444	32.495	3.026	2.108	42.280	4.340	1.939	24.183	3.875	0.453	11.310	0.943
Dirt	1.056	30.917	1.536	1.037	34.120	2.215	1.992	28.958	4.193	1.712	29.041	3.611	0.685	8.116	1.392
Grass	2.560	33.450	3.841	0.890	39.178	1.945	2.387	33.816	4.825	2.162	26.090	4.333	0.523	9.153	1.062
Mud	1.316	29.507	2.038	1.290	35.444	2.709	1.631	34.449	3.467	1.387	21.405	2.885	0.664	19.480	1.461

Table 1: Comparison of ATE, RRE, and RTE on our simulation dataset. Trajectories are categorized by their traversed terrain types.

	TartanDrive Dataset			Rover Dataset			Racer Dataset
	ATE	RRE	RTE	ATE	RRE	RTE	ATE	RRE	RTE
TartanDrive	0.754	5.234	0.771	0.187	38.718	0.368	0.784	39.246	1.656
PhysORD	0.674	5.119	0.586	0.179	19.714	0.354	0.684	20.743	1.453
UKF	0.877	4.986	1.800	0.227	29.738	0.458	0.889	33.632	1.699
PF	0.649	7.320	1.232	0.167	26.005	0.332	0.621	29.721	1.532
AnyNav (Ours)	0.310	2.349	0.543	0.107	5.545	0.177	0.488	11.721	0.901

Table 2: Errors of trajectory estimation on real-world data, including TartanDrive ( Triest et al., 2022 ) and ours Rover and Racer data.

Figure 10: Variation of friction coefficients (a, b) and viscous coefficients (c, d) when driving from one terrain type to another.

Figure 11: Friction and viscosity coefficients estimated by our model. The friction coefficient is evaluated using the Stribeck model at

v_{\text{rel}}=1\text{m/s}

. The muddy area exhibits lower friction and higher viscosity compared to the dry soil region.

Figure 12: Constructed Stribeck maps visualized by mapping

\mu_{v}

,

\mu_{d}

, and

\mu_{s}

to the RGB channels, respectively. The gravel and grass regions exhibit clearly different frictional properties.

Figure 13: Top-down views of steering experiments conducted at varying speeds across different terrains. The results show that the predicted maximum safe speed

v_{\text{safe}}

accurately distinguishes between driftless steering and drifting scenarios.

Figure 14: Experimental setup for the slope-climbing test: conducted on four different surface materials using three types of tires, creating a large range of friction conditions.

	Rubber		Plastic		Paper
	Pred.	Meas.	Pred.	Meas.	Pred.	Meas.
Wood	39.2	43	15.8	16	13.4	14
Foam	41.2	46	23.9	23	27.9	25
Cardboard	39.7	44	11.1	10	11.0	12
Shampoo	16.4	15	-	-	-	-

Table 3: The predicted and measured maximum climb angles (in degrees) across various ground and wheel materials.

	ATE	RRE	RTE
$L_{\text{acc}}$ Only	0.685	15.558	1.394
$L_{\text{prior}}$ Only	0.658	14.839	1.405
AnyNav (Ours)	0.583	11.420	1.216

Table 4: Ablation study of the contributions of the acceleration loss

L_{\text{acc}}

and prior knowledge loss

L_{\text{prior}}

for model training.

	TartanDrive Dataset			Rover Dataset			Racer Dataset
	ATE	RRE	RTE	ATE	RRE	RTE	ATE	RRE	RTE
No sim-to-real transfer	0.675	4.450	1.212	0.117	7.315	0.193	0.645	23.666	1.217
Fixed wheel speed	0.656	3.988	1.177	0.115	5.896	0.188	0.517	12.087	0.953
Fixed inertia matrix	0.366	3.478	0.641	0.109	6.793	0.183	0.605	23.472	1.153
AnyNav (Ours)	0.310	2.349	0.543	0.107	5.545	0.177	0.488	11.721	0.901

Table 5: Ablation study comparing no sim-to-real transfer and fixing wheel speed and inertia matrix during the bilevel optimization.

		Garden			Island
		Succ. Rate	Avg. Time	Time Ratio	Succ. Rate	Avg Time	Time Ratio
Traversibility Estimation	Binary ( Ramirez-Robles et al., 2024 )	72%	31.661	1.116	60%	46.544	1.208
Traversibility Estimation	URA* ( Moore et al., 2023 )	26%	53.774	-	8%	62.605	-
Friction Sensitivity	Uniform friction coeff. $\mu=0.2$	90%	42.820	0.969	92%	73.678	0.962
	Uniform friction coeff. $\mu=0.5$	82%	29.765	1.078	92%	46.183	1.063
	Uniform friction coeff. $\mu=0.8$	82%	28.197	1.127	82%	42.408	1.208

Table 6: Comparison of success rate, average time, and time ratio across two Maps. Note that the Average Time and Time Ratio are computed only on successful tasks. The best success rates are marked in bold.

Figure 15: A navigation task where the target is atop an icy crater. The planner without friction prediction (red) attempted to use a shorter path but failed due to the low friction in the icy valley. In contrast, AnyNav (blue) suggests a safe detour with enough friction, successfully completing the task.

Figure 16: An example highlights the advantage of AnyNav, which optimizes travel time by selecting a low-resistance route.

Figure 17: Two navigation examples on the Island map. Task 1 highlights how incorporating friction knowledge into planning reduces travel time. Task 2 showcases the efficiency of our steering-minimized strategy in improving average speed.

Figure 18: Real-world pre-mapping and navigation tests. (a) The environment map is reconstructed through MVS using drone-captured images. (b) The Stribeck coefficient is estimated using the neuro-symbolic module. Different terrain surfaces, such as lawn and road, are clearly differentiated by their respective friction characteristics. (c-e) Multiple navigation tasks are executed. The red points are selected waypoints, and the blue lines are planned trajectories.

	Garden		Island
	Succ. Rate	Avg. Time	Succ. Rate	Avg. Time
Binary	60.00%	76.85	46.00%	116.58
MPPI-based	72.00%	80.33	54.00%	136.38
AnyNav (Ours)	74.00%	85.55	84.67%	122.04

Table 7: Navigation results in the mapping-from-scratch mode. Each task begins with no environmental knowledge.

	Garden		Island
	Succ. Rate	Avg. Time	Succ. Rate	Avg. Time
Binary	66.00%	78.21	51.33%	114.63
MPPI-based	78.00%	88.36	56.00%	145.57
AnyNav (Ours)	85.33%	86.59	88.67%	121.82

Table 8: Navigation results in the online accumulative mapping mode. The system starts in unknown environments, incrementally builds a global terrain map while executing the navigation tasks. Subsequent tasks can leverage information from mapped regions explored during earlier tasks.

Figure 19: Example of navigating in an initially unknown environment. The friction coefficient map gradually expands as the vehicle moves, while the route is continuously replanned based on the updated map. At

t=3s

, the vehicle moves forward along the road as the sides are too slippery. The path segment outside the mapped region is tentative, providing only a rough direction, and will be updated later as the map expands. At

t=9s

, the vehicle clearly observes the dirt road leading uphill, and the planner selects a path along it. At

t=15s

, the vehicle climbs the hill, and by

t=21s

it reaches the goal.

Figure 20: Example of navigating in a real-world curbside area from the start point to waypoints 1-6. The estimated friction coefficient is

\mu\approx 0.7

on the asphalt road and

\mu\approx 0.5

on the adjacent grass regions.

Figure 21: The roughness map is incrementally built from scratch during a navigation task in a dense vegetation region while the path is replanned accordingly. As the goal is on the right, the vehicle initially attempts to turn right at an early stage (e.g.,

t=1s

). However, as it moves forward, the sensors continuously detect obstacles on the right, causing the planner to delay the turning point (e.g.,

t=20s

), and the vehicle proceeds straight to avoid collisions. At

t=40s

, the planner identifies a flat region suitable for turning right. The vehicle is approaching the goal at

t=60s

after the turn.

Figure 22: Real-world navigation in a forest environment with dense bushes and trees. The roughness map guides the vehicle to avoid collision. The right column shows the roughness map’s building process as the vehicle drives toward waypoint 1.

Figure 23: A long-distance navigation real-world test traversing diverse terrain types. The displayed Stribeck map is incrementally constructed during task execution. Purple waypoints are manually assigned, while red paths are generated by our method.

	Simulation		Real-world
	Racetruck	Rockbouncer	Rover	Racer
$P_{\text{throttle}}$	0.5	0.5	0.5	0.2
$D_{\text{throttle}}$	0.05	0.05	0	0.02
$P_{\text{brake}}$	0.2	0.5	-	0.2
$D_{\text{brake}}$	0.02	0.05	-	0.02
$P_{\text{steering}}$	0.02	0.05	0.05	0.02

AnyNav: Visual Neuro-Symbolic Friction Learning for Off-road Navigation

Abstract

Figures & tables

Appendix

Off-Road Robot Navigation

VertiAdaptor: Online Kinodynamics Adaptation for Vertically Challenging Terrain

NeSAM: Neuro-Symbolic Kinodynamics with Soil Adaptation for Off-Road Mobility

Structured Perception Representations for Off-road Dynamics Learning

Abstract

Off-Road Robot Navigation

Explore similar work

VertiAdaptor: Online Kinodynamics Adaptation for Vertically Challenging Terrain

NeSAM: Neuro-Symbolic Kinodynamics with Soil Adaptation for Off-Road Mobility

Structured Perception Representations for Off-road Dynamics Learning