Off-road navigation is critical for a wide range of field robotics applications from planetary exploration to disaster response. However, it remains a longstanding challenge due to unstructured environments and the inherently complex terrain-vehicle interactions. Traditional physics-based methods struggle to accurately capture the nonlinear dynamics underlying these interactions, while purely data-driven approaches often overfit to specific motion patterns, vehicle geometries, or platforms, limiting their generalization in diverse, real-world scenarios. To address these limitations, we introduce AnyNav, a vision-based friction estimation and navigation framework grounded in neuro-symbolic principles. Our approach integrates neural networks for visual perception with symbolic physical models for reasoning about terrain-vehicle dynamics. To enable self-supervised learning in real-world settings, we adopt the imperative learning paradigm, employing bilevel optimization to train the friction network through physics-based optimization. This explicit incorporation of physical reasoning substantially enhances generalization across terrains, vehicle types, and operational conditions. Leveraging the predicted friction coefficients, we further develop a physics-informed navigation system capable of generating physically feasible, time-efficient paths together with corresponding speed profiles. We demonstrate that AnyNav seamlessly transfers from simulation to real-world robotic platforms, exhibiting strong robustness across different four-wheeled vehicles and diverse off-road environments.
Figures & tables
Figure 1: We tackle the challenges of off-road navigation by developing AnyNav, a neuro-symbolic framework for predicting friction coefficient and integrating these predictions as physical constraints in path and speed planning for reliable navigation.
Figure 2: AnyNav comprises a neuro-symbolic friction learning module and a physics-informed navigation module. The neuro-symbolic module has a neural network to predict friction coefficients from visual inputs, guided by symbolic physics reasoning for self-supervised training. The inputs are captured by camera and LiDAR before entering a terrain patch, while training occurs afterward using the dynamics observed on that patch. The navigation module builds terrain property maps online and leverages the predicted friction information to generate physically feasible and efficient path and speed profiles for off-road driving.
Figure 3: An example of the Stribeck curve . The plot illustrates the friction coefficient of a terrain material with respect to tire sliding speed. The shape of the curve is governed by four coefficients, which vary across different terrain types.
Figure 4: The configurable vehicle model . The blue dots represent the mass distribution, with their size proportional to the magnitude of the mass at each point. M is the total mass, ptCOM is the center of mass, I is the rotational inertia matrix, c are wheel contact points, and r is wheel radius.
Figure 5: Architecture of the AnyNav system. Blue boxes represent ROS nodes; each operates in parallel processes. The odometry and mapping stacks run at 10 Hz, while the planning stack runs at 2 Hz for path and speed planning and at 10 Hz for PID control.
Figure 6: Illustration of the real-time mapping process. For each frame, the colorized points are projected onto the map grid. New grid cells are instantiated as needed. The updated cells are then used to update the three terrain property layers.
Figure 7: (a) The eight discretized orientations of the vehicle. (b) Example of graph edges: vertex P (east-facing) connects to Q1 , Q2 , and Q3 via left-turn, straight, and right-turn actions, respectively. (c) Depiction of the turning curvature factor: η=4 for consecutive turns (e.g., mP=mQ=L ), η=1 for a single turning action (e.g., mP=D,mQ=L ), and η=0 for opposite turns (e.g., mP=R,mQ=L ) that cancel after smoothing.
Figure 8: Driving data collection in simulated environments, each featuring diverse terrain types and friction properties.
Figure 9: The two real-world robots, Rover and Racer , are both equipped with LiDAR, camera, IMU, and wheel encoders, but differ in size and steering mechanism. The Rover uses differential steering, whereas Racer has Ackermann steering.
TartanDrive
PhysORD
UKF ( Vosahlik et al., 2021 )
PF ( Vosahlik et al., 2021 )
AnyNav (Ours)
ATE
RRE
RTE
ATE
RRE
RTE
ATE
RRE
RTE
ATE
RRE
RTE
ATE
RRE
RTE
Asphalt
1.566
27.248
2.524
1.318
28.334
2.594
2.230
27.405
4.479
2.198
17.191
4.483
0.725
9.700
1.523
Rock
0.988
35.496
1.380
1.444
32.495
3.026
2.108
42.280
4.340
1.939
24.183
3.875
0.453
11.310
0.943
Dirt
1.056
30.917
1.536
1.037
34.120
2.215
1.992
28.958
4.193
1.712
29.041
3.611
0.685
8.116
1.392
Grass
2.560
33.450
3.841
0.890
39.178
1.945
2.387
33.816
4.825
2.162
26.090
4.333
0.523
9.153
1.062
Mud
1.316
29.507
2.038
1.290
35.444
2.709
1.631
34.449
3.467
1.387
21.405
2.885
0.664
19.480
1.461
Table 1: Comparison of ATE, RRE, and RTE on our simulation dataset. Trajectories are categorized by their traversed terrain types.
TartanDrive Dataset
Rover Dataset
Racer Dataset
ATE
RRE
RTE
ATE
RRE
RTE
ATE
RRE
RTE
TartanDrive
0.754
5.234
0.771
0.187
38.718
0.368
0.784
39.246
1.656
PhysORD
0.674
5.119
0.586
0.179
19.714
0.354
0.684
20.743
1.453
UKF
0.877
4.986
1.800
0.227
29.738
0.458
0.889
33.632
1.699
PF
0.649
7.320
1.232
0.167
26.005
0.332
0.621
29.721
1.532
AnyNav (Ours)
0.310
2.349
0.543
0.107
5.545
0.177
0.488
11.721
0.901
Table 2: Errors of trajectory estimation on real-world data, including TartanDrive ( Triest et al., 2022 ) and ours Rover and Racer data.
Figure 10: Variation of friction coefficients (a, b) and viscous coefficients (c, d) when driving from one terrain type to another.
Figure 11: Friction and viscosity coefficients estimated by our model. The friction coefficient is evaluated using the Stribeck model at vrel=1m/s . The muddy area exhibits lower friction and higher viscosity compared to the dry soil region.
Figure 12: Constructed Stribeck maps visualized by mapping μv , μd , and μs to the RGB channels, respectively. The gravel and grass regions exhibit clearly different frictional properties.
Figure 13: Top-down views of steering experiments conducted at varying speeds across different terrains. The results show that the predicted maximum safe speed vsafe accurately distinguishes between driftless steering and drifting scenarios.
Figure 14: Experimental setup for the slope-climbing test: conducted on four different surface materials using three types of tires, creating a large range of friction conditions.
Rubber
Plastic
Paper
Pred.
Meas.
Pred.
Meas.
Pred.
Meas.
Wood
39.2
43
15.8
16
13.4
14
Foam
41.2
46
23.9
23
27.9
25
Cardboard
39.7
44
11.1
10
11.0
12
Shampoo
16.4
15
-
-
-
-
Table 3: The predicted and measured maximum climb angles (in degrees) across various ground and wheel materials.
ATE
RRE
RTE
Lacc Only
0.685
15.558
1.394
Lprior Only
0.658
14.839
1.405
AnyNav (Ours)
0.583
11.420
1.216
Table 4: Ablation study of the contributions of the acceleration loss Lacc and prior knowledge loss Lprior for model training.
TartanDrive Dataset
Rover Dataset
Racer Dataset
ATE
RRE
RTE
ATE
RRE
RTE
ATE
RRE
RTE
No sim-to-real transfer
0.675
4.450
1.212
0.117
7.315
0.193
0.645
23.666
1.217
Fixed wheel speed
0.656
3.988
1.177
0.115
5.896
0.188
0.517
12.087
0.953
Fixed inertia matrix
0.366
3.478
0.641
0.109
6.793
0.183
0.605
23.472
1.153
AnyNav (Ours)
0.310
2.349
0.543
0.107
5.545
0.177
0.488
11.721
0.901
Table 5: Ablation study comparing no sim-to-real transfer and fixing wheel speed and inertia matrix during the bilevel optimization.
Garden
Island
Succ. Rate
Avg. Time
Time Ratio
Succ. Rate
Avg Time
Time Ratio
Traversibility Estimation
Binary ( Ramirez-Robles et al., 2024 )
72%
31.661
1.116
60%
46.544
1.208
URA* ( Moore et al., 2023 )
26%
53.774
-
8%
62.605
-
Friction Sensitivity
Uniform friction coeff. μ=0.2
90%
42.820
0.969
92%
73.678
0.962
Uniform friction coeff. μ=0.5
82%
29.765
1.078
92%
46.183
1.063
Uniform friction coeff. μ=0.8
82%
28.197
1.127
82%
42.408
1.208
Table 6: Comparison of success rate, average time, and time ratio across two Maps. Note that the Average Time and Time Ratio are computed only on successful tasks. The best success rates are marked in bold.
Figure 15: A navigation task where the target is atop an icy crater. The planner without friction prediction (red) attempted to use a shorter path but failed due to the low friction in the icy valley. In contrast, AnyNav (blue) suggests a safe detour with enough friction, successfully completing the task.
Figure 16: An example highlights the advantage of AnyNav, which optimizes travel time by selecting a low-resistance route.
Figure 17: Two navigation examples on the Island map. Task 1 highlights how incorporating friction knowledge into planning reduces travel time. Task 2 showcases the efficiency of our steering-minimized strategy in improving average speed.
Figure 18: Real-world pre-mapping and navigation tests. (a) The environment map is reconstructed through MVS using drone-captured images. (b) The Stribeck coefficient is estimated using the neuro-symbolic module. Different terrain surfaces, such as lawn and road, are clearly differentiated by their respective friction characteristics. (c-e) Multiple navigation tasks are executed. The red points are selected waypoints, and the blue lines are planned trajectories.
Garden
Island
Succ. Rate
Avg. Time
Succ. Rate
Avg. Time
Binary
60.00%
76.85
46.00%
116.58
MPPI-based
72.00%
80.33
54.00%
136.38
AnyNav (Ours)
74.00%
85.55
84.67%
122.04
Table 7: Navigation results in the mapping-from-scratch mode. Each task begins with no environmental knowledge.
Garden
Island
Succ. Rate
Avg. Time
Succ. Rate
Avg. Time
Binary
66.00%
78.21
51.33%
114.63
MPPI-based
78.00%
88.36
56.00%
145.57
AnyNav (Ours)
85.33%
86.59
88.67%
121.82
Table 8: Navigation results in the online accumulative mapping mode. The system starts in unknown environments, incrementally builds a global terrain map while executing the navigation tasks. Subsequent tasks can leverage information from mapped regions explored during earlier tasks.
Figure 19: Example of navigating in an initially unknown environment. The friction coefficient map gradually expands as the vehicle moves, while the route is continuously replanned based on the updated map. At t=3s , the vehicle moves forward along the road as the sides are too slippery. The path segment outside the mapped region is tentative, providing only a rough direction, and will be updated later as the map expands. At t=9s , the vehicle clearly observes the dirt road leading uphill, and the planner selects a path along it. At t=15s , the vehicle climbs the hill, and by t=21s it reaches the goal.
Figure 20: Example of navigating in a real-world curbside area from the start point to waypoints 1-6. The estimated friction coefficient is μ≈0.7 on the asphalt road and μ≈0.5 on the adjacent grass regions.
Figure 21: The roughness map is incrementally built from scratch during a navigation task in a dense vegetation region while the path is replanned accordingly. As the goal is on the right, the vehicle initially attempts to turn right at an early stage (e.g., t=1s ). However, as it moves forward, the sensors continuously detect obstacles on the right, causing the planner to delay the turning point (e.g., t=20s ), and the vehicle proceeds straight to avoid collisions. At t=40s , the planner identifies a flat region suitable for turning right. The vehicle is approaching the goal at t=60s after the turn.
Figure 22: Real-world navigation in a forest environment with dense bushes and trees. The roughness map guides the vehicle to avoid collision. The right column shows the roughness map’s building process as the vehicle drives toward waypoint 1.
Figure 23: A long-distance navigation real-world test traversing diverse terrain types. The displayed Stribeck map is incrementally constructed during task execution. Purple waypoints are manually assigned, while red paths are generated by our method.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 24: Analysis of weight transfer of a vehicle on an inclined plane. The vehicle accelerates down an incline at an angle θ . The gravitational force Mg acts on the COM, located at a height h above the slope. The normal forces at the front and rear wheels, FNfront and FNrear , act at contact points cf and cr , respectively. The distances along the slope from the COM to these contact points are denoted as df and dr . The inertial force from the vehicle’s acceleration, −Ma , and the components of gravitational force, Mgsinθ and Mgcosθ , are also shown for analysis of weight transfer.
Autonomous driving in off-road environments presents significant challenges due to the dynamic and unpredictable nature of unstructured terrain. Traditional kinodynamic models often struggle to generalize across diverse geometric and semantic terrain types, underscoring the need for real-time adaptation to ensure safe and reliable navigation. We propose VertiAdaptor (VA), a novel online adaptation framework that efficiently integrates elevation with semantic embeddings to enable terrain-aware kinodynamic modeling and planning via function encoders. VA learns a kinodynamic space spanned by a set of neural ordinary differential equation basis functions, capturing complex vehicle-terrain interactions across varied environments. After offline training, the proposed approach can rapidly adapt to new, unseen environments by identifying kinodynamics in the learned space through a computationally efficient least-squares calculation. We evaluate VA within the Verti-Bench simulator, built on the Chrono multi-physics engine, and validate its performance both in simulation and on a physical Verti-4-Wheeler platform. Our results demonstrate that VA improves prediction accuracy by up to 23.9% and achieves a 5X faster adaptation time, advancing the robustness and reliability of autonomous robots in complex and evolving off-road environments.
Tong Xu, Chenhui Pan, Aniket Datar +1
Department of Computer Science, George Mason University
Accurate prediction of off-road vehicle motion over deformable terrain remains challenging because sinkage, slip, and traction vary with local soil conditions. Existing learning-based kinodynamic models directly approximate vehicle-terrain interactions from data but do not explicitly represent soil mechanics and offer limited physical interpretability. To address these limitations, we present NeSAM, a neuro-symbolic framework that combines differentiable Bekker-Wong terramechanics with learned terrain representations and a Transformer-based residual dynamics model for long-horizon, six degree-of-freedom kinodynamic prediction. The terramechanics component models soil-dependent interaction forces, while the residual model corrects discrepancies between the analytical prediction and the observed vehicle dynamics. NeSAM further estimates physically meaningful soil parameters from terrain observations and updates them online using an extended Kalman filter. We evaluate NeSAM in Verti-Bench, a simulator built on the Chrono multiphysics engine, and validate its performance on a physical Verti-4-Wheeler platform. NeSAM improves prediction accuracy by up to 30% in simulation and 29% on real-world data relative to the strongest compared baselines. When integrated with a close-loop navigation controller, NeSAM further improves traversal success rate through online soil adaptation while reduces Hausdorff distance to the reference trajectory by 69.4%, indicating improved trajectory tracking accuracy.
Chenhui Pan, Tong Xu, Francesco Cancelliere +1
Department of Computer Science, George Mason University, USA · Department of Electrical, Electronic and Computer Engineering, University of Catania, Italy
Autonomously navigating off-road terrain at high speed demands accurate control, which often relies on dynamics models that capture complex interactions between the robot and terrain. In off-road regimes, terrain interaction is a major component of the vehicle's dynamics. Prior work attempts to capture this effect by training learned models of the vehicle's dynamics, conditioned on its perception of the local environment. However, said conditioning is often overly simplistic, ignoring the spatial structure of the dynamics problem. Our approach, SpatialBEV, exploits this spatial structure. At each step, we query the bird's-eye-view (BEV) features around the predicted state, conditioning the next prediction on current geometric and semantic cues. We additionally present a benchmarking approach that mines a dataset's most difficult scenarios. On datasets collected across seasons, SpatialBEV reduces prediction error by 17.1%, compared to physics-based and learned baselines. We further evaluate on hardware on high-speed (≥10 m/s) and geometrically challenging courses with a 25.3% reduction in maximum cross-track error. More details can be found at https://theairlab.org/spatial-bev-dynamics .
Micah Nye, Samuel Triest, Matthew Sivaprakasam +3
The Robotics Institute, Carnegie Mellon University · Pennsylvania State University