Natural-Language to Nav2: Operational Policies for Costmap-Based Navigation
Authors: Jiyeong Chae, Hyunkyo Seo, Heonjae Lee, Kyung-Joon Park
Organizations: Daegu Gyeongbuk Institute of Science and Technology (DGIST) 333 Techno Jungang-daero, Hyeonpung-eup, Dalseong-gun, Daegu, 42988, Republic of Korea
Autonomous mobile robots support material transport in warehouses and industrial facilities, where navigation must adapt to changing operating conditions. Sensor observations reveal physical obstacles, but do not by themselves specify temporary access rules, route preferences, or exceptions known to operators. This paper presents a natural-language policy interface that brings this knowledge into ROS 2 Nav2. A local large language model converts instructions into JSON policies specifying destinations, spatial constraints, and temporal or event conditions. Deterministic checks govern mission acceptance, and three costmap adapters project accepted policies into the existing navigation stack without changing its planner or controller. On 150 researcher-authored English commands, four local models produce 107-142 correct accepted policies, compared with 44 for a rule-based parser, with 3-20 incorrect acceptances per model. Fault injection records rejection of eight endpoint/JSON faults and 29 validator-stage faults. Navigation experiments use correct task policies to evaluate keepouts, zone costs, person-dependent regions, and event-conditioned passage in simulation and on a TurtleBot3. For Tasks 1-3, mean travel times are 18.4-24.5% lower in simulation and 23.1-35.0% lower on the physical testbed than with the baseline lacking those policies. A native KeepoutFilter comparison and two-robot demonstrations examine basic keepout behavior and shared policy projection. The interface connects operator instructions to explicit navigation policies while retaining Nav2's planning and control components.
Figures & tables
Figure 1: The policy interface has three stages. The LLM receives an instruction and registered map labels. Parsing, validation, and execution-context checks govern mission acceptance. Accepted state supplies waypoint goals; policy state and separately maintained restrictions determine the cost contributions. Standard sensor layers, inflation, NavFn, and DWB remain in the navigation stack. The example shows selected policy fields.
Aspect
Native KeepoutFilter [ 6 ]
Proposed policy interface
Policy validation
Filter metadata checks
Whole-candidate schema, conflict, and context checks
Table 1: Keepout filtering and policy management. The native column covers KeepoutFilter; Nav2 task orchestration lies outside that component.
Fields
Interpretation and execution
plan , via_zones
Ordered destinations and required transit zones. The bridge resolves transit waypoints and builds the goal sequence.
exclusions
Zones to keep closed, with optional duration or absolute time interval.
weights
Integer zone costs for traversal preferences.
dynamic_object_rules
Object class and avoidance radius used around detections.
conditional_rules
Zone actions selected by event/state conditions and priority.
Table 2: The JSON policy contract. A candidate may also explicitly reject an instruction using decision and reason_code .
Method
Correct
Incorrect
95% CI (%)
Mean latency
Rule-based parser
44
101
[22.6, 37.1]
0.175 ms
Llama 3.1-8B
107
20
[63.6, 78.0]
2.533 s
Qwen2.5-7B
122
10
[74.3, 86.8]
1.694 s
Qwen3-8B
141
7
[89.0, 96.8]
1.800 s
Qwen3.5-9B
142
3
[89.8, 97.3]
3.011 s
Table 3: Policy conversion on 150 commands per method. Correct and incorrect columns both count accepted policies. The nominal 95% Wilson interval applies to correct acceptance.
Figure 2: Post-validation exact-match accuracy across five phrasing categories, with 30 commands per category. Exact counts and the complete acceptance/rejection breakdown are provided in the Supplementary Material .
Fault class
Injected
Rejected
Stage
Endpoint/JSON
8
8
Before validation
Schema/type
8
8
Validator
Domain/range
11
11
Validator
Time/conflict
10
10
Validator
Table 4: Injected faults and their containment stage. All 37 faults preserve the prior mission state and produce no new policy projection. Six admissible controls are evaluated separately.
Figure 3: Simulation testbed. Aisles are numbered from top to bottom. Tasks 1 and 4 connect WP1 to WP2; Task 2 follows WP5–WP6–WP4–WP3–WP5; Task 3 connects WP7 to WP8. Arrows indicate schematic mission segments.
Figure 4: Physical testbed. Task 1 connects WP1 to WP2, Task 2 WP3 to WP4, Task 3 WP5 to WP6, and Task 4 WP4 to WP1. The map shows the waypoint connections for the four tasks.
Figure 5: Representative simulation Task 1 trajectories. Supplying the closure before navigation changes the initial route and avoids the approach to the blocked aisle.
Figure 6: Task 3 in the physical maze. A detected person activates the policy-defined object region in the costmap used for planning.
Figure 7: Representative physical Task 4 trajectories. The active passage permission allows a more direct route within the same policy-enabled stack. Priority-Off and Priority-On differ only in this policy permission.
Task
Metric
Baseline
Policy
95% CI: Baseline − Policy
1
Time (s)
114.009±0.721
86.125±0.230
[27.356, 28.412]
1
Distance (m)
48.572±0.108
37.245±0.108
[11.226, 11.428]
2
Time (s)
311.369±1.444
238.751±3.643
[69.912, 75.324]
2
Distance (m)
126.152±0.180
96.003±0.177
[29.981, 30.317]
3
Time (s)
142.049±0.315
115.872±0.790
[25.590, 26.764]
3
Distance (m)
61.185±0.140
49.527±0.340
[11.405, 11.911]
Table 5: Simulation navigation: travel time and distance (mean ± sample SD, n=10 per condition). Intervals are approximate 95% Welch confidence intervals for the indicated difference, in the metric’s units.
Task
Metric
Baseline
Policy
95% CI: Baseline − Policy
1
Time (s)
62.179±1.322
40.390±1.213
[20.596, 22.982]
1
Distance (m)
11.993±0.210
8.138±0.190
[3.667, 4.043]
2
Time (s)
55.849±0.687
42.959±1.079
[12.029, 13.751]
2
Distance (m)
10.850±0.080
8.303±0.156
[2.428, 2.666]
3
Time (s)
42.109±2.073
32.160±1.153
[8.341, 11.557]
3
Distance (m)
8.020±0.352
6.394±0.216
[1.348, 1.904]
Table 6: Physical-testbed navigation: travel time and distance (mean ± sample SD, n=10 per condition). Intervals are approximate 95% Welch confidence intervals for the indicated difference, in the metric’s units.
Method
Completed
Distance (m)
Time (s)
Native KeepoutFilter
5/5
37.339±0.103
86.430±0.236
Custom KeepoutLayer
5/5
37.300±0.131
86.351±0.329
Table 7: Direct keepout control, with five restarted runs per condition. Values are mean ± sample SD. Travel time starts at first detected motion and ends at the waypoint-action completion result.
Figure 8: Shared policy projection in the physical two-robot demonstration. Matching policy-induced regions are visible in both costmaps. Each robot retains its own sensor layers, localization, planner, and controller.
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Raw
Post-V
Accuracy (%)
95% CI (%)
Latency
Rule-based parser
44/150
44/150
29.3
[22.6, 37.1]
0.175 ms
Llama 3.1-8B
106/150
107/150
71.3
[63.6, 78.0]
2.533 s
Qwen2.5-7B
122/150
122/150
81.3
[74.3, 86.8]
1.694 s
Qwen3-8B
141/150
141/150
94.0
[89.0, 96.8]
1.800 s
Qwen3.5-9B
142/150
142/150
94.7
[89.8, 97.3]
3.011 s
Appendix
Table S1: Policy-conversion results on 150 commands. Raw and Post-V count policies matching the reference before and after validation, respectively. The 95% Wilson interval is nominal and applies to Post-V. Latency concerns candidate generation only.
Method
L0
L1
L2
L3
L4
Rule-based parser
60.0
20.0
30.0
26.7
10.0
Llama 3.1-8B
83.3
80.0
80.0
53.3
60.0
Qwen2.5-7B
83.3
86.7
83.3
70.0
83.3
Qwen3-8B
100.0
93.3
100.0
90.0
86.7
Qwen3.5-9B
100.0
93.3
100.0
86.7
93.3
Appendix
Table S2: Post-validation exact-match accuracy (%) by phrasing level, with 30 commands per level.
Accepted
Rejected by
Method
Correct
Incorrect
Interpreter
Parsing
Validator
Rule-based parser
44
101
5
0
0
Llama 3.1-8B
107
20
0
0
23
Qwen2.5-7B
122
10
3
0
15
Qwen3-8B
141
7
1
0
1
Qwen3.5-9B
142
3
4
0
1
Appendix
Table S3: Mutually exclusive processing outcomes, each row summing to 150. Interpreter rejection denotes an explicit model or parser rejection.
Method
C1
C2
C3
C4
C5
Recognized
Blocked
Rule-based parser
20
10
0
40
50
24/100
24/100
Llama 3.1-8B
55
55
90
15
10
45/100
96/100
Qwen2.5-7B
85
90
90
55
65
77/100
97/100
Qwen3-8B
100
90
100
85
85
92/100
99/100
Qwen3.5-9B
100
100
100
100
100
100/100
100/100
Appendix
Table S4: Contradiction recognition by category (%, 20 commands each) and overall counts out of 100. Recognized denotes explicit contradiction recognition; Blocked denotes rejection at the policy-acceptance boundary.
Method
Recognition
Boundary containment
Rule-based parser
[16.7, 33.2]
[16.7, 33.2]
Llama 3.1-8B
[35.6, 54.8]
[90.2, 98.4]
Qwen2.5-7B
[67.8, 84.2]
[91.5, 99.0]
Qwen3-8B
[85.0, 95.9]
[94.6, 99.8]
Qwen3.5-9B
[96.3, 100.0]
[96.3, 100.0]
Appendix
Table S5: Nominal 95% Wilson intervals (%) for the 100-command contradiction results. The same command-level and shared-category limitations apply as in the conversion evaluation.
Case
Input and context
Expected behavior
T1
Valid policy
Accept constraints and construct the requested goal sequence.
T2
New mission
Replace the snapshot without inheriting omitted ordinary constraints.
T3a
Exclusion and required transit in one policy
Reject the update and preserve the preceding mission.
T3b
Required transit under a timed keepout
Reject without silently deleting required transit.
T3c
Required transit under a permanent keepout
Reject and retain the preceding mission.
T3d
Required transit after expiry
Accept the candidate with transit retained.
Appendix
Table S6: Node-level checks on the tested implementation. All eight expected outcomes are observed.
Figure S1 : Expiry of a Zone A keepout during one simulation run. The destination remains active as the policy contribution is removed and Nav2 selects a route through the released region.
Task
Metric
Baseline
Policy
95% CI: Baseline − Policy
1
Time (s)
114.009±0.721
86.125±0.230
[27.356, 28.412]
1
Distance (m)
48.572±0.108
37.245±0.108
[11.226, 11.428]
2
Time (s)
311.369±1.444
238.751±3.643
[69.912, 75.324]
2
Distance (m)
126.152±0.180
96.003±0.177
[29.981, 30.317]
3
Time (s)
142.049±0.315
115.872±0.790
[25.590, 26.764]
3
Distance (m)
61.185±0.140
49.527±0.340
[11.405, 11.911]
Appendix
Table S7: Simulation navigation: mean ± sample SD, n=10 per condition. Approximate 95% Welch intervals are for the difference between conditions.
Task
Metric
Baseline
Policy
95% CI: Baseline − Policy
1
Time (s)
62.179±1.322
40.390±1.213
[20.596, 22.982]
1
Distance (m)
11.993±0.210
8.138±0.190
[3.667, 4.043]
2
Time (s)
55.849±0.687
42.959±1.079
[12.029, 13.751]
2
Distance (m)
10.850±0.080
8.303±0.156
[2.428, 2.666]
3
Time (s)
42.109±2.073
32.160±1.153
[8.341, 11.557]
3
Distance (m)
8.020±0.352
6.394±0.216
[1.348, 1.904]
Appendix
Table S8: Physical navigation: mean ± sample SD, n=10 per condition. Approximate 95% Welch intervals are for the difference between conditions.
Task
Metric
Difference
t
Degrees of freedom
p
Simulation
1
Time (s)
27.884
116.5134
10.8129
4.165×10−18
1
Distance (m)
11.327
234.5180
18.0000
7.968×10−33
2
Time (s)
72.618
58.5999
11.7599
7.129×10−16
2
Distance (m)
30.149
377.6629
17.9949
1.535×10−36
3
Time (s)
26.177
97.3314
11.7912
1.689×10−18
Appendix
Table S9: Approximate two-sided Welch test statistics. Differences are Baseline − Policy for Tasks 1–3 and Priority-Off − Priority-On for Task 4. No multiple-comparison adjustment is applied.
Figure S2 : Physical Task 1. The policy-informed route avoids the declared closure.
Figure S3 : Simulation Task 2. Both panels show the complete multi-waypoint mission.
Figure S4 : Physical Task 2. Zone costs change the selected corridor.
Figure S5 : Simulation Task 3 with the person-dependent cost region.
Figure S6 : Physical Task 3. The policy changes the route taken through the maze.
Figure S7 : Simulation Task 4. This event-permission comparison is distinct from Baseline versus Policy.
Figure S8 : Shared policy projection on two simulated robots. The policy contribution agrees even when local sensor information and planned paths differ.
Model
GPU Δ (pp)
VRAM Δ (MiB)
CPU (%)
Host RSS (GiB)
Qwen2.5-7B
54.7
4547
16.8
0.590
Qwen3-8B
58.3
5319
15.6
0.609
Qwen3.5-9B
58.7
6438
17.4
1.005
Llama 3.1-8B
61.8
5043
12.8
0.567
Appendix
Table S10: Local-inference resources. GPU change is in percentage points (pp). Host memory is RSS.
Type
Llama
Qwen2.5
Qwen3
Qwen3.5
D: destination or visit-order error
7
2
2
1
R: requested keepout or person-avoidance omission
2
1
2
1
N: cost omission, target, or role error
3
2
1
0
E: event condition, action, or priority error
0
0
0
0
A: unrequested transit or avoidance constraint
14
7
3
1
Distinct incorrect acceptances
20
10
7
3
Appendix
Table S11: Accepted-error types in stored LLM outputs. Columns follow the model order used in Table 3 ; types overlap.
Command
Policy difference
Potential operational effect
N01-L1-V2
Adds person avoidance with radius 1.5 m.
Additional avoidance may alter route choice.
N02-L1-V2
Keeps WP7 but omits the required preceding WP2 stop.
The requested pickup or visit may be skipped.
N09-L3-V2
Omits the C keepout while retaining transit B, cost 40 in A, and person radius 2 m.
Zone C may remain available to the planner.
Appendix
Table S12: All three Qwen3.5-9B incorrect acceptances and their potential effects, inferred from the recorded policy content.
Human-readable maps provide an intuitive interface for specifying robot destinations, but connecting their schematic geometry to egocentric observations remains challenging. We present NaViRRator, a framework that translates user-specified start and goal locations on such maps into navigation instructions for a pretrained vision-and-language navigation (VLN) policy. Its core method, RouteScribe, separates route inference from verbalization by first generating an explicit route scaffold in map-image coordinates, which a pretrained vision-language model (VLM) converts into a navigation instruction. We construct the scaffold with start--goal line conditional flow matching (SGL-CFM), which deforms a straight start--goal waypoint sequence into a map-conditioned route. During execution, the VLN policy receives only the instruction and egocentric observations, while the map and scaffold remain upstream, allowing executor replacement without retraining the map-to-language modules. Real-world experiments show higher success rates and success weighted by path length (SPL) than direct map-to-instruction generation, A*-based scaffolding, and Gaussian-source conditional flow matching. Qualitative results further show clearer salient turns and better preservation of the intended maneuver sequence, supporting route-grounded language as a modular interface between human-readable maps and pretrained navigation policies.
Ayun Lee, Jiseon Kim, Giseop Kim
Department of Robotics and Mechatronics Engineering, DGIST, Daegu, Republic of Korea
Mobile robots operating in human-centered environments must generate not only collision-free paths but also trajectories that follow local behavioral conventions. Conventional costmap-based navigation emphasizes geometric feasibility and often overlooks such requirements, which can result in socially inappropriate behaviors. This paper presents NORM-Nav, a zero-shot framework that integrates natural language behavioral constraints into costmap-based planning. An LLM parses each instruction into structured constraints and grounds them using real-time vision--LiDAR perception. These constraints are encoded as multi-layer costmaps that represent geometric, semantic, directional, and velocity cues and are directly compatible with standard grid-based planners. Simulation and real-world experiments indicate that NORM-Nav improves task success rates and produces trajectories closer to human references than representative baselines. The project website is available at https://ei-nav.github.io/NORM-Nav.
Dongjie Huo, Junhui Wang, Chao Gao +3
College of Information Science and Technology, Beijing University of Chemical Technology · Institute for AI Industry Research (AIR), Tsinghua University · Institute of Systems Engineering and Collaborative Laboratory for Intelligent Science and Systems, Macau University of Science and Technology +1
Natural language interaction provides an intuitive way for non-expert users to communicate with robotic platforms. However, transforming user requests into executable navigation actions remains a challenging task, requiring the integration of language understanding, environment perception, and autonomous navigation. This work presents a language-driven navigation framework that enables mobile robots to interpret user requests in natural language to move the robot to a destination and autonomously navigate towards it. The framework is composed of modular ROS 2 components that cooperate to transform natural language instructions into navigation actions. Given a natural language request referring to a target in the environment (e.g., "go to the mail box"), the system identifies the referenced object, estimates its position using RGB-D data, and generates a navigation goal, which is then executed through the ROS 2 Nav2 navigation stack. The ROS 2-based implementation facilitates portability across different robotic platforms, requiring only the configuration of the corresponding topics and services. The system is evaluated in both simulation and real-world scenarios using a TurtleBot3 Waffle and a Unitree Go2 robot with a RealSense camera. Experimental results show that the framework successfully interprets both direct commands and contextual requests, generates meaningful natural-language feedback, and navigates towards the desired target. These results demonstrate the feasibility of combining semantic perception and autonomous navigation to provide an intuitive human-robot interaction paradigm. Code will be released as open source upon acceptance.
Jose Martínez-Fajardo, Pablo Pueyo, Fernando Caballero +1