Achieving full automation in self-driving vehicles remains a challenge, especially in dynamic urban environments where navigation requires real-time adaptability. Existing systems struggle to handle navigation plans when faced with unpredictable changes in road layouts, spontaneous detours, or missing map data, due to their heavy reliance on predefined cartographic information. In this work, we explore the use of Large Language Models to generate Answer Set Programming rules by translating informal navigation instructions into structured, logic-based reasoning. ASP provides non-monotonic reasoning, allowing autonomous vehicles to adapt to evolving scenarios without relying on predefined maps. We present an experimental evaluation in which LLMs generate ASP constraints that encode real-world urban driving logic into a formal knowledge representation. By automating the translation of informal navigation instructions into logical rules, our method improves adaptability and explainability in autonomous navigation. Results show that LLM-driven ASP rule generation supports semantic-based decision-making, offering an explainable framework for dynamic navigation planning that aligns closely with how humans communicate navigational intent.
Figures & tables
Fig. 1: Overview of the proposed pipeline. The figure shows two flows for generating ASP rules: a classical path via a programmer and an LLM-assisted path. This work focuses on automating this translation to overcome the knowledge engineering bottleneck.
Fig. 2: Scene interpretation error. The vehicle control system fails to correctly recognize the intersection shown on the map and, as a result, enters the left-hand road in the wrong direction. Images adapted from [ 9 ] .
Fig. 3: Possible scenarios to reach a gas station, given the instructions.
Fig. 4: Possible scenarios derived from “turn left at the next junction, then go straight, and finally turn right” .
ID
Task
Instruction
ID0
Add constraints
Go straight at the first junction, then turn right.
ID1
Add constraints
Turn left at the next junction, then go straight, then turn right.
ID2
Add constraints
Turn right after the first junction
ID3
Generate Rules and constraints
Go straight at the next intersection.
TABLE I: List of navigation instructions used in the experimental validation, ordered by increasing complexity.
LLM Model
Syntax
Semantic
Lines
ChatGPT4-o
✓
✓
7
Grok 3
✓
✓
8
Gemini-2.0-Flash
✓
✓
6
Meta Llama4
✓
✓
8
Qwen2.5-Max
✗
✗
-
Groq Compound Beta Mini
✓
✗
-
TABLE II: Experiment ID0, as validation criteria in Section IV-A .
LLM Model
Syntax
Semantic
Lines
ChatGPT4-o
✓
✓
12
Grok 3
✓
✓
8
Gemini-2.0-Flash
✓
✓
11
Meta Llama4
✓
✓
8
Qwen2.5-Max
✓
✓
10
Groq Compound Beta Mini
✗
✗
-
TABLE III: Experiment ID1, as validation criteria in Section IV-A .
LLM Model
Syntax
Semantic
Lines
ChatGPT4-o
✓
✓
6
Grok 3
✓
✓
8
Gemini-2.0-Flash
✗
✗
-
Meta Llama4
✓
✓
8
Qwen2.5-Max
✓
✓
9
Groq Compound Beta Mini
✗
✗
-
TABLE IV: Experiment ID2, as validation criteria in Section IV-A .
Fig. 5: Experiment ID3. The black car indicates the starting position. Green arrows represent the two navigation scenarios modeled by the DLV-generated rules.
Autonomous driving develops vehicles that perceive road scenes, predict interactions, and plan and control safe movement with limited or no human input.
Recent advancements in Language Models (LMs) have demonstrated strong semantic reasoning capabilities, enabling their application in high-level decision-making for autonomous driving (AD). However, LMs operate over discrete token spaces and lack the ability to generate continuous, physically feasible trajectories required for motion planning. Meanwhile, diffusion models have proven effective at generating reliable and dynamically consistent trajectories, but often lack semantic interpretability and alignment with scene-level understanding. To address these limitations, we propose KnowDiffuser, a knowledge-guided motion planning framework that tightly integrates the semantic understanding of language models with the generative power of diffusion models. The framework employs a language model to infer context-aware meta-actions from structured scene representations, which are then mapped to prior trajectories that anchor the subsequent denoising process. A two-stage truncated denoising mechanism refines these trajectories efficiently, preserving both semantic alignment and physical feasibility. Experiments on the nuPlan benchmark demonstrate that KnowDiffuser significantly outperforms existing planners in both open-loop and closed-loop evaluations, establishing a robust and interpretable framework that effectively bridges the semantic-to-physical gap in AD systems.
Fan Ding, Xuewen Luo, Fengze Yang +4
Monash University Malaysia. · University of Utah, USA.
Vision-language models (VLMs) have emerged as a promising direction for end-to-end autonomous driving (AD) by jointly modeling visual observations, driving context, and language-based reasoning. However, existing VLM-based systems face a trade-off between high-level reasoning and motion planning: large models offer strong semantic understanding but are costly to adapt for precise control, whereas small VLM models can be fine-tuned efficiently but often exhibit weaker reasoning. We propose NaviDriveVLM, a decoupled framework that separates reasoning from action generation using a large-scale Navigator and a lightweight trainable Driver. This design preserves reasoning ability, reduces training cost, and provides an explicit interpretable intermediate representation for downstream planning. Experiments on the nuScenes benchmark show that NaviDriveVLM outperforms large VLM baselines in end-to-end motion planning.
Ximeng Tao, Pardis Taghavi, Dimitar Filev +2
J. Mike Walker ’66 Department of Mechanical Engineering, Texas A&M University, College Station, TX 77843, USA · The Department of Engineering Technology and Industrial Distribution Texas A&M University, College Station, TX 77843, USA
We propose an architecture for integrating high-level, human-provided safety rules and operator-aligned semantic preferences into autonomous robot navigation in unstructured outdoor environments. In our approach, natural-language rules are translated into Signal Temporal Logic (STL) specifications that guide planning and navigation during runtime. Persistent, environment-centric rules and terrain preferences are grounded into a 2D cost map, while temporally dynamic requirements are expressed as STL specifications to be monitored during runtime. We hypothesize the use of Vision-Language Models (VLMs) for zero-shot scene understanding, enabling mapping between human instructions, semantic features, and environmental constraints. Within this framework, we construct an illustrative navigation model that is designed to satisfy a set of STL-encoded specifications and soft operator preferences through formal satisfaction metrics embedded into environmental properties and runtime monitoring.
Kristy Sakano, Kalonji Harrington, Mumu Xu
Department of Computer Science, University of Maryland, College Park, USA. · Department of Aerospace Engineering, University of Maryland, College Park, USA.