cs.ROMar 23, 2026

SafePilot: A Framework for Assuring LLM-enabled Cyber-Physical Systems

Authors: Weizhe Xu, Mengyu Liu, Fanxin Kong

Organizations: University of Notre Dame, Notre Dame, IN, USA · Washington State University, Richland, WA, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Large Language Models (LLMs), deep learning architectures with typically over 10 billion parameters, have recently begun to be integrated into various cyber-physical systems (CPS) such as robotics, industrial automation, and autopilot systems. The abstract knowledge and reasoning capabilities of LLMs are employed for tasks like planning and navigation. However, a significant challenge arises from the tendency of LLMs to produce "hallucinations" - outputs that are coherent yet factually incorrect or contextually unsuitable. This characteristic can lead to undesirable or unsafe actions in the CPS. Therefore, our research focuses on assuring the LLM-enabled CPS by enhancing their critical properties. We propose SafePilot, a novel hierarchical neuro-symbolic framework that provides end-to-end assurance for LLM-enabled CPS according to attribute-based and temporal specifications. Given a task and its specification, SafePilot first invokes a hierarchical planner with a discriminator that assesses task complexity. If the task is deemed manageable, it is passed directly to an LLM-based task planner with built-in verification. Otherwise, the hierarchical planner applies a divide-and-conquer strategy, decomposing the task into sub-tasks, each of which is individually planned and later merged into a final solution. The LLM-based task planner translates natural language constraints into formal specifications and verifies the LLM's output against them. If violations are detected, it identifies the flaw, adjusts the prompt accordingly, and re-invokes the LLM. This iterative process continues until a valid plan is produced or a predefined limit is reached. Our framework supports LLM-enabled CPS with both attribute-based and temporal constraints. Its effectiveness and adaptability are demonstrated through two illustrative case studies.

Figures & tables

Figure 1. The overview of our hierarchical neuro-symbolic LLM-enabled planner.
Figure 2. LLM-driven Task Planning with Verification. Here we use a sub-task received from hierarchical planner.
Figure 3. Illustration of Logic Specification. Here we use an LTL formula as an example of the logic formulas.
Figure 4. An example of a Blocksworld problem. The left side indicates the initial state, while the right side represents the goal state.
Figure 5. Deploy SafePilot on a robotic arm to solve a Blocksworld problem. The left image shows the initial state, the middle image shows the robotic arm executing the plan, and the right image shows the final state.
Figure 6. The performances of LLMs on Blocksworld problem. Fig. 6(a) uses GPT-4o, while Fig. 6(b) uses GPT-3.5-turbo. The blue bars represent the SafePilot, and the orange bars represent the task planner which removes the hierarchical planner from SafePilot. The green bars represent the task planner without reason, which is similar to method ( Jha et al., 2023 ) . The red bars represent the one-shot method.
Figure 7. The performance of GPT-4o on Blocksworld problem using SafePilot with different iteration limits. The blue, orange, green, and red bars represent iteration limits of 10, 20, and 30, respectively.
Figure 8. The performance of GPT-4o on Navigation problem. The blue, orange, and green bars represent the SafePilot, the task planner, the task planner without reason and one-shot methods, respectively.
More in this topic

LLM Safety

Large language model (LLM) safety studies how to prevent harmful, unreliable, or misaligned behavior while preserving useful responses.

See all 426 papers

Explore similar work

CardsList

  1. Large Language Models as Falsifiers for Cyber-Physical Systems

    Sep 17, 2026Ali ArjomandBigdeli, Jiawei Zhou, Stanley BakCyber-Physical SystemsLarge Language Model-Guided Optimization

    Stony Brook University, Stony Brook, New York, USA

  2. Large Language Models in the Loop: A Stability- and Network-Aware Survey in Networked Control, Cyber-Physical, and Multi-Agent Systems

    Sep 15, 2026Haiping Du, Linping ChanLanguage Model-Based ControlCyber-Physical Systems

    School of Engineering, University of Wollongong, Wollongong NSW 2522 Australia

  3. LogiC-Diff: Embedding Security Properties Into AI-Enabled Cyber-Physical Systems

    Sep 29, 2026Ziyan An, John Stankovic, Meiyi MaCyber-Physical SystemsTime Series Forecasting

    Department of Computer Science Vanderbilt University Nashville, TN · Department of Computer Science University of Virginia Charlottesville, VA