A Cognitive Architecture for Shared Autonomy in AUV Operations
Authors: Niamh Ellis, Thi Tran, Ignacio Carlucho, Yvan R. Petillot
Organizations: School of Informatics, University of Edinburgh, Edinburgh, UK · School of Engineering and Physical Sciences, Heriot-Watt University, Edinburgh, UK
Operators remain essential to Remotely Operated Vehicle (ROV) operation, yet often suffer from low situational awareness and high workload, both of which negatively affect safety. This paper presents a cognitive architecture consisting of an ontology and multiple Large Language Models (LLMs) to assist the operator at all stages of the mission. Each LLM is grounded with domain-specific information from the ontology and given a simple role to create a system that can support the operator at all stages of an operation. We are aiming to prove that using the two together will allow decisions to be grounded in the relevant domain knowledge, but also benefit from the reasoning capabilities of the LLM. Our framework determines if a mission is possible for a given Unmanned Underwater Vehicle (UUV), performs mission planning, and executes a given mission in simulation. The operator can be involved in planning and execution, ensuring the resulting plan is valid and that the vehicle behaves safely during execution. We compare different LLMs, Llama3, GPT-OSS, and Qwen2.5, to determine which are best suited to the different roles within our framework. We find that GPT-OSS performs best for feasibility assessment, planning, and execution, while Qwen2.5 is best suited to identifying mission types from natural language input.
Figures & tables
Fig. 1: Diagram of the proposed Cognitive Framework showing communication between different components
Fig. 2: Snapshot of ontology showing connections between mission types and actions along with examples
Configuration
Metric
Llama3
Qwen2.5
GPT-OSS
Fully
Correct Feasibility
60%
20%
100%
w/o requirements
100%
60%
100%
Partially
Correct Feasibility
80%
20%
100%
w/o requirements
40%
0%
40%
Impossible
Correct Feasibility
100%
100%
100%
w/o requirements
100%
100%
100%
TABLE I: Table of mission evaluation results for different configurations and different LLMs.
LLM
Correct Types ID
Llama3
87%
Qwen2.5
100%
GPT-OSS
87%
TABLE II: Table of mission types results for different LLMs.
LLM
Mission Type
Pre-conditions met
Successful completion
Optimal Plan
Llama3
Anode test
20%
0%
0%
Inspection
100%
100%
0%
Qwen2.5
Anode test
0%
0%
20%
Inspection
100%
80%
0%
GPT-OSS
Anode test
40%
40%
80%
Inspection
100%
100%
60%
TABLE III: Table of mission planning results for different missions.
Fig. 3: The ROV performing the inspection manuever in a wind turbine in Stonefish simulation environment.
LLM
Actions called
Incorrect Actions
Times human feedback required
Qwen2.5
7.4
2.4
3.4
GPT-OSS
4
0
0
TABLE IV: Table of mission planning results for different missions. Numbers show average values.
Unmanned underwater vehicles (UUVs) operate persistently in communication-constrained environments, thus requiring high-level autonomous fault-tolerant control under faulty operating conditions. Existing approaches rely heavily on predefined hard-coded rules and struggle to achieve effective fault-tolerant control against unforeseen faults. Although large language models (LLMs) possess powerful cognitive and reasoning capabilities, their inherent hallucinations remain a major obstacle to their application in UUV control systems. This paper proposes an intelligent control method based on the LASSA (LLM-based Agent with Solver, Sensor and Actuator) architecture. Within this architecture, an LLM identifies unknown faults and accomplishes task replanning via autonomous reasoning without hard-coded rules; the intelligent agent undertakes perception, scheduling and decision evaluation; the solver verifies physical boundary feasibility constraints prior to command transmission to the actuators. This architecture suppresses physically infeasible LLM hallucinations and ensures interpretable, verifiable decision-making. Moreover, it enables fast-slow dual closed-loop collaborative control, where the slow loop undertakes high-level dynamic decision-making and the fast loop guarantees high-frequency real-time control, simultaneously balancing decision intelligence and control timeliness. Lake experiments under normal and lower-rudder-fault conditions show that the framework detects trajectory tracking abnormalities, replans the route by adjusting the turning radius from 4m to 12m and reducing speed from 2kn to 1kn, passes all three solver constraints on the first invocation, and guides the UUV to complete the full mission; under normal conditions no false fault alarms are raised throughout the run.
Hong Chen, Zixiang Tang, Yuanbao Chen +1
Wuhan Second Ship Design and Research Institute, Wuhan, 430060, China · School of Aeronautic Science and Engineering, Beihang University, Beijing, 100191, China
Autonomous underwater vehicles (AUVs) operating beyond reliable communications must recover from failures without human intervention. We investigate an architecture in which conventional deterministic layered control autonomy manages normal operations, while an invokable large language model (LLM) serves as a diagnostic and recovery planner when onboard anomaly detection identifies performance outside expected limits. Because language models are stochastic, rigorous evaluation requires ensemble testing rather than individual demonstrations. We present a closed-loop simulation architecture that couples real-time C vehicle software with a higher-level orchestration layer for physics-based fault injection, structured prompting, language-model interaction, mission file generation, validation, execution, and LLM-judge scoring. The framework, which we call SPAR (Simulation Platform for AUV Recovery), supports evaluation across fault realizations, prompt structures, reasoning models, and mission conditions. We vary these for a mass-shift fault over 480 SPAR trials, evaluating a frontier model and three off-the-shelf locally deployable LLMs. Model choice dominates diagnosis: the frontier model places the CG-shift mechanism in its top three hypotheses in 85-90% of trials, versus 60-78% for the best local model. Reasoning analysis indicates that local-model success is associated with following the complete diagnostic procedure, whereas weaker models often commit prematurely to elevator failure even though the actuator tracks its command. Diagnosis and operational decision performance do not appear to be coupled in this dataset. The contributions are an architecture extending unanticipated-fault recovery from detection to mitigation and an ensemble methodology for evaluating LLM-assisted mission management on low-power AUVs.
Khalid Halba, Kylie Cooper, James G. Bellingham
Exploration Robotics Laboratory, Johns Hopkins Institute for Assured Autonomy, Baltimore, MD, USA
Large Language Models (LLMs) are increasingly explored as high-level reasoning engines for cyber-physical systems, yet their application to real-time UAV swarm management remains challenging due to heterogeneous interfaces, limited grounding, and the need for long-running closed-loop execution. This paper presents a mission-agnostic, agent-enhanced LLM framework for UAV swarm control, where users express mission objectives in natural language and the system autonomously executes them through grounded, real-time interactions. The proposed architecture combines an LLM-based Agent Core with a Model Context Protocol (MCP) gateway and a Web-of-Drones abstraction based on W3C Web of Things (WoT) standards. By exposing drones, sensors, and services as standardized WoT Things, the framework enables structured tool-based interaction, continuous state observation, and safe actuation without relying on code generation. We evaluate the framework using ArduPilot-based simulation across four swarm missions and six state-of-the-art LLMs. Results show that, despite strong reasoning abilities, current general-purpose LLMs still struggle to achieve reliable execution - even for simple swarm tasks - when operating without explicit grounding and execution support. Task-specific planning tools and runtime guardrails substantially improve robustness, while token consumption alone is not indicative of execution quality or reliability.
Andrea Iannoli, Lorenzo Gigli, Luca Sciullo +2
Department of Computer Science and Engineering University of Bologna Bologna, Italy