This paper presents the design and validation of a retrieval-based assistant that supports safety assessment, certification activities, and regulatory compliance for unmanned aircraft systems. The work is motivated by the growing complexity of drone operations and the increasing effort required by applicants and aviation authorities to apply established assessment frameworks, including the Specific Operations Risk Assessment and the Pre-defined Risk Assessment, in a consistent and efficient manner. The proposed approach uses a controlled text-based architecture that relies exclusively on authoritative regulatory sources. To enable traceable and auditable outputs, the assistant grounds each response in retrieved passages and enforces citation-driven generation. System-level controls address common failure modes of generative models, including fabricated statements, unsupported inferences, and unclear provenance, by separating evidence storage from language generation and by adopting conservative behavior when supporting documentation is insufficient. The assistant is intentionally limited to decision support; it does not replace expert judgment and it does not make autonomous determinations. Instead, it accelerates context-specific information retrieval and synthesis to improve document preparation and review while preserving human responsibility for critical conclusions. The architecture is implemented using established open-source components, and key choices in retrieval strategy, interaction constraints, and response policies are evaluated for suitability in safety-sensitive regulatory environments. The paper provides technical and operational guidance for integrating retrieval-based assistants into aviation oversight workflows while maintaining accountability, traceability, and regulatory compliance.
Figures & tables
Figure 1: Table extraction process. The table is rendered as an image using ChatGPT 5.2 and converted into a JSON-formatted chunk.
Figure 2: Embedding and retrieval architecture flowchart
Figure 3: Response schema for grounded answers and query preprocessing
Item
Description
Use Case ID
UC-CHAT-01
Use Case Name
Ask SORA or drone operations question via chatbot
Primary Actor
User, operator, safety engineer, student
Supporting Actors or Systems
Web GUI or notebook client; RAG system, vector search and reranker; LLMChatbot; QueryGenerator (optional)
Goal
Get a correct, explainable answer grounded in SORA related documents, with references or snippets where possible
Trigger
User submits a natural language question, for example ground risk buffer, OSOs, or mitigations
Table 1: Chatbot use case, retrieval augmented SORA question answering.
Item
Description
Use Case ID
UC-CLS-01
Use Case Name
Classify initial operation and return indicator answers
Primary Actor
User, operator, analyst, student
Supporting Actors or Systems
Notebook or Web GUI client; RAG system, vector search and reranker; LLM based indicator assistant (LLMIndicatorAssistant)
Goal
Produce consistent initial indicators, for example regulatory pathway and initial ground or air risk orientation, grounded in SORA documentation
Trigger
User provides an InitialOperationInput and selects one or more indicators to evaluate
Table 2: Classification task use case, initial indicators.
Figure 4: Interface for the regulatory assistant, showing a grounded answer with a sources panel.
Variant
Hit@1
Hit@3
Hit@5
Hit@10
MRR
Direct match
100.0%
100.0%
100.0%
100.0%
1.000
Synonym paraphrase
75.0%
75.0%
100.0%
100.0%
0.800
Reworked question
75.0%
75.0%
75.0%
75.0%
0.750
Overall
83.3%
83.3%
91.7%
91.7%
0.850
Table 3: Retrieval metrics by question variant for the five chunk accuracy test.
The integration of Large Language Models (LLMs) into aviation safety decision-making represents a significant technological advancement, yet their standalone application poses critical risks due to inherent limitations such as factual inaccuracies, hallucination, and lack of verifiability. These challenges undermine the reliability required for safety-critical environments where errors can have catastrophic consequences. To address these challenges, this paper proposes a novel, end-to-end framework that synergistically combines LLMs and Knowledge Graphs (KGs) to enhance the trustworthiness of safety analytics. The framework introduces a dual-phase pipeline: it first employs LLMs to automate the construction and dynamic updating of an Aviation Safety Knowledge Graph (ASKG) from multimodal sources. It then leverages this curated KG within a Retrieval-Augmented Generation (RAG) architecture to ground, validate, and explain LLM-generated responses. The implemented system demonstrates improved accuracy and traceability over LLM-only approaches, effectively supporting complex querying and mitigating hallucination. Results confirm the framework's capability to deliver context-aware, verifiable safety insights, addressing the stringent reliability requirements of the aviation industry. Future work will focus on enhancing relationship extraction and integrating hybrid retrieval mechanisms.
Generative AI is increasingly being used informally in Air Traffic Management (ATM) for tasks such as flight plan generation, trajectory interpretation, and constraint checking. Although these tools can reduce workload and accelerate planning, their non-deterministic outputs create safety and operational risks in human-in-the-loop settings. This paper proposes the AI Trust and Assurance Layer (ATAL), a model-agnostic decision assurance architecture that evaluates whether AI-generated flight-planning outputs are sufficiently reliable for operational use. ATAL combines semantic stability under prompt variation, operational consistency of structured outputs, and normative constraint validation against domain rules, and maps these signals to a Decision Readiness Level (DRL) for human operators. An ATM-inspired experimental study shows how unsafe, inconsistent, or misleading outputs can be identified before influencing flight-plan validation or execution. Although demonstrated in aviation, the framework is also transferable to other safety-critical decision-support domains that require human oversight under regulatory constraints.
Developing autonomous drone applications requires developers to compose navigation, sensing, analytics and compute services while satisfying physical and environmental constraints. Large Language Models (LLMs) can generate code from natural-language requirements, but direct prompting remains unreliable for cyber-physical Drone-as-a-Service (DaaS) missions because incorrect API calls, unsafe flight logic or missing analytics choices can cause mission failure. We propose AeroGen, an LLM-driven low-code framework that translates mission requirements into executable DaaS programs using structured guardrails. AeroGen combines static guardrails, such as API signatures and flight constraints, with dynamic robot, runtime and world information so that the LLM can compose navigation, sensing, analytics and execution services in one generation attempt. We evaluate AeroGen on 20 navigation tasks and 5 analytical missions across simulated and real-world environments. With frontier LLMs, AeroGen succeeds on all runs (25/25: 5 analytical missions x five runs) under tested conditions. Its guardrail ablation improves analytical mission success from 8% to 100%, with code-generation time < 200 s and effective cost <US$ 0.50 per mission.
Kautuk Astu, Yogesh Simmhan
Indian Institute of Science, Bangalore 560012 India