Can Large Language Models Explain Flight Safety Events? A Prior-Guided Semantic LLM-based Approach

Lu Xu; Xu Li; Linjiang Zheng; Fan Li; Riquan Zhang; Jiaxing Shang

cs.AIAug 18, 2026

Can Large Language Models Explain Flight Safety Events? A Prior-Guided Semantic LLM-based Approach

Authors: Lu Xu, Xu Li, Linjiang Zheng, Fan Li, Riquan Zhang, Jiaxing Shang

Organizations: College of Computer Science, Chongqing University, Chongqing, China · School of Statistics and Data Science, Shanghai University of International Business and Economics, Shanghai, China

Abstract

Improving flight safety with flight data requires not only accurate detection of risk events, but more importantly, clear interpretation of their underlying causes at the level of pilot control behavior. Existing explainable AI techniques, such as feature importance maps, often require considerable domain knowledge to translate them into operationally meaningful explanations. Large Language Models (LLMs), which excel at language reasoning, bring a promising solution to this issue. However, applying LLMs in this domain presents key challenges such as modal inconsistency, limited classification ability, scarcity of task-specific data for fine-tuning, and lack of domain knowledge. To overcome these challenges, we propose FlightLLM, a prior-guided semantic LLM-based approach for interpretable flight safety analysis. Specifically, we first perform feature engineering to address modal inconsistency, combining statistical descriptors with physically meaningful flight indicators. This representation is further processed by a Semantic Discretization module, which converts abstract numerical patterns into qualitative descriptions that are more compatible with language reasoning. In addition, since LLMs are not inherently strong classifiers, CatBoost is incorporated as a statistical expert, and its prediction results are injected into the prompt as prior guidance. A contrastive few-shot learning strategy is further adopted to compensate for limited data. Finally, we design structured prompts to embed aviation-specific knowledge into the inference process. Using hard landing, a representative risk event with complex causal mechanisms, as an anchor point, we evaluate FlightLLM on a dataset of 704 real-world A320 flight samples. Experimental results show that the proposed approach achieves competitive classification performance while generating direct and reasonable explanations for event causes.

Figures & tables

Fig. 1: The framework of FlightLLM consists of five modules: Data Preprocessing and Feature Engineering, Semantic Discretization, Statistical Expert Hinting, Dynamic Context Retrieval and Prompt Construction and LLM Invocation. Statistical Expert denotes CatBoost model.

Category	Feature Symbol	Description
Part A: Automated Statistical Features ( $\mathbf{F}_{stat}$ )
Vertical Speed	$Var(\Delta\text{IVV}_{0.4}^{1.0})$	The variance of vertical speed in the 40th to 100th percentile range.
	$Var(\|\Delta\text{IVV}\|)$	The absolute variance of the vertical speed change within the full percentile range.
	$Var(\Delta\text{IVV}_{max-min})$	The variance between the maximum and minimum changes in vertical speed.
	$\mathcal{F}_{real}^{12}(\text{IVV})$	The real frequency energy of the vertical speed in the higher frequency band.
Altitude	$Q_{0.1}(\text{RADIO}_{LH})$	The 10th percentile value of the left radio height.

TABLE I: Detailed Description of the Extracted Hybrid Feature Space

Panel of Figure 3 — Fig. 2: High-Dimensional Feature Profile Match

Fig. 3: The Design of the Prior-Guided Prompt Template.

Prompt Module	Content / Instruction Summary
1. System Persona	“You are an expert in aviation flight data. Your task is to determine whether a landing is normal or hard based on strict evidence.”
2. Context Injection	Feature Dictionary: Inject semantic definitions mapping abstract statistical variables to specific aircraft states.
	Auxiliary Report ( $\mathcal{H}$ ): Inject the statistical prior distribution generated by the CatBoost expert model.
	Few-Shot Examples ( $\mathcal{E}$ ): Provide historical normal and hard landing prototypes to construct a contrastive learning context.
3. Input Formulation	Test Sample Features ( $\mathbf{V}_{test}$ ): The sequence of semantic tokens representing the current flight parameters.
4. Task Constraints	1. Rely strictly on provided observable features; avoid speculation or hallucination.

TABLE II: The Design of the Prior-Guided Prompt Template

Parameter	Description	Frequency(Hz)	Parameter	Description	Frequency(Hz)
ALT_QNH	Altitude	1	ROLL	Roll angel	2
ALT_STD	Standard altitude corrected	1	ROLL_CPT	ROLL_CPT	8
RADIO_LH	Left radio height	4	ROLL_FO	Deputy captain roll control	8
RADIO_RH	Right radio height	4	HEAD_MAG	Magnetic heading direction	1
LDGL	Left landing gear state	4	WIN_DIR	Wind direction	1
LDGR	Right landing gear state	4	WIN_SPD	Wind speed	1

TABLE III: The parameters from QAR data

Model	Accuracy	Precision	Recall	F1
LSTM	56.03	45.95	60.71	52.31
SVM	57.72	46.82	54.87	50.53
RF	56.74	50.88	46.77	48.74
KNN	54.61	34.62	16.07	21.95
CNN	77.30	67.14	83.93	74.60
IMTCN	65.96	55.88	67.86	61.29

TABLE IV: Classification results of different methods

Panel of Figure 8 — Fig. 4: The parameters’ curves of the analyzed case and corresponding curves of group.

Fig. 5: The detailed textual output of LLM.

Variant	Accuracy	Precision	Recall	F1
Variant A(w/o Semanticization)	76.60	67.16	80.36	73.17
Variant B(w/o Expert Hint)	75.18	75.61	55.36	63.92
Variant C(w/o Context Retrieval)	73.76	63.01	82.14	71.32
FlightLLM	81.56	85.71	64.29	73.41

TABLE V: The result of ablation experiment

Fig. 6: Feature Attention Heatmap of 3 language models.

Aviation Safety

See all 20 papers

PilotBench: A Benchmark for General Aviation Agents with Safety Constraints
Apr 10, 2026Yalun Wu, Haotian Liu, Zhoujun Li +1AI Agent Safety Benchmarks Trajectory Prediction
School of Computing, National University of Singapore, China · School of Informatics, Xiamen University, China · CESS, Beihang University, China
Building Trust in the Skies: A Knowledge-Grounded LLM-based Framework for Aviation Safety
Apr 10, 2026Anirudh Iyengar, Alisa Tiselska, Dumindu Samaraweera +1Aviation Safety LLM Reliability
Department of Mathematics, Embry-Riddle Aeronautical University, Daytona Beach, FL
FLY-EVAL++: An Evidence-Driven Evaluation Protocol for Safety-Constrained Flight Prediction with Large Language Models
Sep 3, 2026Yalun Wu, Junfeng Fang, Jiawei Wang +6LLM Safety Benchmarks LLM Safety Evaluation
NExT++ Lab, School of Computing, National University of Singapore · 3Xiamen University · University of Manchester +4

Apr 10, 2026J/K move · Enter open · S save

PilotBench: A Benchmark for General Aviation Agents with Safety Constraints

Yalun Wu, Haotian Liu, Zhoujun Li, Boyang Wang

School of Computing, National University of Singapore, China · School of Informatics, Xiamen University, China · CESS, Beihang University, China

AI Agent Safety Benchmarks Trajectory Prediction LLM Safety Benchmarks

Open paper AskPDF arXiv

As Large Language Models (LLMs) advance toward embodied AI agents operating in physical environments, a fundamental question emerges: can models trained on text corpora reliably reason about complex physics while adhering to safety constraints? We address this through PilotBench, a benchmark evaluating LLMs on safety-critical flight trajectory and attitude prediction. Built from 708 real-world general aviation trajectories spanning nine operationally distinct flight phases with synchronized 34-channel telemetry, PilotBench systematically probes the intersection of semantic understanding and physics-governed prediction through comparative analysis of LLMs and traditional forecasters. We introduce Pilot-Score, a composite metric balancing 60% regression accuracy with 40% instruction adherence and safety compliance. Comparative evaluation across 41 models uncovers a Precision-Controllability Dichotomy: traditional forecasters achieve superior MAE of 7.01 but lack semantic reasoning capabilities, while LLMs gain controllability with 86--89% instruction-following at the cost of 11--14 MAE precision. Phase-stratified analysis further exposes a Dynamic Complexity Gap-LLM performance degrades sharply in high-workload phases such as Climb and Approach, suggesting brittle implicit physics models. These empirical discoveries motivate hybrid architectures combining LLMs' symbolic reasoning with specialized forecasters' numerical precision. PilotBench provides a rigorous foundation for advancing embodied AI in safety-constrained domains.

Abstract

Aviation Safety

Explore similar work

PilotBench: A Benchmark for General Aviation Agents with Safety Constraints

Building Trust in the Skies: A Knowledge-Grounded LLM-based Framework for Aviation Safety

FLY-EVAL++: An Evidence-Driven Evaluation Protocol for Safety-Constrained Flight Prediction with Large Language Models