cs.ROFeb 23, 2026
SaveLarge Language Model-Assisted UAV Operations and Communications: A Multifaceted Survey and Tutorial
Abstract
Uncrewed Aerial Vehicles (UAVs) are widely deployed across diverse applications due to their mobility and agility. Recent advances in Large Language Models (LLMs) offer a transformative opportunity to enhance UAV intelligence beyond conventional optimization-based and learning-based approaches. By integrating LLMs into UAV systems, advanced environmental understanding, swarm coordination, mobility optimization, and high-level task reasoning can be achieved, thereby allowing more adaptive and context-aware aerial operations. This survey systematically explores the intersection of LLMs and UAV technologies and proposes a unified framework that consolidates existing architectures, methodologies, and applications for UAVs. We first present a structured taxonomy of LLM adaptation techniques for UAVs, including pretraining, fine-tuning, Retrieval-Augmented Generation (RAG), and prompt engineering, along with key reasoning capabilities such as Chain-of-Thought (CoT) and In-Context Learning (ICL). We then examine LLM-assisted UAV communications and operations, covering navigation, mission planning, swarm control, safety, autonomy, and network management. After that, the survey further discusses Multimodal LLMs (MLLMs) for human-swarm interaction, perception-driven navigation, and collaborative control. Finally, we address ethical considerations, including bias, transparency, accountability, and Human-in-the-Loop (HITL) strategies, and outline future research directions. Overall, this work positions LLM-assisted UAVs as a foundation for intelligent and adaptive aerial systems.
Figures & tables
| Surveys | Focus & Scope | Key Findings / Application Categories |
| [ 22 ] | A broad overview of integrating various LLMs (BERT, GPT, T5, etc.) with UAVs to enhance autonomous systems. | Applications in natural language control, real-time data analysis, and mission planning for domains like surveillance and disaster response. |
| [ 24 ] | Analyzes 17 papers to map the research landscape (2021–2024). | Applications: UAV Optimization (35%), Adaptation (40%), Communication (25%). Models: GPT most used (58%). Highlights challenges, including hardware limitations, latency, security, and ethical alignment. |
| [ 25 ] | Focuses on revolutionizing UAV-assisted wireless communication, information perception, and autonomous control. | Enhances aerial networking, edge computing, multimodal data fusion, scene understanding, and path planning. |
| [ 26 ] | Comprehensively surveys Transformer-based models (ViTs, STTs, etc.) and LLMs for UAV systems. | Categorizes architectures for object detection, tracking, navigation, and mission planning. Highlights LLMs for high-level reasoning and interaction. |
| [ 27 ] | Analyzes 74 papers and 56 projects across 9 core tasks in the UAV workflow. | Academia: Broad, theoretical focus (e.g., multi-UAV planning). Industry: Practical focus on flight control, single-UAV planning, and human–machine interaction. |
| [ 28 ] | A systematic review of LLM and MLLM integration in autonomous driving systems. | Explores applications in perception, prediction, planning, decision-making, multitasking processing, and HMI. Discusses multimodal fusion, prompt engineering, and distillation, outlining challenges and future directions. |
Table I : A summary of existing surveys on the integration of LLMs and MLLMs in UAV Operation and Communication.
Figure 1: Overall organization of the survey, illustrating the research foundations, LLM adaptation stack, core application domains (LLM-enabled communications and MLLM-driven intelligence), and responsible and future intelligence considerations for UAV systems.
Figure 2: LLM fine-tuning pipeline for UAV applications, showing the progression from a pretrained language model through UAV-specific data preparation, model and training setup, parameter-efficient fine-tuning (e.g., adapters, LoRA/QLoRA, and prompt tuning), followed by evaluation, deployment, continuous monitoring, and updates to produce a UAV-adapted LLM.
| Reference | Proposed Framework | Primary Application Domain | Core Technical Approach | Key Performance Results |
| [ 34 ] | Hybrid LLM–RAG Decision-Making System | IoD, Autonomous UAV Missions | LLM integrated with RAG over a knowledge graph built from real-time sensor data and historical mission logs; fully auditable decision tracing | 92% decision accuracy; 94% mission success rate in dynamic environments |
| [ 35 ] | Augmented LLM–RAG IoD Decision Model | IoD, Multi-UAV Network Coordination | Centralized LLM with RAG processing telemetry, mission logs, and environmental data for real-time contextual decisions | BLEU score: 0.82; cosine similarity: 0.87; average decision latency: 120 ms |
| [ 36 ] | LLM-Land | Autonomous UAV Landing | Lightweight LLM with RAG combined with a Vision-Language Encoder (VLE) and Model Predictive Control (MPC) for semantic constraint-aware trajectory replanning | High landing success rate; superior obstacle avoidance compared to baseline MPC and non-RAG LLM-MPC; real-time feasibility on constrained hardware |
| [ 37 ] | LLM–RAG UAV-BS Optimization Framework | UAV-assisted Mobile Networks (UAV-BSs) | GAI-assisted LLM with RAG (e.g., GPT-4) to automate mathematical modeling and optimize UAV placement strategies | Faster convergence and higher efficiency than Genetic Algorithm-based approaches |
| [ 38 ] | GraphRAG | General RAG Enhancement (incl. UAV Analytics) | Knowledge-graph-based retrieval of nodes, subgraphs, and communities instead of isolated text chunks | Improved query-focused summarization; reduced redundancy; enhanced global domain understanding |
| [ 39 ] | HybridRAG | Domain-Specific Analytics (e.g., UAV Systems) | Combination of VectorRAG for semantic similarity and GraphRAG for structured relational reasoning | More faithful, comprehensive, and domain-aligned LLM responses |
Table II : Summary of RAG frameworks applied to UAV operation and communication systems, covering decision-making, network optimization, autonomous landing, edge computing, and evaluation methodologies.
Figure 3: Conventional RAG pipeline for UAV applications, illustrating the workflow from UAV task/query formulation through task-aware query encoding, hybrid knowledge retrieval (vector, graph, and hybrid RAG), context augmentation, and grounded LLM-assisted response generation using domain-specific UAV knowledge sources.
Figure 4: Prompt engineering strategies for UAV applications, comparing in-context learning, chain-of-thought prompting, prompt-based planning, and self-refinement in terms of prompt structure, LLM roles, and resulting capabilities for reliable, scalable, and robust UAV decision-making.
| Reference | Proposed Framework | Primary Application Domain | Core Function of the LLM | Key Advantage over Traditional Methods | Key Performance Results |
| [ 44 ] | ICLDC | UASNets, SAR Emergencies | Edge-deployed LLM uses ICL to generate data collection schedules from natural-language task descriptions derived from sensory logs | Eliminates iterative DRL training; enables rapid adaptation, improved safety via verifier, and robustness through attack detection | Significant reduction in cumulative packet loss vs. DQN and Maximum Channel Gain; faster convergence and lower computational overhead |
| [ 45 ] | FRSICL | UAV-Assisted Wildfire Monitoring | LLM performs real-time flight resource allocation by jointly optimizing data collection schedules and UAV velocities using contextual task descriptions | Joint optimization without retraining; improved responsiveness and stability compared to policy-gradient and heuristic methods | Lower average AoI and more stable velocity control than PPO and Nearest-Neighbor baselines |
| [ 46 ] | LLM-Assisted ICL for Public Safety UAVs | Disaster Response and Wildfire Monitoring | Edge LLM leverages prompts and demonstrations to adaptively control UAV path planning, velocity, scheduling, and power management | Generalizes across tasks without retraining; flexible prompt-based control versus task-specific optimization models | Significant packet loss reduction in data collection; highlights resilience and exposure to jailbreaking vulnerabilities |
| [ 47 ] | AIC-VDS | Multi-UAV Post-Disaster Monitoring (e.g., Tsunami Response) | Attention-guided LLM focuses on urgent sensors to generate task descriptions for joint scheduling and velocity control | Efficient compression of high-dimensional sensory data; scalable coordination beyond multi-agent DRL limits | Up to 91% packet loss reduction compared to Multi-agent DQN and Maximum Channel Gain baselines |
| [ 48 ] | LLM-CRF | Human–Swarm Teaming for Disaster SAR | Central LLM interprets multi-modal human commands, extracts intent, and performs hierarchical task decomposition and mission planning | Transforms human role from low-level control to high-level supervision; improves safety with ICL and HITL verification | 64.2% reduction in task completion time; 94% mission success rate; 42.9% reduction in operator cognitive workload |
Table III : ICL frameworks leveraging LLMs for emergency UAV and sensor network operations, comparing approaches for data collection, wildfire monitoring, disaster response, multi-UAV coordination, and human–swarm teaming in search and rescue.
| Aspect | LLM Pre-training | LLM Fine-tuning | Prompt Engineering | RAG |
| Core Objective | Learn general language patterns, knowledge, and reasoning abilities from a massive corpus. | Adapt a pretrained model’s knowledge to a specific task or domain using a smaller dataset. | Guide the output of a pretrained model for a specific task by carefully designing the input prompt. | Enhance the output of a pretrained model by retrieving and incorporating relevant, up-to-date information from an external knowledge base. |
| Primary Input | Massive, general/specialized text corpora (e.g., web data, books, code). | Smaller, task-specific, often labeled datasets. | A carefully crafted textual prompt (instructions, examples, context). | User query + relevant information retrieved from an external knowledge base. |
| Core Process | Unsupervised next-token prediction on a vast dataset. Training from scratch. | Updating model parameters (fully or via PEFT) on a targeted dataset. | Designing and iteratively refining the input text. No weight updates. | Retrieving relevant documents and augmenting the prompt with this context before generation. |
| Resource Intensity | Extremely High. Vast data, immense compute (GPU clusters, 3D parallelism). | Moderate to High. Lower than pre-training but still significant for full fine-tuning. | Very Low. Requires only a forward pass; no training. | Low to Moderate. Adds a retrieval step and a longer context, but inference-only. |
| Key Advantages | Develops foundational language capabilities and emergent skills. | High task-specific performance, efficient transfer learning. | Highly flexible, fast to implement, resource-efficient. | Provides up-to-date & factual information, reduces hallucinations. |
| Key Challenges | Prohibitively expensive/time-consuming. Requires vast domain-specific data. | Computationally heavy for edge systems. Requires quality labeled data. | Effectiveness depends heavily on the design of prompts. | Requires building/maintaining a high-quality, updated knowledge base. |
Table IV : Comparison of LLM adaptation techniques—pre-training, fine-tuning, prompt engineering, and RAG—in terms of their core objectives, inputs, processes, resource requirements, advantages, challenges, suitability for UAV applications, and basis of output generation.
Figure 5: Diagram of the LLM-assisted UAV navigation, planning, and placement framework, illustrating the integration of mission context, reasoning, decision-making, and multi-agent coordination for autonomous operations.
| Ref. | Proposed Framework | Autonomy Layer | Primary Application Domain | Core Function of the LLM | Key Results / Advantages |
| [ 66 ] | SPINE-based Air–Ground System | Mission-Level Planning | Air–Ground Cooperative Robotics in Unknown Environments | Interprets natural language missions, infers semantic objects, and generates adaptive plans using online semantic–metric mapping | Successfully completes language-specified missions in kilometer-scale urban and rural environments under intermittent connectivity |
| [ 67 ] | MSDTMD-SG-Based Swarm Planning | Planning / Path Optimization | UAV Swarm Path Planning for Industry 5.0 Rescue Scenarios | Learns human-like multi-step reasoning for movement decisions via fine-tuned datasets and symbolic modeling | Highest task completion rate compared to DRL and ACO baselines with competitive latency and improved interpretability |
| [ 68 ] | LLM-assisted Multi-UAV Placement | Network-Level Planning | Multi-UAV IAB Network Deployment | Iteratively generates and refines UAV placement using structured prompting and historical feedback | Near-optimal connectivity and coverage with significantly fewer iterations than conventional optimization methods |
| [ 69 ] | NELV | End-to-End Planning Pipeline | Natural Language–Driven UAV Mission Execution | Maps language instructions to executable flight trajectories via parsing, route planning, control, and monitoring | Reduces pilot workload and enables multi-UAV missions while satisfying regulatory and environmental constraints |
| [ 70 ] | FlockGPT | Execution / Formation Control | Natural Language–Driven UAV Swarm Formation | Translates high-level geometric commands into executable formation control code using SDFs | Enables intuitive swarm control for non-experts; effective formation execution without manual programming |
| [ 71 ] | RALLY | Execution / Hybrid Control | Collaborative UAV Swarm Navigation | Combines LLM reasoning with MARL via role assignment and goal refinement | Improves task success rate, convergence speed, and robustness over MARL-only baselines |
Table V : Comparative overview of recent LLM-assisted UAV planning, coordination, and swarm control frameworks, highlighting their autonomy layers, application domains, core LLM functions, and demonstrated performance benefits across diverse mission scenarios.
| Reference | Proposed Framework | Primary Application Domain | Core Function of the LLM | Key Advantage over Traditional Methods | Key Performance Results |
| [ 74 ] | LLM–HFACS UAV Accident Investigation System | UAV Safety Analysis and Accident Investigation | LLM reasons over unstructured incident narratives and maps findings to the structured HFACS 8.0 taxonomy to identify causal factors across multiple levels | Automates and standardizes human-factor analysis; detects subtle latent errors with reduced time, cost, and human effort compared to manual expert coding | Macro-F1 scores of 0.58–0.76 across 18 categories; best F1 of 0.76 (precision 0.71, recall 0.82); some category accuracies 93% |
| [ 75 ] | LLM-assisted Satellite–UAV–IoT Network Optimization | 6G Integrated Satellite–UAV–IoT Communications | Fine-tuned Llama 3 70B dynamically optimizes device association, resource allocation, and UAV positioning using real-time network telemetry | Reduces action-space complexity and decision latency; enables adaptive, predictive network management beyond static optimization and RL methods | 27% average spectral efficiency gain; 35% reduction in decision latency compared to traditional approaches |
| [ 76 ] | LEDMA (LLM-Assisted Decomposition-Based MOEA) | UAV Networks with ISAC | LLM acts as a black-box search operator to decompose and solve multi-objective optimization problems for UAV deployment and power control | Improves convergence and solution diversity for non-convex, high-dimensional optimization beyond classical evolutionary algorithms | Better Pareto front, faster convergence, and higher hypervolume than traditional multi-objective evolutionary algorithms |
| [ 77 ] | Embodied Aerial Intelligent Agent | Open-World Autonomous UAV Operations | Edge-deployed LLM performs high-level task planning and scene understanding, interfaced with low-level perception, mapping, and motion control | Enables robust autonomy and generalization in communication-constrained environments through hardware–software co-design | 14B-parameter LLM achieves 5–6 tokens/s at 220W; validated across diverse real-world missions (inspection, exploration, monitoring) |
| [ 78 ] | LLM-Assisted Unmanned Traffic Management (UTM) System | UAV Traffic Management and Airspace Safety | LLM analyzes real-time airspace data, checks compliance, and generates actionable alerts and recommendations for human operators | Enhances situational awareness and decision support beyond rule-based monitoring systems | Up to 91.7% decision accuracy with an average response time of 5.7 seconds across evaluated scenarios |
Table VI : LLM-enhanced frameworks for UAV safety, network optimization, and autonomous intelligence, comparing methodologies, core LLM roles, and performance improvements in accident investigation, integrated communications, multi-objective optimization, open-world autonomy, and traffic management.
Figure 6: Architectures for multi-LLM deployment in UAV swarms: parallel/ensemble for robust decision-making, hierarchical for tiered resource-aware intelligence, and distributed specialized agents for decentralized mission execution.
| Reference | Proposed Framework | Primary Application Domain | Core Function of the LLM | Key Advantage over Traditional Methods | Key Performance Results |
| [ 80 ] | Multi-LLM Distributed Coordination Paradigm | Heterogeneous Aerial and UAV Networks | Multiple specialized LLMs collaboratively handle routing, communication optimization, collision avoidance, and mission-level coordination while sharing critical state information | Enables division of labor, real-time adaptation, fault tolerance, and collective optimization beyond single-LLM or centralized controllers | Improved safety, efficiency, resilience, and scalability in heterogeneous aerial networks (conceptual and system-level validation) |
| [ 81 ] | Multi-LLM UAV Swarm Inspection System | UAV Swarm Infrastructure Inspection | Lightweight onboard LLMs perform local path planning and real-time decisions, while a centralized LLM orchestrator handles global reasoning and task allocation | Balances local responsiveness with global optimization; improves robustness, coverage, and energy efficiency compared to traditional planners | Higher inspection coverage, improved energy efficiency, and more robust swarm coordination in dynamic inspection scenarios |
| [ 82 ] | LLM-assisted Multi-Agent Inspection Framework | AEC Industry UAV Visual Inspection | Specialized LLM agents (Router, PathPlanner, Controller, Perceptioner, Retriever) perform 3D reasoning, task delegation, perception, and UAV control | Enables spatial-semantic understanding via semantic point clouds and 3D scene graphs; improves automation over rule-based inspection systems | Demonstrates flexible task coordination and improved automation of complex inspection workflows |
| [ 55 ] | Aero-LLM | Secure and Resilient UAV Operations | Hierarchical LLMs across onboard, edge, and cloud layers perform anomaly detection, forecasting, and cybersecurity analysis | Achieves low-latency response and high-level reasoning while maintaining lightweight onboard computation | Strong performance in threat detection and secure decision-making on resource-constrained UAV platforms |
| [ 84 ] | Hierarchical LLM-Controlled Hybrid Aerial Network | UAV–HAPS Terrestrial–Non-Terrestrial Networks | Meta-controller LLM on HAPS manages global resource allocation and load balancing, while onboard LLMs handle real-time motion and communication control | Joint optimization of motion and communication outperforms DRL and hybrid control schemes in complex aerial networks | Higher system rewards, lower operational cost, and reduced collision rates compared to DRL and hybrid baselines |
Table VII : Hierarchical and multi-LLM architectures for coordinated UAV and aerial network operations, comparing distributed, swarm-based, and layered LLM frameworks in inspection, security, and hybrid terrestrial–non-terrestrial network applications.
Figure 7: End-to-end multimodal reasoning and control frameworks for VLN in UAVs, featuring instruction grounding, hierarchical collaboration, and safety-aware execution for continuous 3D navigation.
| Reference | Purpose | Input Modalities | Models | Key Outcomes / Achievements |
| [ 92 ] | Human-swarm interaction, context-aware command | Text, voice, teleoperation | LLM-assisted modules: Context Generator, Intent Recognition, Task Planner, Modality Selector | Reduces cognitive load, interprets commands accurately, provides flexible communication modes; requires further learning-based intent understanding and large-scale validation |
| [ 93 ] | Autonomous swarm formation, safe command execution | Images, text | GPT-4 multimodal LLM, pre-training on single-UAV | Achieves 82.7% command extraction accuracy; 83.8% formation planning success; handles ambiguous human instructions, integrates perception and swarm status |
| [ 57 ] | Dynamic mission planning, multi-agent coordination, situational awareness | Text, visual, sensor data | MLLM (Doubao 1.5 Vision Pro), chain-of-thought reasoning, multi-source data fusion | Autonomous swarm management, adaptive task execution, natural language interaction; addresses hallucination and inference-latency trade-offs |
| [ 94 ] | Cyber-physical threat detection in UAV swarms | Sensor data, logs, UAV communications | Large multimodal models, hierarchical zero-trust architecture, edge-fog-cloud scheduling | Improves robustness and timeliness in detecting threats; balances inference accuracy, latency, and computational constraints |
| [ 95 ] | Intuitive UAV control via natural language | Text, video, image | MLLMs (Gemini / LLaVA), logistic regression classifier, YOLOv10, DeepSORT, MAVSDK | High-accuracy command execution (97.58% efficiency in video selection), effective autonomous operation, reduces need for specialized programming skills |
| [ 96 ] | Real-time fire monitoring, automated report generation | UAV imagery, ground sensors, meteorological data | LLM (DeepSeek), Grounding DINO, YOLOv8, improved A* path planning | Generates combined text-and-image reports, identifies fireground elements, optimizes evacuation routes, enhances situational awareness and operational efficiency |
Table VIII : Multimodal LLM applications in UAV swarm operations, comparing frameworks for human-swarm interaction, autonomous formation, threat detection, natural language control, and emergency monitoring across diverse sensory and linguistic inputs.
| Reference | Core Innovation / Focus | Key Technical Components | Key Outcomes / Performance | Limitations / Challenges Addressed |
| [ 97 ] | End-to-end VLN; reframes navigation as QA; cross-modal grounding for interpretability. | Fine-tuned LLM, open-vocabulary visual perception, novel cross-modal grounding mechanism. | Significant improvement in instruction-following accuracy & path efficiency; robust zero-shot generalization in diverse indoor/outdoor settings. | Minimizes task-specific supervision; outperforms closed-vocabulary baselines. |
| [ 98 ] | Integrates VLN with NMPC for safe, precise urban flight. | LLM, fine-grained spatial verbalizer, history path memory, NMPC module for control optimization. | Substantial improvement in navigation success rate & operational efficiency; ensures safe, constraint-aware flight. | Reduces human operator cognitive load and advances explainable autonomous decision-making. |
| [ 99 ] | Addresses out-of-distribution (OOD) instructions via an LLM-assisted Instruction Rephraser (LLMIR). | Conditional Transformer (VLCT), SentenceBERT, LLM-assisted Instruction Rephraser (LLMIR). | 1.39% absolute improvement in task success rate; +1.51% improvement on OOD test set. | Enhances reliability with vague/user instructions; improves user-friendliness. |
| [ 100 ] | Dual-UAV collaboration at different altitudes for superior navigation. | Multimodal LLM (Pilot-LLM), orthographic mapping, multi-stage pathfinding pipeline. | Significantly outperforms single-UAV methods; higher success rates & better generalization on unseen environments/objects. | Overcomes limitations of single-UAV perspective via collaborative reasoning. |
| [ 101 ] | Proposes a new VLN benchmark for UAVs in continuous 3D outdoor urban spaces. | Task definition for continuous, open 3D navigation with first-person visual perceptions. | Introduces challenges in 3D spatial reasoning, language understanding, and cross-modal alignment. | Addresses the gap in aerial-agent navigation for complex urban environments. |
Table IX : Comparison of VLN frameworks for UAVs, highlighting innovations in end-to-end reasoning, safety-aware control, instruction robustness, multi-UAV collaboration, and benchmark development for continuous 3D aerial navigation.
Figure 8: Integrated high-level planning and operational intelligence pipeline for UAVs, using multimodal LLMs/VLMs to interpret instructions, infer goals, and generate flight plans, with real-time feedback loops for execution monitoring and situational awareness.
| Core Challenge | Solution Approach | Key Capability | Desired Outcome |
| Model uncertainty and hallucinations of foundation models | Combining structured language reasoning with robust visual perception | Error-resilient, zero-shot task execution across diverse missions | Greater reliability and safety in real-world aerial missions |
| Lack of interpretability and trust in AI decisions | Pairing VLM-based reasoning with real-time depth perception and operator-guided approval loops | Enhanced explainability and human-AI collaboration | Improved transparency and user-aligned operation |
| Limited onboard computational resources for AI models | Using compact, finely tuned LLMs with efficient visual detection and depth estimation pipelines | High-level decision-making on resource-constrained UAVs | Widely deployable AI-powered UAV platforms |
| Complex environment perception and context-aware decision-making | Integrating domain-guided prompt execution with lightweight vision modules | Safe, context-aware decisions in dynamic environments | Intelligent operation in complex scenarios |
| Human intent understanding and mission execution | Systems combining language reasoning with visual perception and human-centric interaction | Understanding human intent and complex environments | More intelligent, interpretable, and reliable systems |
Table X : Key challenges in LLM/VLM-based UAV autonomy—including model uncertainty, interpretability, resource constraints, environmental perception, and intent understanding—paired with solution approaches, required capabilities, and desired operational outcomes.
Figure 9: Multimodal LLM/VLM framework for semantic understanding and analysis in UAV applications, integrating diverse sensor data for scene reasoning, threat assessment, and domain-specific insights.
More in this topic
Language Model-Based Control
Language model-based control uses language models to interpret goals, choose actions, and steer AI systems or physical processes.
See all 180 papers