Organizations: Intelligent Space Robotics Laboratory, Skolkovo Institute of Science and Technology, Moscow, Russia · AiRo Lab, Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University, Hong Kong
Supervising a multi-UAV reconnaissance mission usually means plotting waypoints and watching every drone. We present UAV SwarmAgent, a hierarchical multi-agent system in which an operator states the mission in one natural-language sentence and a ground-station agent writes and executes code that turns it into per-UAV routes. The agent grounds inspection targets in an aerial map with a vision-language model, splits them among N UAVs by balancing route length, and dispatches worker agents that verify incidents with a compact onboard decision model. In simulation over 4 incident types and 3 foundation models, the swarm located the incident in 244 of 255 missions (95.7%). A paired scaling study with one to 5 UAVs shows that flight time follows an Amdahl-type law. The second UAV delivers half of the attainable saving and the speedup is bounded at 1.52x. We discuss what this predictability, and the false alarms we observed, imply for an operator who supervises the swarm, and outline the user study needed to test it.
Figures & tables
Figure 1. UAV-SwarmAgent turns one operator sentence into a multi-UAV reconnaissance mission: the ground-station agent grounds inspection targets on an archival map, allocates them to the UAVs, and each UAV agent verifies the incident onboard and reports its location with the supporting frames. Three steps from left to right, joined by arrows. First, the operator's sentence asking to check houses for a fire. Second, a satellite map on which three routes of different colours leave one base station and visit waypoints on the buildings of a village; a star marks where the fire was located. Third, the camera frame that a UAV reported, showing a burning house.
Figure 2. The two agent tiers (left), their tools (right), and the nine steps of a mission, with outputs of the structure-fire example of Figure 1 . Four boxes in two rows. Top left, the Airspace Management Agent: it reads the operator's request, infers what to inspect, and at the end reports locations and frames. Top right, its tools, called from generated Python: a map thumbnail with waypoints on buildings (marks targets on the archival map) and a map thumbnail with three routes (splits waypoints into routes, dispatches the swarm). Bottom left, the UAV agent, one per drone: it flies its own route, scores every camera frame, and reports high-scoring frames. Bottom right, its tool, the onboard verifier: three camera frames, the last showing a burning house, lead to the question whether a fire is visible and to the answer yes with probability 0.72 at map position 780, 650. Arrows link each agent to its tools; routes go down from the manager to the UAV agents and reports go back up.
Figure 3. Benchmark maps: the archival reference map used for grounding (top) and the target map of the same area with a synthetic incident, over which the drones fly (bottom; incident boxed). Eight satellite image crops in two rows and four columns. The columns show a structure fire, a wildfire, a vehicle collision, and an oil spill. The top row shows each area without the incident; the bottom row shows the same area with the incident marked by a red box: a burning house in a village, a burn scar with flames and smoke in a forest, vehicles stopped at a road junction, and a dark slick on coastal water.
Figure 4. One example mission per scenario ( N=3 ). Buildings and roads are grounded as objects , one waypoint each; forest and water are grounded as areas and covered with an even spread of waypoints. Insets show the reported frame with the highest score; dashed lines are the return legs to the base station. The collision and oil-spill examples use the refined verifier questions. Four satellite maps side by side: a village with one burning house, a forest with a wildfire, a village with vehicles stopped at a road junction, and a river delta with a dark oil slick on the water. On each map three routes in different colours start from a yellow square base station and visit waypoints. On the village maps the waypoints sit on buildings and roads; on the forest and water maps they are spread evenly. A red star marks where the incident was located, and an inset shows the camera frame of the incident.
Scenario
Maps
Gemini
GPT
Qwen
Prec.
Structure fire
30
28/30
30/30
28/30
0.68
Wildfire
5
5/5
5/5
5/5
0.57
Vehicle collision
5
3/5
4/5
4/5
0.74
Oil spill
5
5/5
5/5
5/5
0.95
All scenarios
41/45
44/45
42/45
Agent time, median (s)
152
180
135
Table 1. Missions in which the swarm located the incident, by scenario and model ( N=3 UAVs). Prec.: median share of reported frames that show the incident.
Figure 5. Scaling study (structure fire, 30 maps per swarm size): mean flight, agent, and total mission time with 95% bootstrap confidence intervals; lines are the fitted laws. A line chart of time in seconds against the number of UAVs from one to five. Flight time falls from about 505 to 363 seconds along a curve that flattens, agent time rises slightly from about 150 to 170 seconds along a straight line, and total mission time falls from about 657 to 533 seconds.
Aerial robotics studies the design, perception, control, and coordination of flying robots for navigation, inspection, transport, and physical interaction.
Multi UAV missions in complex environments require the system to understand both the surrounding scene and the intent of a human operator while maintaining feasible task allocation as mission conditions change. This paper presents AgenticSwarm, an agentic framework for semantic perception and adaptive task allocation in heterogeneous multi UAV missions. An agent interprets aerial imagery and natural language instructions to construct a grounded mission representation that links perceived objects and regions with task requirements, capability constraints, and mission dependencies. This information augments a constrained task allocation process in which obstacle aware path feasibility, energy consumption, and protected return home requirements are incorporated before assignment. During execution, changes such as UAV failure, battery degradation, or task modification trigger residual mission reconstruction from the current system state, while completed work and reconnaissance progress are retained. AgenticSwarm is evaluated across five diverse Gazebo environments and an indoor real test environment, demonstrating its ability to connect semantic reasoning with constrained allocation and adaptive multi UAV mission execution. Compared with a Grounding DINO+SAM~2.1 perception baseline, the SAM3-based pipeline improves class-aware recall by 25.2 percentage points (pp) and semantic label accuracy by 29.5 pp. Ablating residual mission replanning increases mean repeated work from 0% to 61.7% and post-event recovery time by 58.6%, highlighting the contribution of adaptive replanning to mission execution.
Muhammad Ahsan Mustafa, Yasheerah Yaqoot, Faryal Batool +3
Unmanned aerial vehicle (UAV) swarms increasingly rely on language-model agents to provide adaptive mission-level reasoning in uncertain environments. Fully distributed control, in which each UAV hosts an independent Small Language Model (SLM), removes reliance on a centralized coordinator but introduces an information-management problem: long-running interaction histories can degrade the reasoning context, while indiscriminate information dissemination increases communication and inference overhead. We address these challenges with a distributed UAV-agent architecture that enables continuous local SLM control through an event-driven reason-act-observe lifecycle. Runtime knowledge is represented as structured atomic notes and organized into core, local, and peer-specific memory. A deterministic interest-aware gossip engine selectively disseminates these notes according to recipient-specific semantic novelty and recency. We evaluate the architecture using ten UAVs in a simulated search-and-rescue mission. Our approach completes all experimental runs, whereas unrestricted flooding messages completes only 70-85%, and delegating forwarding decisions to the SLM prevents mission completion in every run. Compared with unrestricted flooding, our approach approximately halves inference-token consumption, reduces transmitted data, and achieves lower survivor-count error.
Andrea Iannoli, Ivan Zyrianoff, Angelo Trotta +2
Department of Computer Science and Engineering University of Bologna Bologna, Italy · Technology Innovation Institute (TII) United Arab Emirates
Large Language Models (LLMs) are increasingly explored as high-level reasoning engines for cyber-physical systems, yet their application to real-time UAV swarm management remains challenging due to heterogeneous interfaces, limited grounding, and the need for long-running closed-loop execution. This paper presents a mission-agnostic, agent-enhanced LLM framework for UAV swarm control, where users express mission objectives in natural language and the system autonomously executes them through grounded, real-time interactions. The proposed architecture combines an LLM-based Agent Core with a Model Context Protocol (MCP) gateway and a Web-of-Drones abstraction based on W3C Web of Things (WoT) standards. By exposing drones, sensors, and services as standardized WoT Things, the framework enables structured tool-based interaction, continuous state observation, and safe actuation without relying on code generation. We evaluate the framework using ArduPilot-based simulation across four swarm missions and six state-of-the-art LLMs. Results show that, despite strong reasoning abilities, current general-purpose LLMs still struggle to achieve reliable execution - even for simple swarm tasks - when operating without explicit grounding and execution support. Task-specific planning tools and runtime guardrails substantially improve robustness, while token consumption alone is not indicative of execution quality or reliability.
Andrea Iannoli, Lorenzo Gigli, Luca Sciullo +2
Department of Computer Science and Engineering University of Bologna Bologna, Italy