Organizations: Intelligent Space Robotics Laboratory, Skolkovo Institute of Science and Technology, Moscow, Russia · AiRo Lab, Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University, Hong Kong
Supervising a multi-UAV reconnaissance mission usually means plotting waypoints and watching every drone. We present UAV SwarmAgent, a hierarchical multi-agent system in which an operator states the mission in one natural-language sentence and a ground-station agent writes and executes code that turns it into per-UAV routes. The agent grounds inspection targets in an aerial map with a vision-language model, splits them among N UAVs by balancing route length, and dispatches worker agents that verify incidents with a compact onboard decision model. In simulation over 4 incident types and 3 foundation models, the swarm located the incident in 244 of 255 missions (95.7%). A paired scaling study with one to 5 UAVs shows that flight time follows an Amdahl-type law. The second UAV delivers half of the attainable saving and the speedup is bounded at 1.52x. We discuss what this predictability, and the false alarms we observed, imply for an operator who supervises the swarm, and outline the user study needed to test it.
Figures & tables
Figure 1. UAV-SwarmAgent turns one operator sentence into a multi-UAV reconnaissance mission: the ground-station agent grounds inspection targets on an archival map, allocates them to the UAVs, and each UAV agent verifies the incident onboard and reports its location with the supporting frames. Three steps from left to right, joined by arrows. First, the operator's sentence asking to check houses for a fire. Second, a satellite map on which three routes of different colours leave one base station and visit waypoints on the buildings of a village; a star marks where the fire was located. Third, the camera frame that a UAV reported, showing a burning house.
Figure 2. The two agent tiers (left), their tools (right), and the nine steps of a mission, with outputs of the structure-fire example of Figure 1 . Four boxes in two rows. Top left, the Airspace Management Agent: it reads the operator's request, infers what to inspect, and at the end reports locations and frames. Top right, its tools, called from generated Python: a map thumbnail with waypoints on buildings (marks targets on the archival map) and a map thumbnail with three routes (splits waypoints into routes, dispatches the swarm). Bottom left, the UAV agent, one per drone: it flies its own route, scores every camera frame, and reports high-scoring frames. Bottom right, its tool, the onboard verifier: three camera frames, the last showing a burning house, lead to the question whether a fire is visible and to the answer yes with probability 0.72 at map position 780, 650. Arrows link each agent to its tools; routes go down from the manager to the UAV agents and reports go back up.
Figure 3. Benchmark maps: the archival reference map used for grounding (top) and the target map of the same area with a synthetic incident, over which the drones fly (bottom; incident boxed). Eight satellite image crops in two rows and four columns. The columns show a structure fire, a wildfire, a vehicle collision, and an oil spill. The top row shows each area without the incident; the bottom row shows the same area with the incident marked by a red box: a burning house in a village, a burn scar with flames and smoke in a forest, vehicles stopped at a road junction, and a dark slick on coastal water.
Figure 4. One example mission per scenario ( N=3 ). Buildings and roads are grounded as objects , one waypoint each; forest and water are grounded as areas and covered with an even spread of waypoints. Insets show the reported frame with the highest score; dashed lines are the return legs to the base station. The collision and oil-spill examples use the refined verifier questions. Four satellite maps side by side: a village with one burning house, a forest with a wildfire, a village with vehicles stopped at a road junction, and a river delta with a dark oil slick on the water. On each map three routes in different colours start from a yellow square base station and visit waypoints. On the village maps the waypoints sit on buildings and roads; on the forest and water maps they are spread evenly. A red star marks where the incident was located, and an inset shows the camera frame of the incident.
Scenario
Maps
Gemini
GPT
Qwen
Prec.
Structure fire
30
28/30
30/30
28/30
0.68
Wildfire
5
5/5
5/5
5/5
0.57
Vehicle collision
5
3/5
4/5
4/5
0.74
Oil spill
5
5/5
5/5
5/5
0.95
All scenarios
41/45
44/45
42/45
Agent time, median (s)
152
180
135
Table 1. Missions in which the swarm located the incident, by scenario and model ( N=3 UAVs). Prec.: median share of reported frames that show the incident.
Figure 5. Scaling study (structure fire, 30 maps per swarm size): mean flight, agent, and total mission time with 95% bootstrap confidence intervals; lines are the fitted laws. A line chart of time in seconds against the number of UAVs from one to five. Flight time falls from about 505 to 363 seconds along a curve that flattens, agent time rises slightly from about 150 to 170 seconds along a straight line, and total mission time falls from about 657 to 533 seconds.
Aerial robotics studies the design, perception, control, and coordination of flying robots for navigation, inspection, transport, and physical interaction.