cs.ROSep 18, 2026

AgenticSwarm: Semantic Perception and Adaptive Task Allocation for Heterogeneous Multi-UAV Missions

Authors: Muhammad Ahsan Mustafa, Yasheerah Yaqoot, Faryal Batool, Roohan Ahmed Khan, Valerii Serpiva, Dzmitry Tsetserukou

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Multi UAV missions in complex environments require the system to understand both the surrounding scene and the intent of a human operator while maintaining feasible task allocation as mission conditions change. This paper presents AgenticSwarm, an agentic framework for semantic perception and adaptive task allocation in heterogeneous multi UAV missions. An agent interprets aerial imagery and natural language instructions to construct a grounded mission representation that links perceived objects and regions with task requirements, capability constraints, and mission dependencies. This information augments a constrained task allocation process in which obstacle aware path feasibility, energy consumption, and protected return home requirements are incorporated before assignment. During execution, changes such as UAV failure, battery degradation, or task modification trigger residual mission reconstruction from the current system state, while completed work and reconnaissance progress are retained. AgenticSwarm is evaluated across five diverse Gazebo environments and an indoor real test environment, demonstrating its ability to connect semantic reasoning with constrained allocation and adaptive multi UAV mission execution. Compared with a Grounding DINO+SAM~2.1 perception baseline, the SAM3-based pipeline improves class-aware recall by 25.2 percentage points (pp) and semantic label accuracy by 29.5 pp. Ablating residual mission replanning increases mean repeated work from 0% to 61.7% and post-event recovery time by 58.6%, highlighting the contribution of adaptive replanning to mission execution.

Figures & tables

Fig. 1 : AgenticSwarm overview. A human provides a high level mission, an AI agent performs semantic grounding and task allocation over the perceived environment, and the heterogeneous UAV team executes the mission with residual replanning under runtime changes.
Fig. 2 : System architecture of AgenticSwarm.
Fig. 3 : Representative environments used to evaluate heterogeneous UAV-swarm missions: (a) agriculture, (b) earthquake-damaged office, (c) small city, (d) warehouse, and (e) fire outbreak.
MetricGDINO+SAM2.1SAM3Δ\Delta
Precision (%)12.520.2+7.7
Recall (%)21.046.2+25.2
F1 (%)15.427.1+11.7
Label accuracy (%)66.195.6+29.5
Critical recall (%)10.234.5+24.3
TABLE I : Semantic perception performance averaged across five environments. Δ\Delta is the absolute change in percentage points.
EnvironmentGrounded missionRuntime changeAgenticSwarm response
AgricultureClean reachable solar-panel rows, scan the crop field and vegetation, perform an ordered payload transfer, and return home.BigDrone_3 battery reduced to 20.5%; SmallDrone_2 failed.Sent the battery-degraded UAV home, preserved completed work, and reassigned all 11 affected tasks to eligible UAVs.
Earthquake officeVisit six independently grounded people using two camera-and-indicator UAVs and return home.No runtime event.Completed all four reachable person visits and explicitly deferred two targets for which no collision-free route existed.
WarehouseInspect every reachable storage rack with inspection UAVs and return home.A box pickup-and-delivery task was inserted at t=8t=8 s.Assigned the inserted payload task to the capable heavy UAV and completed it; one unreachable rack target was deferred.
Small cityAcquire number-plate imagery for 15 vehicles, roof imagery for 21 structures, and inspect two bridges.SmallDrone_4 and SmallDrone_5 failed at t=10t=10 s and t=20t=20 s.Preserved ongoing work and reassigned all 16 affected tasks; two structure targets with no collision-free route remained deferred.
Fire OutbreakCollect water before suppressing three fires, inspect two damaged structures, deliver medical supplies to all grounded people, avoid smoke, and return home.SmallDrone_5 failed; SmallDrone_3 battery fell to 21%; an SOS task was inserted.Returned the battery-degraded UAV, reassigned all nine affected tasks, and inserted and completed the new SOS delivery task.
TABLE II : Qualitative summary of the five representative mission executions. “Affected tasks” denotes unfinished work assigned to a UAV when a failure or battery event occurred.
MetricAgricultureOfficeWarehouseCityFire
UAVs52388
Tasks completed95/954/615/1636/3836/36
Runtime events20123
Affected tasks recovered11/11––16/169/9
Inserted tasks completed––1/1–1/1
Mission time (s)245.621.060.0157.0206.2
TABLE III : Quantitative outcomes for the representative recorded runs. Affected-task recovery excludes newly inserted tasks, which are reported separately.
MetricAgricultureOfficeWarehouseCityFire
Perception tokens16,29918,96513,96815,81136,082
Perception latency (s)117.1136.489.2102.9172.5
Replan calls20123
Replanning tokens22,482–3,94316,20523,432
Mean replan latency (s)20.6–10.824.919.5
TABLE IV : Recorded language-model inference overhead. Perception latency is cumulative over two calls; replanning tokens are cumulative over the listed events, and replan latency is the mean per event.
Fig. 4 : Semantic perception of the real world scenario with the resulting task allocation and planned paths.
Fig. 5 : Real world mission execution from the natural language instruction.
Ablated variantEffect relative to the full system
w/o capability & dependency reasoningSemantically valid allocations decreased from 5/5 to 1/5 environments; 74 capability mismatches, 1 dependency violation, and 4 coupling violations were introduced.
w/o path-aware allocationExecutable route sets decreased from 5/5 to 2/5 environments. For the two executable pairs, distance increased by 42.3% (agriculture) and 3.6% (fire outbreak).
w/o residual replanningMean repeated work increased from 0% to 61.7%; pooled travel distance and post-event recovery time increased by 41.9% and 58.6%, respectively ( n=3n=3 paired environments).
TABLE V : Component ablation study of AgenticSwarm. Each ablated variant removes one component from the full framework while all other experimental conditions are held fixed.
More in this topic

Robot Task Planning

Robot task planning turns goals and observations into executable actions while coordinating skills, motion, uncertainty, and safety.

See all 239 papers

Explore similar work

CardsList
Oct 23, 2026

UAV-SwarmAgent: Scalable Multi-UAV Reconnaissance and Task Allocation via Code Generation and Vision-Language Reasoning

Supervising a multi-UAV reconnaissance mission usually means plotting waypoints and watching every drone. We present UAV SwarmAgent, a hierarchical multi-agent system in which an operator states the mission in one natural-language sentence and a ground-station agent writes and executes code that turns it into per-UAV routes. The agent grounds inspection targets in an aerial map with a vision-language model, splits them among N UAVs by balancing route length, and dispatches worker agents that verify incidents with a compact onboard decision model. In simulation over 4 incident types and 3 foundation models, the swarm located the incident in 244 of 255 missions (95.7%). A paired scaling study with one to 5 UAVs shows that flight time follows an Amdahl-type law. The second UAV delivers half of the attainable saving and the speedup is bounded at 1.52x. We discuss what this predictability, and the false alarms we observed, imply for an operator who supervises the swarm, and outline the user study needed to test it.

Oleg Sautenkov, Yasheerah Yaqoot, Muhammad Ahsan Mustafa +3

Intelligent Space Robotics Laboratory, Skolkovo Institute of Science and Technology, Moscow, Russia · AiRo Lab, Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University, Hong Kong

Aerial RoboticsMulti-Robot CoordinationMulti-Robot Task Allocation
AskPDF
Sep 16, 2026

AeroWeaver: An Embodied-Agent Harness for Weaving Aerial Skills into Distributed, Adaptive Swarm Execution

Collective intelligence is a collaborative autonomy paradigm in which multiple agents pursue shared objectives through local perception, information exchange, and coordinated action. UAV swarms embody this paradigm by coordinating multiple vehicles in tasks such as search, inspection, and tracking. Recent advances in large language model (LLM) agents have strengthened natural-language task understanding and high-level planning, providing a flexible semantic interface between mission descriptions and collective behavior. While these advances expand semantic reasoning, applying LLM agents to UAV swarms raises challenges in grounding model decisions in executable capabilities, reconciling global task reasoning with distributed execution, and using mission-specific experience for continual adaptation. To address these challenges, we introduce AeroWeaver, an embodied-agent harness that weaves individual UAV skills into coordinated mission-level behavior. AeroWeaver connects semantic decisions to governed skills, organizes role-conditioned local agents for distributed coordination, and uses role-indexed state-action-reward experience to refine skill selection online. Experiments and runtime validation show that AeroWeaver maintains valid skill execution under tested conditions and supports body-local multi-UAV operation without a central agent generating joint actions from global context, while reward-guided online updates provide a training-free path for adaptive learning swarm agents from accumulated execution experience. Code: https://github.com/Admire-ljb/AeroWeaver.

Jiabin Lou, Yirong Yang, Haopeng Wang +6

Beihang University, 37 Xueyuan Road, Haidian District, Beijing 100191, China

Aerial RoboticsSwarm RoboticsLLM Agent Skill LearningMulti-Robot CoordinationLLM Agent Harnesses
AskPDF
Jun 8, 2026

LAEI: Layered Autonomous Edge Intelligence Framework for Robust UAV Swarm Operations

Autonomous UAV swarms require scalable coordination mechanisms that maintain mission performance under limited communication, environmental uncertainty, and component failures. Centralized approaches provide global coordination but suffer from communication bottlenecks and single-node vulnerabilities, whereas fully decentralized methods often lack mission-level consistency. This paper presents Layered Autonomous Edge Intelligence (LAEI), a UAV-swarm framework that combines onboard learned policies with lightweight mission-level supervision. Each UAV performs local perception, obstacle avoidance, and action selection onboard, while the supervisory layer provides adaptive goal reassignment, fault-aware recovery, and context-dependent policy guidance without directly controlling low-level actions. LAEI further incorporates recovery strategies, including dynamic reassociation, backup supervisory support, and fallback local autonomy, to maintain mission continuity under representative failure scenarios. We evaluate LAEI in simulated UAV-swarm scenarios using mission completion time, collision rate, and coverage efficiency. The results show that LAEI reduces mission completion time and improves operational efficiency while maintaining collision-aware distributed UAV-level decision-making.

Changmin Park, Wooyong Jung, Hwangnam Kim

School of Electrical Engineering Korea University Seoul 02841, Korea

Aerial RoboticsSwarm RoboticsMulti-Robot CoordinationMulti-Agent Systems
AskPDF