Organizations: The authors are with the School of Intelligence Science and Engineering, the Guangdong Key Laboratory of Intelligent Morphing Mechanisms and Adaptive Robotics, and the Shenzhen Key Lab for Advanced Motion Control and Modern Automation Equipments, Harbin Institute of Technology, Shenzhen, Guangdong 518055, China
Decentralized cooperative pursuit in cluttered environments is challenging for autonomous aerial swarms, especially under partial and noisy perception. Existing methods often rely on abstracted geometric features or privileged ground-truth states, and therefore sidestep perceptual uncertainty in real-world settings. We propose a decentralized end-to-end multi-agent reinforcement learning (MARL) framework that maps raw LiDAR observations directly to continuous control commands. Central to the framework is the Predictive Spatio-Temporal Observation (PSTO), an egocentric grid representation that aligns obstacle geometry with predictive adversarial intent and teammate motion in a unified, fixed-resolution projection. Built on PSTO, a single decentralized policy enables agents to navigate static obstacles, intercept dynamic targets, and maintain cooperative encirclement. Simulations demonstrate that the proposed method achieves superior capture efficiency and competitive success rates compared to state-of-the-art learning-based approaches relying on privileged obstacle information. Furthermore, the unified policy scales seamlessly across different team sizes without retraining. Finally, fully autonomous outdoor experiments validate the framework on a quadrotor swarm relying on only onboard sensing and computing.
Figures & tables
Fig. 1: Validation in an unstructured outdoor environment. (a) Time-lapse of a fully autonomous 2-vs-1 pursuit. Pursuers (blue) execute a coordinated pincer maneuver to encircle the Evader (orange) among physical obstacles within a 9.0 m virtual boundary (blue ring). Dashed lines visualize the formation geometry at synchronized timestamps. (b) Global state visualization. Trajectories and perception data visualized in the global frame. Note the artificial point cloud wall superimposed to enforce arena constraints.
Fig. 2: System overview of the proposed decentralized end-to-end pursuit framework. The architecture follows the CTDE paradigm. (Top) During decentralized execution, each pursuer generates the Predictive Spatio-Temporal Observation (PSTO), which is fused with proprioceptive data by the policy network to output continuous control commands. (Bottom) During centralized training, a critic network utilizes global state information to guide the policy optimization.
Fig. 3: Visualization of the PSTO generation. (a) Egocentric View: The local environment observed by the Ego-Pursuer (center) in its body frame. It perceives the predicted Evader (pink), Teammates (blue), and static Obstacles (green) relative to itself. (b) & (c) PSTO Channels: The corresponding PSTO generated by the Ego-Pursuer. (b) shows the obstacle depth map projected from the green squares in (a), and (c) shows the intent heatmap combining attraction to the Evader and repulsion from Teammates.
Conditions
Ours
Traditional Heuristics
SOTA (w/ GT)
Learning-Based Ablations
Speed
Obs.
PSTO
Angelani [ 20 ]
APF [ 19 ]
Janosov [ 21 ]
OPEN [ 9 ]
1-vs-1 Abl.
Purely Reactive B.
Velocity-Aware B.
Separated-Input Abl.
SR
CT
SR
CT
SR
CT
SR
CT
SR
CT
SR
CT
SR
CT
SR
CT
SR
CT
0.8
0
100.00%
125.00
100.00%
105.80
100.00%
115.60
100.00%
131.10
100.00%
136.30
100.00%
133.90
100.00%
136.00
100.00%
137.20
100.00%
142.00
3
100.00%
122.13
97.22%
119.05
95.83%
116.47
93.06%
127.13
98.81%
129.20
95.11%
128.88
88.19%
133.30
99.31%
136.95
97.92%
146.30
6
99.31%
116.88
90.28%
128.47
92.36%
115.38
89.58%
123.48
98.41%
129.13
96.53%
122.57
87.35%
129.47
97.92%
126.55
97.22%
142.05
9
95.14%
114.55
84.63%
129.02
89.58%
112.97
81.94%
123.07
98.47%
124.70
90.97%
121.95
85.96%
136.88
96.53%
128.35
90.28%
143.30
TABLE I: Comparison of Ours (PSTO) against baselines in the 2-vs-1 scenario. SR: Success Rate ( ↑ ); CT: Capture Time in steps ( ↓ ). Best results are highlighted in bold.
Fig. 4: Trajectory visualization ( 1.6 m/s, 8 obstacles). (a)–(c) Traditional Heuristics: Collision (‘ × ’) or escape due to poor coordination. (d) OPEN (SOTA) [ 9 ] : Captures (‘ ⋆ ’) but requires privileged obstacle states. (e)–(h) Ablations: (e)–(g) Fail (speed deficit/containment loss) (h) Captures (‘ ⋆ ’) but takes a conservative detour. (i)–(k) Ours (PSTO) & Scalability: (i) Executes rapid capture (‘ ⋆ ’). (j)–(k) The unified policy scales to 3-vs-1 and 4-vs-1, achieving multi-angle containment. (Note: Dashed lines visualize synchronized formation geometry; baselines continue after collision to show trends.)
Obs.
Team Size
SR
CT
0
2-vs-1
100.00%
129.50
3-vs-1
100.00%
110.20
4-vs-1
100.00%
107.90
3
2-vs-1
95.00%
127.22
3-vs-1
93.30%
108.94
4-vs-1
91.49%
101.17
TABLE II: Scalability of the PSTO policy across team sizes at v=2.0 m/s. SR: success rate; CT: capture time (steps).
Fig. 5: Fully autonomous system and outdoor validation. (a) The quadrotor equipped with Livox Mid-360 and NUC 13. (b) Swarm pursuit in an unstructured outdoor environment. (c) Quantitative results corresponding to the mission visualized in Fig. 1 . The distance evolution reveals a pincer maneuver, culminating in successful encirclement.
Autonomous Unmanned Aerial Vehicle (UAV) swarms are increasingly used as rapidly deployable aerial relays and sensing platforms, yet practical deployments must operate under partial observability and intermittent peer-to-peer connectivity. We present a graph-based multi-agent reinforcement learning framework trained under centralized training with decentralized execution (CTDE): a centralized critic and global state are available only during training, while each UAV executes a shared policy using local observations and messages from nearby neighbors. Under restricted communication, neighbor relations are induced by an SNR-threshold connectivity graph. Our architecture encodes local agent state and nearby entities with an agent-entity attention module and aggregates inter-UAV messages with neighbor self-attention over a signal-quality-limited communication graph defined by a channel model. We evaluate the framework on a cooperative relay-deployment task, DroneConnect. Experimental results show that the proposed method achieves an approximately 12% increase in target coverage over MAPPO under restricted communication and partial observability, while remaining competitive with a mixed-integer linear programming (MILP)-based offline upper bound with full node observability.
Enguang Fan, Yifan Chen, Zihan Shan +3
University of Illinois at Urbana-Champaign · Boeing Research and Technology
This article presents a solution to intercept an agile drone by a team of agile drone carrying catching nets. We formulate the problem as a competitive Multi-Agent Reinforcement Learning (MARL) task. To address the problem of nonstationarity and catastrophic forgetting of agents overfitting to the current opponent strategy, we train the pursuers and the evader using Multi-Agent Proximal Policy Optimization (MAPPO) with Prioritized Fictitious Self Play (PFSP). We train the agents in a high-fidelity simulator using low-level control commands, collective thrust and body rates (CTBR), to achieve agile flights for both the pursuers and the evader. We compare the performance of the trained policies in terms of catch rate, time to catch and crash rates, against heuristic baselines and show that our solution outperforms them. Ablation studies show that PFSP lead to more robust policies that can adapt to different opponent strategies, and that a low-level control commands are crucial for learning performing strategies in the pursuit-evasion task. Finally, a qualitative analysis of the learned behaviours highlights the emergence of cooperative tactics among the pursuers.
Timothée Gavin, Murat Bronz
ENAC-LAB, LAAS-RIS · *IAS, Thales LAS, Rungis, France · †Dynamic Systems, OPTIM, Fédération ENAC ISAE-SUPAERO ONERA, Université de Toulouse, Toulouse, France +2
Autonomous UAV swarms require scalable coordination mechanisms that maintain mission performance under limited communication, environmental uncertainty, and component failures. Centralized approaches provide global coordination but suffer from communication bottlenecks and single-node vulnerabilities, whereas fully decentralized methods often lack mission-level consistency. This paper presents Layered Autonomous Edge Intelligence (LAEI), a UAV-swarm framework that combines onboard learned policies with lightweight mission-level supervision. Each UAV performs local perception, obstacle avoidance, and action selection onboard, while the supervisory layer provides adaptive goal reassignment, fault-aware recovery, and context-dependent policy guidance without directly controlling low-level actions. LAEI further incorporates recovery strategies, including dynamic reassociation, backup supervisory support, and fallback local autonomy, to maintain mission continuity under representative failure scenarios. We evaluate LAEI in simulated UAV-swarm scenarios using mission completion time, collision rate, and coverage efficiency. The results show that LAEI reduces mission completion time and improves operational efficiency while maintaining collision-aware distributed UAV-level decision-making.
Changmin Park, Wooyong Jung, Hwangnam Kim
School of Electrical Engineering Korea University Seoul 02841, Korea