Threat-Aware Energy-Efficient Deployment for Dynamic UAV Networks: A Multi-Agent RL Approach

Faisal Al-Kamali; Hussein A. Ammar; Francois Chan; James H. Bayes; Yasser Gadallah; Mohamed H. Ahmed

cs.ITSep 25, 2026

Threat-Aware Energy-Efficient Deployment for Dynamic UAV Networks: A Multi-Agent RL Approach

Authors: Faisal Al-Kamali, Hussein A. Ammar, Francois Chan, James H. Bayes, Yasser Gadallah, Mohamed H. Ahmed

Organizations: School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, ON K1N 6N5, Canada · Department of Electrical Engineering, Ibb University, Ibb, Yemen · Department of Electrical and Computer Engineering, Royal Military College, Kingston, ON, K7K 7B4, Canada · Department of Electronics and Communications Engineering, The American University in Cairo, New Cairo 11835, Egypt

Abstract

Ensuring operational safety in threat-prone environments remains a critical challenge for multi-UAV networks serving as aerial base stations. This paper proposes an efficient framework to maximize global energy efficiency (EE) while promoting safe operation through threat-aware clustering and reward-based safety enforcement. The proposed framework is executed in three steps. First, a threat-aware K-means (TAKM) algorithm determines the minimum required UAVs and computes safe initial placements. Second, an optimal matching stage assigns physical UAVs to these centroids to minimize energy expenditure. Third, a threat-aware multi-agent twin delayed deep deterministic policy gradient (MATD3) algorithm dynamically optimizes trajectories, power, and user associations. Simulation results show that the proposed framework achieves zero observed safety violations in the considered scenarios while achieving superior EE and faster convergence than other learning methods and non-clustering baselines. Compared to heuristic optimization, the proposed framework outperforms the greedy particle swarm optimization (GPSO) and achieves performance comparable to that of the optimized PSO (OPSO), while incurring significantly lower online deployment computational complexity. Furthermore, the proposed framework demonstrates effective generalization to unseen user distributions, large UAV fleets, and different threat geometries, while maintaining zero safety violations.

Figures & tables

Aspect	[ 7 ]	[ 15 ]	[ 16 ]	[ 17 ]	[ 18 ]	This Work
User distribution	Uniform	Uniform	Uniform	Uniform	Real-world	Non-uniform
Threat-zone awareness	✗	✗	✗	✗	✗	✓
UAV count optimization	✗	✗	✗	✗	✗	✓
User association strategy	Dynamic	Fixed	Dynamic	Fixed	Fixed	Dynamic
Clustering scheme	None	MEM	None	Hotspot-based	GAK-means	TAKM
Operational area	$1000\times 1000$ m 2	$1000\times 1000$ m 2	$100\times 100$ m 2	$500\times 500$ m 2	$2000\times 2000$ m 2	$4000\times 4000$ m 2

TABLE I : Comparison Between the Proposed Framework and Related Works

Symbol	Description	Symbol	Description	Symbol	Description
$a,b$	Environment-dependent parameters	$a_{m,k}[t]$	User association indicator	$\mathbf{A}[t]$	User association matrix
$\mathbf{a}_{m}[t]$	Action of UAV $m$	$A_{r}$	Rotor disc area	$B$	System bandwidth
$c_{1},c_{2}$	PSO Acceleration coefficients	$\mathbf{C}_{m}$	TAKM cluster centroid $m$	$C_{m}[t]$	Total power consumption of UAV $m$
$d_{m,k}[t]$	Distance between UAV $m$ and user $k$	$d_{\min}$	Minimum separation distance	$d_{\mathrm{safe}}$	Safety margin around threat zones
$d_{f}$	Fuselage drag ratio	$D_{n}$	Minimum distance to threat $n$	$\mathbf{g}_{n}$	Center of threat zone $n$
$g_{m,k}[t]$	Channel power gain	$H_{m}$	Altitude of UAV $m$	$I$	Max iterations in TAKM

TABLE II : List of Notations

\textbf{a}_{m}[t]=\mathrm{clip}\big(\pi_{\mathbf{U}_{m}}(o_{m})+\mathcal{N}(0,\sigma_{e}^{2}),0,1\big)

Algorithm 3 The Proposed MATD3 with TAKM clustering and pre-placement

Algorithm	Computational Complexity	Approximate Operations
MATD3	$\mathcal{O}(MKI)+\mathcal{O}(M^{3})+\mathcal{O}\!\left(Lt_{\max}\!\left(2M\sum\limits_{i=0}^{L_{a}}N_{a}^{(i)}N_{a}^{(i+1)}+4\sum\limits_{j=0}^{L_{c}}N_{c}^{(j)}N_{c}^{(j+1)}\right)\right)+\mathcal{O}(Lt_{\max}MK)$	Offline Training: $6.4\times 10^{10}$ , Online Deployment: $4.7\times 10^{5}$
MADDPG	$\mathcal{O}(MKI)+\mathcal{O}(M^{3})+\mathcal{O}\!\left(Lt_{\max}\!\left(2M\sum\limits_{i=0}^{L_{a}}N_{a}^{(i)}N_{a}^{(i+1)}+2\sum\limits_{j=0}^{L_{c}}N_{c}^{(j)}N_{c}^{(j+1)}\right)\right)+\mathcal{O}(Lt_{\max}MK)$	Offline Training: $5.8\times 10^{10}$ , Online Deployment: $4.7\times 10^{5}$
PSO	$\mathcal{O}(MKI)+\mathcal{O}(M^{3})+\mathcal{O}\!\left(I_{\text{PSO}}N_{p}(MK+M^{2}+MN)t_{\max}\right)$	Online Deployment: $1.6\times 10^{10}$

TABLE III : Computational Complexity Comparison of Different Algorithms

Parameter	$B$	$K$	$N$	$f$
Value	10 MHz	400	2	1 GHz
Parameter	$\sigma_{n}^{2}$	$P_{\max}$	$V_{\max}$	$\delta_{t}$
Value	-111 dBm	33 dBm	10 m/s	300 ms
Parameter	$d_{safe}$	$\zeta_{saf}$	$d_{\min}$	$H$
Value	10 m	5	10 m	300 m
Parameter	$a$	$b$	$\gamma_{\text{LoS}}$	$\gamma_{\text{NLoS}}$

TABLE IV : Simulation Parameters

Fig. 3 : Clustering using the CKM Algorithm.

Fig. 4 : Clustering using the proposed TAKM Algorithm.

Cluster	CKM		TAKM
Cluster	Threat 1	Threat 2	Threat 1	Threat 2
Cluster 1	3269.50	1134.33	3274.17	1138.50
Cluster 2	2754.85	2139.65	2766.54	2150.54
Cluster 3	151.31 (UNSAFE)	2285.86	510	1784.66
Cluster 4	2018.21	1668.77	2240.12	1785.94

TABLE V : Distance Between CKM/TAKM Cluster Centroids and Threats

Cluster	Cluster Centroid		Number of GUs
Cluster	CKM	TAKM	CKM	TAKM
Cluster 1	[3224.35, 3080.82]	[3224.35, 3088.92]	65	65
Cluster 2	[3239.34, 602.82]	[3249.8, 596]	200	198
Cluster 3	[909.04, 1145]	[1060, 1170]	52	55
Cluster 4	[739.47, 3209.3]	[727.2, 3433.1]	83	82

TABLE VI : Cluster Position and Number of GUs

Parameter	Value
Number of episodes ( $L$ )	400
Episode duration	30 s (100 time slots)
Discount factor $\alpha$	$0.95$
Soft updated factor $\tau$	$0.005$
Actor network	3 hidden layers, 256 neurons each
Critic network	3 hidden layers, 256 neurons each

TABLE VII : MARL Simulation Parameters

Fig. 5 : Safety violations versus Episodes for MATD3 with CKM clustering and Pre-placement.

Fig. 6 : The average reward comparison between the proposed scheme and the benchmark schemes.

Panel of Figure 15 — Fig. 7 : Safety violations versus Episodes for the proposed scheme and the benchmark schemes.

Fig. 8 : Rate violations percentage vs. episodes for the proposed scheme and benchmark schemes.

Fig. 9 : The average reward performance with different altitudes for the proposed scheme.

Fig. 10 : Number of served users at different altitudes for the proposed scheme.

Fig. 11 : The average reward performance with different number of users for the proposed scheme.

Method	Cluster 1	Cluster 2	Cluster 3	Cluster 4	SDNU
CKM	65	200	52	83	58.78
TAKM	65	198	55	82	57.40
MATD3	64	183	67	86	48.66

TABLE VIII : Comparison of User Distribution.

Fig. 12 : The trajectories of UAVs for the proposed scheme.

Fig. 13 : Distribution of user rates at final UAV positions.

Fig. 14 : Energy efficiency versus time steps for the final trajectories in the proposed and PSO schemes.

Scenario	User Distribution	EE (b/J)
Training	rng (1)	$4.48\times 10^{6}$
Testing (Same)	rng (1)	$4.24\times 10^{6}$
Testing (Different)	rng (4)	$4.10\times 10^{6}$
Testing (Different)	rng (10)	$4.06\times 10^{6}$
Testing (Different)	rng (11)	$4.12\times 10^{6}$
Testing (Different)	rng (14)	$4.22\times 10^{6}$

TABLE IX : Reproducibility and Robustness Evaluation of the MATD3 Framework

Fig. 15 : UAV trajectories under a new threat configuration with different zone centers and radii.

Fig. 16 : Average reward versus episode for the proposed scheme at different observation scenarios.

UAV Communication

See all 8 papers

PRO-SPECT: Probabilistically Safe Scalable Planning for Energy-Aware Coordinated UAV-UGV Teams in Stochastic Environments
Apr 2, 2026Roger Fowler, Cahit Ikbal Er, Benjamin Johnsenberg +1Motion Planning Multi-Robot Coordination
Khoury Department of Computer Sciences, Northeastern University, Boston, MA · The Charles Stark Draper Laboratory, Inc., Cambridge, MA · Department of Mechanical and Industrial Engineering, Northeastern University, Boston, MA +1
Dynamic-TD3: A Novel Algorithm for UAV Path Planning with Dynamic Obstacle Trajectory Prediction
Apr 30, 2026Wentao Chen, Jingtang Chen, Mingjian Fu +4Reinforcement Learning UAV Navigation
Communication-Aware Multi-Agent Reinforcement Learning for Decentralized Cooperative UAV Deployment
Mar 17, 2026Enguang Fan, Yifan Chen, Zihan Shan +3Cooperative MARL Decentralized MARL
University of Illinois at Urbana-Champaign · Boeing Research and Technology

Apr 2, 2026J/K move · Enter open · S save

PRO-SPECT: Probabilistically Safe Scalable Planning for Energy-Aware Coordinated UAV-UGV Teams in Stochastic Environments

Roger Fowler, Cahit Ikbal Er, Benjamin Johnsenberg, Yasin Yazicioglu

Khoury Department of Computer Sciences, Northeastern University, Boston, MA · The Charles Stark Draper Laboratory, Inc., Cambridge, MA · Department of Mechanical and Industrial Engineering, Northeastern University, Boston, MA+1

Motion Planning Multi-Robot Coordination

Open paper AskPDF arXiv

We consider energy-aware planning for an unmanned aerial vehicle (UAV) and unmanned ground vehicle (UGV) team operating in a stochastic environment. The UAV must visit a set of air points in minimum time while respecting energy constraints, relying on the UGV as a mobile charging station. Unlike prior work that assumed deterministic travel times or used fixed robustness margins, we model travel times as random variables and bound the probability of failure (energy depletion) across the entire mission to a user-specified risk level. We formulate the problem as a Mixed-Integer Program and propose PRO-SPECT, a polynomial-time algorithm that generates risk-bounded plans. The algorithm supports both offline planning and online re-planning, enabling the team to adapt to disturbances while preserving the risk bound. We provide theoretical results on solution feasibility and time complexity. We also demonstrate the performance of our method via numerical comparisons and simulations.

Abstract

UAV Communication

Explore similar work

PRO-SPECT: Probabilistically Safe Scalable Planning for Energy-Aware Coordinated UAV-UGV Teams in Stochastic Environments

Dynamic-TD3: A Novel Algorithm for UAV Path Planning with Dynamic Obstacle Trajectory Prediction

Communication-Aware Multi-Agent Reinforcement Learning for Decentralized Cooperative UAV Deployment