Organizations: Department of Information and Communication Engineering, Kitami Institute of Technology, Japan. · Graduate School of Informatics and Engineering, the University of Electro-Communications, Japan. · School of Computer Science and Technology, Anhui University of Technology, China. · Department of Information and Communication Engineering and also Information Procession Center, Kitami Institute of Technology, Japan.
Unmanned aerial vehicles serving as aerial base stations can rapidly restore connectivity after disasters, yet abrupt changes in user mobility and traffic demands shift the quality of service trade-offs and induce strong non-stationarity. Deep reinforcement learning policies suffer from plasticity loss under such shifts, as representation collapse and neuron dormancy impair adaptation. We propose plasticity enhanced multi-agent mixture of experts (PE-MAMoE), a centralized training with decentralized execution framework built on multi-agent proximal policy optimization. PE-MAMoE equips each UAV with a sparsely gated mixture of experts actor whose router selects a single specialist per step. A non-parametric Phase Controller injects brief, expert-only stochastic perturbations after phase switches, resets the action log-standard-deviation, anneals entropy and learning rate, and schedules the router temperature, all to re-plasticize the policy without destabilizing safe behaviors. We derive a dynamic regret bound showing the tracking error scales with both environment variation and cumulative noise energy. In a phase-driven simulator with mobile users and 3GPP-style channels, PE-MAMoE improves normalized interquartile mean return by 26.3% over the best baseline, increases served-user capacity by 12.8%, and reduces collisions by approximately 75%. Diagnostics confirm persistently higher expert feature rank and periodic dormant-neuron recovery at regime switches.
Figures & tables
Symbol
Description
System model
Ω
Disaster area [0,S]2
U , U
UAV set and number of UAVs
N(t) , N
Active ground user set and total users
qu(t)
3D position of UAV u at time t
Δt
Time slot duration
TABLE I: Summary of Key Notation
Fig. 1: PE-MAMoE architecture. For each UAV i , the local observation oti is fed into the Actor router . The router produces gates g , dispatches to the Top- k experts, and combines their outputs to form a shared MoE trunk, which parameterizes the policy head (μa,logσ) . A Phase Controller performs non-gradient scheduling at phase switch: router τ -anneal, per group learning rate scheduling and Adam state reset, and log- σ reset → clamp/anneal; it also injects expert noise to enhance plasticity. On the critic side, the centralized state Xt is processed by a Critic MoE encoder to produce the team value Vteam(Xt) . Training follows CTDE with the total loss Ltotal=Lppo+λvLvalue−λentLent+λkl_refLkl_ref . Gradients propagate to routers and experts (conditional computation: only the selected experts execute).
Fig. 2: Evolution of the average expert feature rank over training.
Fig. 3: Evolution of the total dormant neuron fraction over training.
Fig. 4: Mean Return over Training.
Fig. 5: Normalized IQM across five system metrics. Each bar shows the per-metric min–max normalized IQM (middle 50% of episode values); error bars indicate standard error. Higher is better for Coverage, Return, and ServedNumber; lower is better for EnergyUse and Collisions (plotted as 1−normalized value so that taller bars remain preferable).
Fig. 6: Probability distribution of each expert selected for Policy and Value.
Fig. 7: Collisions per Episode over Training.
Fig. 8: Mean UAV Energy Rate over Training.
Fig. 9: Performance comparison based on UAVs served users.
Fig. 10: Mean Action Response to Phase Switches.
Fig. 11: Effect of Noise Injection Threshold on Coverage.
Fig. 12: Ablation Study: IQM Drop When Removing Each Component.
Most reinforcement learning controllers for these networks assume stationary conditions, and the few that handle change react to the external environment while leaving the network's internal state unexamined. We show that sustained non-stationarity damages this internal state directly: as objectives shift, neurons progressively fall dormant and the shared policy loses the capacity to learn. The obvious remedy, resetting dormant neurons, is unsafe under shared-parameter multi-agent training: many neurons that appear inactive are still receiving strong training gradients, and whether a neuron appears dormant depends on which agent's observations it processes. PRIME (Plasticity Recovery In Multi-agent Environments) therefore verifies both directions before intervening. Extending the bidirectional Silent Neuron framework to cooperative multi-agent reinforcement learning, it aggregates activation and gradient statistics over the full team batch, reads the backward signal from the gradient the training loss has already deposited , not from a hand-crafted proxy, and reinitializes only neurons that are simultaneously activation-dormant and gradient-silent. Useful representations are preserved while learning capacity is restored. On a phase-switching UAV emergency communication simulator, PRIME improves interquartile mean return by 24.9% over MAPPO and holds dormant neuron fractions at 10--20% versus 40--45%; ablations attribute the gains to the gradient signal and team-level aggregation rather than to the specific reset operator. A dynamic regret bound shows that the perturbation cost scales with the small silent-subspace dimension rather than the full parameter count.
Wen Qiu, Zhiqiang He, Wei Zhao +1
Department of Information and Communication Engineering, Kitami Institute of Technology, Japan · Graduate School of Informatics and Engineering, University of Electro-Communications, Japan · School of Computer Science and Technology, Anhui University of Technology, China
In post-disaster scenarios, unmanned aerial vehicles (UAVs) are critical for establishing emergency communication networks. For time-critical rescue missions, information freshness is crucial because decisions based on outdated data may lead to ineffective control actions. This paper investigates age of information (AoI) minimization for UAV-assisted emergency communications with heterogeneous emergency services. We model bursty packet arrivals using a Markov-modulated Poisson process and adopt finite blocklength theory to capture the coupling among transmission duration, packet completion, and AoI evolution. To balance delay-tolerant long-packet transmission and urgent short-packet response, we propose a mini-slot-embedded scheduling mechanism with adaptive checkpoint-interval selection. We formulate the joint optimization of UAV trajectory control, user scheduling, and checkpoint-interval selection as a multi-agent decision problem, and develop MA-HEAD-Net, an adaptive rule-guided multi-agent deep reinforcement learning framework. MA-HEAD-Net incorporates communication-domain rule priors into a gated multi-head policy, where adaptive gates regulate the contributions of rule-prior and learned-policy logits for different subtasks. The policy and gating components are jointly optimized under multi-agent proximal policy optimization. Simulation results show that MA-HEAD-Net improves policy-formation efficiency compared with representative multi-agent deep reinforcement learning baselines and achieves lower AoI than both learning-based and heuristic methods in dynamic UAV-assisted emergency communication scenarios.
Yixin Zhang, Zhuohui Yao, Wenchi Cheng +1
State Key Laboratory of Integrated Services Networks, Xidian University, Xi’an, China · Bradley Department of Electrical and Computer Engineering, Virginia Tech, Alexandria, VA, USA
Autonomous Unmanned Aerial Vehicle (UAV) swarms are increasingly used as rapidly deployable aerial relays and sensing platforms, yet practical deployments must operate under partial observability and intermittent peer-to-peer connectivity. We present a graph-based multi-agent reinforcement learning framework trained under centralized training with decentralized execution (CTDE): a centralized critic and global state are available only during training, while each UAV executes a shared policy using local observations and messages from nearby neighbors. Under restricted communication, neighbor relations are induced by an SNR-threshold connectivity graph. Our architecture encodes local agent state and nearby entities with an agent-entity attention module and aggregates inter-UAV messages with neighbor self-attention over a signal-quality-limited communication graph defined by a channel model. We evaluate the framework on a cooperative relay-deployment task, DroneConnect. Experimental results show that the proposed method achieves an approximately 12% increase in target coverage over MAPPO under restricted communication and partial observability, while remaining competitive with a mixed-integer linear programming (MILP)-based offline upper bound with full node observability.
Enguang Fan, Yifan Chen, Zihan Shan +3
University of Illinois at Urbana-Champaign · Boeing Research and Technology