Plasticity-Enhanced Multi-Agent Mixture of Experts for Dynamic Objective Adaptation in UAVs-Assisted Emergency Communication Networks

Wen Qiu; Zhiqiang He; Wei Zhao; Hiroshi Masui

cs.MAApr 10, 2026

Plasticity-Enhanced Multi-Agent Mixture of Experts for Dynamic Objective Adaptation in UAVs-Assisted Emergency Communication Networks

Authors: Wen Qiu, Zhiqiang He, Wei Zhao, Hiroshi Masui

Organizations: Department of Information and Communication Engineering, Kitami Institute of Technology, Japan. · Graduate School of Informatics and Engineering, the University of Electro-Communications, Japan. · School of Computer Science and Technology, Anhui University of Technology, China. · Department of Information and Communication Engineering and also Information Procession Center, Kitami Institute of Technology, Japan.

Abstract

Unmanned aerial vehicles serving as aerial base stations can rapidly restore connectivity after disasters, yet abrupt changes in user mobility and traffic demands shift the quality of service trade-offs and induce strong non-stationarity. Deep reinforcement learning policies suffer from plasticity loss under such shifts, as representation collapse and neuron dormancy impair adaptation. We propose plasticity enhanced multi-agent mixture of experts (PE-MAMoE), a centralized training with decentralized execution framework built on multi-agent proximal policy optimization. PE-MAMoE equips each UAV with a sparsely gated mixture of experts actor whose router selects a single specialist per step. A non-parametric Phase Controller injects brief, expert-only stochastic perturbations after phase switches, resets the action log-standard-deviation, anneals entropy and learning rate, and schedules the router temperature, all to re-plasticize the policy without destabilizing safe behaviors. We derive a dynamic regret bound showing the tracking error scales with both environment variation and cumulative noise energy. In a phase-driven simulator with mobile users and 3GPP-style channels, PE-MAMoE improves normalized interquartile mean return by 26.3% over the best baseline, increases served-user capacity by 12.8%, and reduces collisions by approximately 75%. Diagnostics confirm persistently higher expert feature rank and periodic dormant-neuron recovery at regime switches.

Figures & tables

Symbol	Description
System model
$\Omega$	Disaster area $[0,S]^{2}$
$\mathcal{U}$ , $U$	UAV set and number of UAVs
$\mathcal{N}(t)$ , $N$	Active ground user set and total users
$\mathbf{q}_{u}(t)$	3D position of UAV $u$ at time $t$
$\Delta t$	Time slot duration

TABLE I: Summary of Key Notation

Fig. 1: PE-MAMoE architecture. For each UAV $i$ , the local observation $o_{t}^{i}$ is fed into the Actor router . The router produces gates $g$ , dispatches to the Top- $k$ experts, and combines their outputs to form a shared MoE trunk, which parameterizes the policy head $(\mu_{a},\log\sigma)$ . A Phase Controller performs non-gradient scheduling at phase switch: router $\tau$ -anneal, per group learning rate scheduling and Adam state reset, and log- $\sigma$ reset $\rightarrow$ clamp/anneal; it also injects expert noise to enhance plasticity. On the critic side, the centralized state $X_{t}$ is processed by a Critic MoE encoder to produce the team value $V_{\text{team}}(X_{t})$ . Training follows CTDE with the total loss $L_{\text{total}}=L_{\text{ppo}}+\lambda_{v}L_{\text{value}}-\lambda_{\text{ent}}L_{\text{ent}}+\lambda_{\text{kl\_ref}}L_{\text{kl\_ref}}$ . Gradients propagate to routers and experts (conditional computation: only the selected experts execute).

Method	Actor params	Params vs. MLP	Inference active params
MLP	6 820	1.00 $\times$	6 820 (all)
MoE	20 858	3.06 $\times$	20 858 (all experts)
SMoE	21 392	3.14 $\times$	7 820 (top-1 + router)
PE-MAMoE	21 392	3.14 $\times$	7 820 (top-1 + router)

TABLE III: Computational cost comparison (actor network).

Fig. 2: Evolution of the average expert feature rank over training.

Fig. 3: Evolution of the total dormant neuron fraction over training.

Fig. 5: Normalized IQM across five system metrics. Each bar shows the per-metric min–max normalized IQM (middle 50% of episode values); error bars indicate standard error. Higher is better for Coverage, Return, and ServedNumber; lower is better for EnergyUse and Collisions (plotted as

1-\text{normalized value}

so that taller bars remain preferable).

Panel of Figure 8 — Fig. 6: Probability distribution of each expert selected for Policy and Value.

Fig. 7: Collisions per Episode over Training.

Fig. 8: Mean UAV Energy Rate over Training.

Panel of Figure 11 — Fig. 9: Performance comparison based on UAVs served users.

Fig. 10: Mean Action Response to Phase Switches.

Fig. 11: Effect of Noise Injection Threshold on Coverage.

Fig. 12: Ablation Study: IQM Drop When Removing Each Component.

UAV Communication

See all 8 papers

PRIME: Plasticity Recovery in Multi-Agent Environments for UAV-Assisted Emergency Communication Networks
Jul 20, 2026Wen Qiu, Zhiqiang He, Wei Zhao +1Non-Stationary RL Reinforcement Learning
Department of Information and Communication Engineering, Kitami Institute of Technology, Japan · Graduate School of Informatics and Engineering, University of Electro-Communications, Japan · School of Computer Science and Technology, Anhui University of Technology, China
MA-HEAD-Net: Adaptive Rule-Guided Multi-Agent DRL for AoI Minimization in UAV-Assisted Emergency Networks
Aug 2, 2026Yixin Zhang, Zhuohui Yao, Wenchi Cheng +1Multi-Agent Control Age of Information
State Key Laboratory of Integrated Services Networks, Xidian University, Xi’an, China · Bradley Department of Electrical and Computer Engineering, Virginia Tech, Alexandria, VA, USA
Communication-Aware Multi-Agent Reinforcement Learning for Decentralized Cooperative UAV Deployment
Mar 17, 2026Enguang Fan, Yifan Chen, Zihan Shan +3Cooperative MARL Decentralized MARL
University of Illinois at Urbana-Champaign · Boeing Research and Technology

Jul 20, 2026J/K move · Enter open · S save

PRIME: Plasticity Recovery in Multi-Agent Environments for UAV-Assisted Emergency Communication Networks

Wen Qiu, Zhiqiang He, Wei Zhao, Hiroshi Masui

Department of Information and Communication Engineering, Kitami Institute of Technology, Japan · Graduate School of Informatics and Engineering, University of Electro-Communications, Japan · School of Computer Science and Technology, Anhui University of Technology, China

Non-Stationary RL Reinforcement Learning Cooperative MARL

Open paper AskPDF arXiv

Most reinforcement learning controllers for these networks assume stationary conditions, and the few that handle change react to the external environment while leaving the network's internal state unexamined. We show that sustained non-stationarity damages this internal state directly: as objectives shift, neurons progressively fall dormant and the shared policy loses the capacity to learn. The obvious remedy, resetting dormant neurons, is unsafe under shared-parameter multi-agent training: many neurons that appear inactive are still receiving strong training gradients, and whether a neuron appears dormant depends on which agent's observations it processes. PRIME (Plasticity Recovery In Multi-agent Environments) therefore verifies both directions before intervening. Extending the bidirectional Silent Neuron framework to cooperative multi-agent reinforcement learning, it aggregates activation and gradient statistics over the full team batch, reads the backward signal from the gradient the training loss has already deposited , not from a hand-crafted proxy, and reinitializes only neurons that are simultaneously activation-dormant and gradient-silent. Useful representations are preserved while learning capacity is restored. On a phase-switching UAV emergency communication simulator, PRIME improves interquartile mean return by 24.9% over MAPPO and holds dormant neuron fractions at 10--20% versus 40--45%; ablations attribute the gains to the gradient signal and team-level aggregation rather than to the specific reset operator. A dynamic regret bound shows that the perturbation cost scales with the small silent-subspace dimension rather than the full parameter count.

Abstract

UAV Communication

Explore similar work

PRIME: Plasticity Recovery in Multi-Agent Environments for UAV-Assisted Emergency Communication Networks

MA-HEAD-Net: Adaptive Rule-Guided Multi-Agent DRL for AoI Minimization in UAV-Assisted Emergency Networks

Communication-Aware Multi-Agent Reinforcement Learning for Decentralized Cooperative UAV Deployment