Alleviating Community Fear in Disasters via Multi-Agent Actor-Critic Reinforcement Learning

Yashodhan D. Hakke; Almuatazbellah M. Boker; Lamine Mili; Michael von Spakovsky; Hoda Eldardiry

cs.LGApr 9, 2026

Alleviating Community Fear in Disasters via Multi-Agent Actor-Critic Reinforcement Learning

Authors: Yashodhan D. Hakke, Almuatazbellah M. Boker, Lamine Mili, Michael von Spakovsky, Hoda Eldardiry

Organizations: Bradley Department of Electrical and Computer Engineering, Virginia Tech, Blacksburg, VA, USA · Department of Mechanical Engineering, Virginia Tech, Blacksburg, VA, USA · Department of Computer Science, Virginia Tech, Blacksburg, VA, USA

Abstract

During disasters, cascading failures across power grids, communication networks, and social behavior amplify community fear and undermine cooperation. Existing cyber-physical-social (CPS) models simulate these coupled dynamics but lack mechanisms for active intervention. We extend the CPS resilience model of Valinejad and Mili (2023) with control channels for three agencies, communication, power, and emergency management, and formulate the resulting system as a three-player non-zero-sum differential game solved via online actor-critic reinforcement learning. Simulations based on Hurricane Harvey data show 70% mean fear reduction with improved infrastructure recovery; cross-validation in the case of Hurricane Irma (without refitting) achieves 50% fear reduction, confirming generalizability.

Figures & tables

Index	Variable	Layer
$x_{1}$	Fear	Social
$x_{2}$	Information seeking	Social
$x_{3}$	Flexibility / adaptability	Social
$x_{4}$	Physical health	Social
$x_{5}$	Risk perception	Social
$x_{6}$	Cooperation	Social

TABLE I: Community state vector

x(t)

and component interpretation.

Group	Symbols	Source
CPSS model (refit)	$\kappa,\;\zeta$	Structure from [ 7 ] ; values refit via LS
	$\alpha_{\theta},\;\eta_{\theta},\;\eta_{P},\;\eta_{R},\;\eta_{C},\;\eta_{L}$
	$\{\iota_{k}\},\;\gamma_{8},\;\gamma_{9},\;\gamma_{10}$
Control design (chosen)	$\beta_{1},\;\beta_{8},\;\beta_{9},\;\beta_{10}$	Domain knowledge + drift matching
	$q_{i,k},\;R_{ii},\;R_{ij}$
Learning (tuned)	$\alpha_{c,i},\;\alpha_{a,i},\;p,\;\phi(x)$	RL best practice [ 10 ]

TABLE II: Parameter groups, symbols, and calibration sources.

Panel of Figure 3 — Fig. 1: Ten CPSS state trajectories under three conditions. Black markers : observed data (actual measurements from Hurricane Harvey/Irma via LIWC text mining, utility outage reports, and FEMA logs); blue dashed : open-loop baseline (model simulation with identified parameters and no control input, $u\equiv 0$ ); red solid : closed-loop trajectory under the learned actor–critic policies. The gap between observed data and the open-loop baseline reflects the model prediction error (RMSE $=0.19$ for Harvey, $0.29$ for Irma). (a) Harvey: the controller reduces mean fear ( $x_{1}$ ) by $\sim$ 70% relative to the baseline while improving power availability ( $x_{8}$ ) and physical health ( $x_{4}$ ). (b) Irma (Harvey-fitted parameters, no refitting): mean fear is reduced by $\sim$ 50% despite a higher initial fear ( $x_{1}(0)=0.9$ vs. $0.62$ ); power availability is maintained near 1.0 even during the sharp drop at steps 10–11.

Panel of Figure 4 — Fig. 2: Extended rollouts ( $2\times$ horizon). Beyond the data window, exogenous drivers are held constant at their last observed values. (a) Harvey, 34 steps. (b) Irma, 24 steps. In both cases the learned policies continue to suppress fear and stabilize infrastructure in the extrapolated regime.

Panel of Figure 5 — Fig. 3: Control inputs $u_{1}$ (communication), $u_{2}$ (power), $u_{3}$ (EMS). (a) Harvey, 17 steps. (b) Irma, 12 steps. During the exploration phase ( $t\leq 12$ ), sinusoidal probing is visible. After exploration ends, controls settle to small values as fear is suppressed and infrastructure deficits diminish. Player 1 (communication) dominates in both hurricanes, reflecting the dual coupling of $u_{1}$ to fear and fake news.

Fig. 4: Bellman (Hamiltonian) residuals $\varepsilon_{c,i}$ for each player. Player 2 (power) converges near zero within 12 steps. Player 3 (EMS) shows the largest initial residual due to its multi-objective cost ( $x_{1}$ , $x_{4}$ , $x_{9}$ ) but decreases substantially. Player 1 (communication) stabilizes at a small residual.

Fig. 5: Norms of critic ( $\|W_{c,i}\|$ , solid) and actor ( $\|W_{a,i}\|$ , dashed) weight vectors. All weights start from zero (admissible initialization). The actor weights track the critic weights with a lag set by the ratio $\alpha_{a,i}/\alpha_{c,i}$ , as consistent with the two-timescale nature of the learning algorithm.

	Harvey		Irma
Metric	Base	Ctrl	Base	Ctrl
Mean fear ( $x_{1}$ )	0.523	0.158	0.745	0.369
Fear reduction (%)	—	69.8	—	50.4
Power deficit ( $1{-}x_{8}$ )	0.127	0.087	0.113	0.014
Health deficit ( $1{-}x_{4}$ )	0.410	0.371	0.647	0.616
EMS deficit ( $1{-}x_{9}$ )	0.438	0.496	0.602	0.790

TABLE III: Performance comparison across both hurricanes.

Method	Fear % $\uparrow$	Pow	EMS	Effort	$\Sigma J$
Open-loop	0.0	0.127	0.438	0	17.17
Constant $\bar{u}$	92.1	0.052	0.346	51.0	56.76
Prop. (tuned)	85.7	0.121	0.420	1.60	9.06
Centralized AC	70.7	0.085	0.495	1.27	14.01
NZS AC (ours)	69.8	0.087	0.496	0.97	13.81

TABLE IV: Controller comparison (Hurricane Harvey).

\Sigma J

: total integrated cost across all players. Effort:

\sum u_{i}^{2}

Configuration	Fear % $\uparrow$	Pow	EMS	$\mathcal{E}_{1}$ %
Default	69.8	0.087	0.496	5.4
$2{\times}q_{1,1}$	70.0	0.087	0.496	8.5
$0.5{\times}q_{1,1}$	69.7	0.087	0.496	3.4
$5{\times}q_{3,9}$	69.8	0.087	0.491	5.4
$2{\times}\beta_{1}$	80.5	0.087	0.496	2.4
$0.5{\times}\beta_{1}$	45.5	0.087	0.495	1.9

TABLE V: Sensitivity analysis (Hurricane Harvey). Default:

q_{1,1}{=}1

\beta_{1}{=}0.5

\Delta{=}6

. Fear %: mean fear reduction; Pow/EMS: mean deficit;

\mathcal{E}_{1}

: Player 1 exploitability at

\epsilon{=}5\%

	Improvement (%)
Player	5%	10%	20%	$J_{i}^{\mathrm{base}}$	$\Delta J_{i}$
P1 (Comm)	5.4	14.5	17.7	4.29	0.23
P2 (Power)	0.1	0.1	0.1	1.86	$<$ 0.01
P3 (EMS)	0.1	0.1	0.1	6.74	$<$ 0.01

TABLE VI: Unilateral deviation test: maximum cost improvement found by perturbing each player’s actor weights at scale

\epsilon\cdot\|\hat{W}_{a,i}\|

\Delta J_{i}

: absolute improvement at

\epsilon{=}5\%

Fig. 6: PE proxy diagnostics. Top: minimum eigenvalue of the running average covariance of the normalized regressor. Bottom: windowed variant. Both remain near zero because $p{=}66$ exceeds the 17-sample trajectory length. Vertical dashed line marks the end of the exploration phase ( $T_{\mathrm{ex}}{=}12$ ).

Multi-Agent Control

Multi-agent control coordinates interacting autonomous agents to achieve shared goals while respecting safety, communication, and resource constraints.

See all 108 papers

PRIME: Plasticity Recovery in Multi-Agent Environments for UAV-Assisted Emergency Communication Networks
Jul 20, 2026Wen Qiu, Zhiqiang He, Wei Zhao +1Non-Stationary RL Reinforcement Learning
Department of Information and Communication Engineering, Kitami Institute of Technology, Japan · Graduate School of Informatics and Engineering, University of Electro-Communications, Japan · School of Computer Science and Technology, Anhui University of Technology, China
Decentralized Multi-agent Reinforcement Learning for Resilient Critical Infrastructures
Jul 20, 2026Minghui Ding, Evangelos PournarasMulti-Agent Coordination Credit Assignment in RL
School of Computer Science, University of Leeds, Leeds, UK
Two-Timescale Hierarchical Reinforcement Learning for Resilient Operations
Jul 26, 2026Young Hyun Cho, Franz Stoll, Will Wei Sun +2Non-Stationary RL Hierarchical RL
Department of Statistics, Harvard University · Edwardson School of Industrial Engineering, Purdue University · Mitch Daniels School of Business, Purdue University +3

Jul 20, 2026J/K move · Enter open · S save

PRIME: Plasticity Recovery in Multi-Agent Environments for UAV-Assisted Emergency Communication Networks

Wen Qiu, Zhiqiang He, Wei Zhao, Hiroshi Masui

Department of Information and Communication Engineering, Kitami Institute of Technology, Japan · Graduate School of Informatics and Engineering, University of Electro-Communications, Japan · School of Computer Science and Technology, Anhui University of Technology, China

Non-Stationary RL Reinforcement Learning Cooperative MARL

Open paper AskPDF arXiv

Most reinforcement learning controllers for these networks assume stationary conditions, and the few that handle change react to the external environment while leaving the network's internal state unexamined. We show that sustained non-stationarity damages this internal state directly: as objectives shift, neurons progressively fall dormant and the shared policy loses the capacity to learn. The obvious remedy, resetting dormant neurons, is unsafe under shared-parameter multi-agent training: many neurons that appear inactive are still receiving strong training gradients, and whether a neuron appears dormant depends on which agent's observations it processes. PRIME (Plasticity Recovery In Multi-agent Environments) therefore verifies both directions before intervening. Extending the bidirectional Silent Neuron framework to cooperative multi-agent reinforcement learning, it aggregates activation and gradient statistics over the full team batch, reads the backward signal from the gradient the training loss has already deposited , not from a hand-crafted proxy, and reinitializes only neurons that are simultaneously activation-dormant and gradient-silent. Useful representations are preserved while learning capacity is restored. On a phase-switching UAV emergency communication simulator, PRIME improves interquartile mean return by 24.9% over MAPPO and holds dormant neuron fractions at 10--20% versus 40--45%; ablations attribute the gains to the gradient signal and team-level aggregation rather than to the specific reset operator. A dynamic regret bound shows that the perturbation cost scales with the small silent-subspace dimension rather than the full parameter count.

Abstract

Multi-Agent Control

Explore similar work

PRIME: Plasticity Recovery in Multi-Agent Environments for UAV-Assisted Emergency Communication Networks

Decentralized Multi-agent Reinforcement Learning for Resilient Critical Infrastructures

Two-Timescale Hierarchical Reinforcement Learning for Resilient Operations