Cyber Deception for Mission Surveillance via Hypergame-Theoretic Deep Reinforcement Learning

Zelin Wan; Jin-Hee Cho; Mu Zhu; Ahmed H. Anwar; Charles Kamhoua; Munindar P. Singh

cs.CRMar 21, 2026

Cyber Deception for Mission Surveillance via Hypergame-Theoretic Deep Reinforcement Learning

Authors: Zelin Wan, Jin-Hee Cho, Mu Zhu, Ahmed H. Anwar, Charles Kamhoua, Munindar P. Singh

Organizations: Department of Computer Science, Virginia Tech, Falls Church, VA, USA · Computer Network Information Center, Chinese Academy of Sciences, Beijing, China · US Army Research Laboratory, Adelphi, MD, USA · Department of Computer Science, North Carolina State University, Raleigh, NC 27695

Abstract

Unmanned Aerial Vehicles (UAVs) are valuable for mission-critical systems like surveillance, rescue, or delivery. Not surprisingly, such systems attract cyberattacks, including Denial-of-Service (DoS) attacks to overwhelm the resources of mission drones (MDs). How can we defend UAV mission systems against DoS attacks? We adopt cyber deception as a defense strategy, in which honey drones (HDs) are proposed to bait and divert attacks. The attack and deceptive defense hinge upon radio signal strength: The attacker selects victim MDs based on their signals, and HDs attract the attacker from afar by emitting stronger signals, despite this reducing battery life. We formulate an optimization problem for the attacker and defender to identify their respective strategies for maximizing mission performance while minimizing energy consumption. To address this problem, we propose a novel approach, called HT-DRL. HT-DRL identifies optimal solutions without a long learning convergence time by taking the solutions of hypergame theory into the neural network of deep reinforcement learning. This achieves a systematic way to intelligently deceive attackers. We analyze the performance of diverse defense mechanisms under different attack strategies. Further, the HT-DRL-based HD approach outperforms existing non-HD counterparts up to two times better in mission performance while incurring low energy consumption.

Figures & tables

Panel of Figure 1 — Fig. 1: (1) Example drone fleet with multi-hop communications consisting of MDs, HDs, RLD, and GCS. (2) Example of the Neighbor Table (NT) and Fleet Table (FT) where NT provides information about the neighboring drones of a given drone, while FT tracks the membership of the mission team, including drones executing missions, leaving the team to recharge batteries, or compromised by the attacker. Drones with a battery level lower than the energy threshold $T_{e}$ will leave the mission team and move to the charging station.

Fig. 2: The procedures generating the solutions by a HT-DRL agent: $\mathcal{S}_{t}$ is the state at round $t$ and $\pi(a_{t}|\mathcal{S}_{t})$ is the probability of all actions.

Fig. 3: Hypergame-Theoretic Strategy Selection Methods by the Attacker and Defender.

Fig. 4: Conceptual model of the proposed honey-drone mission system in which both the DoS attacker and the defender employ Deep Reinforcement Learning to select their optimal signal strengths. This figure is adapted from our prior design [ 11 ] .

Symbol	Meaning	Default
$\rho$	Signal strength decrement interval	5
$T_{C}$	Time duration of battery being charged	30
$T_{M}^{\mathrm{max}}$	Maximum mission duration	150
$[\tau_{l},\tau_{u}]$	The maximum number of MDs that an HD can protect simultaneously	$[2,4]$
$E_{P}$	Energy consumption rate by a drone	7,900 mW
$E_{C}$	Energy consumption rate by a camera	4 mW

TABLE I: Key Design Parameters and Default Values

Panel of Figure 6 — Fig. 5: Performance comparison of the considered defense approaches with respect to the ratio of completed mission tasks ( $\mathcal{R}_{MC}$ ), given an attack strategy.

Panel of Figure 7 — Fig. 6: Performance analysis of HD-HT-DRL, IDS, CD, and fixed defense, given an attack strategy with respect to energy consumption ( $\mathcal{EC}$ ).

Panel of Figure 8 — Fig. 7: Performance analysis of honey drone defense strategies with respect to the ratio of completed mission tasks ( $\mathcal{R}_{MC}$ ), given an attack strategy.

Panel of Figure 9 — Fig. 8: Performance analysis of honey drone defense strategies with respect to energy consumption ( $\mathcal{EC}$ ), given an attack strategy.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 16 — Fig. 1: Performance analysis of HD-HT-DRL, IDS, CD, and fixed defense, given an attack strategy, with respect to the number of active, connected drones ( $\mathcal{N}_{AC}$ ).

Panel of Figure 17 — Fig. 2: Performance analysis of HD-HT-DRL, IDS, CD, and fixed defense, given an attack strategy, with respect to the attacker’s accumulated reward ( $\mathcal{G}^{A}$ ).

Panel of Figure 18 — Fig. 3: Performance analysis of HD-HT-DRL, IDS, CD, and fixed defense, given an attack strategy, with respect to the defender’s accumulated reward ( $\mathcal{G}^{D}$ ).

Panel of Figure 19 — Fig. 4: Performance analysis of honey done-based defenses with respect to the number of active, connected drones ( $\mathcal{N}_{AC}$ ), under a given attack.

Panel of Figure 20 — Fig. 5: Performance analysis of honey done-based defenses with respect to the attacker’s accumulated reward ( $\mathcal{G}^{A}$ ), under a given attack.

Panel of Figure 21 — Fig. 6: Performance analysis of honey done-based defenses with respect to the defender’s accumulated reward ( $\mathcal{G}^{D}$ ), under a given attack.

Panel of Figure 22 — Fig. 7: Effect of varying the attack budget ( $\zeta$ ) on the ratio of completed mission tasks ( $\mathcal{R}_{MC}$ ).

Panel of Figure 23 — Fig. 8: Effect of varying the attack budget ( $\zeta$ ) on energy consumption ( $\mathcal{EC}$ ).

Panel of Figure 24 — Fig. 9: Effect of varying the attack budget ( $\zeta$ ) on the number of active, connected drones ( $\mathcal{N}_{AC}$ ).

Panel of Figure 25 — Fig. 10: Effect of varying the attack budget ( $\zeta$ ) on the attacker’s accumulated reward ( $\mathcal{G}^{A}$ ).

Panel of Figure 26 — Fig. 11: Effect of varying the attack budget ( $\zeta$ ) on the defender’s accumulated reward ( $\mathcal{G}^{D}$ ).

Panel of Figure 27 — Fig. 12: Probability distributions of the strategy selection in HT, DRL, and HT-DRL after the training with the two episodes.

Panel of Figure 28 — Fig. 13: Probability distributions of the strategy selection in HT, DRL, and HT-DRL after the models are fully trained.

Cybersecurity

Cybersecurity develops methods to protect digital and cyber-physical systems from attacks, vulnerabilities, data loss, and misuse.

See all 354 papers

Intent-Context Synergy Reinforcement Learning for Autonomous UAV Decision-Making in Air Combat
Mar 1, 2026Jiahao Fu, Feng YangReinforcement Learning Partially Observable RL
National Elite Institute of Engineering,Northwestern Polytechnical University, Xi’an, Shanxi, China · School of Automation, Northwestern Polytechnical University, Xi’an, Shanxi, China
TASER: Task-Aware Spectral Energy Refine for Backdoor Suppression in UAV Swarms Decentralized Federated Learning
Mar 10, 2026Sizhe Huang, Shujie YangBackdoor Attacks in Federated Learning Federated Learning
Threat-Aware Energy-Efficient Deployment for Dynamic UAV Networks: A Multi-Agent RL Approach
Sep 25, 2026Faisal Al-Kamali, Hussein A. Ammar, Francois Chan +3UAV Communication UAV Trajectory Optimization
School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, ON K1N 6N5, Canada · Department of Electrical Engineering, Ibb University, Ibb, Yemen · Department of Electrical and Computer Engineering, Royal Military College, Kingston, ON, K7K 7B4, Canada +1

Mar 1, 2026J/K move · Enter open · S save

Intent-Context Synergy Reinforcement Learning for Autonomous UAV Decision-Making in Air Combat

Jiahao Fu, Feng Yang

National Elite Institute of Engineering,Northwestern Polytechnical University, Xi’an, Shanxi, China · School of Automation, Northwestern Polytechnical University, Xi’an, Shanxi, China

Reinforcement Learning Partially Observable RL Deep Q-Learning Hierarchical RL

Open paper AskPDF arXiv

Autonomous UAV infiltration in dynamic contested environments remains a significant challenge due to the partially observable nature of threats and the conflicting objectives of mission efficiency versus survivability. Traditional Reinforcement Learning (RL) approaches often suffer from myopic decision-making and struggle to balance these trade-offs in real-time. To address these limitations, this paper proposes an Intent-Context Synergy Reinforcement Learning (ICS-RL) framework. The framework introduces two core innovations: (1) An LSTM-based Intent Prediction Module that forecasts the future trajectories of hostile units, transforming the decision paradigm from reactive avoidance to proactive planning via state augmentation; (2) A Context-Analysis Synergy Mechanism that decomposes the mission into hierarchical sub-tasks (safe cruise, stealth planning, and hostile breakthrough). We design a heterogeneous ensemble of Dueling DQN agents, each specialized in a specific tactical context. A dynamic switching controller based on Max-Advantage values seamlessly integrates these agents, allowing the UAV to adaptively select the optimal policy without hard-coded rules. Extensive simulations demonstrate that ICS-RL significantly outperforms baselines (Standard DDQN) and traditional methods (PSO, Game Theory). The proposed method achieves a mission success rate of 88% and reduces the average exposure frequency to 0.24 per episode, validating its superiority in ensuring robust and stealthy penetration in high-dynamic scenarios.

Abstract

Cybersecurity

Explore similar work

Intent-Context Synergy Reinforcement Learning for Autonomous UAV Decision-Making in Air Combat

TASER: Task-Aware Spectral Energy Refine for Backdoor Suppression in UAV Swarms Decentralized Federated Learning

Threat-Aware Energy-Efficient Deployment for Dynamic UAV Networks: A Multi-Agent RL Approach