Probabilistic Robustness-driven Universal Adversarial Perturbations with Explainability against Deep Reinforcement Learning-based Intrusion Detection System

Hongsen Zhang; Lu Zhang; Mingjing Xu; Yi Zhang; Gregory Epiphaniou; Carsten Maple

cs.LGSep 24, 2026

Probabilistic Robustness-driven Universal Adversarial Perturbations with Explainability against Deep Reinforcement Learning-based Intrusion Detection System

Authors: Hongsen Zhang, Lu Zhang, Mingjing Xu, Yi Zhang, Gregory Epiphaniou, Carsten Maple

Organizations: WMG University of Warwick Coventry, CV4 7AL · Wolfson School of Mechanical, Electrical and Manufacturing Engineering Loughborough University Loughborough, LE11 3TU · Faculty of Science and Engineering Swansea university Swansea, SA2 8PP

Abstract

Deep reinforcement learning (DRL) enables adaptive intrusion detection in dynamic network environments but also exposes intrusion detection systems (IDS) to adversarial threats such as universal adversarial perturbations (UAPs), which apply a single input-agnostic perturbation to degrade detection performance across traffic. Probabilistic Robustness (PR), as a post-hoc evaluation metric, provides a principled, population-level measure of adversarial impact that conceptually aligns with the universality objective of UAPs, i.e., PR quantifies the prevalence of misclassification in the input space, making it a natural signal for guiding UAP generation. Hence, we propose PR-based UAP, which represents the first integration of an explicit PR-driven objective into generating UAPs against DRL-based IDS. Building on this formulation, we introduce PX-UAP, which leverages explainable artificial intelligence (XAI) to guide perturbation shaping under realistic domain constraints, and provides a rigorous theoretical analysis of its design. Extensive experiments demonstrate that PX-UAP consistently outperforms state-of-the-art UAP methods in attack effectiveness.

Figures & tables

Figure 1: (a): Adversarial scenario for a network-flow-based IDS. (b): Comparison between worst-case robustness (i) and probabilistic robustness (ii), adapted from [ 15 ] .

Figure 2: (a) : Illustration of mapping between PR-based variable $k$ and Misclassification-Guaranteed Region in the UAP setting. Two examples $\delta_{1}$ and $\delta_{2}$ are provided in (i) and (ii), respectively. UAP by $\delta_{1}$ (larger k) induces a larger size Misclassification-Guaranteed Region $\mathcal{B}(x+\delta_{1},k_{1})$ , while smaller size $\mathcal{B}(x+\delta_{2},k_{2})$ by $\delta_{2}$ . (b) : Main Procedures of the PR-based UAP: (1) Attack initialization. (2) AE candidate selection. (3) $k$ -driven optimization.

Index	Baseline UAP	Loss function to maximize
(1)	PD_mean_UAP [ 34 ]	$\log\!\left(\prod_{i=1}^{k}\mathrm{mean}(l_{i}+\mathrm{eps})\right)$ $\text{where }l_{i}=\mathrm{activation}(x+\delta),\ \mathrm{eps}=1\times 10^{-8}$
(2)	PD_L2_UAP [ 35 ]	$\log\!\left(\prod_{i=1}^{k}\left\lVert l_{i}\right\rVert_{2}+\mathrm{eps}\right)$ $\text{where }l_{i}=\mathrm{activation}(x+\delta),\ \mathrm{eps}=1\times 10^{-8}$
(3)	COSSIM_L3_UAP [ 36 ]	$-\mathrm{cossim}\!\left(\mathrm{activation}_{l_{i}}(x+\delta),\ \mathrm{activation}_{l_{i}}(x)\right)$ , $\text{where }l_{i}=l_{3}$
(4)	COSSIM_L4_UAP [ 36 ]	$-\mathrm{cossim}\!\left(\mathrm{activation}_{l_{i}}(x+\delta),\ \mathrm{activation}_{l_{i}}(x)\right)$ , $\text{where }l_{i}=l_{4}$
(5)	PCC_UAP [ 10 ]	$\mathrm{PCCsim}\!\left(\mathrm{activation}_{l_{i}}(x+\delta),\ \mathrm{activation}_{l_{i}}(\delta)\right)$ , $\text{where }l_{i}=l_{4}$

Table 1: UAP baseline with loss functions for comparison with PR-based UAP.

Figure 3: (a): False Negative Rate (FNR) and (b): Accuracy of the proposed PR-based UAP and other UAP baselines. (c): FNR and (d): ACC of PX-UAP with different $\mathrm{Topn}$ values. (e): FNR and (f): ACC for ablation study.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Feature Name	Description	Recalculate Formulation	Group
Tot Fwd Pkts	Total packets in the forward direction	–	Modified Features
Tot Bwd Pkts	Total packets in the backward direction	–
TotLen Fwd Pkts	Total size of packet in forward direction	–
TotLen Bwd Pkts	Total size of packet in backward direction	–
Flow Duration	Duration of the flow in Microsecond	–
Fwd Pkts/s	Number of forward packets per second	${\text{Tot Fwd Pkts}\times 10^{6}}\;/\;{\text{Flow Duration}}$

Appendix

Table 2: Feature categorization, descriptions, and formulations

Feature Name	Feature Description	Feature Value
$\epsilon$	Adversarial Budget	$[0.04,\,0.15]$
$size$	Seedset size	$1\times 10^{-4}$
$max\_iter$	Max Iteration Number	20
$N$	PGD Attack Number	10
$n_{\mathrm{step}}$	PGD Iteration Step	50
$\alpha$	PGD Step Size	$\epsilon/n_{\mathrm{step}}$

Appendix

Table 3: Hyperparameter Setting for algorithms

Index Name	AE Initia.	K-Sel.	Optz.
(1): FUAP	FGM(1)	$\times$	$\times$
(2): KFUAP	FGM(N)	$\surd$	$\times$
(3): PUAP	PGD(1)	$\times$	$\times$
(4): KPUAP	PGD(N)	$\surd$	$\times$
(5): PR-based UAP	PGD(N)	$\surd$	$\surd$

Appendix

Table 4: Deatils of PR-based UAP attack and ablation variants.

Figure 4: F1 score of the proposed PR-based UAP and other UAP baselines

Figure 5: F1 score of (a): PX-UAP with different $\mathrm{Topn}$ values. (b): The ablation study

Figure 6: FNR of proposed PR-based UAP on DRL-based IDS (DQN-4-64) on (a): NSL-KDD over $\epsilon\in[0,0.45]$ , (b) : UNSW-NB15 over $\epsilon\in[0,0.15]$ .

Figure 7: FNR of proposed PR-based UAP on different DRL-based IDS on CICIDS2018 dateset with the taget model of (a): DQN-6-128, (b): PPO-4-64, (c): A2C-4-64.

Figure 8: FNR for Cross-model Transferability of AE (PR-based UAP) among different DRL-based IDS model on CICIDS2018 at $\epsilon=0.15$

Figure 9: FNR of (a): PR-based UAP (b): PCC_UAP (c): UAP with different stepsize on CICIDS2018 dateset. The applied stepsize are show in parentheses.

Figure 10: FNR of best-step-size comparison for PR-based UAP, PCC_UAP and UAP attack.

Adversarial Attacks

Adversarial attacks exploit crafted inputs, poisoned data, or system interactions to manipulate AI predictions, behavior, privacy, or reliability.

See all 451 papers

Probabilistic Adversarial Training
Sep 30, 2026Andi Zhang, Xingyu Zhao, Siddartha KhastgirAdversarial Training Adversarial Robustness
WMG, University of Warwick · Wuhan University
Robust Policy Optimization via Adversarial Importance Sampling
Sep 11, 2026Amine Andam, Jamal Bentahar, Mustapha HedabouRobust RL Policy Learning
Mohammed VI Polytechnic University · Khalifa University · Concordia University
DeepXplain: XAI-Guided Autonomous Defense Against Multi-Stage APT Campaigns
Mar 22, 2026Trung V. Phan, Thomas BauschertAutonomous Cyber Defense Reinforcement Learning
Chair of Communication Networks, Technische Universität Chemnitz, 09126 Chemnitz, Germany

Sep 30, 2026J/K move · Enter open · S save

Probabilistic Adversarial Training

Andi Zhang, Xingyu Zhao, Siddartha Khastgir

WMG, University of Warwick · Wuhan University

Adversarial Training Adversarial Robustness

Open paper AskPDF arXiv

Building on a probabilistic perspective in which adversarial examples arise from the overlap between a distance-based distribution

p_{\mathrm{dis}}

and a victim-classifier-induced distribution

p_{\mathrm{vic}}

, we start from a simple intuition: adversarial examples become harder to generate when these two distributions are pushed apart, as their overlap becomes smaller, thereby increasing robustness. This intuition naturally motivates a KL-based robustness objective. We then prove that

\mathrm{KL}(p_{\mathrm{dis}}\|p_{\mathrm{vic}})-\log Z_{\mathrm{vic}}

is a lower bound on probabilistic robustness (PR), where

Z_{\mathrm{vic}}

denotes the normalizing constant of

p_{\mathrm{vic}}

. Since PR is generally intractable to compute directly, maximizing this KL-based lower bound provides a tractable surrogate objective for improving PR. We further show that this objective recovers a scaled form of adversarial training, offering a probabilistic interpretation of adversarial training and a principled route to robustness improvement. We call the resulting method probabilistic adversarial training. Experiments show that it consistently improves PR, and ablation studies demonstrate that the induced scaling factor can even enhance the PR of non-probabilistic adversarial training methods.

Abstract

Adversarial Attacks

Explore similar work

Probabilistic Adversarial Training

Robust Policy Optimization via Adversarial Importance Sampling

DeepXplain: XAI-Guided Autonomous Defense Against Multi-Stage APT Campaigns