Machine Learning for Network Attacks Classification and Statistical Evaluation of Adversarial Learning Methodologies for Synthetic Data Generation

Iakovos-Christos Zarkadis; Christos Douligeris

cs.CRMar 18, 2026

Machine Learning for Network Attacks Classification and Statistical Evaluation of Adversarial Learning Methodologies for Synthetic Data Generation

Authors: Iakovos-Christos Zarkadis, Christos Douligeris

Organizations: University of Piraeus, Athens, Greece · Dept. of Informatics, University of Piraeus, Piraeus, Greece

Abstract

Supervised detection of network attacks has always been a critical part of network intrusion detection systems (NIDS). Nowadays, in a pivotal time for artificial intelligence (AI), with even more sophisticated attacks that utilize advanced techniques, such as generative artificial intelligence (GenAI) and reinforcement learning, it has become a vital component if we wish to protect our personal data, which are scattered across the web. In this paper, we address two tasks, in the first unified multi-modal NIDS dataset, which incorporates flow-level data, packet payload information and temporal contextual features, from the reprocessed CIC-IDS-2017, CIC-IoT-2023, UNSW-NB15 and CIC-DDoS-2019, with the same feature space. In the first task we use machine learning (ML) algorithms, with stratified cross validation, in order to prevent network attacks, with stability and reliability. In the second task we use adversarial learning algorithms to generate synthetic data, compare them with the real ones and evaluate their fidelity, utility and privacy using the SDV framework, f-divergences, distinguishability and non-parametric statistical tests. The findings provide stable ML models for intrusion detection and generative models with high fidelity and utility, by combining the Synthetic Data Vault framework, the TRTS and TSTR tests, with non-parametric statistical tests and f-divergence measures.

Figures & tables

Model	Train/Test-Precision	Train/Test-Recall
LightGBM	97.74/96.36	97.69/96.25
AutoGluon	92.34/93.53	91.32/92.30
XGBoost	98.67/96.40	98.64/96.29
XGBoost-RF	96.04/95.09	95.99/94.74
Random Forest	96.64/95.88	96.56/95.63
HB-Gradient Boosting	94.74/94.74	93.91/93.91

TABLE I: Train/Test Precision and Recall Results (

\%

)

Model	Train/Test-F1	Train/Test-Roc-Auc
LightGBM	97.69/96.25	98.81/98.08
AutoGluon	91.12/92.31	95.56/9606
XGBoost	98.65/96.33	99.30/98.11
XGBoost-RF	95.99/94.86	98.95/97.32
Random Forest	95.56/95.49	96.56/95.63
HB-Gradient Boosting	94.66/94.55	97.28/97.18

TABLE II: Train/Test F1 and Roc-Auc Results (

\%

)

Fig. 1: XGBoost Precision-Recall Stratified 10-Fold Cross Validation.

Divergence	$f(u)$	$f^{*}(t)$
Kullback-Leibler	$ulog(u)$	$\exp\{t-1\}$
Squared-Hellinger	$\big(\sqrt{u}-1\big)^{2}$	$\frac{t}{1-t}$

TABLE III:

f

-GAN generator functions and their conjugates

Model	Quality Report ( $\%$ )	Diagnostic Report ( $\%$ )
Vanilla-GAN	60.43	78.37
WGAN	57.07	81.24
WGAN-GP	63.55	78.72
KLD-GAN	63.55	78.72
H2D-GAN	49.85	80.22
CTGAN-1	75.82	83.79

TABLE IV: SDV Overall Scores

Model	Test-F1	Test-Roc-Auc
CTGAN-2	54.24	64.24
XGB-DF	53.39	65.07
DistilGPT2	52.80	63.81
PATE-CTGAN	100.00	100.00

TABLE V: Random Forest - Distinguishability Test Results (

\%

)

Model	Test-F1	Test-Roc-Auc
CTGAN-2	0.00	50.85
XGB-DF	0.13	52.80
DistilGPT2	0.20	53.70
PATE-CTGAN	14.65	63.18

TABLE VI: Logistic Regression - Distinguishability Test Results (

\%

)

Model	Precision	Recall
CTGAN-2	96.59	96.51
XGB-DF	96.67	96.59
DistilGPT2	96.67	96.59
PATE-CTGAN	00.54	0.02

TABLE VII: TRTS Results (

\%

)

Model	Train/Test Precision	Train/Test Recall
CTGAN-2	96.60/95.65	96.51/95.38
XGB-DF	96.76/95.56	96.67/95.26
DistilGPT2	96.77/95.56	96.67/95.26
PATE-CTGAN	75.22/00.18	11.27/3.60

TABLE VIII: TSTR Results (

\%

)

Divergence	Jensen-Shannon	Hellinger	Wasserstein
CTGAN-2	0.000967	0.031219	0.129161
XGB-DF	3.541914e-10	1.881997e-5	0.000106
DistilGPT2	3.541914e-10	1.881997e-5	0.000106
PATE-CTGAN	0.004467	0.066443	0.729779

TABLE IX: Divergence-Based Evaluation Metrics

Model	p-values
CTGAN-2	0.99
XGB-DF	1.00
DistilGPT2	1.00
PATE-CTGAN	0.99

TABLE X: Regularized Hotelling’s

T^{2}

Model	p-values
CTGAN-2	0.0019
XGB-DF	0.9381
DistilGPT2	0.9261
PATE-CTGAN	0.14

TABLE XI: Frobenius Norm Covariance Test Results

Model	p-values
CTGAN-2	0.0829
XGB-DF	1.0000
DistilGPT2	1.0000
PATE-CTGAN	0.0000

TABLE XII: MMD Test Results

Model	Train-NNDR	Test-NNDR
CTGAN-2	0.6136	0.7171
XGB-DF	0.6122	0.7194
DistilGPT2	0.6140	0.7193
PATE-CTGAN	0.9634	0.9731

TABLE XIII: NNDR Results

Explore similar work

Enhancing Network Intrusion Detection Systems: A Multi-Layer Ensemble Approach to Mitigate Adversarial Attacks
Mar 11, 2026Nasim Soltani, Shayan Nejadshamsi, Zakaria Abou El Houda +4Network Intrusion Detection Adversarial Robustness
Institut national de la recherche scientifique (INRS-EMT), Université du Québec, Montréal, Québec, Canada · INRS-UQO Mixed Research Unit on Cybersecurity, Gatineau, Québec, Canada · Concordia Institute for Information Systems Engineering (CIISE), Concordia University, Montréal, Québec, Canada +2
Generative AI and Federated Learning for Intrusion Detection Systems: A Survey
Jul 1, 2026Jiefei Liu, Abu Saleh Md Tayeen, Pratyay Kumar +5Network Intrusion Detection Synthetic Anomaly Generation
Department of Computer Science, New Mexico State University, Las Cruces, NM, USA · University of Hartford, CT, USA · DEVCOM Analysis Center, WSMR, NM, USA
Synthetic Network Packet Generation through Statistical Learning and Genetic Algorithms
Jun 18, 2026Mayank Raj, Nathaniel D. Bastian, Lance Fiondella +1Synthetic Data Generation Synthetic Tabular Data Generation
University of Massachusetts Dartmouth Dartmouth, MA USA · United States Military Academy West Point, NY USA

Mar 11, 2026J/K move · Enter open · S save

Enhancing Network Intrusion Detection Systems: A Multi-Layer Ensemble Approach to Mitigate Adversarial Attacks

Nasim Soltani, Shayan Nejadshamsi, Zakaria Abou El Houda, Raphael Khoury+3

Institut national de la recherche scientifique (INRS-EMT), Université du Québec, Montréal, Québec, Canada · INRS-UQO Mixed Research Unit on Cybersecurity, Gatineau, Québec, Canada · Concordia Institute for Information Systems Engineering (CIISE), Concordia University, Montréal, Québec, Canada+2

Network Intrusion Detection Adversarial Robustness Adversarial Attacks

Open paper AskPDF arXiv

Adversarial examples can represent a serious threat to machine learning (ML) algorithms. If used to manipulate the behaviour of ML-based Network Intrusion Detection Systems (NIDS), they can jeopardize network security. In this work, we aim to mitigate such risks by increasing the robustness of NIDS towards adversarial attacks. To that end, we explore two adversarial methods for generating malicious network traffic. The first method is based on Generative Adversarial Networks (GAN) and the second one is the Fast Gradient Sign Method (FGSM). The adversarial examples generated by these methods are then used to evaluate a novel multilayer defense mechanism, specifically designed to mitigate the vulnerability of ML-based NIDS. Our solution consists of one layer of stacking classifiers and a second layer based on an autoencoder. If the incoming network data are classified as benign by the first layer, the second layer is activated to ensure that the decision made by the stacking classifier is correct. We also incorporated adversarial training to further improve the robustness of our solution. Experiments on two datasets, namely UNSW-NB15 and NSL-KDD, demonstrate that the proposed approach increases resilience to adversarial attacks.

Abstract

Explore similar work

Enhancing Network Intrusion Detection Systems: A Multi-Layer Ensemble Approach to Mitigate Adversarial Attacks

Generative AI and Federated Learning for Intrusion Detection Systems: A Survey

Synthetic Network Packet Generation through Statistical Learning and Genetic Algorithms