Enhancing Network Intrusion Detection Systems: A Multi-Layer Ensemble Approach to Mitigate Adversarial Attacks
Authors: Nasim Soltani, Shayan Nejadshamsi, Zakaria Abou El Houda, Raphael Khoury, Kelton A. P. Costa, Tiago H. Falk, Anderson R. Avila
Organizations: Institut national de la recherche scientifique (INRS-EMT), Université du Québec, Montréal, Québec, Canada · INRS-UQO Mixed Research Unit on Cybersecurity, Gatineau, Québec, Canada · Concordia Institute for Information Systems Engineering (CIISE), Concordia University, Montréal, Québec, Canada · Department of Computer Science and Engineering, Université du Québec en Outaouais, Gatineau, Québec, Canada · Department of Computing, São Paulo State University, São Paulo, Brazil
Adversarial examples can represent a serious threat to machine learning (ML) algorithms. If used to manipulate the behaviour of ML-based Network Intrusion Detection Systems (NIDS), they can jeopardize network security. In this work, we aim to mitigate such risks by increasing the robustness of NIDS towards adversarial attacks. To that end, we explore two adversarial methods for generating malicious network traffic. The first method is based on Generative Adversarial Networks (GAN) and the second one is the Fast Gradient Sign Method (FGSM). The adversarial examples generated by these methods are then used to evaluate a novel multilayer defense mechanism, specifically designed to mitigate the vulnerability of ML-based NIDS. Our solution consists of one layer of stacking classifiers and a second layer based on an autoencoder. If the incoming network data are classified as benign by the first layer, the second layer is activated to ensure that the decision made by the stacking classifier is correct. We also incorporated adversarial training to further improve the robustness of our solution. Experiments on two datasets, namely UNSW-NB15 and NSL-KDD, demonstrate that the proposed approach increases resilience to adversarial attacks.
Figures & tables
Fig. 1 : Under a small perturbation, ϵ , input x becomes an adversarial example that is misclassified into f(x+ϵ) , which falls on the wrong side of the decision boundary.
Fig. 2 : The Generative Adversarial Network against NIDS is composed of a generator G , a discriminator D and a voting classifier. While G is trained to produce mutable adversarial vectors, D is optimized to identify malicious generated network traffic in collaboration with the voting classifier.
TABLE I : Performance on the NSL-KDD dataset for unmodified, GAN and FGSM conditions (p-value < 0.01).
Unmodified
GAN
FGSM
All
Precision
Recall
F1
Precision
Recall
F1
Precision
Recall
F1
Precision
Recall
F1
DT
61.18
58.32
57.69
65.87
62.87
63.18
67.90
64.88
65.18
62.98
55.29
54.89
GB
76.52
76.15
75.75
74.33
74.58
74.36
75.03
75.28
75.02
74.43
74.68
74.51
BGG
59.26
56.66
56.05
60.01
54.52
54.18
61.81
57.25
57.29
69.73
68.02
68.42
KNN
82.99
81.88
81.47
82.69
82.40
81.99
84.03
83.47
83.01
83.53
83.30
82.86
LR
83.40
77.76
76.12
73.73
73.55
72.38
73.96
73.73
72.54
71.72
72.23
71.28
TABLE II : Performance on the UNSW-NB15 dataset for unmodified, GAN and FGSM conditions (p-value < 0.01).
Dataset
Models
Acc
NSL-KDD
BAT-MC [ 20 ]
84.15
Autoencoder [ 21 ]
84.21
Ours
87.35
UNSW-NB15
Scaling SVM [ 22 ]
75.77
DT (C5) [ 23 ]
75.53
Ours
77.71
TABLE III : Comparison with deep neural network models (accuracy).
Dataset
OD
AT
SC
AE
DR
NSL-KDD
✓
✗
✓
✗
72.42
✓
✗
✓
✓
77.80
✓
✓
✓
✗
90.29
✓
✓
✓
✓
92.99
UNSW-NB15
✓
✗
✓
✗
86.05
✓
✗
✓
✓
86.23
TABLE IV : Ablation study considering original data (OD), adversarial (AT) training, the stacking classifier (SC) and the latter with anomaly detection including the autoencoder (AE).
Fig. 4 : Detection rate comparison of ML classifiers on the NSL-KDD and UNSW-NB15 datasets under unmodified, GAN and FGSM conditions.
The new wave of adversarial attacks that utilize gradient-related vulnerabilities in neural network-based classifiers makes Network Intrusion Detection Systems more open to such threats. Although state-of-the-art adversarial training methods have shown promising results in producing more robust classifiers, their interpretability and defense ability are limited due to their lack of understanding of how adversarial attacks propagate in different layers of network classifiers. In this paper, we present an insightful approach, called LARAR (Layer-wise Adversarial Robustness using Adaptive Regularization), that incorporates additional layer-wise vulnerability analysis and adaptive weighting in conventional adversarial training methods. Additionally, we utilize 'Auxiliary Classifiers' in our approach. LARAR provides interpretable layer-wise vulnerability scores, achieves a clean accuracy of 95.01%, and provides better robustness against adversarial attacks (FGSM, PGD, and transfer attacks) on the UNSW-NB15 dataset. Through the identification of vulnerable layers, the proposed framework reduces computational complexity and enables the early detection of adversarial samples, thus enhancing the effectiveness and interpretability of adversarial defense mechanisms in NIDS.
Hira Nasir, Eiman Javed, Balawal Shabir +2
National Cyber Security Academy (NCSA), Air University, Islamabad, Pakistan. · School of Science, Edith Cowan University, Australia.
It is crucial to safeguard computer networks from evolving network security threats and unknown cyberattacks. An essential tool for protecting computer networks against unknown cyber threats is Network Intrusion Detection System (NIDS). However, NIDS faces a major security concern due to its susceptibility to adversarial attacks. Adversarial attacks aim to deceive NIDS by crafting and injecting adversarial examples into the system. These adversarial inputs can deceive the NIDS into misclassifying benign network traffic as malicious. We developed a resilient hybrid defense mechanism aimed to mitigate the impact of two potent adversarial attacks: Fast Gradient Sign Method (FGSM) and Carlini & Wagner (C&W) attack. Our hybrid defense approach leverages the combined strength of two heuristic defense methods: Adversarial Training (AT) and Gaussian Data Augmentation (GDA). GDA provides multi-directional defense, while AT enhances NIDS robustness against specific adversarial vectors. Under pre-attack scenarios, NIDS demonstrated good accuracy and f1-score. However, in the post-attack scenario, its accuracy significantly dropped under FGSM and C&W attacks (0.2649 and 0.4961, respectively). Our proposed hybrid defense method effectively mitigated these adversarial threats, with post-defense accuracy of 96.57% and 89.20% for FGSM and C&W attacks. We evaluated the defense strategy across a range of epsilon and confidence noise factor values (ranging from 0.0001 to 0.0009). This research provides a good direction for future researchers in the emerging area of adversarial machine learning from a security perspective.
Khushnaseeb Roshan
Department of Computer Science, Aligarh Muslim University (AMU), Aligarh, INDIA
Deep learning (DL)-based Network Intrusion Detection System (NIDS) has demonstrated great promise in detecting malicious network traffic. However, they face significant security risks due to their vulnerability to adversarial examples (AEs). Most existing adversarial attacks maliciously perturb data to maximize misclassification errors. Among AEs, natural adversarial examples (NAEs) are particularly difficult to detect because they closely resemble real data, making them challenging for both humans and machine learning models to distinguish from legitimate inputs. Creating NAEs is crucial for testing and strengthening NIDS defenses. This paper proposes NetDiffuser1, a novel framework for generating NAEs capable of deceiving NIDS. NetDiffuser consists of two novel components. First, a new feature categorization algorithm is designed to identify relatively independent features in network traffic. Perturbing these features minimizes changes while preserving network flow validity. The second component is a novel application of diffusion models to inject semantically consistent perturbations for generating NAEs. NetDiffuser performance was extensively evaluated using three benchmark NIDS datasets across various model architectures and state-of-the-art adversarial detectors. Our experimental results show that NetDiffuser achieves up to a 29.93% higher attack success rate and reduces AE detection performance by at least 0.267 (in some cases up to 0.534) in the Area under the Receiver Operating Characteristic Curve (AUC-ROC) score compared to the baseline attacks.
Pratyay Kumar, Abu Saleh Md Tayeen, Satyajayant Misra +4
Department of Computer Science, New Mexico State University, Las Cruces, NM, USA · University of Hartford, CT, USA · DEVCOM Analysis Center, WSMR, NM, USA