Enhancing Network Intrusion Detection Systems: A Multi-Layer Ensemble Approach to Mitigate Adversarial Attacks
Authors: Nasim Soltani, Shayan Nejadshamsi, Zakaria Abou El Houda, Raphael Khoury, Kelton A. P. Costa, Tiago H. Falk, Anderson R. Avila
Organizations: Institut national de la recherche scientifique (INRS-EMT), Université du Québec, Montréal, Québec, Canada · INRS-UQO Mixed Research Unit on Cybersecurity, Gatineau, Québec, Canada · Concordia Institute for Information Systems Engineering (CIISE), Concordia University, Montréal, Québec, Canada · Department of Computer Science and Engineering, Université du Québec en Outaouais, Gatineau, Québec, Canada · Department of Computing, São Paulo State University, São Paulo, Brazil
Adversarial examples can represent a serious threat to machine learning (ML) algorithms. If used to manipulate the behaviour of ML-based Network Intrusion Detection Systems (NIDS), they can jeopardize network security. In this work, we aim to mitigate such risks by increasing the robustness of NIDS towards adversarial attacks. To that end, we explore two adversarial methods for generating malicious network traffic. The first method is based on Generative Adversarial Networks (GAN) and the second one is the Fast Gradient Sign Method (FGSM). The adversarial examples generated by these methods are then used to evaluate a novel multilayer defense mechanism, specifically designed to mitigate the vulnerability of ML-based NIDS. Our solution consists of one layer of stacking classifiers and a second layer based on an autoencoder. If the incoming network data are classified as benign by the first layer, the second layer is activated to ensure that the decision made by the stacking classifier is correct. We also incorporated adversarial training to further improve the robustness of our solution. Experiments on two datasets, namely UNSW-NB15 and NSL-KDD, demonstrate that the proposed approach increases resilience to adversarial attacks.
Figures & tables
Fig. 1 : Under a small perturbation, ϵ , input x becomes an adversarial example that is misclassified into f(x+ϵ) , which falls on the wrong side of the decision boundary.
Fig. 2 : The Generative Adversarial Network against NIDS is composed of a generator G , a discriminator D and a voting classifier. While G is trained to produce mutable adversarial vectors, D is optimized to identify malicious generated network traffic in collaboration with the voting classifier.
TABLE I : Performance on the NSL-KDD dataset for unmodified, GAN and FGSM conditions (p-value < 0.01).
Unmodified
GAN
FGSM
All
Precision
Recall
F1
Precision
Recall
F1
Precision
Recall
F1
Precision
Recall
F1
DT
61.18
58.32
57.69
65.87
62.87
63.18
67.90
64.88
65.18
62.98
55.29
54.89
GB
76.52
76.15
75.75
74.33
74.58
74.36
75.03
75.28
75.02
74.43
74.68
74.51
BGG
59.26
56.66
56.05
60.01
54.52
54.18
61.81
57.25
57.29
69.73
68.02
68.42
KNN
82.99
81.88
81.47
82.69
82.40
81.99
84.03
83.47
83.01
83.53
83.30
82.86
LR
83.40
77.76
76.12
73.73
73.55
72.38
73.96
73.73
72.54
71.72
72.23
71.28
TABLE II : Performance on the UNSW-NB15 dataset for unmodified, GAN and FGSM conditions (p-value < 0.01).
Dataset
Models
Acc
NSL-KDD
BAT-MC [ 20 ]
84.15
Autoencoder [ 21 ]
84.21
Ours
87.35
UNSW-NB15
Scaling SVM [ 22 ]
75.77
DT (C5) [ 23 ]
75.53
Ours
77.71
TABLE III : Comparison with deep neural network models (accuracy).
Dataset
OD
AT
SC
AE
DR
NSL-KDD
✓
✗
✓
✗
72.42
✓
✗
✓
✓
77.80
✓
✓
✓
✗
90.29
✓
✓
✓
✓
92.99
UNSW-NB15
✓
✗
✓
✗
86.05
✓
✗
✓
✓
86.23
TABLE IV : Ablation study considering original data (OD), adversarial (AT) training, the stacking classifier (SC) and the latter with anomaly detection including the autoencoder (AE).
Fig. 4 : Detection rate comparison of ML classifiers on the NSL-KDD and UNSW-NB15 datasets under unmodified, GAN and FGSM conditions.
Department of Computer Science, New Mexico State University, Las Cruces, NM, USA · University of Hartford, CT, USA · DEVCOM Analysis Center, WSMR, NM, USA