Targeted Adversarial Traffic Generation : Black-box Approach to Evade Intrusion Detection Systems in IoT Networks

Islam Debicha; Tayeb Kenaza; Ishak Charfi; Salah Mosbah; Mehdi Sehaki; Jean-Michel Dricot

cs.CRMar 24, 2026

Targeted Adversarial Traffic Generation : Black-box Approach to Evade Intrusion Detection Systems in IoT Networks

Authors: Islam Debicha, Tayeb Kenaza, Ishak Charfi, Salah Mosbah, Mehdi Sehaki, Jean-Michel Dricot

Abstract

The integration of machine learning (ML) algorithms into Internet of Things (IoT) applications has introduced significant advantages alongside vulnerabilities to adversarial attacks, especially within IoT-based intrusion detection systems (IDS). While theoretical adversarial attacks have been extensively studied, practical implementation constraints have often been overlooked. This research addresses this gap by evaluating the feasibility of evasion attacks on IoT network-based IDSs, employing a novel black-box adversarial attack. Our study aims to bridge theoretical vulnerabilities with real-world applicability, enhancing understanding and defense against sophisticated threats in modern IoT ecosystems. Additionally, we propose a defense scheme tailored to mitigate the impact of evasion attacks, thereby reinforcing the resilience of ML-based IDSs. Our findings demonstrate successful evasion attacks against IDSs, underscoring their susceptibility to advanced techniques. In contrast, we proposed a defense mechanism that exhibits robust performance by effectively detecting the majority of adversarial traffic, showcasing promising outcomes compared to current state-of-the-art defenses. By addressing these critical cybersecurity challenges, our research contributes to advancing IoT security and provides insights for developing more resilient IDS.

Figures & tables

Study	Contribution	Limitations
Islam et al. Islam et al. (2021)	Feature selection for lightweight IDS	Limited to specific IoT protocols
Sarhan et al. Sarhan et al. (2024)	Novel framework for IoT-specific challenges	Requires large labeled datasets
Altulaihan et al. Altulaihan et al. (2024)	Real-time anomaly detection	Susceptible to adversarial attacks
Alslman et al. Alslman et al. (2024)	Robust IDS using adversarial training	High resource requirements
Yuan et al. Yuan et al. (2024)	Simple yet effective adversarial defenses	Limited evaluation on IoT datasets
Barik et al. Barik et al. (2024)	Adversarial robustness for tabular data	Not IoT-specific

Table 1: Summary of Recent Works on ML-based IDS

Figure 1: Schematic illustration of the threat scenario, showing how the attacker replicates a substitute IoT environment, builds a substitution model, and leverages transferability to craft adversarial examples that evade the defender’s IoT-based NIDS.

Features	Description
Dur	Duration of the flow
spkts	Number of packets from source to destination
sbytes	Number of bytes from source to destination

Table 2: Description of Common Modifiable Features

Features	Description	Dependency
Pkts	Total number of packets	Spkts, Dpkts
Bytes	Total number of bytes	Sbytes, Dbytes
Rate	Total number of packets per second	Dur, Pkts
Srate	Packets from source to destination per second	Dur, Spkts
Drate	Packets from destination to source per second	Dur, Dpkts

Table 3: Description of Common Dependent Features

Features	Description
Dpkts	Number of packets from destination to source
Dbytes	Number of bytes from destination to source
Proto	Textual representation of protocols
Sport	Source port number
Dport	Destination port number

Table 4: Description of Common Non-Modifiable Features

Figure 2: Conceptual illustration of the D2TC approach, where malicious instances are iteratively shifted toward the benign mean and projected into valid ranges. Once inside the benign region, the instance is misclassified as benign, forming an adversarial example

Figure 3: Adversarial instance generation process. The attacker’s IoT dataset is split to build a substitution model that approximates the defender’s IDS. Iterative perturbations (and mask switching) are then applied to malicious flows until successful evasion is achieved, producing adversarial traffic ready to bypass the target system.

Mask ID	Mask Value	Manipulated Features
1	001	$Dur$
2	010	$Spkts$
3	100	$Sbytes$
4	011	$Dur$ , $Spkts$
5	101	$Dur$ , $Sbytes$
6	110	$Sbytes$ , $Spkts$

Table 5: Binary masks for modifiable network features ( Dur , Spkts , Sbytes ), indicating which combination of features is perturbed for each mask ID

Figure 4: Workflow for creating the defense dataset. The original dataset is split into benign and malicious flows, with malicious instances undergoing adversarial instance generation (D2TC) to produce adversarial samples. These adversarial and clean subsets are then combined to form a comprehensive defense dataset.

Figure 5: Partitioning of the defense dataset into training/evaluation (derived from the defender’s data) and testing (sourced from the attacker’s data)

Figure 6: Overview of the proposed defense process. Sub-detectors are trained and evaluated in parallel on labeled clean and adversarial data, with each sub-detector assigned a reliability weight based on its detection performance. The final, ensemble-based decision filter is placed before the NIDS to intercept and mitigate adversarial traffic

Figure 7: Overview of the proposed adversarial detector architecture. Each incoming instance is split into feature-specific sub-detectors, whose decisions are weighted by reliability scores and fused to determine whether the traffic is benign or adversarial

Figure 8: Partitioning of the BoT-IoT and ToN-IoT datasets into attacker and defender subsets, each further split into training (75%) and test (25%) sets

Port Range	Category
0 to 1023	Well-Known Ports
1024 to 49151	Registered Ports
49152 to 65535	Dynamic/Private Ports

Table 6: Port range categorization for the Sport and Dport features prior to one-hot encoding

Figure 9: Machine Learning pipeline for both attacker and defender

Figure 10: Initial performance of the trained models (KNN, RF, DT, XGBoost) on both the attacker and defender sides for the Bot-IoT dataset. All models exhibit high metric scores (precision, recall, F1-score), with minimal variations across different hyperparameters.

Figure 11: Initial performance of the trained models (KNN, RF, DT, XGBoost) on both the attacker and defender sides for the Ton-IoT dataset. Similar to the Bot-IoT results, each model achieves consistently high metric scores, demonstrating negligible differences despite variations in hyperparameters.

Figure 12: Impact of adversarial perturbations on IDS detection rates for the Bot-IoT dataset. The bars compare detection rates of KNN, RF, DT, and XGBoost across four attack types (DDOS, DOS, Service_scan, and OS_fingerprinting) before and after adversarial perturbations. A marked drop in detection rate indicates the effectiveness of the adversarial modifications in evading each IDS model.

Figure 13: Impact of adversarial perturbations on IDS detection rates for the Ton-IoT dataset. Similar to Figure 12 , this figure highlights how the detection performance of KNN, RF, DT, and XGBoost significantly declines once adversarial samples are introduced, demonstrating the robustness of our attack across multiple attack types (DDOS, DOS, Scanning, XSS, Password, Injection, and Backdoor).

Figure 14: Percentage usage of different mask combinations for each attack in the Bot-IoT dataset. Each bar represents how frequently a particular set of features (e.g., dur , spkts , or sbytes ) is manipulated to create adversarial instances for a specific attack. Higher percentages indicate that those features play a more critical role in evading detection for that attack.

Figure 15: Proportion of mask use for every attack type in the Ton-IoT dataset. Similar to Figure 14 , this figure reveals which feature manipulations ( dur , spkts , sbytes , or their combinations) are most frequently employed to generate successful adversarial samples for each attack category.

Figure 16: Number of steps required to generate successful adversarial instances across different attacks. The bars represent the percentage of instances requiring each step count (1 to 10) to achieve evasion. A large proportion of attacks become adversarial within the first step, indicating that only minimal perturbations are necessary to fool the IDS.

Attack Type	KNN	RF	DT	XGBoost
DDOS	0.05	0.11	0.01	0.02
DOS	0.02	1.40	1.29	1.53
Service Scan	0.07	0.22	0.01	0.02
OS Fingerprinting	0.07	0.27	0.01	0.02

Table 7: Average generation time (second) of adversarial instances - Bot-IoT

Attack Type	KNN	RF	DT	XGBoost
DDOS	1.47	0.40	0.01	0.09
DOS	0.08	0.21	0.01	0.74
Scanning	0.06	0.24	0.01	0.12
XSS	1.02	1.62	0.03	0.85
Password	0.43	1.85	0.02	0.40
Injection	1.65	1.88	0.08	1.25

Table 8: Average generation time (second) of adversarial instances - Ton-IoT

Defender models	Attacker’s substitute models				Average
Defender models	KNN	RF	DT	XGBoost
KNN	49%	47%	50%	51%	49%
RF	56%	49%	53%	52%	53%
DT	55%	50%	52%	52%	52%
XGBoost	69%	53%	69%	63%	64%

Table 9: Detection rates of the defender’s NIDS without adversarial defense across four attacker substitute models (KNN, RF, DT, XGBoost). The final column shows the average detection rate.

Defender models	Attacker’s substitute models				Average
Defender models	KNN	RF	DT	XGBoost
KNN	65%	61%	74%	71%	68%
RF	76%	63%	69%	58%	67%
DT	66%	39%	68%	68%	60%
XGBoost	79%	56%	74%	72%	70%

Table 10: Detection rates of the defender’s NIDS with our adversarial defense (Bayesian fusion) against four attacker substitute models. The final column shows the average detection rate.

Defender models	Attacker’s substitute models				Average
Defender models	KNN	RF	DT	XGBoost
KNN	65%	61%	75%	71%	68%
RF	76%	63%	69%	71%	70%
DT	66%	56%	76%	60%	63%
XGBoost	82%	66%	76%	73%	74%

Table 11: Detection rates of the defender’s NIDS with our adversarial defense (Dempster-Shafer fusion) against four attacker substitute models. The final column shows the average detection rate.

Figure 17: Defender’s NIDS performance under adversarial traffic, with and without our proposed defense (using Bayesian and Dempster-Shafer fusion).

Figure 18: Comparison of our Bayesian and Dempster-Shafer fusion-based defenses with a state-of-the-art adversarial detection method across various ML models

Explore similar work

Jun 30, 2026

Comparative Analysis of Machine Learning based Intrusion Detection in Realistic IoT Networks

The Internet of Things (IoT) is rapidly growing and expanding into various sectors, such as healthcare, transportation, smart homes, and more. Despite the benefits of using IoT devices, they present several challenges. Given the significant role these devices play in our lives, it is crucial to address issues related to their security and privacy. These devices are limited in resources, which complicates their security and the protection of the data that they manage. The paper aims to examine intrusion detection systems using the Gotham2025 dataset, generated through the Gotham testbed, which consists of 78 emulated IoT devices utilising various protocols, including MQTT, CoAP, and RTSP, to assist in safeguarding IoT networks from attacks. We conduct a comparative analysis between five machine learning algorithms, including Random Forest, XGBoost, Logistic Regression, Naive Bayes, and Deep Neural Network. We demonstrate that the Random Forest Classifier was the top-performing model, achieving an F1-score of 0.99 in classifying attacks.

Rana Alharbi, Chuadhry Mujeeb Ahmed

Newcastle University, UK

IoT Intrusion Detection Network Intrusion Detection Internet of Things

Ask PDF

Sep 25, 2026

Evasion Attacks on Cost-Utility-Based Adversarial Training for Online AutoML in IoT Networks

As Internet of Things (IoT) networks increasingly depend on machine learning for anomaly, malware, intrusion detection, and network monitoring, such systems have become attractive targets for evasion attacks. Evasion attacks pose a major security risk because an adversary intentionally modifies input data to mislead a trained model into producing incorrect predictions while evading detection. This study evaluates the impact of black-box evasion attacks on a cost-utility-based adversarial training defense strategy in an Online AutoML context for IoT networks. Specifically, evasion attacks were applied to online learners, including Hoeffding Tree (HT), Leveraging Bagging (LB), Streaming Random Patches (SRP), Hoeffding Adaptive Tree (HAT), and Adaptive Random Forest (ARF). By developing naive and adversarially trained (AT) versions of these online learners, we generated clean and adversarial accuracies for each model. The results show that the AT versions of LB and SRP performed best, achieving the highest adversarial accuracy (0.985) and high clean accuracy (0.993) at the highest cost budget of 1.00, with a maximum accuracy reduction of only 0.8%. Finally, drift detection was conducted using the Early Drift Detection Method (EDDM).

Chukwunonso Henry Nwokoye, Wajiha Zaheer, Khalil El-Khatib +1

Faculty of Business and Information Technology, Ontario Tech University, Oshawa, Ontario, Canada · Department of Computer Science, Alex Ekwueme Federal University, Nigeria

Adversarial Robustness Adversarial Training Black-Box Adversarial Attacks Online Learning IoT Intrusion Detection

Ask PDF

Apr 15, 2026

Robustness Analysis of Machine Learning Models for IoT Intrusion Detection Under Data Poisoning Attacks

Ensuring the reliability of machine learning-based intrusion detection systems remains a critical challenge in Internet of Things (IoT) environments, particularly as data poisoning attacks increasingly threaten the integrity of model training pipelines. This study evaluates the susceptibility of four widely used classifiers, Random Forest, Gradient Boosting Machine, Logistic Regression, and Deep Neural Network models, against multiple poisoning strategies using three real-world IoT datasets. Results show that while ensemble-based models exhibit comparatively stable performance, Logistic Regression and Deep Neural Networks suffer degradation of up to 40% under label manipulation and outlier-based attacks. Such disruptions significantly distort decision boundaries, reduce detection fidelity, and undermine deployment readiness. The findings highlight the need for adversarially robust training, continuous anomaly monitoring, and feature-level validation within operational Network Intrusion Detection Systems. The study also emphasizes the importance of integrating resilience testing into regulatory and compliance frameworks for AI-driven IoT security. Overall, this work provides an empirical foundation for developing more resilient intrusion detection pipelines and informs future research on adaptive, attack-aware models capable of maintaining reliability under adversarial IoT conditions.

Fortunatus Aabangbio Wulnye, Justice Owusu Agyemang, Kwame Opuni-Boachie Obour Agyekum +3

Department of Telecommunication, Kwame Nkrumah University of Science and Technology (KNUST), Kumasi, Ghana

IoT Intrusion Detection Adversarial Robustness Data Poisoning Attacks

Ask PDF