cs.CRMar 24, 2026
SaveTargeted Adversarial Traffic Generation : Black-box Approach to Evade Intrusion Detection Systems in IoT Networks
Abstract
The integration of machine learning (ML) algorithms into Internet of Things (IoT) applications has introduced significant advantages alongside vulnerabilities to adversarial attacks, especially within IoT-based intrusion detection systems (IDS). While theoretical adversarial attacks have been extensively studied, practical implementation constraints have often been overlooked. This research addresses this gap by evaluating the feasibility of evasion attacks on IoT network-based IDSs, employing a novel black-box adversarial attack. Our study aims to bridge theoretical vulnerabilities with real-world applicability, enhancing understanding and defense against sophisticated threats in modern IoT ecosystems. Additionally, we propose a defense scheme tailored to mitigate the impact of evasion attacks, thereby reinforcing the resilience of ML-based IDSs. Our findings demonstrate successful evasion attacks against IDSs, underscoring their susceptibility to advanced techniques. In contrast, we proposed a defense mechanism that exhibits robust performance by effectively detecting the majority of adversarial traffic, showcasing promising outcomes compared to current state-of-the-art defenses. By addressing these critical cybersecurity challenges, our research contributes to advancing IoT security and provides insights for developing more resilient IDS.
Figures & tables
| Study | Contribution | Limitations |
|---|---|---|
| Islam et al. Islam et al. (2021) | Feature selection for lightweight IDS | Limited to specific IoT protocols |
| Sarhan et al. Sarhan et al. (2024) | Novel framework for IoT-specific challenges | Requires large labeled datasets |
| Altulaihan et al. Altulaihan et al. (2024) | Real-time anomaly detection | Susceptible to adversarial attacks |
| Alslman et al. Alslman et al. (2024) | Robust IDS using adversarial training | High resource requirements |
| Yuan et al. Yuan et al. (2024) | Simple yet effective adversarial defenses | Limited evaluation on IoT datasets |
| Barik et al. Barik et al. (2024) | Adversarial robustness for tabular data | Not IoT-specific |
Table 1: Summary of Recent Works on ML-based IDS
Figure 1: Schematic illustration of the threat scenario, showing how the attacker replicates a substitute IoT environment, builds a substitution model, and leverages transferability to craft adversarial examples that evade the defender’s IoT-based NIDS.
| Features | Description |
|---|---|
| Dur | Duration of the flow |
| spkts | Number of packets from source to destination |
| sbytes | Number of bytes from source to destination |
Table 2: Description of Common Modifiable Features
| Features | Description | Dependency |
|---|---|---|
| Pkts | Total number of packets | Spkts, Dpkts |
| Bytes | Total number of bytes | Sbytes, Dbytes |
| Rate | Total number of packets per second | Dur, Pkts |
| Srate | Packets from source to destination per second | Dur, Spkts |
| Drate | Packets from destination to source per second | Dur, Dpkts |
Table 3: Description of Common Dependent Features
| Features | Description |
|---|---|
| Dpkts | Number of packets from destination to source |
| Dbytes | Number of bytes from destination to source |
| Proto | Textual representation of protocols |
| Sport | Source port number |
| Dport | Destination port number |
Table 4: Description of Common Non-Modifiable Features
Figure 2: Conceptual illustration of the D2TC approach, where malicious instances are iteratively shifted toward the benign mean and projected into valid ranges. Once inside the benign region, the instance is misclassified as benign, forming an adversarial example
Figure 3: Adversarial instance generation process. The attacker’s IoT dataset is split to build a substitution model that approximates the defender’s IDS. Iterative perturbations (and mask switching) are then applied to malicious flows until successful evasion is achieved, producing adversarial traffic ready to bypass the target system.
| Mask ID | Mask Value | Manipulated Features |
|---|---|---|
| 1 | 001 | |
| 2 | 010 | |
| 3 | 100 | |
| 4 | 011 | , |
| 5 | 101 | , |
| 6 | 110 | , |
Table 5: Binary masks for modifiable network features ( Dur , Spkts , Sbytes ), indicating which combination of features is perturbed for each mask ID
Figure 4: Workflow for creating the defense dataset. The original dataset is split into benign and malicious flows, with malicious instances undergoing adversarial instance generation (D2TC) to produce adversarial samples. These adversarial and clean subsets are then combined to form a comprehensive defense dataset.
Figure 5: Partitioning of the defense dataset into training/evaluation (derived from the defender’s data) and testing (sourced from the attacker’s data)
Figure 6: Overview of the proposed defense process. Sub-detectors are trained and evaluated in parallel on labeled clean and adversarial data, with each sub-detector assigned a reliability weight based on its detection performance. The final, ensemble-based decision filter is placed before the NIDS to intercept and mitigate adversarial traffic
Figure 7: Overview of the proposed adversarial detector architecture. Each incoming instance is split into feature-specific sub-detectors, whose decisions are weighted by reliability scores and fused to determine whether the traffic is benign or adversarial
Figure 8: Partitioning of the BoT-IoT and ToN-IoT datasets into attacker and defender subsets, each further split into training (75%) and test (25%) sets
| Port Range | Category |
|---|---|
| 0 to 1023 | Well-Known Ports |
| 1024 to 49151 | Registered Ports |
| 49152 to 65535 | Dynamic/Private Ports |
Table 6: Port range categorization for the Sport and Dport features prior to one-hot encoding
Figure 9: Machine Learning pipeline for both attacker and defender
Figure 10: Initial performance of the trained models (KNN, RF, DT, XGBoost) on both the attacker and defender sides for the Bot-IoT dataset. All models exhibit high metric scores (precision, recall, F1-score), with minimal variations across different hyperparameters.
Figure 11: Initial performance of the trained models (KNN, RF, DT, XGBoost) on both the attacker and defender sides for the Ton-IoT dataset. Similar to the Bot-IoT results, each model achieves consistently high metric scores, demonstrating negligible differences despite variations in hyperparameters.
Figure 12: Impact of adversarial perturbations on IDS detection rates for the Bot-IoT dataset. The bars compare detection rates of KNN, RF, DT, and XGBoost across four attack types (DDOS, DOS, Service_scan, and OS_fingerprinting) before and after adversarial perturbations. A marked drop in detection rate indicates the effectiveness of the adversarial modifications in evading each IDS model.
Figure 13: Impact of adversarial perturbations on IDS detection rates for the Ton-IoT dataset. Similar to Figure 12 , this figure highlights how the detection performance of KNN, RF, DT, and XGBoost significantly declines once adversarial samples are introduced, demonstrating the robustness of our attack across multiple attack types (DDOS, DOS, Scanning, XSS, Password, Injection, and Backdoor).
Figure 14: Percentage usage of different mask combinations for each attack in the Bot-IoT dataset. Each bar represents how frequently a particular set of features (e.g., dur , spkts , or sbytes ) is manipulated to create adversarial instances for a specific attack. Higher percentages indicate that those features play a more critical role in evading detection for that attack.
Figure 15: Proportion of mask use for every attack type in the Ton-IoT dataset. Similar to Figure 14 , this figure reveals which feature manipulations ( dur , spkts , sbytes , or their combinations) are most frequently employed to generate successful adversarial samples for each attack category.
Figure 16: Number of steps required to generate successful adversarial instances across different attacks. The bars represent the percentage of instances requiring each step count (1 to 10) to achieve evasion. A large proportion of attacks become adversarial within the first step, indicating that only minimal perturbations are necessary to fool the IDS.
| Attack Type | KNN | RF | DT | XGBoost |
|---|---|---|---|---|
| DDOS | 0.05 | 0.11 | 0.01 | 0.02 |
| DOS | 0.02 | 1.40 | 1.29 | 1.53 |
| Service Scan | 0.07 | 0.22 | 0.01 | 0.02 |
| OS Fingerprinting | 0.07 | 0.27 | 0.01 | 0.02 |
Table 7: Average generation time (second) of adversarial instances - Bot-IoT
| Attack Type | KNN | RF | DT | XGBoost |
|---|---|---|---|---|
| DDOS | 1.47 | 0.40 | 0.01 | 0.09 |
| DOS | 0.08 | 0.21 | 0.01 | 0.74 |
| Scanning | 0.06 | 0.24 | 0.01 | 0.12 |
| XSS | 1.02 | 1.62 | 0.03 | 0.85 |
| Password | 0.43 | 1.85 | 0.02 | 0.40 |
| Injection | 1.65 | 1.88 | 0.08 | 1.25 |
Table 8: Average generation time (second) of adversarial instances - Ton-IoT
| Defender models | Attacker’s substitute models | Average | |||
|---|---|---|---|---|---|
| KNN | RF | DT | XGBoost | ||
| KNN | 49% | 47% | 50% | 51% | 49% |
| RF | 56% | 49% | 53% | 52% | 53% |
| DT | 55% | 50% | 52% | 52% | 52% |
| XGBoost | 69% | 53% | 69% | 63% | 64% |
Table 9: Detection rates of the defender’s NIDS without adversarial defense across four attacker substitute models (KNN, RF, DT, XGBoost). The final column shows the average detection rate.
| Defender models | Attacker’s substitute models | Average | |||
|---|---|---|---|---|---|
| KNN | RF | DT | XGBoost | ||
| KNN | 65% | 61% | 74% | 71% | 68% |
| RF | 76% | 63% | 69% | 58% | 67% |
| DT | 66% | 39% | 68% | 68% | 60% |
| XGBoost | 79% | 56% | 74% | 72% | 70% |
Table 10: Detection rates of the defender’s NIDS with our adversarial defense (Bayesian fusion) against four attacker substitute models. The final column shows the average detection rate.
| Defender models | Attacker’s substitute models | Average | |||
|---|---|---|---|---|---|
| KNN | RF | DT | XGBoost | ||
| KNN | 65% | 61% | 75% | 71% | 68% |
| RF | 76% | 63% | 69% | 71% | 70% |
| DT | 66% | 56% | 76% | 60% | 63% |
| XGBoost | 82% | 66% | 76% | 73% | 74% |
Table 11: Detection rates of the defender’s NIDS with our adversarial defense (Dempster-Shafer fusion) against four attacker substitute models. The final column shows the average detection rate.
Figure 17: Defender’s NIDS performance under adversarial traffic, with and without our proposed defense (using Bayesian and Dempster-Shafer fusion).
Figure 18: Comparison of our Bayesian and Dempster-Shafer fusion-based defenses with a state-of-the-art adversarial detection method across various ML models