Quantifying Catastrophic Forgetting in IoT Intrusion Detection Systems

Sourasekhar Banerjee; David Bergqvist; Salman Toor; Christian Rohner; Andreas Johnsson

cs.LGFeb 27, 2026

Quantifying Catastrophic Forgetting in IoT Intrusion Detection Systems

Authors: Sourasekhar Banerjee, David Bergqvist, Salman Toor, Christian Rohner, Andreas Johnsson

Organizations: Uppsala University, Department of Information Technology, Sweden · Scaleout Systems AB, Sweden · Ericsson Research, Research Area AI, Sweden

Abstract

Distribution shifts in attack patterns within RPL-based IoT networks pose a critical threat to the reliability and security of large-scale connected systems. Intrusion Detection Systems (IDS) trained on static datasets often fail to generalize to unseen threats and suffer from catastrophic forgetting when updated with new attacks. Ensuring continual adaptability of IDS is therefore essential for maintaining robust IoT network defence. In this focused study, we formulate intrusion detection as a domain continual learning problem and propose a method-agnostic IDS framework that can integrate diverse continual learning strategies. We systematically benchmark five representative approaches across multiple domain-ordering sequences using a comprehensive multi-attack dataset comprising 48 domains. Results show that continual learning mitigates catastrophic forgetting while maintaining a balance between plasticity, stability, and efficiency, a crucial aspect for resource-constrained IoT environments. Among the methods, Replay-based approaches achieve the best overall performance, while Synaptic Intelligence (SI) delivers near-zero forgetting with high training efficiency, demonstrating strong potential for stable and sustainable IDS deployment in dynamic IoT networks.

Figures & tables

Fig. 1: (a) Sequential training suffers from catastrophic forgetting, while (b) continual learning retains knowledge of prior attacks to sustain performance.

Fig. 2: System model of continual learning based intrusion detection system.

Fig. 3: T-SNE 2D-visualizations of attack domains for four IoT network attacks—Blackhole (BH), DIS-Flooding (DF), Local Repair (LR), and Worst Parent (WP)—under three attack types: base (left), on-off (center), and gradual decrease (right). Network comprises 20 nodes, simulated using Cooja [ 5 ] .

Method	Blackhole (BH)		DIS-Flooding (DF)		Local Repair (LR)		Worst Parent (WP)
	$F1\uparrow$	$AUC\uparrow$	$F1\uparrow$	$AUC\uparrow$	$F1\uparrow$	$AUC\uparrow$	$F1\uparrow$	$AUC\uparrow$
W/O CL	0.30	0.49	0.78	0.96	0.63	0.75	0.45	0.57
Replay	0.64	0.69	0.97	1.00	0.88	0.95	0.63	0.70
EWC	0.52	0.64	0.82	1.00	0.63	0.74	0.48	0.58
SI	0.42	0.61	0.88	1.00	0.58	0.77	0.47	0.59
LwF	0.33	0.52	0.71	0.81	0.54	0.72	0.43	0.58

TABLE I: Average performance across four scenarios (Random, B2W, W2B, Toggle) for each method and attack type.

Fig. 4: Change of BWT per domain for domain sequence Random (Top left), B2W (Top right), W2B (Bottom left), Toggle (Bottom right).

Method	Random			B2W			W2B			Toggle
	$\bar{S}\uparrow$	$\bar{P}\uparrow$	$TE\uparrow$	$\bar{S}\uparrow$	$\bar{P}\uparrow$	$TE\uparrow$	$\bar{S}\uparrow$	$\bar{P}\uparrow$	$TE\uparrow$	$\bar{S}\uparrow$	$\bar{P}\uparrow$	$TE\uparrow$
W/O CL	-0.32	0.25	0.94	-0.31	0.13	0.96	-0.38	0.40	0.85	-0.35	0.31	0.96
Replay	-0.11	0.31	0.54	-0.11	0.32	0.52	-0.11	0.31	0.66	-0.11	0.31	0.57
EWC	-0.15	0.08	0.18	-0.28	0.19	0.22	-0.17	0.16	0.20	-0.17	0.14	0.18
SI	-0.06	0.03	0.83	-0.08	0.03	0.80	-0.09	0.06	0.92	-0.09	0.05	0.78
LwF	-0.22	0.07	0.64	-0.26	0.08	0.61	-0.10	0.14	0.76	-0.16	0.13	0.64

TABLE II: Average Stability (

\bar{S}

), Plasticity (

\bar{P}

), and Training Efficiency (TE) after training on

48

domains.

IoT Intrusion Detection

IoT intrusion detection identifies malicious network activity targeting connected devices and industrial systems using traffic and device data.

See all 51 papers

ThreatFormer-IDS: Robust Transformer Intrusion Detection with Zero-Day Generalization and Explainable Attribution
Feb 26, 2026Srikumar NayakExplainable Intrusion Detection IoT Intrusion Detection
Incedo Inc, USA
Deep learning based intelligent IDS for Large-scale IoT networks
Mar 17, 2026Isha Andrade, Shalaka S Mahadik, Mithun Mukherjee +2IoT Intrusion Detection Network Intrusion Detection
Department of Computer Science, Birla Institute of Technology and Science, Pilani, Dubai Campus, Dubai International Academic City, Dubai, United Arab Emirates
MI $^2$ DAS: A Multi-Layer Intrusion Detection Framework with Incremental Learning for Securing Industrial IoT Networks
Feb 27, 2026Wei Lian, Alejandro Guerra-ManzanaresIoT Intrusion Detection Network Intrusion Detection
School of Computer Science, University of Nottingham, Ningbo, China

Feb 26, 2026J/K move · Enter open · S save

ThreatFormer-IDS: Robust Transformer Intrusion Detection with Zero-Day Generalization and Explainable Attribution

Srikumar Nayak

Incedo Inc, USA

Explainable Intrusion Detection IoT Intrusion Detection Network Intrusion Detection Adversarial Robustness Internet of Things Transformer

Open paper AskPDF arXiv

Intrusion detection in IoT and industrial networks requires models that can detect rare attacks at low false-positive rates while remaining reliable under evolving traffic and limited labels. Existing IDS solutions often report strong in-distribution accuracy, but they may degrade when evaluated on future traffic, unseen (zero-day) attack families, or adversarial feature manipulations, and many systems provide limited evidence to support analyst triage. To address these gaps, we propose ThreatFormer- IDS, a Transformer-based sequence modeling framework that converts flow records into time-ordered windows and learns contextual representations for robust intrusion screening. The method combines (i) weighted supervised learning for imbalanced detection, (ii) masked self-supervised learning to improve representation stability under drift and sparse labels, (iii) PGDbased adversarial training with scale-normalized perturbations to strengthen resilience against feature-level evasion, and (iv) Integrated Gradients attribution to highlight influential time steps and features for each alert. On the ToN IoT benchmark with chronological evaluation, ThreatFormer-IDS achieves AUCROC 0.994, AUC-PR 0.956, and Recall@1%FPR 0.910, outperforming strong tree-based and sequence baselines. Under a zero-day protocol with held-out attack families, it maintains superior generalization (AUC-PR 0.721, Recall@1%FPR 0.783). Robustness tests further show slower degradation in AUCPR as the adversarial budget increases, confirming improved stability under bounded perturbations. Overall, ThreatFormer- IDS provides a unified, deployment-oriented IDS pipeline that balances detection quality, zero-day behavior, robustness, and explainability.

Abstract

IoT Intrusion Detection

Explore similar work

ThreatFormer-IDS: Robust Transformer Intrusion Detection with Zero-Day Generalization and Explainable Attribution

Deep learning based intelligent IDS for Large-scale IoT networks

MI2^22DAS: A Multi-Layer Intrusion Detection Framework with Incremental Learning for Securing Industrial IoT Networks

MI $^2$ DAS: A Multi-Layer Intrusion Detection Framework with Incremental Learning for Securing Industrial IoT Networks