Leakage-free evaluation of residual graph convolutional networks for network intrusion detection

Zahid Iqbal

cs.LGNov 3, 2026

Leakage-free evaluation of residual graph convolutional networks for network intrusion detection

Authors: Zahid Iqbal

Organizations: Department of Computer Science, Faculty of Computing and Information Technology, University of Gujrat, 50700, Punjab, Pakistan

Abstract

Graph neural networks are increasingly proposed for network intrusion detection, yet many reported results come from protocols in which test flows shape the graph used for training, duplicated flows sit on both sides of the split, or a single split is reported. This study measures what a residual graph convolutional network gains over simpler models once these sources of optimism are removed. CICIDS2017 and NF-UNSW-NB15-v2 were cleaned with label-aware removal of exact duplicates, removing 69% of NF-UNSW-NB15-v2 flows. Flows were linked in a nearest-neighbour graph built from training flows only, and validation and test flows were attached to their nearest training flows at prediction time. A two-layer residual network was compared with eight models under the same settings, including a nearest-neighbour vote over the same graph, gradient-boosted trees and a random forest, over three seeds that each draw a new split, with ablations, leave-one-family-out and feature-noise tests. On CICIDS2017 the residual network reached a binary F1 of 98.11% (SD 0.18) and on NF-UNSW-NB15-v2 90.57% (0.38), indistinguishable from a multilayer perceptron without the graph and from other graph networks. Tree ensembles were 1.48 to 1.57 F1 points higher on CICIDS2017 and led on the area under the precision-recall curve on both datasets; their CICIDS2017 multiclass macro-F1 reached 93.08% against 64.51%, a gap mainly from benign flows the residual network assigned to small attack classes. No ablation, including transductive graph construction, changed F1 beyond seed variation. Under Gaussian feature noise (SD 0.5), the residual network kept an F1 of 96.16%, while XGBoost fell to 0.67%. Under a leakage-controlled protocol, message passing over flow graphs adds no measurable accuracy over a perceptron, and tree ensembles remain stronger on both; the advantage of graph models lies in robustness to feature noise.

Figures & tables

Study	Nodes; edges	Model; learning	Edges of test flows	Split	Repeated runs	Datasets
E-GraphSAGE [ 14 ]	hosts; flows	E-GraphSAGE; supervised	–	70/30	–	BoT-IoT, ToN-IoT, NF-BoT-IoT, NF-ToN-IoT
Anomal-E [ 2 ]	hosts; flows	E-GraphSAGE + DGI; self-supervised	–	–	–	NF-UNSW-NB15-v2, NF-CSE-CIC-IDS2018-v2
E-ResSAGE, E-ResGAT [ 23 ]	endpoints; flows	GraphSAGE, GAT + feature concatenation; supervised	–	50/20/30	–	UNSW-NB15, CIC-DarkNet, CSE-CIC-IDS, ToN-IoT
TKSGF [ 4 ]	flows; Top- $K$ cosine	GraphSAGE (GCN, GAT); supervised	among test flows (graph per partition)	60/20/20, 70/15/15	$\geq$ 5	NF-ToN-IoT, NF-BoT-IoT
GraphIDS [ 3 ]	hosts; flows	E-GraphSAGE + masked autoencoder; benign-only	among test flows (graph per partition)	80/10/10	several seeds, mean $\pm$ SD	NF-UNSW-NB15, NF-CSE-CIC-IDS2018 (v2, v3)
This study	flows; $k$ NN, $k=10$	R-GCN and 8 baselines; supervised	to training flows only	60/20/20	3 seeds, new split each	CICIDS2017, NF-UNSW-NB15-v2 (de-duplicated)

Table 1: Graph-based intrusion detection studies closest to the present work, as described in the cited papers (–: not stated).

Figure 1: Evaluation protocol. The flow records are cleaned and exact duplicates are removed before a stratified split forms the training, validation and test partitions (one lane each). Everything learned from data (the scaling, the neighbours of the $k$ NN graph and the model) comes from training flows and is passed down to the other lanes (dashed arrows); validation and test flows link only to training flows. Seeds 0, 1 and 2 repeat the protocol with new splits.

Table 3

Figure 2: Inductive $k$ NN flow graph (schematic with $k=3$ ; the experiments use $k=10$ ). (a) During training, each training flow receives messages from its nearest training flows. (b) A validation or test flow receives messages from its nearest training flows only; query flows are never linked to each other and never send messages to training flows.

Figure 3: Two-layer residual GCN. Each layer applies mean aggregation $\hat{A}=D^{-1}(A+I)$ over the inductive $k$ NN graph, a linear map, batch normalisation, ReLU and dropout; the output of the first layer is added to that of the second before the classifier.

Neural models (shared)
Setting	Value
Graph	$k$ NN of flows, $k=10$ , Euclidean distance, inductive (training flows only)
Layers / hidden units	2 / 64
Normalisation / dropout	batch normalisation; dropout 0.2
Residual (R-GCN only)	$H^{(l)}=Z^{(l)}+H^{(l-1)}$ for $l\geq 2$
GAT heads / GraphSAGE	4 heads / mean aggregation

Table 4: Settings of all models. The same values were used for every seed and for both datasets.

Model	F1	MCC	PR-AUC	FPR	Recall at 0.1% FPR	Recall at 1% FPR
R-GCN	98.11 $\pm$ 0.18	0.977 $\pm$ 0.002	99.86 $\pm$ 0.01	0.64 $\pm$ 0.07	97.39 $\pm$ 0.14	99.52 $\pm$ 0.04
GCN	98.15 $\pm$ 0.15	0.978 $\pm$ 0.002	99.86 $\pm$ 0.01	0.62 $\pm$ 0.06	97.67 $\pm$ 0.18	99.53 $\pm$ 0.03
GraphSAGE	98.14 $\pm$ 0.17	0.978 $\pm$ 0.002	99.86 $\pm$ 0.00	0.62 $\pm$ 0.09	97.40 $\pm$ 0.29	99.49 $\pm$ 0.01
GAT	98.33 $\pm$ 0.26	0.980 $\pm$ 0.003	99.85 $\pm$ 0.02	0.51 $\pm$ 0.17	97.29 $\pm$ 0.85	99.52 $\pm$ 0.05
$k$ NN vote	99.27 $\pm$ 0.02	0.991 $\pm$ 0.000	99.55 $\pm$ 0.01	0.15 $\pm$ 0.01	98.71 $\pm$ 0.07	99.81 $\pm$ 0.01
MLP	97.92 $\pm$ 0.04	0.975 $\pm$ 0.001	99.84 $\pm$ 0.01	0.71 $\pm$ 0.02	97.11 $\pm$ 0.28	99.43 $\pm$ 0.04

Table 5: Binary detection on the CICIDS2017 test partitions (%, MCC without unit; mean

\pm

SD over three seeds). Recall at 0.1% or 1% FPR is the highest recall at a false-positive rate not above that value; bold marks the best mean.

Model	F1	MCC	PR-AUC	FPR	Recall at 0.1% FPR	Recall at 1% FPR
R-GCN	90.57 $\pm$ 0.38	0.904 $\pm$ 0.004	95.11 $\pm$ 0.14	1.22 $\pm$ 0.06	43.62 $\pm$ 1.25	97.97 $\pm$ 0.35
GCN	90.82 $\pm$ 0.41	0.906 $\pm$ 0.004	95.04 $\pm$ 0.22	1.19 $\pm$ 0.05	44.33 $\pm$ 0.24	98.27 $\pm$ 0.21
GraphSAGE	90.87 $\pm$ 0.16	0.907 $\pm$ 0.002	94.82 $\pm$ 0.24	1.19 $\pm$ 0.02	42.31 $\pm$ 3.19	98.27 $\pm$ 0.46
GAT	90.96 $\pm$ 0.22	0.908 $\pm$ 0.002	95.01 $\pm$ 0.06	1.17 $\pm$ 0.03	43.53 $\pm$ 1.39	98.17 $\pm$ 0.21
$k$ NN vote	92.47 $\pm$ 0.14	0.920 $\pm$ 0.001	94.95 $\pm$ 0.10	0.64 $\pm$ 0.02	0.00 $\pm$ 0.00	98.42 $\pm$ 0.03
MLP	90.57 $\pm$ 0.50	0.904 $\pm$ 0.005	94.90 $\pm$ 0.17	1.23 $\pm$ 0.07	40.79 $\pm$ 1.19	98.42 $\pm$ 0.23

Table 6: Binary detection on the NF-UNSW-NB15-v2 test partitions (%, MCC without unit; mean

\pm

SD over three seeds). Recall at 0.1% or 1% FPR is the highest recall at a false-positive rate not above that value; bold marks the best mean. E-GraphSAGE uses a host graph and is listed separately. The

k

NN-vote score takes only 11 values, so a recall at a fixed false-positive rate may not exist and is then 0.

Figure 4: Binary F1 per model on the test partitions. Small points: individual seeds; large points: mean $\pm$ SD over three seeds. The F1 axis does not start at zero. E-GraphSAGE (right of the dashed line) uses a host graph and is reported separately.

R-GCN minus	$\Delta$ F1	95% CI	Holm $p$	$\Delta$ PR-AUC	95% CI	Holm $p$
CICIDS2017
GCN	−0.04	[−0.56, +0.48]	1.000	0.00	[−0.02, +0.01]	0.595
GraphSAGE	−0.04	[−0.77, +0.70]	1.000	0.00	[−0.02, +0.02]	1.000
GAT	−0.22	[−1.13, +0.68]	1.000	0.00	[−0.02, +0.03]	1.000
$k$ NN vote	−1.16	[−1.57, −0.75]	0.033	+0.31	[+0.28, +0.34]	0.004
MLP	+0.19	[−0.27, +0.65]	0.887	+0.01	[+0.01, +0.02]	0.061

Table 7: Paired per-seed differences in binary detection, R-GCN minus each model (percentage points, three seeds), with 95% confidence intervals from the

t

distribution and paired

t

-test

p

-values Holm-corrected within each dataset and metric.

Figure 5: Paired per-seed differences in binary detection, R-GCN minus each model, with 95% confidence intervals from the $t$ distribution (three seeds). (a) F1; (b) PR-AUC. Points left of zero favour the other model.

Model	Macro-F1	Weighted F1	Accuracy
R-GCN	64.51 $\pm$ 0.23	97.80 $\pm$ 0.05	96.60 $\pm$ 0.04
GCN	64.65 $\pm$ 0.38	97.85 $\pm$ 0.15	96.65 $\pm$ 0.21
$k$ NN vote	81.52 $\pm$ 1.81	99.74 $\pm$ 0.01	99.74 $\pm$ 0.01
MLP	62.39 $\pm$ 0.59	96.47 $\pm$ 0.71	94.65 $\pm$ 1.09
XGBoost	93.08 $\pm$ 1.22	99.85 $\pm$ 0.01	99.84 $\pm$ 0.01
LightGBM	81.57 $\pm$ 0.75	99.66 $\pm$ 0.01	99.58 $\pm$ 0.04

Table 8: Multiclass detection on the CICIDS2017 test partitions (%, mean

\pm

SD over three seeds). Macro-F1 averages the per-class F1 over the classes present in the test partition; weighted F1 weights each class by its number of flows; bold marks the best mean.

Model	Macro-F1	Weighted F1	Accuracy
R-GCN	44.21 $\pm$ 1.07	97.04 $\pm$ 0.11	96.25 $\pm$ 0.21
$k$ NN vote	48.72 $\pm$ 0.31	97.79 $\pm$ 0.01	97.88 $\pm$ 0.01
MLP	45.85 $\pm$ 1.24	97.02 $\pm$ 0.17	96.19 $\pm$ 0.33
XGBoost	55.98 $\pm$ 0.88	97.90 $\pm$ 0.06	97.48 $\pm$ 0.08
LightGBM	53.82 $\pm$ 1.04	97.83 $\pm$ 0.05	97.39 $\pm$ 0.07

Table 9: Multiclass detection on the NF-UNSW-NB15-v2 test partitions (%, mean

\pm

SD over three seeds). Macro-F1 averages the per-class F1 over the classes present in the test partition; weighted F1 weights each class by its number of flows; bold marks the best mean.

Figure 6: Per-class F1 in multiclass detection on the test partitions (%, mean over three seeds). (a) CICIDS2017; (b) NF-UNSW-NB15-v2. Classes follow the order of Tables 3 and 3 .

Figure 7: Confusion matrices of multiclass detection on CICIDS2017, summed over the three seeds and normalised by the true class (rows, %). (a) R-GCN; (b) XGBoost. Cells below 0.05% are left blank.

Configuration	F1	PR-AUC	FPR	$\Delta$ F1 [95% CI]	Holm $p$
R-GCN reference (2 layers, residual, $k=10$ )	98.11 $\pm$ 0.18	99.86 $\pm$ 0.01	0.64 $\pm$ 0.07	–	–
2 layers, no skip (GCN)	98.15 $\pm$ 0.15	99.86 $\pm$ 0.01	0.62 $\pm$ 0.06	+0.04 [−0.48, +0.56]	1.000
$k=5$	98.03 $\pm$ 0.10	99.85 $\pm$ 0.01	0.67 $\pm$ 0.03	−0.07 [−0.30, +0.16]	1.000
$k=20$	98.06 $\pm$ 0.10	99.84 $\pm$ 0.01	0.66 $\pm$ 0.04	−0.04 [−0.27, +0.18]	1.000
$k=50$	98.22 $\pm$ 0.35	99.82 $\pm$ 0.01	0.53 $\pm$ 0.20	+0.11 [−0.30, +0.53]	1.000
1 layer	97.66 $\pm$ 0.14	99.75 $\pm$ 0.02	0.75 $\pm$ 0.13	−0.45 [−1.24, +0.34]	1.000

Table 10: Ablation of the R-GCN on CICIDS2017, binary detection, one setting changed at a time (%, mean

\pm

SD over three seeds).

\Delta

F1 is the paired difference to the reference in percentage points with its 95% confidence interval; Holm

p

is corrected over all rows.

Figure 8: Ablation on CICIDS2017, binary F1 (mean $\pm$ SD over three seeds). (a) Number of neighbours $k$ (logarithmic axis); the dashed line is the MLP without a graph. (b) Number of graph convolution layers without a skip, with the identity skip of the R-GCN, and with an additional input projection.

Held-out family	Flows	R-GCN	$k$ NN vote	MLP	XGBoost
DoS	38,749	49.37 $\pm$ 0.67	50.36 $\pm$ 0.68	10.49 $\pm$ 8.56	3.19 $\pm$ 0.07
DDoS	25,603	63.42 $\pm$ 0.27	63.42 $\pm$ 0.27	63.16 $\pm$ 0.44	59.42 $\pm$ 5.47
PortScan	18,139	0.24 $\pm$ 0.08	0.18 $\pm$ 0.03	0.28 $\pm$ 0.10	0.39 $\pm$ 0.11
BruteForce	1,830	0.24 $\pm$ 0.14	0.09 $\pm$ 0.08	0.24 $\pm$ 0.17	0.26 $\pm$ 0.03
Web	428	0.31 $\pm$ 0.36	1.01 $\pm$ 0.36	0.31 $\pm$ 0.36	83.36 $\pm$ 1.81
Bot	389	0.00 $\pm$ 0.00	0.00 $\pm$ 0.00	0.00 $\pm$ 0.00	0.00 $\pm$ 0.00

Table 11: Recall (%) on the attack family withheld from training and validation, CICIDS2017, threshold 0.5 (mean

\pm

SD over three seeds). Flows: test flows of the withheld family; the last two rows summarise the families seen in training and the benign flows over all six runs.

Figure 9: Recall on the attack family removed from training and validation (CICIDS2017, threshold 0.5, mean $\pm$ SD over three seeds).

Model	$\sigma=0$	$\sigma=0.1$	$\sigma=0.25$	$\sigma=0.5$
R-GCN	98.26 $\pm$ 0.24	98.21 $\pm$ 0.21	97.91 $\pm$ 0.25	96.16 $\pm$ 0.29
GCN	98.12 $\pm$ 0.07	98.11 $\pm$ 0.08	97.78 $\pm$ 0.05	96.04 $\pm$ 0.08
$k$ NN vote	99.32 $\pm$ 0.03	98.71 $\pm$ 0.01	98.07 $\pm$ 0.07	95.47 $\pm$ 0.05
MLP	97.94 $\pm$ 0.04	97.76 $\pm$ 0.13	93.47 $\pm$ 0.78	77.70 $\pm$ 2.65
XGBoost	99.69 $\pm$ 0.04	5.84 $\pm$ 2.51	1.84 $\pm$ 0.96	0.67 $\pm$ 0.40

Table 12: Binary F1 (%) on CICIDS2017 under Gaussian noise with standard deviation

\sigma

added to the standardised test features (up to 100,000 test flows; mean

\pm

SD over three seeds).

Figure 10: Binary F1 under Gaussian noise added to the standardised test features (CICIDS2017, up to 100,000 test flows per run, mean $\pm$ SD over three seeds).

Network Intrusion Detection

Network intrusion detection identifies malicious or abnormal network activity to help protect computers, connected devices, and critical infrastructure.

See all 174 papers

Jun 15, 2026

Timestamp-Aware Spatio-Temporal Graph Contrastive Learning for Network Intrusion Detection

Given their effectiveness in modeling the relational structure among network traffic flows, graph neural networks (GNNs) have been widely adopted in network intrusion detection systems (NIDSs). However, most existing GNN-based NIDS approaches focus on the relational structure of traffic flows, and treat them as temporally independent, which limits their ability to cope with evolving attack behaviors. Moreover, their reliance on supervised or semi-supervised learning often restricts generalization to unseen attacks. To address these limitations, we propose a novel self-supervised GNN-based framework. To the best of our knowledge, the proposed model is among the first self-supervised GNN-based NIDS models to explicitly leverage real timestamps, which provides faithful temporal dependencies for representation learning. We first construct a series of temporal graphs from network traffic flows according to their timestamps, and then employ an E-GraphSAGE and LSTM based encoder to fully extract temporal information and spatial dependencies of network traffic, without introducing time-costly attention mechanisms. A multi-view graph contrastive learning (GCL) scheme is introduced, where temporal, spatial, and feature contrasts are jointly performed to capture temporal continuity, preserve structural consistency, and improve the generalization and robustness of the learned representations, respectively. In addition, a gradient-norm-based adaptive weighting strategy is designed to optimize the contrastive loss weights. Experimental results on four representative NIDS datasets with real timestamps demonstrate that our method significantly outperforms existing self-supervised approaches and achieves performance comparable to the supervised state-of-the-art GNN method, while maintaining high computational efficiency.

Jianli Dai, Guangwei Wu, Jiacheng Li +3

Central South University of Forestry and Technology, School of Computer Science and Mathematics, Changsha, 410004, Hunan, P.R. China · Central South University, School of Computer Science and Engineering, Changsha, 410083, Hunan, P.R. China

Temporal GNNs Graph SSL Network Intrusion Detection Graph Contrastive Learning Spatiotemporal GNNs

Ask PDF

May 8, 2026

GESR: Graph-Based Edge Semantic Reconstruction for Stealthy Communication Detection with Benign-Only Training

Detecting stealthy malicious communications from flow logs under benign-only training remains a critical challenge in network security. Malicious communications often camouflage as normal traffic like standard HTTPS flows. Conventional intrusion detectors rely strictly on known labeled attacks. Alternatively, they score flows completely independently. These approaches fail against sparse and context-dependent suspicious activity. To capture this essential context, graph anomaly detectors have been introduced to add valuable relational information to the analysis. However, existing methods fail to test the structural consistency of specific communication edges. To overcome these fundamental limitations, we present GESR, a novel graph-based framework for detecting suspicious communications and anomalous hosts under a benign-only training setting. GESR models complex network activity as attributed communication graphs. It cleverly reconstructs edge semantics entirely from local structural context rather than isolated features. This non-intuitive design forces the framework to predict expected communication patterns from neighborhood topologies. Attackers cannot easily manipulate this deep structural dependency. The model then converts the resulting structural inconsistencies into host-level anomaly scores. It utilizes robust Median Absolute Deviation (MAD) calibration for this final step. We evaluate GESR extensively on CTU-13 and CICIDS2017 datasets. These evaluations strictly impose tight false-positive operating constraints. On CICIDS2017, GESR achieves an outstanding ROC-AUC of 0.9753. It also yields a high TPR of 0.8569 at a strict 5% FPR threshold. GESR consistently outperforms existing methods across both evaluated benchmarks. The results prove that structure-conditioned edge reconstruction is a credible direction for practical intrusion detection.

Henghui Xu, Yuchen Zhang, Xiaobo Ma

MOE Key Laboratory for Intelligent Networks and Network Security, Xi’an Jiaotong University, Xi’an, China · Faculty of Electronic and Information Engineering, Xi’an Jiaotong University, Xi’an, China · Shaanxi Province Key Laboratory of Computer Network, Xi’an Jiaotong2026 University, Xi’an, China

Network Intrusion Detection Reconstruction-Based Anomaly Detection Anomaly Detection

Ask PDF

Oct 31, 2026

Adversarial Robustness Is Not Zero-Day Generalization: A Controlled Evaluation of Flow-Based Intrusion Detectors on Corrected CIC-IDS2017

Synthetic variants of known attacks, including feature-space adversarial perturbations, are sometimes used as stand-ins for unseen ("zero-day") attacks when evaluating machine-learning network intrusion detection systems (NIDS). We test whether this proxy holds. On the corrected CIC-IDS2017 release we train Random Forest, XGBoost, a 1D-CNN and an FT-Transformer and compare, at a fixed 1% false-positive rate over five seeds, detection under constrained perturbations of known attacks (gradient-based and black-box attacks on 42 attacker-controllable features) with leave-one-attack-family-out (LOAO) detection, where an entire family is absent from training. Detection under the strongest perturbation we tested, with budgets of at most 0.2, stays between 90.9% and 99.7%, yet mean detection of held-out families falls to 76.2% (RF), 40.2% (XGBoost), 25.8% (1D-CNN) and 22.8% (FT-Transformer). Robustness exceeds LOAO detection for every model (gaps of 24 to 72 percentage points, 23 to 66 when robustness is also averaged over families; Holm-adjusted p

\le

0.00096), and the two measures agree at most on a coarse ranking of the models. An exact mixed-integer evasion search confirms the tree ensembles' scores. Benign-only anomaly detectors and a simple multi-class open-set baseline do not close the gap. A temporal split on 5 seeds (3 of the 4 models) leaves the gap in place. On CSE-CIC-IDS2018 the gap recurs for every model (16 to 68 points). On these benchmarks and under this threat model, a robustness score should not be read as a zero-day score, and the two should be reported separately. Code and per-seed results are publicly available.

Abdulhamid Zaro

College of Professions and Applied Sciences, Hebron University, Hebron, Palestine

Network Intrusion Detection Adversarial Robustness Adversarial Attacks

Ask PDF

Abstract

Network Intrusion Detection

Explore similar work

Timestamp-Aware Spatio-Temporal Graph Contrastive Learning for Network Intrusion Detection

GESR: Graph-Based Edge Semantic Reconstruction for Stealthy Communication Detection with Benign-Only Training

Adversarial Robustness Is Not Zero-Day Generalization: A Controlled Evaluation of Flow-Based Intrusion Detectors on Corrected CIC-IDS2017