Organizations: Joint Research Centre, European Commission, Ispra, 21027, Italy. · Department of Computer Science, American University, 4400 Massachusetts Avenue NW, Washington, 20016, DC, USA. · Department of Computer Science, University of Bari Aldo Moro, Via Orabona, 4, Bari, 70125, Italy.
Network intrusion detection systems play a crucial role in the security strategy employed by organisations to detect and prevent cyberattacks. Such systems usually combine pattern detection signatures with anomaly detection techniques powered by machine learning methods. However, the commonly proposed machine learning methods present drawbacks such as over-reliance on labeled data and limited generalization capabilities. To address these issues, embedding-based methods have been introduced to learn representations from network data, such as DNS traffic, mainly due to its large availability, that generalise effectively to many downstream tasks. However, current approaches do not properly consider contextual information among DNS queries. In this paper, we tackle this issue by proposing DNS-GT, a novel Transformer-based model that learns embeddings for domain names from sequences of DNS queries. The model is first pre-trained in a self-supervised fashion in order to learn the general behavior of DNS activity. Then, it can be finetuned on specific downstream tasks, exploiting interactions with other relevant queries in a given sequence. Our experiments with real-world DNS data showcase the ability of our method to learn effective domain name representations. A quantitative evaluation on domain name classification and botnet detection tasks shows that our approach achieves better results compared to relevant baselines, creating opportunities for further exploration of large-scale language models for intrusion detection systems. Our code is available at: https://github.com/m-altieri/DNS-GT.
Figures & tables
Figure 1: Conceptual overview of domain classification task with model training phases: pre-training (a) and fine-tuning (b).
Figure 2: Workflow of the proposed DNS-GT method. Each stage is represented in red. Input and output data are depicted in green.
Figure 3: Model architecture. The colour is used to emphasise model input and output (green), tensors (yellow), operations (red) and learnable neural networks (blue).
Models (w/ External Classifiers)
Fixed
Time
Density
AUC
F1 @ 0.5
F1 @ Best
AUC
F1 @ 0.5
F1 @ Best
AUC
F1 @ 0.5
F1 @ Best
W2V-CBOW + AdaBoost
0.790
0.448
0.448
0.722
0.396
0.396
0.748
0.411
0.411
W2V-CBOW + Decision Tree
0.620
0.348
0.348
0.590
0.320
0.320
0.606
0.335
0.335
W2V-CBOW + GNB
0.821
0.494
0.508
0.678
0.348
0.356
0.738
0.427
0.433
W2V-CBOW + SVM
0.846
0.524
0.531
0.799
0.457
0.479
0.811
0.492
0.535
W2V-CBOW + VAE
0.600
0.110
0.318
0.581
0.076
0.316
0.598
0.144
0.305
Table 1: Performance results of the considered methods on the domain names classification task, measured on the ROC AUC, F1 score with threshold 0.5 , and F1 score with the best threshold, for the three sequencing approaches. Bold text highlights the best results for each metric and each sequencing approach. The best results for each metric are surrounded by a rectangle.
Configuration
AUC
F1 @ 0.5
F1 @ Best
DNS-GT (No Attention)
0.410
0.313
0.331
DNS-GT (No Hosts)
0.762
0.467
0.508
DNS-GT (Complete)
0.848
0.567
0.654
Table 2: Ablation study on the domain name classification task, comparing the proposed model in its full configuration with two variants where key components are deactivated.
Models
Accuracy
AUC
F1
W2V-CBOW (End-to-End)
0.851
0.967
0.851
W2V-SkipGram (End-to-End)
0.877
0.970
0.877
DNS-GT (End-to-End)
0.877
0.970
0.877
Table 3: Results of the considered methods on the botnet detection task, measured in terms of Accuracy, AUC, and F1 score (density-based sequencing approach).
Model
Seq. strategy
Training time (minutes)
Pre-training
Finetuning
W2V-CBOW
Fixed
224
412
Time
204
401
Density
45
99
W2V-SkipGram
Fixed
482
529
Time
224
355
Table 4: Empirical analysis of time complexity for all considered models on end-to-end training.
Figure 4: ROC curves for all considered methods with end-to-end training and evaluation.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Concepts
NLP
DNS
Token
Word a
Domain name
Sequence
Sentence
Set of queries
Order matters
Yes
No
Semantics
Meaning
User activity
Appendix
Table 5: Analogies between NLP and DNS concepts.
Figure 5: High-level overview of the training process and the data pipeline for the proposed methodology to learn representations of DNS queries. The colour encodes the function of the stage: input and output (green), data (yellow), operations (red) and learnable neural networks (blue).
Figure 6: Architecture of the graph attention network for a single graph topology. The colour is used to emphasise input (green), tensors (yellow), operations (red) and learnable neural networks (blue).
Figure 7: Stacking of multiple model blocks. The color is used to emphasise tensors (yellow), operations (red) and learnable neural networks (blue).
Figure 8: Examples of different graph topologies for a given sequence, with their corresponding adjacency matrices. Respectively, fully connected graph (top), a graph where the masked query is connected to all other queries and q1 and q2 are connected (middle), and a graph with where each node is only connected to the masked query (bottom). In these examples all nodes have self-loops.
Figure 9: Comparison of average intra-distance among the top 17 most common benign domains (blue line) and average inter-distance between common benign domains and blacklisted domains (red line) using the cosine similarity (a) and the Euclidean distance (b).
Figure 10: Average pairwise cosine similarity and Euclidean distance between domain name embeddings appearing in the same sequence vs. randomly sampled domains.
Figure 11: Cold start, latency and throughput of DNS-GT during both training and testing stages, with varying batch sizes, on the standard configuration used in the experiments. Steps per second corresponds to the number of batches processed in the time unit (second).
Figure 12: Box plot showing the classification scores for a random set of domain names across different sequences in which they appear. For each domain name, the assigned classification scores vary according to the contextual information. In the plot, domains are sorted in descending order according to the median value of their classification scores.
Figure 13: Embedding space visualization using t-SNE with perplexity 30 and 500 iterations. The red dots show the location of the blacklisted domain names, and the green dots (clustered at the very centre) represent the most common benign domain names in our dataset.
Figure 14: Classification score of DNS-GT for download.cdn.mozilla.net in three different contexts. In (a) and (b), the context is either absent or benign, and the classification score is close to 0 (benign), while in (c), the context is suspicious, potentially indicating spoofing or other malicious activities, and the classification score is close to 1 (malicious).
Network intrusion detection identifies malicious or abnormal network activity to help protect computers, connected devices, and critical infrastructure.
Julius-Maximilians-Universität Würzburg Würzburg, Germany · Data Science Chair Center for Artificial Intelligence and Data Science Julius-Maximilians-Universität Würzburg Würzburg, Germany
Royal Military College of Canada, Kingston, Canada · Department of Electrical and Computer Engineering, Faculty of Engineering, Royal Military College of Canada (RMC), Kingston, Ontario, K7K 7B4, Canada.