Comparative Evaluation of Static Embedding Models for HTTP Request Anomaly Detection
Organizations: Tilsor SA · Departamento de Informática e Inteligencia Artificial, Universidad Católica del Uruguay
Abstract
Web applications are increasingly targeted by cyberattacks that exploit HTTP requests to evade security mechanisms. Traditional web application firewalls (WAFs) rely on rule-based approaches that often exhibit high false positive rates and limited adaptability. Recent studies have explored machine learning techniques and word embedding models to improve anomaly detection in HTTP traffic. This paper presents a benchmark for static embedding models, specifically Word2Vec, FastText, and Doc2Vec, within a unified, single-class classification framework. We propose HEDA (HTTP Embedding-Based Detection Architecture), a modular detection pipeline that combines static embedding representations with single-class anomaly detection models to detect anomalies at the request level. The approach operates in an unsupervised environment, where both the embedding models and detectors are trained exclusively on benign HTTP traffic. The proposed methodology is evaluated on three datasets with heterogeneous characteristics, including both synthetic and real traffic. The experimental results show that the choice of embedding representation significantly affects detection performance, and that FastText-based embeds produce the most consistent results across all datasets, achieving high detection rates while keeping false positive rates under control.
Figures & tables
| Author(s) | Embeddings Compared | Task / Domain | Main Findings |
|---|---|---|---|
| Bulbul et al. [ 6 ] | Word2Vec, FastText, GloVe | Bengali Language, Analogy Tasks | Skip-gram outperforms CBOW; FastText is best for syntax; training methodology and hyperparameters significantly impact embedding quality. |
| Pak et al. [ 7 ] | Word2Vec, FastText, GloVe, BERT, GPT | General NLP | Survey and taxonomy; contextual embeddings outperform static ones in polysemy and semantics; highlights evaluation practices. |
| Vianna et al. [ 8 ] | Word2Vec, FastText, GloVe | Portuguese Tweets, Sentiment Analysis | FastText yields best performance on short, noisy text; language-specific resources and preprocessing are crucial. |
| Siagh et al. [ 9 ] | Word2Vec, FastText, GloVe, BERT | Social Media, Education (Sentiment Classification) | Static embeddings perform better on preprocessed data; BERT excels on raw input; embedding effectiveness depends on input quality. |
| Dharma et al. [ 10 ] | Word2Vec, FastText, GloVe | News Classification (CNN) | FastText achieves the highest accuracy (97.2%); OOV handling via subwords is advantageous. |
| Kumar et al. [ 11 ] | Word2Vec, FastText, GloVe (Ensemble) | Product Reviews, Sentiment Analysis | Vector ensemble enhances semantic robustness; applied to Amazon reviews with SVM classifier. |
| Author(s) | Embedding | Feature Representation | Classifier / Model | Dataset(s) | Main Contribution |
| Aharon et al. [ 14 ] | FastText (custom) | API token embeddings | ANN retrieval (few-shot) | CSIC 2010, ATRDF 2023 | Unsupervised anomaly detection using custom FastText and approximate nearest neighbors. |
| Yamada & Kawahara [ 15 ] | FastText | Token embeddings from HTTP | CNN Autoencoder | CSIC 2010 | Demonstrated that FastText outperforms ASCII input in the WAF context. |
| S. Amutha et al. [ 16 ] | Doc2Vec | Document-level vectors | RF, KNN | Synthetic XSS Dataset | Effective semantic feature extraction for XSS with explainability. |
| Wang et al. [ 17 ] | Word2Vec | Averaged sentence embeddings | Logistic Regression, XGBoost | Kaggle SQLi dataset | Semantic-aware detection of SQLi using classical ML. |
| Montes et al. [ 5 ] | RoBERTa | Contextual embeddings | SVM | CSIC 2010, DRUPAL, PKDD | Combines semantic embeddings with term-weighting. |
| Khalil et al. [ 18 ] | Doc2Vec, RoBERTa | Request-level embeddings | SVM | CSIC 2010 | Proposed HTTP2Vec for embedding complete HTTP requests. |
| Split | Class | DRUPAL | CSIC 2010 | SR-BH 2020 | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Req | Words | Uniq | Req | Words | Uniq | Req | Words | Uniq | ||
| Train | Valid | 45 609 | 6 425 269 | 13 972 | 36 000 | 7 117 978 | 58 055 | 367 547 | 43 702 525 | 12 560 |
| Test | Valid | 19 533 | 2 745 495 | 8 777 | 36 000 | 7 118 738 | 58 744 | 157 648 | 18 752 952 | 7 146 |
| Attacks | 325 | 44 884 | 573 | 25 065 | 5 498 727 | 40 821 | 114 697 | 19 045 348 | 35 879 | |
| DRUPAL | |||||
|---|---|---|---|---|---|
| Embedding | FPR (%) | TPR (%) | AUC | ||
| Word2Vec | 100 | 3 | 0.35 | 70.26 | 0.82 |
| 5 | 0.40 | 71.40 | 0.85 | ||
| 7 | 0.36 | 71.40 | 0.88 | ||
| 150 | 3 | 0.27 | 68.37 | 0.83 | |
| 5 | 0.36 | 70.45 | 0.85 | ||
| Metric (vocabulary=16,488) | Doc2Vec | FastText | Word2Vec |
|---|---|---|---|
| Total token coverage (test) | 99.0% | 100.0% | 99.0% |
| Direct vocabulary coverage | 99.0% | 99.0% | 99.0% |
| Coverage via n-grams | 0.0% | 1.0% | 0.0% |
| OOV tokens without vector | 1.0% | 0.0% | 1.0% |
| Requests with representation | 100.0% | 100.0% | 100.0% |
| Fully covered requests | 83.1% | 100.0% | 83.1% |
| Embedding | DRUPAL | SR-BH 2020 | CSIC | ||||||
|---|---|---|---|---|---|---|---|---|---|
| FPR (%) | TPR (%) | AUC | FPR (%) | TPR (%) | AUC | FPR (%) | TPR (%) | AUC | |
| Word2Vec | 0.36 | 71.40 | 0.88 | 5.1 | 85.95 | 0.82 | 41.66 | 91.43 | 0.74 |
| FastText | 0.79 | 80.11 | 0.96 | 3.4 | 90.90 | 0.92 | 3.07 | 62.99 | 0.79 |
| Doc2Vec | 0.39 | 55.68 | 0.87 | 24.57 | 87.22 | 0.76 | 16.70 | 72.39 | 0.82 |
| Embedding | Algorithm | DRUPAL | SR-BH 2020 | CSIC | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| TPR (%) | FPR (%) | AUC | TPR (%) | FPR (%) | AUC | TPR (%) | FPR (%) | AUC | ||
| FastText | OCSVM | 80.11 | 0.79 | 0.96 | 90.09 | 3.43 | 0.92 | 62.00 | 3.07 | 0.79 |
| LOF | 12.31 | 2.28 | 0.87 | 91.02 | 11.09 | 0.94 | 86.94 | 3.69 | 0.96 | |
| IF | 39.01 | 0.40 | 0.83 | 76.53 | 25.05 | 0.84 | 62.52 | 17.38 | 0.81 | |
| Word2Vec | OCSVM | 71.40 | 0.36 | 0.88 | 85.95 | 5.10 | 0.88 | 91.43 | 41.66 | 0.74 |
| LOF | 7.38 | 2.97 | 0.86 | 91.97 | 11.53 | 0.93 | 60.39 | 0.11 | 0.67 | |
| DRUPAL | ||
|---|---|---|
| Approach | TPR (%) | FPR (%) |
| ModSecurity OWASP CRS v3 - PL1 | 29.55 | 15.57 |
| ModSecurity OWASP CRS v3 - PL2 | 77.89 | 49.93 |
| Expert-Selected features-OCSVM [ 4 ] | 94.40 | 6.00 |
| Automatic-Selected features-OCSVM [ 24 ] | 91.76 | 2.29 |
| FastText-OCSVM | 80.11 | 0.79 |