A Large-Scale Benchmark and Risk Assessment of Traffic Analysis Attacks on Cloud LLM Services
Authors: Shahrooz Pouryousef, Jesus Lopez, Saeefa Rubaiyat Nowmi, Md Mahmuduzzaman Kamol, Moinul Hossain, Muoi Tran, Mohammad Saidur Rahman
Organizations: Chalmers University of Technology, Sweden · University of Texas at El Paso, USA · Old Dominion University, USA · George Mason University, USA
Cloud-based Large language model (LLM) services create a network-level traffic side channel that can expose model, prompt, and task behavior despite encryption. From packet sizes, directions, timing, and burst structure alone, a passive local observer can infer the serving model, the user's prompt category, and the task executed by a collaborative multi-agent system. Yet current evidence is fragmented across separate datasets and settings, limiting reproducibility and comparison. We present, to our knowledge, the first unified measurement study and public benchmark of encrypted LLM traffic across both user--LLM and multi-agent executions. The large-scale benchmark contains 60,000 user--LLM interactions across 10 models and 6 prompt categories, plus 2,838 multi-agent executions covering 10 task categories and two coordination topologies. Using only encrypted packet metadata, we assess the risk of traffic analysis attack by characterizing traffic signatures, identifying the features most associated with leakage, and testing robustness under prompt reformulation, decoding-temperature changes, larger candidate model sets, and partial traffic observation. Model fingerprinting achieves 97.7% balanced accuracy, prompt-category fingerprinting reaches 76.7% mean accuracy, and multi-agent task fingerprinting achieves up to 90.7% accuracy. Prompt reformulation weakens but does not remove model-specific leakage, and task fingerprints remain detectable even from a single agent's traffic.
Figures & tables
Fig. 1 : Figure 1 (a) illustrates the direct user–LLM setting. Figure 1 (b) illustrates the collaborative multi-agent setting, where the adversary observes encrypted agent–LLM traffic but not agent-to-agent communication.
Fig. 2 : Workflow of the traffic-analysis attack. The adversary trains a fingerprint model via labeled encrypted traffic traces and applies it to observed encrypted traffic to infer the serving model, prompt category, or collaborative task.
Fig. 3 : Encrypted user–LLM traffic properties. (a) Response-packet inter-arrival timing across prompt categories for a fixed LLM. Each point represents one prompt; boxes show medians and interquartile ranges. (b) Temporal response-packet signatures of 10 LLMs for a fixed prompt. Each line is the execution closest to the model’s median behavior over 100 trials. Payload-bearing LLM-to-user packets are aggregated into 1 s intervals.
Fig. 4 : Burst-level traffic representation. Consecutive packets transmitted in the same direction form a burst.
Feature family
No. features
Family only (%)
Without family (%)
All features
168
95.67±0.70
–
Burst
75
94.17±1.56
86.33±2.67
Timing
28
66.50±1.09
95.67±1.49
Packet size
21
87.83±1.62
95.83±1.32
Ordering
22
48.00±5.67
95.83±0.83
Volume/direction
10
30.17±4.87
96.33±1.12
TABLE I : Feature-family analysis for LLM-model fingerprinting using one repetition per model–prompt pair. Values are percentages (mean ± std dev across five prompt-grouped folds).
Features
No.
Only (%)
Without (%)
Drop (pp)
Traffic-view sanity check
All traffic
168
65.10±7.93
–
–
Request only
16
51.68±0.81
–
–
Response only
22
67.37±11.24
–
–
Response-only feature families
Volume
2
37.12±7.08
67.32±11.37
0.05
TABLE II : Prompt-category fingerprinting using different traffic views and response-only feature families. “Only” uses only the indicated response-feature family, whereas “Without” removes it from the complete 22-feature response representation. Values are mean accuracy across the ten LLM-specific classifiers.
Category
Precision
Recall
F1
Main confusion
C1
75.65%
79.91%
77.72%
C4 (7.92%)
C2
58.31%
52.82%
55.43%
C3 (16.21%)
C3
66.74%
67.10%
66.92%
C2 (15.07%)
C4
69.07%
73.15%
71.05%
C6 (9.14%)
C5
66.08%
59.57%
62.66%
C6 (12.11%)
C6
66.37%
71.15%
68.68%
C4 (8.72%)
TABLE III : Per-category performance using response-only traffic and all repetitions. Metrics are computed from pooled out-of-fold predictions. The final column reports the most frequent wrong prediction as a percentage of the true category’s traces.
Fig. 5 : t-SNE visualization of encrypted user–LLM traffic. Panel (a) compares ten LLMs executing the same prompt using all 168 features. Panel (b) compares six prompt categories for a fixed LLM using all 22 response-side features. Each point represents one traffic trace; colors denote models or prompt categories.
Fig. 6 : Response-side model leakage and fingerprinting performance. Families are ordered by estimated leakage, with F1–F9 defined in the legend. (a) Response-family leakage I(X;M) ; the dashed line marks the maximum possible leakage H(M)=3.32 bits. (b) Fano-derived accuracy upper bound and held-out Random Forest test accuracy. Error bars denote 95% confidence intervals across five prompt-grouped folds.
Rank
Graph
Star
1
Outgoing burst bytes, Q75
Outgoing burst bytes, Q75
2
Outgoing packet size, std.
Burst bytes, median
3
First-30 outgoing bytes
Outgoing packet size, mean
4
Incoming burst bytes, median
Incoming burst bytes, median
5
Burst bytes, Q75
Outgoing packet size, Q75
6
Outgoing burst bytes, std.
Outgoing packet size, std.
TABLE IV : Top-ranked task-fingerprinting features. The 75th percentile is denoted by Q75 ; outgoing and incoming indicate agent-to-LLM and LLM-to-agent traffic.
Fig. 7 : Multi-agent task-fingerprinting features. (a) Importance of the 10 highest-ranked features under graph and star topologies. (b) Joint t-SNE using the 100 highest-ranked workflow-only features. Colors denote task categories; marker shapes denote topology.
Fig. 8 : Multi-agent task fingerprinting under topology changes and partial traffic visibility. (a) Within-topology and cross-topology accuracy as the number of traffic features increases. (b) Accuracy when the attacker observes traffic from up to k agents. Error bars summarize variation across feature subsets and agent selections, respectively. The dashed line denotes the 10% random-guessing baseline.
Fig. 9 : (a) Accuracy reduction from prompt reformulation and decoding-temperature randomization relative to the 93.2% baseline across four prompt-grouped folds. (b) Fingerprinting accuracy for 5 – 20 LLMs using 168 engineered features vs. a 35 -feature baseline.
Large language models (LLMs) are increasingly deployed through hosted APIs, making model extraction a practical threat to model ownership and service security. However, individual extraction queries often resemble benign requests, and existing evaluations often focus on single-query anomaly scoring or pure benign-versus-attacker user settings. We formulate model extraction monitoring as benign-calibrated traffic-window distribution testing and show that an embarrassingly simple detector is effective: embed incoming queries into a semantic space and test whether their aggregate distribution deviates from historical benign traffic. We instantiate the detector with maximum mean discrepancy (MMD), using only benign-vs-benign comparisons to set the decision threshold. We evaluate on fourteen attacker-normal query pairs from four extraction scenarios and compare with adapted PRADA, SEAT, CAP, DATE, and marginal Mahalanobis baselines. Across three random seeds, MMD achieves 0.3% benign FPR, 100.0% pure-attacker TPR, 90.5% average TPR over attacker fractions, and 95.1% balanced accuracy. These results show that benign-calibrated distribution testing is a strong empirical baseline for model extraction detection in both user-level and mixed multi-user LLM API traffic. Code is released at: https://github.com/LabRAI/mmd-llm-mea-detection.
We introduce a reusable framework for auditing whether LLM attack benchmarks collectively cover the threat surface: a 4×6 Target × Technique matrix grounded in STRIDE, constructed from a 507-leaf taxonomy -- 401 data-populated and 106 threat-model-derived leaves -- of inference-time attacks extracted from 932 arXiv security studies (2023--2026). The matrix enables benchmark-external validation -- auditing collective coverage rather than individual benchmark consistency. Applying it to six public benchmarks reveals that the three primary frameworks (HarmBench, InjecAgent, AgentDojo) occupy non-overlapping cells covering at most 25% of the matrix, while entire STRIDE threat categories (Service Disruption, Model Internals) lack any standardized evaluation, despite published attacks in these categories achieving 46× token amplification and 96% attack success rates through mechanisms which no benchmark tests. The corpus of 2,521 unique attack groups further reveals pervasive naming fragmentation (up to 29 surface forms for a single attack) and heavy concentration in Safety & Alignment Bypass, structural properties invisible at smaller scale. The taxonomy, attack records, and coverage mappings are released as extensible artifacts; as new benchmarks emerge, they can be mapped onto the same matrix, enabling the community to track whether evaluation gaps are closing.
Karthik Raghu Iyer, Yazdan Jamshidi, Nicholas Bray +1
Published evaluations of prompt-injection and jailbreak detectors for Large Language Models often suffer from two systematic weaknesses: per-dataset threshold tuning and undisclosed operating points. We describe an evaluation harness that addresses both. The detector under evaluation is scored across 16 public benchmarks (12,111 samples) using 5-fold cross-validation. StratifiedKFold (by row) is the headline pass; a parallel StratifiedGroupKFold pass over a composite key (parent-prompt id plus MinHash + LSH near-duplicate clusters at Jaccard ≳0.8) runs alongside it as a leakage-premium diagnostic. A single global operating point is selected on the held-out folds (max F1 subject to FPR ≤1%) and applied uniformly to every dataset, so per-dataset results reflect one threshold rather than per-benchmark optimisation. Generalisation is examined through a battery of diagnostics (leave-one-dataset-out cross-validation, a random-label control, adversarial validation, permutation feature importance, length-bias correlation, classifier-head agreement, cross-source near-duplicate detection, threshold transferability, train-vs-OOF agreement, and a paraphrase-invariance probe), most with a quantitative pass threshold and the remainder with a stated failure mode. For every external comparison, the detector's threshold is re-tuned to the competitor's published false-positive rate so head-to-head values are evaluated at matched operating points.