Rethinking Latency Denial-of-Service: Attacking the LLM Serving Framework, Not the Model

Tianyi Wang; Huawei Fan; Yuanchao Shu; Peng Cheng; Cong Wang

cs.CRFeb 8, 2026

Rethinking Latency Denial-of-Service: Attacking the LLM Serving Framework, Not the Model

Authors: Tianyi Wang, Huawei Fan, Yuanchao Shu, Peng Cheng, Cong Wang

Organizations: Zhejiang University

Abstract

LLM inference is inherently expensive, even a modest slowdown can translate into substantial operating costs and severe availability risks. Recently, a growing body of research known as latency attacks focuses on crafting inputs to trigger worst-case output lengths. However, we report a contrary finding that these algorithmic-level latency attacks are largely ineffective against modern LLM serving systems. We reveal that system-level optimization such as continuous batching provides a logical isolation to mitigate contagious latency impact on co-located users. Thus, in this paper, we shift our focus from the algorithm to the system layer, and introduce a new Fill and Squeeze attack strategy targeting the state transition of the scheduler. Fill'' first exhausts the global KV cache to induce Head-of-Line blocking, while Squeeze'' forces the system into repetitive preemption. By manipulating output lengths using different attack prompts, and leveraging side-channel probing of memory status, we demonstrate that the attack can succeed in a practical black-box setting with much less cost. Extensive evaluations on vLLM indicate up to $75-742\times$ TTFT degradation relative to benign baselines and $1.5-4\times$ average slowdown on Time Per Output Token compared to existing attacks with 30-40% lower attack cost. Code: https://github.com/Phil-Fan/FS-attack

Figures & tables

Figure 1 : Impact of latency attacks on normal users.

Figure 2 : CUDA kernel execution under the latency attack.

Figure 3 : Overview of F&S attack steps and scheduling states, in which a key is the transition between the FCFS-based Waiting Queue and the continuously-batched Running Queue .

Panel of Figure 4 — Figure 4: Black-box estimation of KV usage on Qwen3-8B. (a) Similar latency imposed on all users due to shared memory contention; (b) strong linear correlation between ITL and global KV-cache usage ( $U_{\mathrm{sys}}$ ); (c) trace of KV-cache estimation accuracy using LightGBM.

Figure 5 : Visualization of TTFT, ITL, Global KV-Cache usage and queue size of F&S attack over time. The Waiting Queue Size (bottom) accumulates as opposed to the Running Queue, which directly correlates with the massive outliers observed in the TTFT Scatter Plot (top right), where user requests are blocked for tens of seconds before processing begins, confirming that the TTFT surge stems from HOL blocking.

Target Model	Attack Method	TTFT (↑)	TTFT P99 (↑)	TPOT (↑)	TPOT P99 (↑)	Preempt#	Attack Request# (↓)	Cost ($) (↓)
Qwen3-8B	Benign (No Attack)	0.15	0.27	34.78	49.66	0	0	—
	Engorgio	0.12	0.24	48.17	59.51	0	4613	1.18
	LoopLLM	1.80	58.05	114.12	1249.20	1104	295	0.80
	ExtendAttack	0.85	11.90	80.03	474.01	538	529	0.92
	F&S+plain-text	11.05	400.13	194.45	2159.74	618	180	0.75
	F&S+ExtendAttack	35.95	408.86	181.93	991.19	524	73	0.42

Table 1 : Comparison of attack effectiveness. ↑ and ↓ denote the higher/lower the better, respectively. TTFT/TTFT P99 measured in seconds ( s ), TPOT/TPOT P99 measured in milliseconds ( ms ). Each cell is the mean over five independent runs.

Model	Workload	TTFT	$\Delta$ T	TPOT	$\Delta$ P
Qwen3-8B	Alpaca	11.05	75.6 $\times$	194	5.6 $\times$
Qwen3-8B	ShareGPT	2.57	41.0 $\times$	254	7.3 $\times$
Gemma3-12B-it	Alpaca	97.83	742.8 $\times$	244	3.6 $\times$
Gemma3-12B-it	ShareGPT	65.89	720.6 $\times$	359	5.3 $\times$
DeepSeek-R1- D-Llama-8B	Alpaca	5.31	68.2 $\times$	90	2.4 $\times$
DeepSeek-R1- D-Llama-8B	ShareGPT	5.28	67.8 $\times$	80	2.1 $\times$

Table 2 : Workload sensitivity of F&S attack strategy.

\Delta

indicates degradation ratio. TTFT/TPOT in s/ms.

Figure 6 : Impact of malicious user rate on attack efficacy and cost across different model architectures.

Figure 7 : Attack impact on normal users under real-world traffic patterns. Left: Hourly request distribution from BurstGPT traces with three representative traffic intensities. Right: TTFT and TPOT distributions for normal users under varying malicious ratios.

Panel of Figure 8 — Figure 8 : Multi-GPU scalability analysis under varying hardware resources and load configurations.

Model	Method	TTFT	TPOT	Cost($)
Qwen3-8B	Engorgio	0.10	45.9	1.23
	ExtendAttack	20.11	191.0	0.75
	LoopLLM	2.11	64.7	0.95
	F&S+plain-text	21.32	135.2	0.78
	F&S+ExtendAttack	27.65	196.99	0.68
DeepSeek*	Engorgio	0.10	50.19	1.13

Table 3 : Attack effectiveness and cost analysis under reasoning.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 9 — Figure 9 : Attack cost and effectiveness. (a) Monetary cost (USD) vs. computational time (seconds). Direct Generation (green star) incurs minimal overhead, whereas optimization-based methods show log-linear cost scaling. (b) Distribution of output token lengths. Optimization methods (e.g., AutoDoS ) produce significantly longer sequences than the concise outputs of direct heuristics.

Figure 10 : Hierarchical design of adversarial prompt construction by categorizing prompts into a tiered set ( $\mathcal{P}_{High},\mathcal{P}_{Mid},\mathcal{P}_{Low}$ ).

Figure 11 : Illustration of performance metrics (TTFT, TPOT, ITL).

Package	Version	Package	Version
Python	3.11.0	Transformers	4.57.6
PyTorch	2.9.0	vLLM	0.11.2
TorchVision	0.24.0	xFormers	0.0.33.post1
Torchaudio	2.9.0	LightGBM	4.6.0
CUDA	13.0	Scikit-learn	1.7.2
Accelerate	1.12.0	NumPy	1.26.4

Appendix

Table 4 : Key Software Dependencies

Setting	TTFT (s)	TTFT P99 (s)	TPOT (ms)	TPOT P99 (ms)	Preempt#	Req#	Cost ($)
Full F&S	12.124	403.282	205.90	2892.10	268	205	0.74
Fill-only	3.572	92.553	111.39	1122.38	854	508	0.81
Squeeze-only	0.095	0.139	48.94	58.93	0	3462	1.19
No adaptive back-off	0.803	24.608	78.75	322.67	583	448	0.94
Non-tiered adaptive	0.272	0.921	73.39	246.61	384	510	0.99

Appendix

Table 5 : Component ablation of F&S on Qwen3-8B, five-run averages. Setting definitions in text.

F&S mechanism	vLLM (v0.11.2)	SGLang (commit 99c0b62 )
Admission gate	can_allocate()	add_one_req() $\to$ budget_state()
HOL-blocking trigger	Free blocks $<M_{\text{req}}$	rem_total_tokens $\leq 0$
Budget type	Block count (reactive)	Token count (forward-looking)
Budget reservation	None	$+$ CLIP_MAX_NEW_TOKENS $\,{=}\,4096$ per req
Default queue order	FCFS	LPM (reverts to FCFS at queue ${>}\,128$ )
Long-output attack amplification	KV growth over time	KV growth plus immediate budget inflation

Appendix

Table 6 : Mapping of F&S mechanisms between vLLM and SGLang. Source: vLLM v0.11.2 BlockSpaceManager ; SGLang commit 99c0b62 schedule_policy.py / scheduler.py .

Target Model	Attack Method	TTFT (↑)	TPOT (↑)	Attack Request# (↓)	Cost ($) (↓)
Qwen3-8B	Engorgio	0.09	48.40	4487	3.88
	LoopLLM	12.65	55.40	607	3.01
	ExtendAttack	6.74	57.55	422	3.31
	F&S+plain-text	42.40	56.44	202	2.71
Gemma3-12B-it	Engorgio	201.03	65.88	317	0.85
	LoopLLM	267.30	70.26	262	0.99

Appendix

Table 7: Comparison of attack effectiveness on SGLang under attack settings. ↑ and ↓ denote the higher/lower the better, respectively. TTFT is measured in seconds ( s ), and TPOT is measured in milliseconds ( ms ).

Figure 12 : Scheduler dynamics of SGLang under different attacks. From top to bottom, the panels show request events, queue sizes, per-traffic queue composition, and KV-cache usage over time. Gray regions indicate KV usage above the

97.5\%

threshold. F&S induces sustained KV pressure, oscillating queues, and repeated saturation–recovery behavior, illustrating Fill-induced HOL blocking and Squeeze-induced recovery amplification.

Figure 13 : Temporal dynamics of ITL and KV-cache usage under single-user load (Concurrency=1). The blue trace represents per-token ITL, while the red trace indicates KV-cache utilization. Both metrics exhibit synchronized periodic patterns, suggesting a strong temporal correlation.

Panel of Figure 14 — Figure 14 : KV-ITL correlation analysis. (a) Scatter plot with linear regression showing positive correlation between KV-cache usage and ITL. (b) Region-wise breakdown reveals that global correlation is driven primarily by cross-region variations rather than intra-region dynamics.

Concurrency	$R^{2}$	ITL (ms)	KV (%)
C = 1	0.649	7 – 18	0 – 1.14
C = 2	0.848	7 – 22	0 – 2.29
C = 4	0.958	5 – 45	0 – 4.57
C = 8	0.978	8 – 80	0 – 9.13
C = 16	0.980	8 – 65	0 – 9.13

Appendix

Table 8 : Linear correlation (

R^{2}

) between KV-cache usage and ITL under different concurrency levels.

Figure 15 : KV-ITL correlation under different system load configurations (Concurrency $=2,4,8,16$ ). Each subplot shows the scatter plot with linear regression fit. The correlation strengthens significantly as concurrency increases, with $R^{2}$ exceeding $0.95$ for $C\geq$ 4.

Model	Acc.	F1	Train (s)	Infer (s)
LightGBM	0.87	0.88	6.94	0.01
tsai-LSTM	0.80	0.79	83.25	0.55
tsai-GRU	0.80	0.79	83.25	0.55
tsai-Transformer	0.75	0.75	158.91	0.56

Appendix

Table 9 : Qwen3-8B classification performance comparison on 10-class dataset

Setting	Accuracy	Macro-F1
Controlled L40S, Qwen3-8B	0.870	0.880
Production H20, single replica	0.771	0.765

Appendix

Table 10 : Single-replica prober: controlled L40S bench vs. single-replica production H20 deployment. Each setting trains and evaluates on its own aligned (ITL, KV-usage) data.

Stage	Requests (completed / dispatched)	Observed KV usage	Windows
Low	320 / 320	0.19–1.35%	4,031
Mid	637 / 640	2.43–34.73%	153,328
High	950 / 960	2.96–62.58%	179,108
Total	1,907 / 1,920	0.19–62.58%	336,467

Appendix

Table 11 : Production round-robin collection: request counts, observed KV usage, and aligned window counts per load stage.

Test set	Windows	Accuracy	Macro-F1
Single-replica in-domain reference	15,355	0.771	0.765
Two-replica round-robin	336,467	0.681	0.646

Appendix

Table 12 : Zero-shot transfer from the single-replica prober to the two-replica round-robin test set of Table 11 . No target-specific recalibration.

Stage	Chunks scanned	Max ITL	$>$ 200 ms	$>$ 500 ms	$>$ 1000 ms
Low	120,119	65.5 ms	0	0	0
Mid	3,505,123	410.9 ms	2,954	0	0
High	3,955,238	1,332.9 ms	828	222	56

Appendix

Table 13 : Client-observed stalls by load stage. Threshold columns count streaming chunks whose ITL exceeds the header value (in milliseconds).

Method	TTFT	TPOT	Preempt	Req.#	Cost
	(ms)	(ms)	(#)	(#)	($)
Benign (No Attack)	144.16	32.28	0	0	—
Engorgio	122.28	39.22	0	5576	1.25
LoopLLM	137.16	57.53	86	2489	4.16
ExtendAttack	131.05	45.10	0	1850	2.10
F&S+plain-text	126.68	54.35	27	1006	3.73

Appendix

Table 14 : Attack effectiveness on Llama-3.1-8B-Instruct. Unlike other models, Llama-3.1 exhibits resilience due to strict generation length constraints.

Explore similar work

Continuous Discovery of Vulnerabilities in LLM Serving Systems with Fuzzing
May 11, 2026Yunze Zhao, Yibo Zhao, Yuchen Zhang +2LLM Serving LLM Security
University of Maryland · New York University
LLMVisor: A Real-Time Latency Attribution Model for Multi-Tenant LLM Serving
Aug 9, 2026Shuowei Jin, Xueshen Liu, Jiaxin Shan +4LLM Serving
University of Michigan · ByteDance
Token Latency Fairness: Performance Isolation for Multi-Tenant LLM Serving
Sep 16, 2026Dev Bali, Soujanya Ponnapalli, Yichuan Wang +3LLM Serving LLM Inference Scheduling
University of California, Berkeley

May 11, 2026, cs.CRJ/K move · Enter open · S save

Continuous Discovery of Vulnerabilities in LLM Serving Systems with Fuzzing

Yunze Zhao, Yibo Zhao, Yuchen Zhang, Zaoxing Liu+1

University of Maryland · New York University

LLM Serving LLM Security

Open paper AskPDF arXiv

LLM inference and serving systems have become security-critical infrastructure; however, many of their most concerning failures arise from the serving layer rather than from model behavior alone. Modern inference engines combine KV cache, batching, prefix sharing, speculative decoding, adapters, and multi-tenant scheduling, creating shared-state behavior that only emerges under realistic concurrent workloads and is missed by standard model, safety, and API tests. We present GRIEF, a greybox fuzzer for LLM inference engines that treats timed multi-request traces as first-class inputs, uses lightweight oracles to detect crashes, hangs, performance pathologies, and silent output corruption, and applies controlled replay with log-probability checks to confirm reproducible serving-layer failures. Across early campaigns on vLLM and SGLang, GRIEF discovers 15 vulnerabilities, 10 confirmed by engine developers, including 2 CVEs, spanning KV-cache isolation failures, cross-request performance interference, and crash or liveness bugs. These results show that concurrency, caching, and state reuse can induce silent cross-request contamination, noisy-neighbor denial of service, and delayed crashes without malformed inputs or explicit server errors, making concurrent serving behavior a first-class security and reliability boundary for LLM infrastructure.

Abstract

Explore similar work

Continuous Discovery of Vulnerabilities in LLM Serving Systems with Fuzzing

LLMVisor: A Real-Time Latency Attribution Model for Multi-Tenant LLM Serving

Token Latency Fairness: Performance Isolation for Multi-Tenant LLM Serving