Rethinking Latency Denial-of-Service: Attacking the LLM Serving Framework, Not the Model

Tianyi Wang; Huawei Fan; Yuanchao Shu; Peng Cheng; Cong Wang

cs.CRFeb 8, 2026

Rethinking Latency Denial-of-Service: Attacking the LLM Serving Framework, Not the Model

Authors: Tianyi Wang, Huawei Fan, Yuanchao Shu, Peng Cheng, Cong Wang

Organizations: Zhejiang University

Abstract

LLM inference is inherently expensive, even a modest slowdown can translate into substantial operating costs and severe availability risks. Recently, a growing body of research known as latency attacks focuses on crafting inputs to trigger worst-case output lengths. However, we report a contrary finding that these algorithmic-level latency attacks are largely ineffective against modern LLM serving systems. We reveal that system-level optimization such as continuous batching provides a logical isolation to mitigate contagious latency impact on co-located users. Thus, in this paper, we shift our focus from the algorithm to the system layer, and introduce a new Fill and Squeeze attack strategy targeting the state transition of the scheduler. Fill'' first exhausts the global KV cache to induce Head-of-Line blocking, while Squeeze'' forces the system into repetitive preemption. By manipulating output lengths using different attack prompts, and leveraging side-channel probing of memory status, we demonstrate that the attack can succeed in a practical black-box setting with much less cost. Extensive evaluations on vLLM indicate up to $75-742\times$ TTFT degradation relative to benign baselines and $1.5-4\times$ average slowdown on Time Per Output Token compared to existing attacks with 30-40% lower attack cost. Code: https://github.com/Phil-Fan/FS-attack

Figures & tables

Figure 1 : Impact of latency attacks on normal users.

Figure 2 : CUDA kernel execution under the latency attack.

Figure 3 : Overview of F&S attack steps and scheduling states, in which a key is the transition between the FCFS-based Waiting Queue and the continuously-batched Running Queue .

Panel of Figure 4 — Figure 4: Black-box estimation of KV usage on Qwen3-8B. (a) Similar latency imposed on all users due to shared memory contention; (b) strong linear correlation between ITL and global KV-cache usage ( $U_{\mathrm{sys}}$ ); (c) trace of KV-cache estimation accuracy using LightGBM.

Figure 5 : Visualization of TTFT, ITL, Global KV-Cache usage and queue size of F&S attack over time. The Waiting Queue Size (bottom) accumulates as opposed to the Running Queue, which directly correlates with the massive outliers observed in the TTFT Scatter Plot (top right), where user requests are blocked for tens of seconds before processing begins, confirming that the TTFT surge stems from HOL blocking.

Target Model	Attack Method	TTFT (↑)	TTFT P99 (↑)	TPOT (↑)	TPOT P99 (↑)	Preempt#	Attack Request# (↓)	Cost ($) (↓)
Qwen3-8B	Benign (No Attack)	0.15	0.27	34.78	49.66	0	0	—
	Engorgio	0.12	0.24	48.17	59.51	0	4613	1.18
	LoopLLM	1.80	58.05	114.12	1249.20	1104	295	0.80
	ExtendAttack	0.85	11.90	80.03	474.01	538	529	0.92
	F&S+plain-text	11.05	400.13	194.45	2159.74	618	180	0.75
	F&S+ExtendAttack	35.95	408.86	181.93	991.19	524	73	0.42

Table 1 : Comparison of attack effectiveness. ↑ and ↓ denote the higher/lower the better, respectively. TTFT/TTFT P99 measured in seconds ( s ), TPOT/TPOT P99 measured in milliseconds ( ms ). Each cell is the mean over five independent runs.

Model	Workload	TTFT	$\Delta$ T	TPOT	$\Delta$ P
Qwen3-8B	Alpaca	11.05	75.6 $\times$	194	5.6 $\times$
Qwen3-8B	ShareGPT	2.57	41.0 $\times$	254	7.3 $\times$
Gemma3-12B-it	Alpaca	97.83	742.8 $\times$	244	3.6 $\times$
Gemma3-12B-it	ShareGPT	65.89	720.6 $\times$	359	5.3 $\times$
DeepSeek-R1- D-Llama-8B	Alpaca	5.31	68.2 $\times$	90	2.4 $\times$
DeepSeek-R1- D-Llama-8B	ShareGPT	5.28	67.8 $\times$	80	2.1 $\times$

Table 2 : Workload sensitivity of F&S attack strategy.

\Delta

indicates degradation ratio. TTFT/TPOT in s/ms.

Figure 6 : Impact of malicious user rate on attack efficacy and cost across different model architectures.

Figure 7 : Attack impact on normal users under real-world traffic patterns. Left: Hourly request distribution from BurstGPT traces with three representative traffic intensities. Right: TTFT and TPOT distributions for normal users under varying malicious ratios.

Panel of Figure 8 — Figure 8 : Multi-GPU scalability analysis under varying hardware resources and load configurations.

Model	Method	TTFT	TPOT	Cost($)
Qwen3-8B	Engorgio	0.10	45.9	1.23
	ExtendAttack	20.11	191.0	0.75
	LoopLLM	2.11	64.7	0.95
	F&S+plain-text	21.32	135.2	0.78
	F&S+ExtendAttack	27.65	196.99	0.68
DeepSeek*	Engorgio	0.10	50.19	1.13

Table 3 : Attack effectiveness and cost analysis under reasoning.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 9 — Figure 9 : Attack cost and effectiveness. (a) Monetary cost (USD) vs. computational time (seconds). Direct Generation (green star) incurs minimal overhead, whereas optimization-based methods show log-linear cost scaling. (b) Distribution of output token lengths. Optimization methods (e.g., AutoDoS ) produce significantly longer sequences than the concise outputs of direct heuristics.

Figure 10 : Hierarchical design of adversarial prompt construction by categorizing prompts into a tiered set ( $\mathcal{P}_{High},\mathcal{P}_{Mid},\mathcal{P}_{Low}$ ).

Figure 11 : Illustration of performance metrics (TTFT, TPOT, ITL).

Package	Version	Package	Version
Python	3.11.0	Transformers	4.57.6
PyTorch	2.9.0	vLLM	0.11.2
TorchVision	0.24.0	xFormers	0.0.33.post1
Torchaudio	2.9.0	LightGBM	4.6.0
CUDA	13.0	Scikit-learn	1.7.2
Accelerate	1.12.0	NumPy	1.26.4

Appendix

Table 4 : Key Software Dependencies

Setting	TTFT (s)	TTFT P99 (s)	TPOT (ms)	TPOT P99 (ms)	Preempt#	Req#	Cost ($)
Full F&S	12.124	403.282	205.90	2892.10	268	205	0.74
Fill-only	3.572	92.553	111.39	1122.38	854	508	0.81
Squeeze-only	0.095	0.139	48.94	58.93	0	3462	1.19
No adaptive back-off	0.803	24.608	78.75	322.67	583	448	0.94
Non-tiered adaptive	0.272	0.921	73.39	246.61	384	510	0.99

Appendix

Table 5 : Component ablation of F&S on Qwen3-8B, five-run averages. Setting definitions in text.

F&S mechanism	vLLM (v0.11.2)	SGLang (commit 99c0b62 )
Admission gate	can_allocate()	add_one_req() $\to$ budget_state()
HOL-blocking trigger	Free blocks $<M_{\text{req}}$	rem_total_tokens $\leq 0$
Budget type	Block count (reactive)	Token count (forward-looking)
Budget reservation	None	$+$ CLIP_MAX_NEW_TOKENS $\,{=}\,4096$ per req
Default queue order	FCFS	LPM (reverts to FCFS at queue ${>}\,128$ )
Long-output attack amplification	KV growth over time	KV growth plus immediate budget inflation

Appendix

Table 6 : Mapping of F&S mechanisms between vLLM and SGLang. Source: vLLM v0.11.2 BlockSpaceManager ; SGLang commit 99c0b62 schedule_policy.py / scheduler.py .

Target Model	Attack Method	TTFT (↑)	TPOT (↑)	Attack Request# (↓)	Cost ($) (↓)
Qwen3-8B	Engorgio	0.09	48.40	4487	3.88
	LoopLLM	12.65	55.40	607	3.01
	ExtendAttack	6.74	57.55	422	3.31
	F&S+plain-text	42.40	56.44	202	2.71
Gemma3-12B-it	Engorgio	201.03	65.88	317	0.85
	LoopLLM	267.30	70.26	262	0.99

Appendix

Table 7: Comparison of attack effectiveness on SGLang under attack settings. ↑ and ↓ denote the higher/lower the better, respectively. TTFT is measured in seconds ( s ), and TPOT is measured in milliseconds ( ms ).

Figure 12 : Scheduler dynamics of SGLang under different attacks. From top to bottom, the panels show request events, queue sizes, per-traffic queue composition, and KV-cache usage over time. Gray regions indicate KV usage above the

97.5\%

threshold. F&S induces sustained KV pressure, oscillating queues, and repeated saturation–recovery behavior, illustrating Fill-induced HOL blocking and Squeeze-induced recovery amplification.

Figure 13 : Temporal dynamics of ITL and KV-cache usage under single-user load (Concurrency=1). The blue trace represents per-token ITL, while the red trace indicates KV-cache utilization. Both metrics exhibit synchronized periodic patterns, suggesting a strong temporal correlation.

Panel of Figure 14 — Figure 14 : KV-ITL correlation analysis. (a) Scatter plot with linear regression showing positive correlation between KV-cache usage and ITL. (b) Region-wise breakdown reveals that global correlation is driven primarily by cross-region variations rather than intra-region dynamics.

Concurrency	$R^{2}$	ITL (ms)	KV (%)
C = 1	0.649	7 – 18	0 – 1.14
C = 2	0.848	7 – 22	0 – 2.29
C = 4	0.958	5 – 45	0 – 4.57
C = 8	0.978	8 – 80	0 – 9.13
C = 16	0.980	8 – 65	0 – 9.13

Appendix

Table 8 : Linear correlation (

R^{2}

) between KV-cache usage and ITL under different concurrency levels.

Figure 15 : KV-ITL correlation under different system load configurations (Concurrency $=2,4,8,16$ ). Each subplot shows the scatter plot with linear regression fit. The correlation strengthens significantly as concurrency increases, with $R^{2}$ exceeding $0.95$ for $C\geq$ 4.

Model	Acc.	F1	Train (s)	Infer (s)
LightGBM	0.87	0.88	6.94	0.01
tsai-LSTM	0.80	0.79	83.25	0.55
tsai-GRU	0.80	0.79	83.25	0.55
tsai-Transformer	0.75	0.75	158.91	0.56

Appendix

Table 9 : Qwen3-8B classification performance comparison on 10-class dataset

Setting	Accuracy	Macro-F1
Controlled L40S, Qwen3-8B	0.870	0.880
Production H20, single replica	0.771	0.765

Appendix

Table 10 : Single-replica prober: controlled L40S bench vs. single-replica production H20 deployment. Each setting trains and evaluates on its own aligned (ITL, KV-usage) data.

Stage	Requests (completed / dispatched)	Observed KV usage	Windows
Low	320 / 320	0.19–1.35%	4,031
Mid	637 / 640	2.43–34.73%	153,328
High	950 / 960	2.96–62.58%	179,108
Total	1,907 / 1,920	0.19–62.58%	336,467

Appendix

Table 11 : Production round-robin collection: request counts, observed KV usage, and aligned window counts per load stage.

Test set	Windows	Accuracy	Macro-F1
Single-replica in-domain reference	15,355	0.771	0.765
Two-replica round-robin	336,467	0.681	0.646

Appendix

Table 12 : Zero-shot transfer from the single-replica prober to the two-replica round-robin test set of Table 11 . No target-specific recalibration.

Stage	Chunks scanned	Max ITL	$>$ 200 ms	$>$ 500 ms	$>$ 1000 ms
Low	120,119	65.5 ms	0	0	0
Mid	3,505,123	410.9 ms	2,954	0	0
High	3,955,238	1,332.9 ms	828	222	56

Appendix

Table 13 : Client-observed stalls by load stage. Threshold columns count streaming chunks whose ITL exceeds the header value (in milliseconds).

Method	TTFT	TPOT	Preempt	Req.#	Cost
	(ms)	(ms)	(#)	(#)	($)
Benign (No Attack)	144.16	32.28	0	0	—
Engorgio	122.28	39.22	0	5576	1.25
LoopLLM	137.16	57.53	86	2489	4.16
ExtendAttack	131.05	45.10	0	1850	2.10
F&S+plain-text	126.68	54.35	27	1006	3.73

Appendix

Table 14 : Attack effectiveness on Llama-3.1-8B-Instruct. Unlike other models, Llama-3.1 exhibits resilience due to strict generation length constraints.

Explore similar work

May 11, 2026cs.CR

Continuous Discovery of Vulnerabilities in LLM Serving Systems with Fuzzing

LLM inference and serving systems have become security-critical infrastructure; however, many of their most concerning failures arise from the serving layer rather than from model behavior alone. Modern inference engines combine KV cache, batching, prefix sharing, speculative decoding, adapters, and multi-tenant scheduling, creating shared-state behavior that only emerges under realistic concurrent workloads and is missed by standard model, safety, and API tests. We present GRIEF, a greybox fuzzer for LLM inference engines that treats timed multi-request traces as first-class inputs, uses lightweight oracles to detect crashes, hangs, performance pathologies, and silent output corruption, and applies controlled replay with log-probability checks to confirm reproducible serving-layer failures. Across early campaigns on vLLM and SGLang, GRIEF discovers 15 vulnerabilities, 10 confirmed by engine developers, including 2 CVEs, spanning KV-cache isolation failures, cross-request performance interference, and crash or liveness bugs. These results show that concurrency, caching, and state reuse can induce silent cross-request contamination, noisy-neighbor denial of service, and delayed crashes without malformed inputs or explicit server errors, making concurrent serving behavior a first-class security and reliability boundary for LLM infrastructure.

Yunze Zhao, Yibo Zhao, Yuchen Zhang +2

University of Maryland · New York University

LLM Serving LLM Security

Ask PDF

Aug 9, 2026cs.AI

LLMVisor: A Real-Time Latency Attribution Model for Multi-Tenant LLM Serving

As LLM inference shifts to multi-tenant GPU clusters, co-batching improves throughput but obscures per-tenant usage and limits control. Enabling fractional sharing of the inference engine requires a real-time, per-request attribution primitive that is accurate and light enough to run inside the scheduling loop. We present LLMVisor, a roofline-guided latency attribution model that captures the memory-bound and compute-bound phases via a concise piecewise-linear form over features proportional to FLOPs and memory I/O traffic. LLMVisor decomposes batch latency into additive, per-request shares and runs efficiently at microsecond scale. We evaluate LLMVisor across Llama 3.1-8B and Qwen 2.5-14B/32B on A100/H100 GPUs under varying tensor parallelism and workload mixes. Compared to a token-count baseline, LLMVisor attains near-perfect R-squared and reduces relative error by up to 2.5x and 3.3x at p90 and p99, respectively, for prefill, and by up to 3.5x and 4.4x for decode, despite batching variability and sequence divergence.

Shuowei Jin, Xueshen Liu, Jiaxin Shan +4

University of Michigan · ByteDance

LLM Serving

Ask PDF

Sep 16, 2026cs.DC

Token Latency Fairness: Performance Isolation for Multi-Tenant LLM Serving

LLM serving is typically offered as a shared, multi-tenant service, where high-demand workloads from one client can cause latency SLO violations for others. Existing solutions for performance isolation equalize client throughput in the long run, for example through queueing and batching fairness. However, these approaches do not provide latency isolation guarantees; as a result, well-behaved clients can still experience significant degradation to their token-level latencies. In this paper, we present FairInference, which provides the novel δ-token fairness guarantee: for a well-behaved client, if a token is generated in d time units in isolation, it will be generated within d + δ time units in multi-tenant execution, providing strong latency isolation guarantees for LLM serving. To achieve this, FairInference addresses a key challenge of LLM serving: bounding delays from sharing GPU resources without support for fine-grained scheduling or resource allocation. In FairInference, the scheduler enforces per-token deadlines, while bounding the delays from GPU compute sharing and accounting for the additional delays introduced by the shared KV caching in GPU memory. We show that FairInference effectively bounds token-level latency spikes for well-behaved clients and improves overall throughput compared to state-of-the-art LLM serving systems.

Dev Bali, Soujanya Ponnapalli, Yichuan Wang +3

University of California, Berkeley

LLM Serving LLM Inference Scheduling

Ask PDF