Characterizing Contention-Induced Reliability Collapse in KV-Cache Timing Side Channels for Multi-Tenant LLM Serving

Rana Abu Bakar

cs.CRSep 6, 2026

Characterizing Contention-Induced Reliability Collapse in KV-Cache Timing Side Channels for Multi-Tenant LLM Serving

Authors: Rana Abu Bakar

Abstract

Shared key--value (KV) cache reuse improves large language model (LLM) serving, but it can also create a timing side channel that reveals whether a prefix is already cached. Previous work shows that such attacks are possible, but their reliability under realistic multi-tenant contention is less understood. We study this problem through seven experiments on live shared LLM-serving systems. On a vLLM server running DeepSeek-R1-Distill-Llama-8B on NVIDIA GB10, mean Cohen's d drops from 0.7789 with no synthetic workers to 0.2109 with two workers (t=8.412), while higher worker counts cause no statistically detectable further loss. A 120-run sparse-overlap experiment places the best breakpoint at the boundary of the measured range (tau=0, 95% CI [0.000,0.113]), supporting an ambient-versus-loaded regime change rather than an internal physical threshold. AUROC falls from 0.650 at ambient to 0.531 near 61% overlap and partially recovers to 0.574 at saturation. Concurrency-depth variance is the strongest measured correlate of effect size (r=-0.416) and hit consistency (r=-0.637). An interleaved control preserves the same non-monotonic ordering. The main collapse is also reproduced on a real two-node, two-GPU tensor-parallel vLLM setup, where mean d falls from 3.418 to 0.511 (p<0.01). Two SGLang pilots are statistically inconclusive. Overall, KV-cache timing reliability depends strongly on the load regime and serving stack, and measurements on quiet systems can overestimate operational attack reliability.

Figures & tables

Study	Runs	Controlled factor	Primary measurement	Main purpose / result
1	10	Ambient vs. 4 workers	$d$ , $H$	Pilot establishes a large first-order loss when sustained synthetic contention is introduced.
2	48	0/2/4/8 workers	$d$ , $H$ , Welch tests	Dose response shows the dominant transition is 0 $\rightarrow$ 2 workers; higher worker counts add no statistically detectable monotonic loss.
3	36	Worker count and output length	$d$ , $H$	Disentangles filler shape and exposes the unusually high variance of one short-output worker.
4	20	Workload shape with interval logging	$O$ , $d$ , $H$	Shows request-overlap fraction rapidly saturates and cannot explain residual loaded-regime variance.
5	120	Sparse measured overlap across $[0,1]$	AUROC, $H$ , $P_{succ}$ , $C(t)$ , breakpoint	Tests the missing transition region, rejects an interior threshold, and identifies concurrency-depth variance as the strongest measured correlate.
6	45	Interleaved 0/40/100% conditions	$d$ vs. round/order	Rechecks the non-monotonic ordering while reducing long-timescale time-of-day confounding.

TABLE I: Experimental progression. Each study addresses a limitation exposed by the previous one rather than repeating the same load sweep.

Condition	Mean $d$ [95% CI]	Std. dev.	Mean $H$ [95% CI]
Ambient	1.126 [0.923, 1.329]	0.269	88.0% [83.2, 92.8]
$+4$ workers	0.213 [0.152, 0.298]	0.096	63.2% [53.6, 72.8]

TABLE II: Study 1: pilot effect-size and hit-consistency summary, with bootstrap 95% CIs (2,000 resamples).

Workers	Mean $d$ [95% CI]	Std. dev.	Mean $H$ [95% CI]
0	0.7789 [0.713, 0.845]	0.1241	79.0% [78.0, 80.2]
2	0.2109 [0.107, 0.318]	0.1982	63.3% [57.7, 68.8]
4	0.1399 [0.016, 0.262]	0.2250	61.0% [53.8, 67.7]
8	0.0820 [ $-$ 0.028, 0.201]	0.2093	58.7% [49.3, 67.7]

TABLE III: Study 2: effect size and hit consistency versus configured worker count, with bootstrap 95% CIs (2,000 resamples,

n=12

per level).

Comparison	$t$	Welch df
0 vs. 2 workers	8.412	18.5
2 vs. 4 workers	0.820	21.7
4 vs. 8 workers	0.652	21.9

TABLE IV: Study 2: Welch tests between consecutive worker levels.

Fig. 1: Study 2: every run plotted as a jittered point against configured worker count, with the condition mean and bootstrap 95% CI in red. The near-total drop occurs between 0 and 2 workers; the 2/4/8-worker means remain within one another’s CIs.

Condition	Workers	Max tok.	Mean $d$ [95% CI]	Std. $d$	Mean $H$ [95% CI]
1w_small16	1	16	0.0568 [ $-$ 0.317, 0.448]	0.7112	57.8% [46.7, 69.2]
1w_big128	1	128	0.0854 [0.023, 0.156]	0.1207	62.3% [57.5, 66.7]
2w_tiny4	2	4	0.1089 [0.068, 0.147]	0.0727	61.8% [60.3, 63.5]

TABLE V: Study 3: independently varied filler configurations, with hit consistency and bootstrap 95% CIs (2,000 resamples,

n=12

per condition).

Condition	Measured synthetic overlap	$d$ range	Mean $d$ [95% CI]	Mean $H$ [95% CI]
ambient	0.0000 in all four runs	0.34–0.79	0.496 [0.364, 0.700]	75.5% [73.0, 78.5]
1w_small16	0.965–0.997	$-0.91$ –0.79	0.124 [ $-$ 0.551, 0.658]	53.5% [36.5, 70.0]
2w_16	0.970–1.000	0.17–0.41	0.281 [0.174, 0.388]	73.5% [69.0, 78.0]
1w_big128	0.760–0.930	$-0.11$ –0.22	0.099 [ $-$ 0.049, 0.202]	69.5% [67.0, 72.5]
2w_tiny4	0.990–1.000	0.12–0.18	0.151 [0.131, 0.168]	67.5% [60.5, 74.0]

TABLE VI: Study 4: measured synthetic request-overlap fraction, effect-size range, and mean effect size / hit consistency with bootstrap 95% CIs (2,000 resamples,

n=4

per condition). “Ambient” means no synthetic workers; uncontrolled production traffic may still be present.

Fig. 2: Study 4: effect size versus measured overlap, labeled by workload shape. The loaded cluster sits at $O\gtrsim 0.76$ regardless of workload, but 1w_small16 (red) spans nearly the full range of observed $d$ , from the single most negative point to among the most positive, despite similar overlap to the other loaded conditions.

Bin	Mean $O$	AUROC	95% CI	$p_{\mathrm{hit}}$	$P_{\mathrm{succ}}(1)$	$P_{\mathrm{succ}}(3)$	$P_{\mathrm{succ}}(5)$	$P_{\mathrm{succ}}(9)$
0%	0.000	0.650	[0.628, 0.675]	0.695	0.695	0.777	0.830	0.895
5%	0.076	0.619	[0.595, 0.644]	0.648	0.648	0.716	0.762	0.825
10%	0.096	0.628	[0.602, 0.652]	0.652	0.652	0.721	0.768	0.832
20%	0.184	0.586	[0.564, 0.611]	0.639	0.639	0.703	0.747	0.809
40%	0.392	0.564	[0.541, 0.587]	0.575	0.575	0.611	0.638	0.678
60%	0.613	0.531	[0.507, 0.556]	0.525	0.525	0.538	0.547	0.562

TABLE VII: Study 5: pooled attack reliability per bin (750 trials per bin, 15 runs

\times

50 trials).

Metric	$r$ with $d$	$r$ with $H$
Measured overlap $O$	$-0.106$	$-0.504$
Mean binary overlap	$-0.122$	$-0.510$
Mean concurrency depth	$+0.039$	$-0.337$
Max concurrency depth	$-0.235$	$-0.419$
Concurrency-depth variance	$\mathbf{-0.416}$	$\mathbf{-0.637}$

TABLE VIII: Study 5: correlation of contention metrics with Cohen’s

d

and hit consistency (n=120 runs).

Fig. 3: Study 5: effect size versus mean concurrency depth (left, $r=+0.039$ ) and concurrency-depth variance (right, $r=-0.416$ , the strongest correlate found in this study). Mean depth clusters into discrete bands set by worker count; variance spreads continuously and tracks Cohen’s $d$ far better.

Fig. 4: Study 5: all 120 runs plotted against measured overlap, with the fitted breakpoint $\hat{\tau}=0$ and the segment-2 fit ( $O>\hat{\tau}$ ). The near-flat, slightly positive segment-2 slope and the breakpoint sitting at the left edge of the range are visually consistent with an ambient-vs-loaded regime change rather than an interior threshold.

Bin	Mean $d$	Std. $d$	corr( $d$ ,round)
0%	$+0.2489$	0.0778	$-0.366$
40%	$+0.0331$	0.0584	$-0.040$
100%	$+0.0695$	0.0642	$-0.317$

TABLE IX: Study 6: interleaved conditions, 15 rounds each. “corr(d,round)” and “slope” test for a within-condition trend across the interleaved rounds, i.e. time-of-day drift isolated from the contention effect.

Bin	Study 5 (sequential) $d$	Study 6 (interleaved) $d$
0%	$0.3007\pm 0.0998$	$0.2489\pm 0.0778$
40%	$0.0177\pm 0.0995$	$0.0331\pm 0.0584$
100%	$0.1424\pm 0.0888$	$0.0695\pm 0.0642$

TABLE X: Study 6 vs. Study 5: same three conditions, sequential-block design (Study 5) versus interleaved design (Study 6).

Fig. 5: Study 5’s pooled AUROC per bin (blue, with 95% CI) versus Study 6’s interleaved mean $p_{\mathrm{hit}}$ for the three re-run conditions (red diamonds). The 40% dip and partial 100% recovery appear in both the sequential and interleaved designs.

Deployment	Framework	GPUs	Ambient $d$	Loaded $d$	Welch $t$	Inference
Primary node, Study 1	vLLM 0.28.0	1	$1.126\pm 0.269$	$0.213\pm 0.096$	–	collapse
Independent physical node	vLLM 0.27.1	1	$\approx 1.03$	–	–	ambient replicated
MPS co-resident pilot	SGLang 0.5.17	1	$0.073\pm 0.104$	$0.190\pm 0.142$	1.50 (NS)	inconclusive
Two-node tensor parallel	SGLang 0.5.17	2	$0.356\pm 0.212$	$0.468\pm 0.173$	0.91 (NS)	inconclusive
Two-node tensor parallel	vLLM 0.26.0	2	$3.418\pm 1.433$	$0.511\pm 0.687$	$\approx 4.09$	collapse, $p<0.01$

TABLE XI: Study 7: compact replication across physical instances, frameworks, and deployment topologies. All loaded pilots use

+4

synthetic workers except the independent-node attempt, for which only the ambient result is retained. “NS” denotes no statistically significant ambient–loaded difference at the pilot sample size.

LLM Security

Large language model (LLM) security studies attacks, privacy risks, and defenses across model training, inference, and deployment.

See all 327 papers

Aug 10, 2026

Governing the KV Cache: Preventing Timing Side-Channel Leakage in Multi-Tenant LLM Inference

The key-value (KV) cache is the primary throughput optimization in modern large language model (LLM) inference, enabling prefix reuse across requests. In multi-tenant deployments this cache is shared across tenants, creating a timing side channel: an adversarial tenant can reconstruct another tenant's private prompt by probing cache-hit latency. Three published attacks exploit it -- PROMPTPEEK, EarlyBird and InputSnatch -- reaching up to 100% attack success rate against unprotected vLLM and SGLang, with rates varying by cache architecture and prompt structure. We present KVGov, a governance layer addressing all three attack families' prefix-cache paths under one mechanism. A per-principal salt sigma_p = HMAC_K(secret, principal_id) seeds the block-hash chain, making cache keys cryptographically disjoint across principals. An ablation (N=1000 trials, seed 2026, deterministic judges) isolates this salt as the necessary and sufficient component. KVGov adds ORIGAMI, a Stackelberg water-filling audit scheduler that reduces adversary expected utility by 12.6% at realistic tenant heterogeneity (Gini 0.63), and an evolutionary stability analysis giving a 31.6% adversary-prevalence tipping point below which global caching remains stable. On real hardware (Qwen2.5-7B-Instruct, vLLM 0.26.0, NVIDIA A100) we measure a gate-verified cold/cached TTFT ratio of 0.22, confirming the channel is exploitable at production scale; the defense itself is evaluated in simulation calibrated to those measurements. We replicate the channel on an independent stack (llama.cpp on Apple Metal, ratio 0.093). Finally, isolation and cache efficiency need not conflict: identifying information resides only where prompts diverge, so injecting the salt at that boundary rather than the chain root retains an estimated 93% of the prefix-cache benefit with no cross-principal signal.

Tejasvi C. Addagada

KV Caching KV-Cache Management Privacy Leakage in Language Models LLM Security KV-Cache Reuse

Ask PDF

Sep 28, 2026

TempoKV: Timely Staging of LLM KV Caches for Memory-Semantic Flash

Reusable prefix key-value (KV) caches can outgrow GPU memory in large language model (LLM) serving. A memory-semantic flash hierarchy offers SSD-backed capacity with a limited fast tier, but a logical KV hit is not necessarily ready for GPU retrieval. Demand staging exposes SSD latency, whereas immediate staging can reserve fast-tier capacity long before retrieval begins. We present TempoKV, a timing-aware resource-commitment layer that separates early knowledge of reuse from the acquisition of staging resources. It records reusable-KV hits as metadata-only claims and requests commitment when the runtime-estimated time until retrieval falls to the storage-estimated time needed to make KV resident and protected against eviction. These estimates adapt to runtime progress and staging state, while commitment remains subject to available protected capacity. We implement TempoKV in vLLM and LMCache on an SSD-backed CXL memory device without changing request scheduling. Across two models and three prefix cache ratios, TempoKV reduces protected fast-tier byte-time per request by 63-91% versus immediate staging while retaining much of the serving benefit of advance staging. In a fast-tier capacity sweep, output throughput and p95 time to first token (TTFT) remain nearly unchanged as capacity decreases from 100 to 25 GiB. Compared with unmodified LMCache's Device-DAX L1 configuration, TempoKV reduces p95 TTFT by up to 48.0% and increases output throughput by up to 27.8%.

Jay H. Park, Hyungjun Kim, Dong Kim

Samsung Semiconductor

KV-Cache Offloading KV-Cache Management LLM Inference Acceleration Prefix Caching

Ask PDF

Jun 30, 2026

From Tensor Buffer to Distributed Memory Hierarchy: A Survey of KV Cache Management for LLM Serving

The key-value (KV) cache has become a first-order memory object in LLM serving rather than a temporary per-request tensor. This survey classifies more than thirty KV-management systems and frameworks using four axes: locality, lifetime, ownership, and substrate. The axes reveal five architectural archetypes -- local-paged, disaggregated-pipeline, shared-store, memory-pool, and hybrid-tier. Once workload and hardware are fixed, ownership accounts for much of the remaining design variance among distributed systems. The survey also audits current evaluations and identifies seven missing KV-specific measurements, linking them to open problems in fault tolerance, isolation, tiered eviction, speculative decoding, MoE serving, and shared-cache semantics.

Jie Li, Tongyang Wang, Yong Chen

Texas Tech University, Department of Computer Science, Lubbock, Texas, USA

LLM Inference Efficiency LLM Serving KV Caching KV-Cache Management Distributed Inference

Ask PDF

Abstract

LLM Security

Explore similar work

Governing the KV Cache: Preventing Timing Side-Channel Leakage in Multi-Tenant LLM Inference

TempoKV: Timely Staging of LLM KV Caches for Memory-Semantic Flash

From Tensor Buffer to Distributed Memory Hierarchy: A Survey of KV Cache Management for LLM Serving