LLM agents drive inference engines with closed-loop sessions whose context grows every turn, and serving them economically depends on prefix caching: a hit costs an incremental prefill of about a thousand tokens, a miss recomputes a 10--40K-token context. We show that this makes prefix-cached agent serving metastable. With an identical base workload, a few minutes of extra arrivals leaves a vLLM server in a degraded regime--a ~10% hit ratio, about half the throughput, and a backlog that grows for the remaining 70 minutes--while its no-burst twin stays healthy (9 of 12 pairs; none in the reverse direction). The regime appears at loads the cache-warm server sustains, is absent without prefix caching, and also appears with Qwen3-32B on SWE-bench agent trace shapes and on SGLang, which tolerates larger triggers. Slower service keeps more sessions in the system, their working set outgrows the KV pool, and every turn becomes a recomputation; a first-order workload-balance condition, calibrated independently, brackets when the regime can drain. Getting out is the harder problem: throttling new admissions recovered 1 of 7 matched collapsed runs, while suspending admitted sessions against a capacity-based limit recovered all of them within minutes. A CPU offload tier delays the collapse but does not prevent it. A budgeter that bounds the resident KV demand of active sessions to the pool, with thresholds that need no per-workload tuning, prevented the failure in every run we tested.
Figures & tables
Figure 1: A burst that outlasts itself. Same model, same server, same base arrivals; the red run additionally receives a 5-minute injected burst (shaded). Its hit ratio and throughput never recover after the burst ends, while its blue no-burst twin stays healthy.
Experiment
Model / engine
KV pool
Base λ (/s)
Trigger
Control
Runs
Paired history dependence (§ 3 )
8B / vLLM
400K
0.035
+1.5 λ , 5 min @15
–
12 pairs
No-cache control
8B / vLLM, no cache
400K
0.0086
+16 sessions, 5 min @15
–
4 pairs + 4
Flush trigger
8B / vLLM
400K
0.04
cache flush @10
none / budgeter
5 + 3
Trace shapes
32B / vLLM
≈ 365K
0.025
2 × , 4 min @10
none / budgeter / cap
4 each
Second engine
8B / SGLang
400K
0.04
2 × 2 min; +1.5 λ 5–10 min; flush
–
4+6+3, 4 pairs
Recovery (§ 6 )
8B / vLLM
400K
0.04
2 × , 4 min @10
5 arms @20
16 seeds
Table 1: Experiments. Times are minutes from the start of a run; runs last 30–120 minutes. Synthetic sessions: 25 turns, 6K → 37K tokens (session-mix shift: 12 turns, 3K/22K start); trace shapes: mini-SWE-agent on SWE-bench, truncated at 38K tokens.
No burst (twin)
Burst
Seed
Hit
Sessions
Hit
Sessions
501
0.95
8
0.12 ∗
109
502
0.95
8
0.11 ∗
142
503
0.95
8
0.12 ∗
144
504
0.95
9
0.12 ∗
142
505
0.95
6
0.95
5
Table 2: Paired runs with an identical base workload: token-weighted hit ratio over the last 10 minutes and mean number of sessions in the system over the last 5 minutes of the 90-minute run. ∗ persistently degraded from its first degraded window after the burst to the end of the run.
Figure 2: Qwen3-32B, automatically sized KV pool, SWE-bench trace shapes, λ=0.025 (same seed): a 2 × burst from minute 10 to 14, the same burst with the budgeter, and no burst.
Figure 3: SGLang at λ=0.04 absorbs a 2 × , 2-minute burst (from minute 10) but not an extra 1.5λ for 5 or 10 minutes (from minute 15).
Figure 4: The loop in one pair (seed 501): resident demand relative to the KV pool, recomputed prompt tokens, and hit ratio, per minute. The burst (shaded) pushes demand past the pool; the twin never crosses it.
Control (from 20 min)
Recovered
Time to recover (min)
Hit (end)
Cohort done
None
0/8
–
0.12–0.13
0%
Admission-only cap
2/8
21–23
0.07–0.93
0–92%
Latency AIMD + park
5/7
14–25
0.84–0.89
50–100%
Session cap + park
7/7
≤ 6
0.95
100%
budgeter + park
7/7
< 5
0.93–0.95
100%
Table 3: Recovery when control starts at minute 20 (the 4-minute burst ran from minute 10 to 14), over runs that were collapsed at that point; denominators differ because arms share arrivals but not engine state. Time: from when control starts to the start of the first 5-minute window from which every later window has a hit ratio ≥ 0.8; “ < 5” means within the first window. Cohort: fraction of sessions arriving in the next 20 minutes that completed.
Figure 5: Recovery with no control and four controllers enabled at minute 20, one seed (identical arrivals; this seed was collapsed at minute 20 in every arm).
Figure 6: Workload shift from short to long sessions. Each point is a run: loose fixed caps collapse after the shift (bottom), tight ones make first-phase sessions wait (right); only the budgeter and the adaptive cap sit at the top left. Two seeds per configuration; points offset by ≤ 0.4 min for visibility.
Figure 7: Qwen3-32B, full KV pool, SWE-bench trace shapes: recovery when control starts at minute 25 (one seed).
Figure 8: Fraction of each arrival cohort (5-minute cohorts for 8B, 10-minute for 32B) that completed by the end of the run (minute 60 for 8B, 90 for 32B), averaged over the runs collapsed when control began. Later cohorts have less time to complete.
Loop
Same-load
Persistence
Recovery of
observed
history dependence
after trigger
admitted sessions
CONCUR Chen et al. (2026)
✓
–
–
–
KAIROS Yuan et al. (2026)
✓
–
argued
–
MISA-T Hong et al. (2026)
✓
–
–
–
VAMP Kim et al. (2026)
✓
–
–
–
Metronome Meng & Li (2026)
KV-pool exhaustion
–
–
–
Table 4: What prior agent-serving work establishes about cache-induced collapse. “Same-load history dependence”: identical subsequent load, different outcome depending on a past trigger. “Recovery of admitted”: measured recovery once collapsed, with control over already-admitted sessions. “–”: not demonstrated in the cited evaluation; CONCUR has agent pause/resume, but does not measure recovery from an established collapse.