KV-Repurpose: A Testbed for Evaluating Non-Prefix KV-Cache Reuse

Taiga Ikeda; Daisuke Miyashita; Jun Deguchi

cs.CLOct 9, 2026

KV-Repurpose: A Testbed for Evaluating Non-Prefix KV-Cache Reuse

Authors: Taiga Ikeda, Daisuke Miyashita, Jun Deguchi

Organizations: Kioxia Corporation, Japan

Abstract

KV-cache reuse in large language model serving is commonly limited to exact prompt prefixes, preventing reuse when the same retrieved document appears after different system prompts, conversation histories, or dynamic metadata. We present KV-Repurpose, an open-source, vLLM-based system for developing and evaluating segment-wise KV-cache reuse beyond exact prefixes. The system combines cache-aware segment scheduling, prompt construction, capacity management, workload generation, and runtime monitoring. Experiments on long-context benchmarks show that naive segment reuse largely preserves output quality for single-document question answering and summarization, while exposing clear limitations in cross-document reasoning. Under a Zipfian RAG-style workload in which 20% of requests share a fixed prefix, KV-Repurpose achieves a substantially higher token-level cache hit rate than prefix caching at the same 30 GB cache capacity, 63.9% versus 3.3%, resulting in lower TTFT. Even with 1 TB of cache capacity, prefix caching reaches only a 17.9% hit rate in this workload. Source code is available under the Apache-2.0 license at https://github.com/taiga333/eval-kv-repurpose.

Figures & tables

Task (Metrics)	QA (Token-level F1 score)				Summary (ROUGE-L)
# Document	Single	Multi			Single		Multi
Dataset	QASPER	HQA	2WMQA	MuSiQue	QMSum	GovRepo	M-News
w/o context	0.04	0.11	0.12	0.11	0.07	0.03	0.04
Recompute	0.42	0.71	0.80	0.61	0.20	0.26	0.22
Repurp	0.42	0.60	0.50	0.40	0.19	0.26	0.17
Repurp (Opus 3)	0.42	0.59	0.50	0.41	0.19	0.21	0.17

Table 1: Performance comparison of KV-Repurpose and KV recomputation on LongBench datasets.

Dataset	LongBench Pro (Ave.)
w/o context	0.11
Recompute	0.45
Repurp	0.43
Repurp (Opus 3)	0.45
Repurp (Haiku 3.5)	0.44

Table 2: Comparison of the average performance of KV-Repurpose and KV recomputation across the 25 LongBench Pro tasks.

Figure 3: Token-level cache hit rate vs storage capacity.

Figure 4: p50/p95 TTFT under a Zipfian workload with Poisson request arrivals and a 30 GB cache capacity.

Figure 5: p50 TTFT, p95 TTFT, and throughput across request rates and context lengths.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Storage vs hit ratio and prefill saving.

Figure 7: Performance comparison of KV-Repurpose and KV recomputation on LongBench Pro datasets.

Dataset	QA1	QA2	QA3
Recompute	0.71	0.78	0.60
Repurp	0.67	0.57	0.47
Repurp (Opus 3)	0.68	0.59	0.47
Repurp (Haiku 3.5)	0.68	0.56	0.49

Appendix

Table 3: Performance comparison of KV-Repurpose and KV recomputation on HotpotQA (QA1), 2WikiMultihopQA (QA2), and MuSiQue (QA3), using only supporting passages as input documents.

KV-Cache Reuse

Key-value (KV) cache reuse lets language models reuse previously computed attention states to cut repeated prefill work and serving latency.

See all 73 papers

KVShareArena: KV-Cache Reuse Across Contexts and Model Checkpoints
Sep 9, 2026Xi Shi, Mengxin Zheng, Qian LouKV Caching LLM Inference Acceleration
University of Central Florida
SAFE-KV: Causal Suffix Repair for Behavior-Preserving Non-Prefix KV Cache Reuse in RAG
Oct 10, 2026Xingjian ZhangKV-Cache Reuse KV-Cache Management
Carnegie Mellon University, Pittsburgh, PA, USA
KV Packet: Recomputation-Free Context-Independent KV Caching for LLMs
Apr 14, 2026Chuangtao Chen, Grace Li Zhang, Xunzhao Yin +3KV Caching KV-Cache Reuse
Technical University of Munich Munich, Germany · Technical University of Darmstadt Darmstadt, Germany · Zhejiang University Hangzhou, China +1

Sep 9, 2026J/K move · Enter open · S save

KVShareArena: KV-Cache Reuse Across Contexts and Model Checkpoints

Xi Shi, Mengxin Zheng, Qian Lou

University of Central Florida

KV Caching LLM Inference Acceleration Prefix Caching KV-Cache Reuse

Open paper AskPDF arXiv

Reusing key-value (KV) caches speeds up LLM inference by avoiding repeated computation on shared text. Standard prefix caching reuses a KV cache only when the LLM is the same and all preceding text is identical, but real workloads often break both conditions: RAG systems place different documents before the same one, agents with different system prompts read the same file or tool output, multi-agent workflows use specialized LLMs on shared material, and an updated model reads documents cached by its previous version. Because KV caches depend on both the preceding text and the model weights, direct reuse can reduce answer quality. Many methods repair or compress the reused cache, but each paper uses its own tasks, models, and cost measures, and existing benchmarks mainly test long-context processing or reuse of an unchanged prefix. We introduce KVShareArena, a benchmark and open evaluation framework for comparing them under the same conditions. KVShareArena has (1) reuse tests on 2,150 questions from three QA datasets, where the preceding text, the cache-writing LLM, or both change while the answering LLM and input stay fixed; (2) five dense and mixture-of-experts LLMs (4B-30B) and six LLM pairs where one version of an LLM reads caches written by another, for 33 model-dataset settings; (3) 11 repair and compression methods from six method classes; (4) four evaluation perspectives: answer quality, prefill computation, KV-cache memory, and latency; and (5) a common interface for adding new methods and an interactive leaderboard. Experiments yield two findings. First, both the quality loss from reuse and which repairs help depend on the LLM, even between two 8B models. Second, most repairs keep their quality when another LLM version wrote the cache, but a trained repair adapter loses quality in 12 of 18 pair-dataset tests. Code and data: https://github.com/xishi404/KVShare-Arena

Abstract

KV-Cache Reuse

Explore similar work

KVShareArena: KV-Cache Reuse Across Contexts and Model Checkpoints

SAFE-KV: Causal Suffix Repair for Behavior-Preserving Non-Prefix KV Cache Reuse in RAG

KV Packet: Recomputation-Free Context-Independent KV Caching for LLMs