KVCMAS: Efficient KV cache Correction for Shared Context in Multi-Agent Systems

Hyesung Jeon; Hyeongju Ha; Seoyoung Lee; Beomseok Kang; Jae-Joon Kim

cs.LGSep 28, 2026

KVCMAS: Efficient KV cache Correction for Shared Context in Multi-Agent Systems

Authors: Hyesung Jeon, Hyeongju Ha, Seoyoung Lee, Beomseok Kang, Jae-Joon Kim

Organizations: Department of Electrical and Computer Engineering Seoul National University

Abstract

Prompt-specialized multi-agent systems enable multiple agents to share a model while performing complementary roles to solve complex tasks. However, agent-specific prefixes change the KV cache generated for the same shared context, causing each agent to repeatedly prefill the growing context and construct a separate cache with high computation and memory overhead. Selective recomputation reduces this redundancy but still retains substantial model execution, while existing delta correction methods either support only recurring context relations or maintain memory-intensive online correction states for dynamically changing context. For first seen shared context, these methods also construct a reference cache outside the agent workflow, and an approximate correction at the first agent affects the outputs passed to subsequent agents. We present KVCMAS, an online KV cache correction framework that represents cross-agent cache deviations using compact low-rank states and seamlessly chains corrections along the agent workflow without an additional reference prefill. This design supports dynamically changing shared context while preserving an exact first-agent cache. Across multiple language and vision-language workloads, KVCMAS matches or improves the accuracy of prior KV cache sharing methods while achieving the lowest TTFT under highly concurrent serving. Under controlled serving traces, it provides a 2.0x TTFT speedup over inference without KV cache sharing and reduces peak GPU memory by up to 3.7x relative to a prior KV cache correction method. These results establish KVCMAS as an accurate and scalable KV cache sharing approach for prompt-specialized multi-agent serving.

Figures & tables

Figure 1: Overview of the context sharing convention and KVCMAS on an agent workflow edge from agent $i$ to agent $j$ . (a) Agent trajectories interleave agent-specific prefix segments (PF) with shared context segments (PH). KVCMAS reuses the preceding agent’s KV cache for a PH segment when its correction is reliable and otherwise falls back to dense prefill. (b) KVCMAS matches the current cache representation to the online anchors of the corresponding shared segment and estimates the required KV cache correction by combining their low-rank corrections with similarity-based weights. (c) The corrected cache becomes the reference for the next workflow edge, chaining corrections along the agent workflow without constructing a separate context-free reference cache.

Figure 2: Effective rank of the base cache and delta correction across layers and workloads. Effective rank is defined as the minimum rank retaining 90% of the singular-value energy. The annotated values report the effective rank averaged across layers, and the dotted line marks rank 32.

Figure 3: KV cache reconstruction error normalized by the Frobenius norm of the target tensor, using a context-free reference (non-chained) and the source agent cache (chained). The error is measured against the exact cache obtained through full prefill. (a) reports the error across layers, and (b) reports the error across agents in execution order A0 $\rightarrow$ A1 $\rightarrow$ A2 $\rightarrow$ A3, with an exact A0 cache under chained correction.

Figure 4: Benchmark accuracy and latency trade-off across the evaluated workloads. The shaded region spans FullShared and NonShared, with the upper left indicating a better trade-off. Out-of-range results are placed on the plot boundaries and annotated with their values. KVCMAS consistently remains near the upper-left Pareto frontier.

Figure 5: Median (p50) TTFT under concurrent serving across shared context lengths and QPS. The top row compares selective recomputation at $\rho=0.9$ with delta correction at $\rho=0.8$ , while the bottom row compares $\rho=0.8$ with $\rho=0.6$ , respectively. KVComm runs out of memory at 8K and 32K shared context.

Figure 6: Median (p50) TTFT of chained and non-chained correction at $\rho=0.8$ across shared context lengths and QPS. Non-chained correction uses a context-free reference, while chained correction follows the preceding agent cache.

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

	Layer Selection		Deviation Selection		Attention Selection
Dataset / Model	Coverage	Relative Coverage	Coverage	Relative Coverage	Coverage	Relative Coverage
GSM8K Llama-3.1-8B	0.13	1.33 $\times$	0.16	1.61 $\times$	0.15	1.45 $\times$
MMLU Llama-3.1-8B	0.03	0.87 $\times$	0.27	2.66 $\times$	0.17	1.65 $\times$
HumanEval Qwen2.5-Coder-7B	0.29	2.67 $\times$	0.32	3.24 $\times$	0.13	1.33 $\times$
MathVista LLaVA-OneVision-7B	0.50	2.35 $\times$	0.22	1.10 $\times$	0.20	1.02 $\times$
Video-MME LLaVA-OneVision-7B	0.39	1.82 $\times$	0.21	1.03 $\times$	0.20	0.99 $\times$

Appendix

Table 2: Reconstruction coverage of selective recomputation across datasets and models. Coverage measures the fraction of KV cache reuse error removed by recomputation, and Relative Coverage normalizes it by random selection under the same budget. Higher is better for both.

Figure 7: Effective rank and reconstruction error of PF-induced delta corrections. (a) reports the layer-wise effective rank, defined as the minimum rank retaining 90% of the singular-value energy, together with the average across layers for each workload. (b) reports the relative reconstruction error as the retained rank increases.

	MMLU	GSM8K	HumanEval	MathVista	Video-MME
Agent Family	AnalyzeAgent	MathSolver	CodeWriting	MathSolver	AnalyzeAgent
# Samples	1,531	1,319	161	1,000	1,296
Threshold $\tau$	0.29	0.32	0.34	0.50	0.60

Appendix

Table 3: Detailed workload configuration used in the accuracy evaluation. MathVista and Video-MME inherit the agent prompt structures of GSM8K and MMLU, respectively.

	MMLU		GSM8K		HumanEval		MathVista		Video-MME
Method	Average	Maximum	Average	Maximum	Average	Maximum	Average	Maximum	Average	Maximum
NonShared	2,324	3,176	2,267	3,460	1,809	3,277	4,216	9,032	6,594	7,659
FullShared	1,986	2,947	2,597	3,911	1,774	3,563	4,199	8,904	7,666	9,194
DroidSpeak	2,115	3,116	2,312	3,965	1,738	3,434	4,223	9,033	7,415	9,150
CacheBlend	2,345	3,432	2,275	3,919	1,774	3,104	4,213	9,033	6,943	9,206
RelayCaching	2,169	3,851	2,105	3,994	1,672	3,155	4,282	9,414	8,745	9,276

Appendix

Table 4: Average and maximum trajectory lengths during the accuracy evaluation, reported in model tokens. Visual tokens are included for MathVista and Video-MME.

Method	MMLU	GSM8K	HumanEval	MathVista	Video-MME
NonShared	0.26	0.22	0.59	0.35	0.37
FullShared	0.27	0.28	0.66	0.35	0.36
DroidSpeak	0.43	0.46	1.03	0.54	0.58
CacheBlend	0.43	0.40	1.03	0.55	0.58
RelayCaching	0.43	0.40	1.01	0.55	0.57
GraphFlow	0.37	0.38	1.14	0.46	0.45

Appendix

Table 5: Standard deviation of accuracy across three runs, reported in percentage points.

MMLU			Video-MME
$\tau$	Accuracy	$\rho$	$\tau$	Accuracy	$\rho$
0.10	67.47	0.623	0.20	52.98	0.590
0.20	66.82	0.705	0.40	52.88	0.610
0.25	66.62	0.720	0.50	52.80	0.630
0.27	66.82	0.762	0.55	52.64	0.630
0.29	65.45	0.790	0.58	52.44	0.630

Appendix

Table 6: Effect of

\tau

on accuracy (%) and reuse ratio. The settings used in the main evaluation are shown in bold .

Figure 8: Tail (p90) TTFT under concurrent serving across shared context lengths and QPS. The reuse ratio configurations follow Figure 5 .

Reference structure	MMLU	Video-MME
Chained	65.45	52.36
Non-chained	59.01	50.83

Appendix

Table 8: Accuracy of chained and non-chained KVCMAS. The correction representation, anchor pool, reliability threshold, and reuse ratio follow the main accuracy evaluation settings.

Figure 9: Tail (p90) TTFT of chained and non-chained correction across shared context lengths and request rates at $\rho=0.8$ . Chained correction reduces tail latency by avoiding the separate context-free reference prefill.

Pct.	Shared	Method	QPS
Pct.	Shared	Method	0	0.25	0.5	1	2	4	8
p50	2K	Chained	0.14	0.15	0.15	0.15	0.16	0.16	2.06
	2K	Non-chained	0.15	0.15	0.15	0.15	0.15	0.16	3.42
	8K	Chained	0.38	0.50	0.50	0.52	0.55	3.94	8.15
	8K	Non-chained	0.49	0.55	0.55	0.58	1.14	9.16	13.32
	32K	Chained	1.02	1.90	3.78	33.52	49.75	57.93	61.95

Appendix

Table 9: Median (p50) and tail (p90) TTFT of chained and non-chained correction across shared context lengths and request rates at

\rho=0.8

. All values are in seconds.

Figure 10: Median (p50) TTFT across request rates with 0.375K, 1.5K, and 6K shared generated tokens. The non-generated shared context is fixed to 2K tokens. Selective recomputation at $\rho=0.9$ and $0.8$ is matched with delta correction at $\rho=0.8$ and $0.6$ , respectively.

Figure 11: Tail (p90) TTFT under the same accumulated-output configurations as Figure 10 .

Shared context	Method	Reuse ratio $\rho$
		0.95	0.90	0.80	0.70	0.60
2K	NonShared	16.9	16.9	16.9	16.9	16.9
	FullShared	16.4	16.4	16.4	16.4	16.4
	DroidSpeak	17.0	17.0	17.0	17.1	17.2
	CacheBlend	17.2	17.2	17.2	17.3	17.4
	RelayCaching	17.4	17.4	17.4	17.5	17.5

Appendix

Table 11: Peak GPU memory in GB under single-stream execution across shared context lengths and reuse ratios. KVComm † exceeds the A6000 capacity, so its requirement is estimated from its full-dimensional anchor states.

Figure 12: Single-stream TTFT, E2E throughput, and peak GPU memory across shared context lengths at matched reuse ratios. Lower TTFT and memory and higher throughput indicate better efficiency. KVComm exceeds the NVIDIA A6000 48 GB memory capacity.

		TTFT (s)					Throughput (tokens/s)
Shared context	Method	$\rho$					$\rho$
		0.95	0.90	0.80	0.70	0.60	0.95	0.90	0.80	0.70	0.60
2K	NonShared	0.51	0.51	0.51	0.51	0.51	900	900	900	900	900
	FullShared	0.16	0.16	0.16	0.16	0.16	1,004	1,004	1,004	1,004	1,004
	DroidSpeak	0.62	0.62	0.62	0.62	0.62	868	868	866	866	863
	CacheBlend	0.40	0.42	0.44	0.46	0.48	935	933	920	917	906

Appendix

Table 12: Average reuse-request TTFT and E2E trajectory throughput under single-stream execution. TTFT is reported in seconds, and throughput in tokens/s.

Figure 13: Effect of the base cache and delta correction ranks on accuracy and single-stream efficiency. The upper panels report accuracy on MMLU and Video-MME, while the lower panels show TTFT, throughput, and peak GPU memory across rank combinations. “Full” denotes the corresponding uncompressed KV feature dimension, and the default $(r_{C},r_{\Delta})=(32,32)$ is highlighted.

$V$	TTFT (s)	Throughput	Memory	MMLU	Video-MME
1	2.69	4,050	26.21	62.51	49.40
5	3.93	3,821	26.68	64.42	51.89
10	4.02	3,780	29.24	65.45	52.36
20	4.05	3,769	31.94	65.80	53.15

Appendix

Table 13: Effect of anchor pool capacity

V

on single-stream efficiency and accuracy. TTFT is reported in seconds, throughput in tokens/s, peak GPU memory in GB, and accuracy in percent.

	TTFT (s)			Throughput (token/s)			Memory (GB)
Method	$N=2$	$N=4$	$N=8$	$N=2$	$N=4$	$N=8$	$N=2$	$N=4$	$N=8$
NonShared	8.25	8.37	oom	2,729	2,760	oom	27.25	35.63	oom
FullShared	3.87	2.01	1.08	3,830	4,922	5,904	23.06	23.06	23.06
DroidSpeak	6.82	6.19	5.91	2,657	2,852	3,021	27.12	27.93	29.64
CacheBlend	5.63	4.49	3.90	3,627	3,900	4,153	27.03	27.79	29.54
RelayCaching	5.53	4.28	3.65	3,480	3,883	4,093	26.61	27.22	28.44

Appendix

Table 14: Single-stream efficiency as the number of agents increases. Selective recomputation uses

\rho=0.9

, and delta correction uses

\rho=0.8

. KVComm is omitted because its anchor pool exceeds GPU capacity at 32K shared tokens. OOM denotes out of memory.

	TTFT (s)			Throughput (token/s)			Memory (GB)
Method	$R=2$	$R=4$	$R=8$	$R=2$	$R=4$	$R=8$	$R=2$	$R=4$	$R=8$
NonShared	8.22	8.33	8.19	2,728	2,747	2,769	31.44	31.44	31.44
FullShared	1.39	0.74	0.43	5,421	6,445	6,948	23.06	23.06	23.06
DroidSpeak	5.87	5.61	5.54	2,756	2,777	2,805	27.66	27.75	28.03
CacheBlend	4.03	3.57	3.36	3,927	4,110	4,278	27.50	27.65	27.90
RelayCaching	3.75	3.37	3.08	3,824	3,972	4,138	27.00	27.16	27.38

Appendix

Table 15: Single-stream efficiency as the number of interaction rounds increases. Selective recomputation uses

\rho=0.9

, and delta correction uses

\rho=0.8

. KVComm is omitted because its anchor pool exceeds GPU capacity at 32K shared tokens.

Explore similar work

PReCache: Efficient KV Cache Sharing for Multi-LoRA Agents via Low-Rank Precomputation and Neutral Reconstruction
Sep 28, 2026Hyesung Jeon, Hyeongju Ha, Jae-Joon KimKey-Value Cache Cache
Department of Electrical and Computer Engineering Seoul National University
PolyKV: A Shared Asymmetrically-Compressed KV Cache Pool for Multi-Agent LLM Inference
Apr 27, 2026Ishan Patel, Ishan JoshiKey-Value Cache Compression Key-Value Cache
Independent Researcher
KVShareArena: KV-Cache Reuse Across Contexts and Model Checkpoints
Sep 9, 2026Xi Shi, Mengxin Zheng, Qian LouKv-Cache Management Large Language Model Serving
University of Central Florida

Sep 28, 2026, cs.LGJ/K move · Enter open · S save

PReCache: Efficient KV Cache Sharing for Multi-LoRA Agents via Low-Rank Precomputation and Neutral Reconstruction

Hyesung Jeon, Hyeongju Ha, Jae-Joon Kim

Department of Electrical and Computer Engineering Seoul National University

Key-Value Cache Cache

Open paper AskPDF arXiv

Multi-LoRA agent systems enable efficient role specialization by sharing a common backbone model. However, each agent repeatedly processes the growing shared trajectory and constructs its own KV cache, introducing substantial memory and computation redundancy in long-horizon tasks. Existing KV cache sharing methods reduce this repeated prefill, but they either require additional training or architectural constraints or retain substantial model computation. Moreover, direct cache reuse causes the current agent to rely on cache states generated by the previous agent's adapter, weakening the role-specific behavior encoded by its own LoRA. We present PReCache, a training-free KV cache sharing framework with two designs, namely PreLRShared and ReBaseShared, that share the base cache computed using the pretrained weights and precompute a compact agent-specific low-rank (LR) cache. To remove repeated prefill, PreLRShared precomputes each agent's LR cache when the shared context is first processed, allowing the current agent to use its own LR cache without reprocessing context processed by previous agents. To improve sharing accuracy, ReBaseShared reconstructs the shared base cache from adapter-free hidden states, reducing the remaining error caused by the previous agent's adapted representation. To minimize its reconstruction cost, we propose two inference schemes tailored to single-stream inference and concurrent serving, performing the same reconstruction after each agent's turn or alongside its execution, respectively. Across multiple models and agent benchmarks, PreLRShared achieves up to a 3.1x TTFT speedup and a 2.3x improvement in per-request throughput over inference without KV cache sharing. ReBaseShared best preserves accuracy overall among the evaluated cache-sharing methods, with an average drop of only 1.1 points relative to inference without cache sharing.

Abstract

Explore similar work

PReCache: Efficient KV Cache Sharing for Multi-LoRA Agents via Low-Rank Precomputation and Neutral Reconstruction

PolyKV: A Shared Asymmetrically-Compressed KV Cache Pool for Multi-Agent LLM Inference

KVShareArena: KV-Cache Reuse Across Contexts and Model Checkpoints