Non-prefix KV cache reuse is a promising approach for improving inference efficiency in multi-agent systems as agents share the same context blocks. However, when agents have conflicting tasks, directly reusing non-prefix KV caches across agents can severely degrade model performance, causing up to a 90% drop. Existing remedies, including selective recomputation and offset-based calibration, recover a limited fraction of this loss. We argue that the problem lies not only in contaminated KV representations but also in the agent's difficulty interpreting reused context under its own instructions. To this end, we introduce LatentRemedy, which uses a few task-conditioned latent tokens to help the target agent leverage reused context without recomputing the shared block. Specifically, we use an MLP to map the last hidden state of the target prefix into a small number of latent tokens, which are placed after the reused KV block to interact with the shared context. To guide subsequent predictions toward the target task, we train the MLP through full-prefill distillation. Experiments show that LatentRemedy consistently outperforms existing reuse baselines on conflicting tasks while maintaining TTFT close to direct reuse as cache size grows. Code and data are available at https://github.com/chenzhwsysu57/kvreuse.
Figures & tables
Figure 1: Illustration of reusing KV-cache of the same block from task A to task B.
Figure 2: Performance of different KV cache reuse methods under conflicting tasks.
Figure 3: Failure modes of different KV cache reuse methods on Deal or No Deal.
Reuse
Restate
Repeat
Full
No-reas.
Argkp
2.27
2.73
32.27
85.91
deal
3.18
4.55
9.09
20.00
Job
7.27
7.73
11.36
69.09
Reason.
Argkp
70.91
70.00
74.09
90.00
deal
79.09
57.73
99.09
100.00
Job
86.82
85.91
92.27
93.18
Table 1: Performance comparison between original full, reuse, restate and repeat.
Figure 4: The overall framework of our latent remedy method.
Qwen3-1.7B
Qwen3-4B
Qwen3-8B
Approach
No reasoning
Reasoning
No reasoning
Reasoning
No reasoning
Reasoning
Reuse
17.00
40.39
16.00
50.94
17.28
44.91
Repeat
19.11
38.33
20.81
58.39
23.33
40.00
EPIC-16
17.03
17.56
15.69
29.77
16.53
34.99
KVCOMM
15.64
9.38
13.94
14.97
13.89
16.64
CacheBlend
17.17
17.39
15.81
29.57
16.67
41.51
Table 2: Accuracy across different settings in ConflictInstruct test set. Best are bold, second to best are underlined.
ArgKP
Deal or No Deal
Job Interview
Approach
No reasoning
Reasoning
No reasoning
Reasoning
No reasoning
Reasoning
Reuse
2.27
70.91
3.18
79.09
7.27
86.82
Repeat
32.27
74.09
9.09
99.09
11.36
92.27
EPIC-16
6.82
69.55
3.18
89.09
7.27
85.91
KVCOMM
1.36
28.64
5.45
55.91
6.82
88.64
CacheBlend
6.82
71.82
3.64
92.73
9.55
88.18
Table 3: Accuracy across different settings in Qwen3-4B under different settings in ArgKP, Deal and Job. Best are bold, second to best are underlined.
Figure 5: TTFT accuracy comparison across representative tasks under various settings.
Figure 6: TTFT benchmark results for different Qwen3 model sizes.
Figure 7: TTFT decomposition pipeline of LatentRemedy.
reuse KV length = 256
Approach
Prefix
Method
KV operation
Suffix
Sampling
Total TTFT
Full
29.35
Reuse
19.02
–
1.04
21.00
0.08
41.14
Ours
18.60
0.21
0.94
21.67
0.05
41.46
EPIC-16
19.66
19.74
1.04
20.99
0.08
61.52
KVCOMM
19.14
4.63
1.08
20.32
0.07
45.23
Table 4: TTFT decomposition for Qwen3-4B as shared KV length increases (ms; 10 runs).
Figure 8: Case study of reasoning traces of different methods.
Setting
No reasoning
Reasoning
Reuse
16.00
50.94
Latent-1
40.56
63.83
Latent-4
43.11
61.28
Latent-8
43.61
62.11
Latent-16
41.64
62.42
Latent-32
42.33
59.53
Table 5: Ablation for number of latent remedy tokens in Qwen3-4B for test set.
# pairs
No reasoning
Reasoning
Reuse
16.00
50.94
1
34.83
54.61
3
42.36
61.61
5
41.64
62.42
10
43.19
53.86
15
46.08
62.75
Table 6: Accuracy on test set ablation for number of training pairs for each task in Qwen3-4B.
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
ArgKP
Deal or No Deal
Job Interview
Approach
No reasoning
Reasoning
No reasoning
Reasoning
No reasoning
Reasoning
Reuse
8.18
49.55
10.91
36.36
17.27
46.82
Repeat
18.64
59.55
13.18
80.91
18.18
55.00
EPIC-16
10.45
46.36
10.00
39.09
17.27
43.24
KVCOMM
5.00
21.36
15.00
24.09
18.64
42.73
CacheBlend
8.18
45.00
10.45
49.09
16.82
49.55
Appendix
Table 7: Performance comparison on OOD tasks with Qwen3-1.7B under no-reasoning and reasoning settings. Best results are bold and second-best results are underlined.
ArgKP
Deal or No Deal
Job Interview
Approach
No reasoning
Reasoning
No reasoning
Reasoning
No reasoning
Reasoning
Reuse
2.27
82.27
4.09
95.45
6.82
88.64
Repeat
53.18
86.82
7.73
100.00
9.09
94.09
EPIC-16
7.27
81.82
5.00
97.73
6.36
88.83
KVCOMM
1.82
35.45
4.09
92.73
7.27
90.83
CacheBlend
7.27
85.45
4.09
78.18
8.64
88.64
Appendix
Table 8: Performance comparison on OOD tasks with Qwen3-8B under no-reasoning and reasoning settings. Best results are bold and second-best results are underlined.
Qwen3-1.7B; shared KV length = 256
Approach
Prefix
Method
KV operation
Suffix
Sampling
Total TTFT
Full
16.16
Reuse
13.84
0.80
14.77
0.06
29.47
Ours
13.69
0.18
0.72
16.12
0.05
30.77
EPIC-16
13.82
14.13
0.80
14.70
0.06
43.51
KVCOMM
14.18
3.63
0.79
15.02
0.07
33.69
Appendix
Table 9: Mean TTFT decomposition for Qwen3-1.7B at shared KV lengths 256, 1,024, 2,048, and 4,096 (ms; 10 runs).
Qwen3-8B; shared KV length = 256
Approach
Prefix
Method
KV operation
Suffix
Sampling
Total TTFT
Full
48.33
Reuse
22.33
1.03
23.62
0.08
47.06
Ours
22.18
0.25
0.98
27.35
0.07
50.82
EPIC-16
21.82
21.95
1.03
23.71
0.08
68.60
KVCOMM
22.85
4.94
1.15
24.59
0.08
53.61
Appendix
Table 10: Mean TTFT decomposition for Qwen3-8B at shared KV lengths 256, 1,024, 2,048, and 4,096 (ms; 10 runs).
Figure 9: TTFT accuracy comparison across all tasks under no-reasoning settings using Qwen3-1.7B, 4B and 8B.
Figure 10: TTFT accuracy comparison across all tasks under reasoning settings using Qwen3-1.7B, 4B and 8B.
Figure 11: TTFT accuracy comparison across all tasks under no-reasoning settings using Qwen3-1.7B, 4B and 8B.
Figure 12: TTFT accuracy comparison across all tasks under reasoning settings using Qwen3-1.7B, 4B and 8B.
Figure 13: TTFT accuracy comparison across all tasks under no-reasoning settings using Qwen3-1.7B, 4B and 8B.
Figure 14: TTFT accuracy comparison across all tasks under reasoning settings using Qwen3-1.7B, 4B and 8B.
Figure 15: Case study of reasoning traces of different methods in the Argument Key Point task.
Figure 16: Case study of reasoning traces of different methods in the Argument Key Point task.
Figure 17: Case study of reasoning traces of different methods in the Argument Key Point task.
Figure 18: Case study of reasoning traces of different methods in the Argument Key Point task.
Figure 19: Case study of reasoning traces of different methods in the Deal or no deal task.
Figure 20: Case study of reasoning traces of different methods in the Deal or no deal task.
Figure 21: Case study of reasoning traces of different methods in the Deal or no deal task.
Figure 22: Case study of reasoning traces of different methods in the Deal or no deal task.
Task group
Families
Train / Test
Pairs
Instruction
Shared block
Audited algorithmic families
95
69 / 26
2,850
123.7
117.7
Basic instruction switches
16
13 / 3
480
46.7
170.0
Scene-augmented switches
40
30 / 10
1,200
53.1
167.9
Structured reasoning
68
52 / 16
2,040
85.5
439.5
Long-form role conflicts
6
4 / 2
180
33.6
1,775.7
Argument-perspective scenes
5
5 / 0
150
20.1
84.2
Appendix
Table 11: Statistics of ConflictInstruct across different task groups.
Split
Families
Pairs
Instruction
Shared
train75
180
5,400
86.7
253.7
test25
60
1,800
93.1
281.0
Appendix
Table 12: Family-level 75/25 split of ConflictInstruct. Instruction and Shared are mean token lengths under the measurement convention in Table 11 .
Beijing University of Posts and Telecommunications, Beijing, China · Tsinghua University, Beijing, China · University of Electronic Science and Technology of China, Chengdu, China +1