Large language models (LLMs) have demonstrated remarkable capabilities in solving complex tasks in coding, mathematical logic, and iterative reasoning, especially by coordinating effectively within multi-agent systems (MAS). However, language agents traditionally communicate via text exchange, which introduces a bottleneck in information transfer and adds prefill latency for downstream agents. Recent work proposes that directly sharing the key-value (KV) cache between language models can enable efficient latent collaboration, entirely bypassing standard text-based communication. To enable latent communication between heterogeneous models of different sizes, prior work introduces trained fusion adapters for each model pair, which limits practical deployment. We introduce CacheFuse, a training-free method which demonstrates that augmenting a downstream model with an upstream model's KV cache using linear interpolation consistently enables superior performance than the standalone downstream model can achieve. We systematically analyze CacheFuse across four reasoning and knowledge benchmarks and demonstrate up to 8.6 percentage points higher accuracy compared to single-model baselines and up to 12.2x TTFT speedup compared to text-based communication in multi-model inference.
Figures & tables
Figure 1: CacheFuse pipeline. The sharer and receiver each prefill the same prompt independently, producing per-layer KV caches of identical geometry (nkv,dhead,L) . Every layer’s key and value tensors are linearly interpolated with a single scalar θ to form a fused cache, which the downstream receiver model then decodes from directly to produce the response.
Pair
Benchmark
Receiver
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Replace
Sharer
T2T
8B → 4B
ARC-C
87.03
87.97
88.82
89.33
90.70
91.47
91.72
91.21
90.96
90.02
83.45
91.21
90.19
OBQA
78.00
79.00
80.80
83.40
84.40
86.60
85.20
84.00
83.00
82.60
71.40
85.00
83.80
MMLU-R
69.95
71.18
72.40
73.28
74.09
74.70
74.68
73.98
73.42
72.18
63.11
73.88
72.26
MMLU-P
41.08
42.20
43.46
44.33
45.07
45.33
44.93
44.22
42.80
37.43
27.81
44.41
43.87
4B → 8B
ARC-C
91.21
91.30
91.30
91.64
91.55
91.72
90.61
89.76
89.51
77.30
39.16
87.03
91.98
OBQA
85.00
85.60
85.00
86.00
85.80
85.80
84.80
82.60
84.00
78.40
54.60
78.00
87.80
Table 1: Full θ sweep across benchmarks on Qwen3 model pairs. Receiver corresponds to θ=0 (pure receiver cache). Columns 0.1–0.9 are θ values for the CacheFuse linear interpolation. Replace corresponds to θ=1 (pure sharer cache). Sharer denotes the standalone sharer model (reference). T2T denotes the text-to-text transfer baseline (reference). Bold underlined marks the best score in the θ sweep (across the Receiver to Replace endpoints).
Pair
Method
ARC-C
OBQA
MMLU-R
MMLU-P
Qwen3-8B → Qwen3-4B
Sharer
91.21
85.00
73.88
44.41
Receiver
87.03
78.00
69.95
41.08
CacheFuse
91.72
86.60
74.70
45.33
T2T
90.19
83.80
72.26
43.87
Qwen3-4B → Qwen3-8B
Sharer
87.03
78.00
69.95
41.08
Receiver
91.21
85.00
73.88
44.41
Table 2: Accuracy across all four benchmarks for the best-performing θ per pair (exact θ per benchmark given in Table 1 ). Bold underlined marks where CacheFuse exceeds Receiver (otherwise Receiver is marked).
Pair
Method
ARC-C
OBQA
TTFT
TTEoA
TTFT
TTEoA
Qwen3-8B → Qwen3-4B
Receiver
78.8
2518.8
78.0
2497.2
Replace
79.5
2518.3
78.6
2477.1
Sharer
80.2
2569.0
79.3
2545.7
CacheFuse
120.5
2560.3
119.0
2536.8
T2T
1468.5
3915.6
1259.1
3680.0
Table 3: Latency in milliseconds (ms) across model pairs and transfer methods. TTFT denotes time-to-first-token. TTEoA denotes time-to-end-of-answer.
Figure 2: MMLU-R accuracy sweep for Qwen3-8B → Qwen3-4B. Receiver ( θ=0 ) and Replace ( θ=1 ) are included as endpoints. Fusion peaks at θ=0.5 , exceeding the receiver model individually (Qwen3-4B), direct KV cache replacement, and even the larger sharer model individually (Qwen3-8B).
Figure 6
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Pair
Benchmark
Receiver
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Replace
Sharer
Qwen3-8B → Qwen3-4B
ARC-C
87.8
88.8
89.7
90.0
91.2
91.6
91.5
92.0
91.9
90.9
83.6
92.0
OBQA
74.5
75.9
77.3
78.5
79.8
79.4
79.3
79.6
78.8
77.6
70.1
79.5
Qwen3-4B → Qwen3-8B
ARC-C
92.0
92.1
92.3
92.4
92.1
91.7
91.8
90.5
89.4
77.6
41.4
87.8
OBQA
79.5
80.1
80.0
80.3
79.9
80.3
79.6
78.5
77.3
71.6
57.4
74.5
Qwen3-1.7B → Qwen3-0.6B
ARC-C
57.5
61.6
65.7
64.9
51.4
38.4
30.3
26.0
23.5
21.8
23.2
75.2
OBQA
46.7
49.0
50.8
53.7
50.9
43.1
34.7
31.5
28.3
25.5
26.6
60.8
Appendix
Table 4: Accuracy of offline θ calibration sweep on the first 1000 examples of each train split (ARC-C and OBQA), averaged over seeds. Column conventions match Table 1 : Receiver corresponds to θ=0 . Columns 0.1–0.9 are linear interpolation between the sharer and receiver. Replace corresponds to θ=1 . Sharer denotes the standalone sharer. Bold underlined marks the best score in the θ sweep (across Receiver to Replace endpoints).
Figure 5: Calibration vs. test θ sweeps on ARC-C. Left: Qwen3-8B → Qwen3-4B (calibration peaks at θ=0.7 while test peaks at θ=0.6 ). Right: Qwen3-1.7B → Qwen3-0.6B (both peak at θ=0.2 ).
Pair
Benchmark
Receiver
Test peak
Top-1
Top-2
Top-3
Qwen3-8B → Qwen3-4B
ARC-C
87.03
91.72 ( θ=0.6 )
91.21 ( θ=0.7 )
91.21 ( θ=0.7 )
91.47 ( θ=0.5 )
OBQA
78.00
86.60 ( θ=0.5 )
84.40 ( θ=0.4 )
84.40 ( θ=0.4 )
86.60 ( θ=0.5 )
Qwen3-4B → Qwen3-8B
ARC-C
91.21
91.72 ( θ=0.5 )
91.64 ( θ=0.3 )
91.64 ( θ=0.3 )
91.64 ( θ=0.3 )
OBQA
85.00
86.00 ( θ=0.3 )
86.00 ( θ=0.3 )
86.00 ( θ=0.3 )
86.00 ( θ=0.3 )
Qwen3-1.7B → Qwen3-0.6B
ARC-C
52.65
61.09 ( θ=0.2 )
61.09 ( θ=0.2 )
61.09 ( θ=0.2 )
61.09 ( θ=0.2 )
OBQA
53.60
54.60 ( θ=0.2 )
53.40 ( θ=0.3 )
53.40 ( θ=0.3 )
54.60 ( θ=0.2 )
Appendix
Table 5: Top- k offline calibration for reducing the test-time θ search. Test peak is the best accuracy (%) over the full test θ sweep (Table 1 ). Top- k selects the k highest-accuracy θ values from the averaged 1000-example train calibration sweep (Table 4 ), then reports the best test accuracy among only those k candidates (selected θ in parentheses). Bold underlined denotes where the top- k sweep exactly recovers the oracle score found on the test time search.
Multi-LLM collaboration studies how large language models share work, exchange information, and combine outputs to improve capability, reliability, or cost.