Swarm Telepathy: Prefill-Free KV-Cache Transfer Across Families

Abdulrahman Alabdulkareem; Irene Tenison; Yang Zhang; Yada Zhu; Boris Katz; Dan Gutfreund

cs.LGOct 10, 2026

Swarm Telepathy: Prefill-Free KV-Cache Transfer Across Families

Authors: Abdulrahman Alabdulkareem, Irene Tenison, Yang Zhang, Yada Zhu, Boris Katz, Dan Gutfreund

Organizations: MIT CSAIL · MIT-IBM Computing Research Lab

Abstract

LLM multi-agent systems split work across agents that communicate through text. An agent summarizes its findings, another agent receives the text. This constrains communication to discrete tokens and costs redundant compute, limiting bandwidth and capabilities of multi-agent systems. Recent works demonstrate KV-cache transfer between LLMs but they either require the LLMs to be within the same family or cannot function across context, thus requiring the receiver to prefill text. We present SwaT (Swarm Telepathy), the first cross-family KV-cache transfer method that allows prefill-free communication in multi-family multi-agent systems. In an agent-to-agent communication setting, combining SwaT with recent advances in KV-cache compaction beats text-to-text communication in both accuracy and speed, with a mean of $40\%$ higher accuracy and a mean of $4\times$ faster wall-clock time. Experiments show SwaT achieves high performance on 17 benchmarks across 12 LLMs and 6 families. Additionally, given a sender Vision-Language Model (VLM), SwaT is able to transfer the vision capabilities to a text-only receiver LLM exceeding text-to-text communication despite SwaT being trained on text-only data. SwaT keeps both sender and receiver models frozen without fine-tuning and only requires $<50M$ tokens and $<1$ GPU-day to train.

Figures & tables

Panel of Figure 1 — Figure 1. Communication between two LLM agents from different families. Top: communicating in words is slow and the receiver only gets the gist. Bottom: \swat translates the sender’s latent state (KV-cache) to the receiver’s latent space. Combined with equal-budget cache compression, \swat is on average 40% more accurate than text summarization, at $4\times$ the speed. Left two thirds, two rows. Top row, labelled Text Communication: a blue square-headed robot labelled Family A with a neutral face speaks a text bubble reading The summary is, and an orange dome-headed robot labelled Family B with a neutral face ends up with a grey thought cloud reading I get the gist. Bottom row, labelled SwaT (ours): both robots smile; the blue robot's thought cloud of blue circles, labelled latent state, passes through a green SwaT box and becomes a full thought cloud of orange squares next to the orange robot. Right third, a small scoreboard with columns speed and performance: the words row has an hourglass and two check marks plus an empty slot; the SwaT row has a lightning bolt marked 4 times and three check marks.

Paper	Date	Different	Cross-	Skip
Paper	Date	Sizes	Family	Prefill
Closed Form ( Heo et al., 2026 )	Aug 2026	✓	✗	✓
CacheBridge ( Qu et al., 2026 )	Sep 2026	✓	✗	✓
KV Alignment ( Dery et al., 2026 )	Jan 2026	✓	✗	✓
Dense Latent ( Chen et al., 2026 )	Jun 2026	✓	✗	✓
LCF ( Rossi et al., 2026 )	May 2026	✓	✗	✗

Table 1. Cross-model KV transfer. \swat is the first method for cross-family KV-cache transfer with no receiver prefill (i.e., the receiver does not read the text).

Panel of Figure 2 — Figure 2. \swat between a frozen sender agent ( $\mathcal{S}$ ) and receiver agent ( $\mathcal{R}$ ). Left: $\mathcal{S}$ prefills the document $d$ once. \swat has $B$ blocks where block $b$ cross-attends to $\mathcal{S}$ ’s KV-cache for the layer group $\mathcal{S}_{b}$ and outputs $\mathcal{R}$ ’s KV-cache for the layer group $\mathcal{R}_{b}$ . Only \swat is trained where underlining marks learned parameters or modules. Right: block $b$ maps $h_{b}$ to $h_{b+1}$ through causal self-attention on $h_{b}$ , then causal cross-attention on a linear projection of $\mathcal{S}_{b}$ (RoPE removed), then an FFN. Positions and the causal mask in the cross-attention module are aligned across the tokenizers as described in section 3.2 . The output of $b$ is linearly projected to $\mathcal{R}_{b}$ with receiver RoPE applied to the keys. Left: three towers drawn with abstract depth. Blue Sender Agent (S) with L_S layers shows groups S_0, S_b and S_B separated by vertical dots; a small green SwaT box with B blocks and about 300M parameters, trained in under 6 GPU-hours on under 50M tokens, marked with a flame, shows blocks 0, b and B with b highlighted; orange Receiver Agent (R) with L_R layers shows groups R_0, R_b and R_B. Snowflakes mark both agents as frozen. Arrows connect each sender group to its block and each block to its receiver group. The context enters S as S's tokens and SwaT as R's embeddings; R reads an optional query or nothing. Dashed lines zoom from block b into a green panel on the right: K and V of the sender layers in S_b, K de-RoPE'd, are concatenated and pass through a LayerNorm-plus-linear block into the causal cross-attention of a residual stream with causal self-attention, cross-attention and FFN; The residual stream goes from h_b to h_{b+1}. Underlining identifies learned modules; memory normalization has no learned parameters. An output head of LayerNorm, linear, RMS calibration and receiver RoPE writes the keys and values of the receiver layers in R_b into Agent R.

Sender		Receiver	Short Acc	Context Acc	Context F1	Short Acc	Context Acc	Context F1	Mean
			In-domain (9)			OOD (8)
Small	$\boldsymbol{\rightarrow}$	Large
Qwen 3 4B	$\rightarrow$	Llama 3 8B	100.6%	100.2%	110.9%	88.7%	91.6%	98.7%	98.5%
Llama 3 8B	$\rightarrow$	Qwen 3 14B	89.6%	98.2%	116.1%	86.5%	98.9%	103.5%	97.5%
Ministral 3 8B	$\rightarrow$	Llama 3.2 11B	104.2%	97.5%	108.5%	89.7%	92.8%	99.0%	99.4%
Gemma 3 4B	$\rightarrow$	Granite 3.3 8B	95.4%	93.6%	159.8%	82.0%	95.0%	131.4%	108.5%

Table 2. Zero-shot retention (%,

\text{score}_{\text{\swat{}}}/\text{score}_{\text{receiver}}

) across 17 benchmarks.

Figure 3. Comparing \swat (ours) with AM ( Zweiger et al., 2026 ) against text-to-text communication ( T2T ) across three model–pairs, five budgets, and seven datasets. \swat + AM exceeds T2T in all metrics (task score, sender GPU time, receiver GPU TTFT). Description

		Qwen3-4B $\rightarrow$ Llama3-8B			Llama3-8B $\rightarrow$ Qwen3-14B			Gemma3-12B $\rightarrow$ Qwen3-4B
Metric	Budget	T2T	\swat	Change	T2T	\swat	Change	T2T	\swat	Change
	Lowest	$\approx$ 16.2 $\,\times$ compression			$\approx$ 12.7 $\,\times$			$\approx$ 17.2 $\,\times$
Score		34.2	53.1	+18.9 $\pm$ 4.6	37.2	58.6	+21.4 $\pm$ 4.5	29.8	49.0	+19.3 $\pm$ 5.5
Tokens		117	113	$-$ 3.1 $\pm$ 2.1%	161	149	$-$ 7.4 $\pm$ 3.3%	114	110	$-$ 3.8 $\pm$ 2.2%
$\mathcal{S}$ GPU (ms)		3,608	1,527	$\times$ 2.4 $\pm$ 0.4	3,675	1,897	$\times$ 1.9 $\pm$ 0.2	7,329	2,094	$\times$ 3.5 $\pm$ 0.6
$\mathcal{R}$ TTFT (ms)		9.2	3.8	$\times$ 2.4 $\pm$ 0.2	21.8	7.3	$\times$ 3.0 $\pm$ 0.3	6.7	3.2	$\times$ 2.1 $\pm$ 0.1

Table 3. Performance of \swat vs T2T over three model–pairs, five budgets, and the mean over seven datasets (QuALITY, QASPER, HotpotQA, MuSiQue, SQuAD, NarrativeQA, LongHealth). All dataset samples are

(d,q)

: document

d

is given to the sender agent (

\mathcal{S}

) and question

q

to the receiver agent (

\mathcal{R}

). T2T :

\mathcal{S}

summarizes

d

\mathcal{R}

prefills the summary along with

q

and proceeds to answer. \swat +AM: the KV-cache of

d

from

\mathcal{S}

is transferred ( \swat ) and compacted (AM) then given to

\mathcal{R}

, which prefills only

q

and proceeds to answer. Budgets are set by the T2T summary from

\mathcal{S}

under five different summary prompts, and \swat +AM is compressed to a similar budget. Metrics: task score (accuracy or F1), tokens communicated between the agents (Tokens), GPU time of

\mathcal{S}

per request (

\mathcal{S}

GPU), and time to first token of

\mathcal{R}

per request (

\mathcal{R}

TTFT), both in milliseconds. Change is reported

\pm

bootstrap standard error. Green: \swat is statistically better than T2T (95% bootstrap CI).

Gemma 3 4B $\rightarrow$ Granite 3.3 8B
Method	VQAv2	TQA	OKVQA	GQA	VW	Avg.
Sender	45.3 ∗	53.4 ∗	22.0 ∗	31.4 ∗	19.0	34.2 $\pm$ .4 ∗
T2T	33.8 ∗	32.0 ∗	15.7 ∗	16.3 ∗	7.7 ∗	21.1 $\pm$ .4 ∗
\swat	59.5	61.8	28.7	39.5	18.3	41.6 $\pm$ .4
Gemma 3 4B $\rightarrow$ Llama 3 8B
Sender	45.3 ∗	53.4 ∗	22.0 ∗	31.4 ∗	19.0	34.2 $\pm$ .4 ∗

Table 4. Test accuracy (%) of 3 VLM

\rightarrow

text-only LLM pairs on 5 VQA benchmarks (TQA

=

TallyQA, VW

=

VizWiz).

\pm

is bootstrap standard error. Sender : VLM answers from the image and question. T2T : the receiver LLM answers from the sender VLM’s caption. \swat : the sender VLM’s KV-cache is transferred to the text-only receiver. \swat is significantly better than results marked with ∗ (95% paired bootstrap).

CMMLU subject	Qwen	Llama	\swat Q $\rightarrow$ L
CMMLU subject	3 4B	3 8B	Acc.	Change
Arts	90.0	55.6	88.8	+59.6%
Security study	83.7	55.6	77.0	+38.7%
Concept. physics	83.7	45.6	73.5	+61.2%
Prof. psychology	82.8	56.0	77.6	+38.5%
All 67 subjects	69.7	51.2	65.1	+27.1%

Table 5. Zero-shot specialized knowledge transfer from Qwen 3 4B to Llama 3 8B on CMMLU (Chinese MMLU; accuracy, %). Although our training data contained only English text, \swat transfers most of Qwen’s advantage on foreign text never seen during training. Change is relative to Llama 3 8B.

	Qwen 3 4B $\rightarrow$ Llama 3 8B						Llama 3 8B $\rightarrow$ Qwen 3 14B
	FLOPs (TFLOPs)			Latency (ms)			FLOPs (TFLOPs)			Latency (ms)
Seq Len	Re-prefill	\swat	Ratio	Re-prefill	\swat	Speedup	Re-prefill	\swat	Ratio	Re-prefill	\swat	Speedup
64	0.89	0.04	$\times$ 24.7	26.0	2.1	$\times$ 12.3	1.69	0.04	$\times$ 45.3	43.5	2.2	$\times$ 20.2
256	3.59	0.15	$\times$ 24.4	31.7	2.6	$\times$ 12.0	6.79	0.15	$\times$ 44.7	47.8	2.7	$\times$ 18.0
1K	14.57	0.63	$\times$ 23.2	52.9	5.7	$\times$ 0 9.3	27.49	0.65	$\times$ 42.5	95.8	5.7	$\times$ 16.7
4K	61.57	3.13	$\times$ 19.7	209.9	21.7	$\times$ 0 9.7	115.11	3.20	$\times$ 35.9	394.1	22.3	$\times$ 17.7

Table 6. Receiver re-prefill vs \swat FLOP count and latency.

Sender $\rightarrow$ Receiver	Mode	Closed form ( Heo et al., 2026 )	Dense latent ( Chen et al., 2026 )	\swat (ours)
Qwen 3 4B $\rightarrow$ Qwen 3 8B	Same Family	59.9 $\pm$ 0.2	73.5 $\pm$ 0.1	72.3 $\pm$ 0.2
Qwen 3 4B $\rightarrow$ Qwen 2 7B	Cross Gen.	41.0 $\pm$ 0.5	47.3 $\pm$ 0.2	68.1 $\pm$ 0.1
Qwen 3 4B $\rightarrow$ Llama 3 8B	Cross Family	30.5 $\pm$ 2.2	43.5 $\pm$ 0.3	63.5 $\pm$ 0.1

Table 7. Mean (%, F1) over 17 benchmarks (

\pm

SE over 3 training seeds). Other methods are provided cross-tokenizer-aware alignment to function across model families. \swat exceeds others by

\geq

20 points in the cross-generation/family cases.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Small $\boldsymbol{\rightarrow}$ Large
		Short					Context
Pair	Row	ARC-C Acc	HSwag Acc	WGrande Acc	MMLU Acc	GSM8K Acc	QuALITY Acc	QASPER F1	Hotpot F1	MuSiQue F1
Qwen 3 4B	Receiver	80.6	77.2	59.2	62.2	75.2	64.1	49.0	67.4	34.6
$\rightarrow$ Llama 3 8B	\swat	85.2	83.6	58.8	62.0	67.6	64.2	51.1	70.6	42.6
	Retention (%)	105.7	108.3	99.3	99.7	89.9	100.2	104.4	104.8	123.4
Llama 3 8B	Receiver	92.6	91.4	69.4	73.2	89.8	80.1	49.1	73.1	43.4

Appendix

Table 8. In-domain (9): per-benchmark receiver score with its own context, \swat score, and zero-shot retention (italic; \swat over receiver). Scores in percent. HSwag = HellaSwag; WGrande = WinoGrande; Hotpot = HotpotQA.

Small $\boldsymbol{\rightarrow}$ Large
		Short			Context
Pair	Row	MedQA Acc	OpenBookQA Acc	MATH-500 Acc	BoolQ Acc	RACE Acc	LongHealth Acc	SQuAD F1	NarrQA F1
Qwen 3 4B	Receiver	60.0	76.0	28.4	83.7	81.3	66.8	88.9	74.2
$\rightarrow$ Llama 3 8B	\swat	54.8	79.2	20.0	79.8	76.4	57.0	87.5	73.4
	Retention (%)	91.3	104.2	70.4	95.3	94.0	85.4	98.4	98.9
Llama 3 8B	Receiver	64.4	90.2	76.6	89.1	89.4	72.0	86.9	72.7

Appendix

Table 9. OOD (8): per-benchmark receiver score with its own context, \swat score, and zero-shot retention (italic; \swat over receiver). Scores in percent.

		Qwen3-4B $\rightarrow$ Llama3-8B			Llama3-8B $\rightarrow$ Qwen3-14B			Gemma3-12B $\rightarrow$ Qwen3-4B
Metric	Budget	T2T	\swat	Change	T2T	\swat	Change	T2T	\swat	Change
	Lowest	$\approx$ 10 $\,\times$ compression ( $P_{\mathrm{sum},1}$ )			$\approx$ 9 $\,\times$ ( $P_{\mathrm{sum},1}$ )			$\approx$ 10 $\,\times$ ( $P_{\mathrm{sum},1}$ )
Score		44.6	62.8	+18.2	47.2	67.9	+20.6	37.9	53.8	+15.9
Tokens		144	132	$-$ 8.2%	170	154	$-$ 9.6%	151	138	$-$ 8.3%
$\mathcal{S}$ GPU (ms)		4,731	997	$\times$ 4.7	3,687	1,438	$\times$ 2.6	9,452	1,336	$\times$ 7.1
$\mathcal{R}$ TTFT (ms)		9.8	3.6	$\times$ 2.7	21.8	6.5	$\times$ 3.4	7.2	2.9	$\times$ 2.5

Appendix

Table 10. Experiment 2, HotpotQA. T2T : sender summarizes to receiver. \swat +AM: the sender’s cache rendered and compressed with Attention Matching. Budgets are determined by the sender agent in T2T given one of five summary prompts and sorted from lowest to highest per pair. Change: score difference in points, relative token difference, and speedup.

		Qwen3-4B $\rightarrow$ Llama3-8B			Llama3-8B $\rightarrow$ Qwen3-14B			Gemma3-12B $\rightarrow$ Qwen3-4B
Metric	Budget	T2T	\swat	Change	T2T	\swat	Change	T2T	\swat	Change
	Lowest	$\approx$ 58 $\,\times$ compression ( $P_{\mathrm{sum},1}$ )			$\approx$ 26 $\,\times$ ( $P_{\mathrm{sum},3}$ )			$\approx$ 39 $\,\times$ ( $P_{\mathrm{sum},1}$ )
Score		28.7	37.2	+8.5	39.0	47.8	+8.8	30.8	40.5	+9.8
Tokens		130	129	$-$ 0.5%	308	303	$-$ 1.5%	203	202	$-$ 0.4%
$\mathcal{S}$ GPU (ms)		4,299	3,410	$\times$ 1.3	7,481	4,840	$\times$ 1.5	13,791	5,567	$\times$ 2.5
$\mathcal{R}$ TTFT (ms)		11.7	5.9	$\times$ 2.0	38.0	12.0	$\times$ 3.2	11.2	5.1	$\times$ 2.2

Appendix

Table 11. Experiment 2, LongHealth. T2T : sender summarizes to receiver. \swat +AM: the sender’s cache rendered and compressed with Attention Matching. Budgets are determined by the sender agent in T2T given one of five summary prompts and sorted from lowest to highest per pair. Change: score difference in points, relative token difference, and speedup.

		Qwen3-4B $\rightarrow$ Llama3-8B			Llama3-8B $\rightarrow$ Qwen3-14B			Gemma3-12B $\rightarrow$ Qwen3-4B
Metric	Budget	T2T	\swat	Change	T2T	\swat	Change	T2T	\swat	Change
	Lowest	$\approx$ 14 $\,\times$ compression ( $P_{\mathrm{sum},1}$ )			$\approx$ 10 $\,\times$ ( $P_{\mathrm{sum},1}$ )			$\approx$ 15 $\,\times$ ( $P_{\mathrm{sum},1}$ )
Score		11.8	24.0	+12.2	11.8	32.6	+20.8	8.2	20.0	+11.7
Tokens		168	168	$-$ 0.2%	254	248	$-$ 2.3%	171	166	$-$ 3.4%
$\mathcal{S}$ GPU (ms)		5,287	1,568	$\times$ 3.4	5,722	2,290	$\times$ 2.5	10,607	2,441	$\times$ 4.3
$\mathcal{R}$ TTFT (ms)		9.9	3.7	$\times$ 2.7	27.0	6.6	$\times$ 4.1	7.8	2.9	$\times$ 2.6

Appendix

Table 12. Experiment 2, MuSiQue. T2T : sender summarizes to receiver. \swat +AM: the sender’s cache rendered and compressed with Attention Matching. Budgets are determined by the sender agent in T2T given one of five summary prompts and sorted from lowest to highest per pair. Change: score difference in points, relative token difference, and speedup.

		Qwen3-4B $\rightarrow$ Llama3-8B			Llama3-8B $\rightarrow$ Qwen3-14B			Gemma3-12B $\rightarrow$ Qwen3-4B
Metric	Budget	T2T	\swat	Change	T2T	\swat	Change	T2T	\swat	Change
	Lowest	$\approx$ 7 $\,\times$ compression ( $P_{\mathrm{sum},1}$ )			$\approx$ 7 $\,\times$ ( $P_{\mathrm{sum},1}$ )			$\approx$ 9 $\,\times$ ( $P_{\mathrm{sum},1}$ )
Score		29.2	65.6	+36.4	29.8	66.7	+36.9	19.1	53.1	+34.0
Tokens		108	103	$-$ 4.9%	115	103	$-$ 10.7%	84	80	$-$ 4.3%
$\mathcal{S}$ GPU (ms)		3,286	1,085	$\times$ 3.0	2,806	1,023	$\times$ 2.7	5,590	1,083	$\times$ 5.2
$\mathcal{R}$ TTFT (ms)		8.3	3.0	$\times$ 2.8	16.8	5.9	$\times$ 2.8	5.2	2.6	$\times$ 2.0

Appendix

Table 13. Experiment 2, NarrativeQA. T2T : sender summarizes to receiver. \swat +AM: the sender’s cache rendered and compressed with Attention Matching. Budgets are determined by the sender agent in T2T given one of five summary prompts and sorted from lowest to highest per pair. Change: score difference in points, relative token difference, and speedup.

		Qwen3-4B $\rightarrow$ Llama3-8B			Llama3-8B $\rightarrow$ Qwen3-14B			Gemma3-12B $\rightarrow$ Qwen3-4B
Metric	Budget	T2T	\swat	Change	T2T	\swat	Change	T2T	\swat	Change
	Lowest	$\approx$ 41 $\,\times$ compression ( $P_{\mathrm{sum},1}$ )			$\approx$ 27 $\,\times$ ( $P_{\mathrm{sum},1}$ )			$\approx$ 41 $\,\times$ ( $P_{\mathrm{sum},1}$ )
Score		21.7	34.3	+12.6	26.5	36.3	+9.8	23.0	31.9	+8.9
Tokens		106	106	+0.2%	165	164	$-$ 0.5%	108	108	$-$ 0.2%
$\mathcal{S}$ GPU (ms)		3,269	2,125	$\times$ 1.5	3,850	2,781	$\times$ 1.4	6,953	3,337	$\times$ 2.1
$\mathcal{R}$ TTFT (ms)		8.0	2.9	$\times$ 2.8	19.8	5.5	$\times$ 3.6	5.4	2.4	$\times$ 2.2

Appendix

Table 14. Experiment 2, QASPER. T2T : sender summarizes to receiver. \swat +AM: the sender’s cache rendered and compressed with Attention Matching. Budgets are determined by the sender agent in T2T given one of five summary prompts and sorted from lowest to highest per pair. Change: score difference in points, relative token difference, and speedup.

		Qwen3-4B $\rightarrow$ Llama3-8B			Llama3-8B $\rightarrow$ Qwen3-14B			Gemma3-12B $\rightarrow$ Qwen3-4B
Metric	Budget	T2T	\swat	Change	T2T	\swat	Change	T2T	\swat	Change
	Lowest	$\approx$ 42 $\,\times$ compression ( $P_{\mathrm{sum},1}$ )			$\approx$ 30 $\,\times$ ( $P_{\mathrm{sum},1}$ )			$\approx$ 52 $\,\times$ ( $P_{\mathrm{sum},1}$ )
Score		51.6	59.2	+7.6	57.8	71.8	+13.9	53.6	61.6	+8.0
Tokens		132	128	$-$ 2.5%	184	180	$-$ 2.0%	106	104	$-$ 2.3%
$\mathcal{S}$ GPU (ms)		3,721	1,476	$\times$ 2.5	4,174	2,085	$\times$ 2.0	7,229	2,208	$\times$ 3.3
$\mathcal{R}$ TTFT (ms)		11.8	6.0	$\times$ 2.0	27.3	11.6	$\times$ 2.3	8.2	4.9	$\times$ 1.7

Appendix

Table 15. Experiment 2, QuALITY. T2T : sender summarizes to receiver. \swat +AM: the sender’s cache rendered and compressed with Attention Matching. Budgets are determined by the sender agent in T2T given one of five summary prompts and sorted from lowest to highest per pair. Change: score difference in points, relative token difference, and speedup.

		Qwen3-4B $\rightarrow$ Llama3-8B			Llama3-8B $\rightarrow$ Qwen3-14B			Gemma3-12B $\rightarrow$ Qwen3-4B
Metric	Budget	T2T	\swat	Change	T2T	\swat	Change	T2T	\swat	Change
	Lowest	$\approx$ 3 $\,\times$ compression ( $P_{\mathrm{sum},1}$ )			$\approx$ 4 $\,\times$ ( $P_{\mathrm{sum},1}$ )			$\approx$ 4 $\,\times$ ( $P_{\mathrm{sum},1}$ )
Score		51.6	88.7	+37.1	48.4	87.5	+39.1	35.8	82.4	+46.6
Tokens		64	60	$-$ 5.5%	60	47	$-$ 22.9%	50	47	$-$ 7.2%
$\mathcal{S}$ GPU (ms)		1,852	1,068	$\times$ 1.7	1,272	935	$\times$ 1.4	2,923	1,220	$\times$ 2.4
$\mathcal{R}$ TTFT (ms)		6.2	3.1	$\times$ 2.0	11.5	5.9	$\times$ 1.9	4.0	2.5	$\times$ 1.6

Appendix

Table 16. Experiment 2, SQuAD. T2T : sender summarizes to receiver. \swat +AM: the sender’s cache rendered and compressed with Attention Matching. Budgets are determined by the sender agent in T2T given one of five summary prompts and sorted from lowest to highest per pair. Change: score difference in points, relative token difference, and speedup.

	Qwen3-4B $\rightarrow$ Llama3-8B			Llama3-8B $\rightarrow$ Qwen3-14B			Gemma3-12B $\rightarrow$ Qwen3-4B			All pairs
Metric	T2T	SwaT	Change	T2T	SwaT	Change	T2T	SwaT	Change	T2T	SwaT	Change
Score	38.8	54.8	+41.2%	39.2	59.1	+50.6%	34.1	52.8	+54.9%	37.3	55.5	+48.8%
Score $\Delta$ (pts)	42.9	57.8	+14.9	42.9	61.9	+19.0	38.0	56.2	+18.2	41.3	58.6	+17.4
Tokens	293	254	$-$ 13.1%	269	243	$-$ 9.7%	309	269	$-$ 13.1%	290	255	$-$ 12.0%
$\mathcal{S}$ GPU (ms)	8,752	2,150	$\times$ 4.1	6,198	2,360	$\times$ 2.6	19,761	2,888	$\times$ 6.8	10,234	2,447	$\times$ 4.2
$\mathcal{R}$ TTFT (ms)	18.0	4.0	$\times$ 4.5	31.8	7.5	$\times$ 4.3	14.1	3.4	$\times$ 4.2	20.1	4.7	$\times$ 4.3

Appendix

Table 17. Geometric mean performance of SwaT over three sender–receiver pairs, seven datasets and five budgets. T2T and SwaT columns are geometric means over all datasets and budgets; Change is their ratio. Score

\Delta

is the arithmetic mean score difference in points.

Panel of Figure 4 — Figure 4. Scatter plot for all individual cells in tables 10 , 11 , 12 , 13 , 14 , 15 and 16 . We note that points in the tokens plot are close to the diagonal by construction as the two methods were picked to have close to matching budgets. Description

Ablation	$\Delta$ \swat
\swat	$0$
\swat (w/ c2c ( Fu et al., 2026 ) token alignment)	$-3.20$
Headwise MLP	$-14.71$
Phase 1 only	$-36.97$
Phase 2 only	$-36.96$
Dense loss only	$-55.11$

Appendix

Table 18. Ablations of our \swat architecture. Each method is compared against the average of two independently trained \swat models. Differences are in percentage points on the mean benchmark score (accuracy or F1). Metrics calculated over 13 benchmarks with the pair Qwen3-4B

\rightarrow

Llama-3-8B.

Baselines
Method	Seq. len	EM	F1
Qwen 3 4B	28K	20.4	36.75
Llama 3 8B truncate	8K	5.4	13.32
Llama 3 8B $\mathrm{AM}_{20}$	5.6K	21.2	34.95
\swat
Qwen $\rightarrow^{\mathrm{AM}_{20}}$ Llama	5.6K	24.8	41.87

Appendix

Table 19. Interleaved HotpotQA (500 questions; EM/F1 in %). Sequence lengths is number of text tokens or KV-cache tokens.

Gemma 3 4B $\rightarrow$ Gemma 3 4B
Method	VQAv2	TQA	OKVQA	GQA	VW	Avg.
Sender	45.3 ∗	53.4 ∗	22.0	31.4 ∗	19.0	34.2 $\pm$ .4 ∗
\swat	53.8	56.6	22.4	35.3	19.0	37.4 $\pm$ .4
Gemma 3 4B $\rightarrow$ Granite 3.3 8B
Sender	45.3 ∗	53.4 ∗	22.0 ∗	31.4 ∗	19.0	34.2 $\pm$ .4 ∗
T2T	33.8 ∗	32.0 ∗	15.7 ∗	16.3 ∗	7.7 ∗	21.1 $\pm$ .4 ∗

Appendix

Table 20. Similar to table 4 . Test accuracy (%) for multiple pairs including a control where sender and receiver are the same VLM (Gemma 3 4B

\rightarrow

Gemma 3 4B pair).

Gemma 3 4B $\rightarrow$ Gemma 3 4B
Method	DocVQA	InfoVQA	Avg.
Sender	73.3	41.6	57.4 $\pm$ .8
\swat	40.9	26.8	33.8 $\pm$ .8
Gemma 3 4B $\rightarrow$ Granite 3.3 8B
Sender	73.3	41.6	57.4 $\pm$ .8
T2T	39.3 ∗	25.5 ∗	32.4 $\pm$ .8 ∗

Appendix

Table 21. Similar to table 4 . Test accuracy (%) for multiple pairs on DocVQA and InfoVQA, which depend on dense text and fine detail in the image.

Multi-Agent Communication

Multi-agent communication studies how autonomous agents exchange information to coordinate decisions, share knowledge, and act together under constraints.

See all 380 papers

Prefill-Free Cross-Family KV Cache Transfer for Heterogeneous Multi-Agent LLMs
Sep 26, 2026Vincent-Daniel Yun, Woosang Lim, Haneul Yoo +3Multi-Agent LLM Systems LLM Inference Acceleration
University of Southern California, USA · Seoul National University, Republic of Korea · New York University, USA
Latent Cache Flow: Model-to-Model Communication Without Text
May 19, 2026Maximillian Rossi, Prajwal Raghunath, Eugene WuKV Caching KV-Cache Compression
Columbia University
See What I See, Know What I Think: Dense Latent Communication Across Heterogeneous Agents
Jun 11, 2026Siyi Chen, Xiaoyan Zhang, Meng Wu +7Multi-Agent Communication Multi-Agent LLMs
University of Michigan · NVIDIA · University of Pennsylvania +2

Sep 26, 2026J/K move · Enter open · S save

Prefill-Free Cross-Family KV Cache Transfer for Heterogeneous Multi-Agent LLMs

Vincent-Daniel Yun, Woosang Lim, Haneul Yoo, Sungjoo Yoo+2

University of Southern California, USA · Seoul National University, Republic of Korea · New York University, USA

Multi-Agent LLM Systems LLM Inference Acceleration Multi-LLM Collaboration

Open paper AskPDF arXiv

Recent multi-agent LLM systems increasingly combine heterogeneous models for specialized agent roles. However, text-based communication requires each receiver to prefill shared context already processed by the sender. Reusing the sender's key-value (KV) cache avoids this redundancy, but prefill-free transfer across model families must handle differences in tokenization, model depth, and KV representations. To address these issues, we propose HeteroFold, a prefill-free cross-family KV cache transfer method that keeps both the sender and receiver frozen. HeteroFold aligns model structures, maps the sender cache into the receiver space, and calibrates it to preserve receiver behavior. Across six transfer directions, HeteroFold achieves the best cache-transfer performance on all four long-context benchmarks and most short-context settings. It also matches text-based communication on the multi-agent benchmark. At 32K context length, Llama-3.1-8B

\rightarrow

Ministral-3-14B transfer is

10.7\times

faster than Native Prefill and

1.18

1.47\times

faster than the state-of-the-art prefill-free baselines, Dense Latent and KV Ridge. These results show that HeteroFold enables efficient cross-family KV reuse without receiver prefill.

Abstract

Multi-Agent Communication

Explore similar work

Prefill-Free Cross-Family KV Cache Transfer for Heterogeneous Multi-Agent LLMs

Latent Cache Flow: Model-to-Model Communication Without Text

See What I See, Know What I Think: Dense Latent Communication Across Heterogeneous Agents