Dual-Cache Latent Space Communication between Heterogeneous Language Models

Jiyao Liu; Qi Zhang; Yaoyi Jia; Ziwen Kan; Song Wang

cs.AIAug 20, 2026

Dual-Cache Latent Space Communication between Heterogeneous Language Models

Authors: Jiyao Liu, Qi Zhang, Yaoyi Jia, Ziwen Kan, Song Wang

Organizations: Independent Researcher · Amazon Web Services (AWS) · University of Central Florida (UCF)

Abstract

Multi-agent LLM systems split work across models, so answering often requires knowledge that sits in another agent's context: a Sharer has encoded information that a Receiver needs to complete its task. They usually communicate by exchanging text, which puts autoregressive decoding on the critical path and reduces the exchange to a discrete message written without sight of the receiver's state. Recent latent protocols instead translate the sharer's key-value (KV) cache into the receiver's: C2C supports heterogeneous models but requires both to read the same input, while LCF-X removes this shared-context requirement through position-free sharer-cache pooling. Three restrictions remain: LCF-X compresses the sharer alone, supplies the same layer-local summary to every receiver position with no joint cross-layer memory to retrieve from, and assumes matched layer count and KV geometry. We introduce XKV, which lifts all three: learned-query attention pools both caches; self-attention over receiver-aligned layer tokens, with a learned layer map reconciling different depths, mixes the pooled summaries into a compact joint memory; and a shared position decoder lets every raw receiver cache position retrieve its own per-head-gated residual in the receiver's native KV geometry. Both models stay frozen and may differ in family, depth, KV-head count, head dimension, and tokenizer; only the translator is trained. Across 45 dataset-model-pair settings (six heterogeneous and three same-model ordered pairings, five datasets), XKV attains the highest macro score and best average rank, improving on LCF-X on every dataset (by 4.6 exact-match and 4.2 F1 points on ROPES) and surpassing text communication on four of the five, while training 76% fewer parameters and translating a cache pair 10.3x faster (5.8 vs. 59.9 ms); end to end, XKV is 26% faster than LCF-X and 6.8x faster than text communication.

Figures & tables

Figure 1: Communication protocols for two models holding complementary contexts, annotated with the challenges each raises (

\times

) or resolves (

\checkmark

). T2T decodes a message on the critical path without access to the receiver’s state. LCF-X pools a summary from the sharer cache alone and supplies the same summary to every position, although its downstream projector also conditions on the local receiver cache row. XKV (ours) builds the communication memory jointly from both caches, and each receiver position retrieves its own receiver-native residual.

Figure 2: Answer quality versus communication cost, aggregated over all 45 dataset–model-pair settings (details in the technical appendix); the ordering on both axes is faithful, the spacing is not to scale. XKV dominates both baselines: it improves over LCF-X on every dataset with a

10.3\times

faster translator and 76% fewer trainable parameters, and is

6.8\times

faster end to end than T2T , which leads only on QASC. C2C is omitted: it requires token-aligned shared context.

Figure 3: Overview of XKV . Both frozen models prefill their private contexts. Stage 1 ( what to communicate ): learned-query attention pools each cache from

N_{a}

positions to

k

slots per layer and head, a learned map aligns the sharer layer axis, and the concatenated summaries are bottlenecked and mixed by self-attention into one joint memory token per receiver layer. Stage 2 ( where to write ): every raw receiver cache position queries the memory through a shared cross-attention decoder, and shared, zero-initialized output heads emit per-head-gated KV residuals in the receiver’s native cache geometry. Only the XKV translator is trained.

Sharer $\rightarrow$ Receiver	Method	ROPES		MuSiQue		QASC	StrategyQA	HotpotQA-bridge		Cross-dataset Avg.
Sharer $\rightarrow$ Receiver	Method	EM	F1	EM	F1	Acc.	Acc.	EM	F1	Cross-dataset Avg.
	T2T	35.8	45.2	4.6	12.2	78.6	64.1	16.6	27.1	45.44
	LCF-X	45.2	52.4	7.7	15.4	82.4	58.8	29.1	40.6	49.92
Qwen $\rightarrow$ Qwen	XKV	47.9	55.6	8.1	16.0	82.2	61.4	29.5	40.7	51.18
Qwen $\rightarrow$ Gemma	T2T	42.4	48.6	6.8	14.5	65.8	51.9	22.4	34.0	42.96
	LCF-X	43.9	51.3	6.0	13.3	49.2	60.8	23.3	34.0	41.72

Table 1: Full results for all nine ordered sharer–receiver settings. Each setting occupies three rows, one per communication method. We report EM/F1 for generative QA and accuracy (Acc.) for QASC and StrategyQA; higher is better. Cross-dataset Avg. averages the primary metric (F1 for generative QA and accuracy for classification) across all five datasets. The technical appendix provides the same results as per-dataset tables with cell-wise highlighting.

Figure 4: Dataset-level mean primary scores over the full

3\times 3

sharer–receiver grid. Each dataset forms one group and each bar is a communication method; we use F1 for generative QA and accuracy for classification. XKV is best on four of the five datasets.

Sharer $\rightarrow$ Receiver	Method	ROPES		MuSiQue		QASC		StrategyQA		HotpotQA-bridge		Cross-dataset Avg.
Sharer $\rightarrow$ Receiver	Method	Comm.	E2E	Comm.	E2E	Comm.	E2E	Comm.	E2E	Comm.	E2E	Comm.	E2E
	T2T	–	955	–	974	–	328	–	848	–	995	–	820.0
	LCF-X	31	165	115	376	42	156	42	154	64	259	58.8	222.0
Qwen $\rightarrow$ Qwen	XKV	6	139	6	218	6	119	10	122	6	201	6.8	159.8
	T2T	–	1549	–	1225	–	790	–	1884	–	1372	–	1364.0
	LCF-X	39	213	84	427	45	188	44	215	58	294	54.0	267.4

Table 2: Latency results for same-model sharer–receiver settings. Each dataset reports latency in milliseconds per example; lower is better. Communication (Comm.) is fusor-only time for LCF-X and translator-only time for XKV. Since T2T has neither module, only its end-to-end (E2E) latency is reported. Cross-dataset Avg. averages the available measurement over five datasets.

Figure 5: Pair-level communication and end-to-end latency savings relative to the faster baseline, averaged over five datasets. Rows denote sharers and columns receivers. (a) Communication latency saved by XKV (ms). (b) End-to-end latency saved by XKV. Positive values mean XKV is faster.

Variant	QASC $\Delta$	ROPES F1 $\Delta$	StrategyQA $\Delta$	Mean $\Delta$	Comm. (ms)	E2E (ms)	Params
Full XKV	0.00	0.00	0.00	0.00	5.8	131.7	4.55M
$-\mathrm{RP}$	-0.41	-0.82	-0.16	-0.46	4.7	128.7	3.58M
$-\mathrm{RX}$	-1.98	-0.44	-0.08	-0.83	3.7	127.1	3.43M
LCFP	-0.82	-2.00	+0.01	-0.94	39.1	183.7	3.53M

Table 3: Ablation results over QASC, ROPES, and StrategyQA. Score changes are relative to full XKV and use accuracy for QASC and StrategyQA and F1 for ROPES. “Comm.” is translator-only latency.

Sharer $\rightarrow$ Receiver	Method	ROPES		MuSiQue		QASC		StrategyQA		HotpotQA-bridge		Cross-dataset Avg.
Sharer $\rightarrow$ Receiver	Method	Comm.	E2E	Comm.	E2E	Comm.	E2E	Comm.	E2E	Comm.	E2E	Comm.	E2E
	T2T	–	955	–	974	–	328	–	848	–	995	–	820.0
	LCF-X	31	165	115	376	42	156	42	154	64	259	58.8	222.0
Qwen $\rightarrow$ Qwen	XKV	6	139	6	218	6	119	10	122	6	201	6.8	159.8
Qwen $\rightarrow$ Gemma	T2T	–	914	–	971	–	291	–	855	–	1001	–	806.4
	LCF-X	35	193	118	439	42	177	41	177	65	295	60.2	256.2

Table 4: Appendix backup: full latency results for all nine ordered sharer–receiver settings. Each dataset reports latency in milliseconds per example; lower is better. Communication (Comm.) is fusor-only time for LCF-X and translator-only time for XKV. Because T2T has neither module, only its end-to-end (E2E) latency is reported. Cross-dataset Avg. averages the available measurement over five datasets.

Setting	Value
Optimizer	AdamW
Learning rate	$1\times 10^{-4}$
LR schedule	linear warmup ( $10\%$ of total steps)
Effective batch size	256
Gradient clipping	$1.0$
Pooling spans $P$	$4$

Table 5: Default training / inference hyperparameters.

Sharer	Receiver	LCF-X	XKV	T2T
Qwen-0.6B	Qwen-0.6B	45.2/52.4	47.9 / 55.6	35.8/45.2
Qwen-0.6B	Gemma-1B	43.9/51.3	46.2 / 54.3	42.4/48.6
Qwen-0.6B	Llama-3B	51.4/58.5	57.3 / 61.8	50.6/58.0
Gemma-1B	Qwen-0.6B	47.2/53.7	49.1 / 55.4	31.9/42.5
Gemma-1B	Gemma-1B	45.5/51.4	48.1 / 55.6	31.8/42.1
Gemma-1B	Llama-3B	42.1/49.8	54.0 / 58.7	46.9/55.7

Table 6: ROPES EM/F1 scores (%). Higher is better; cell-wise best values are bold. Mean across the nine model pairs: LCF-X 45.86/52.79; XKV 50.48/56.99; T2T 40.74/49.31.

Sharer	Receiver	LCF-X		XKV		T2T
Sharer	Receiver	Comm.	E2E	Comm.	E2E	E2E
Qwen-0.6B	Qwen-0.6B	31	165	6	139	955
Qwen-0.6B	Gemma-1B	35	193	5	160	914
Qwen-0.6B	Llama-3B	36	148	6	119	927
Gemma-1B	Qwen-0.6B	42	182	6	159	1605
Gemma-1B	Gemma-1B	39	213	5	171	1549

Table 7: ROPES latency in milliseconds per example. Communication is measured only for the latent modules (LCF-X fusor and XKV translator); T2T therefore reports end-to-end latency only. Lower is better; cell-wise minima at the displayed precision are bold. Means across the nine model pairs: LCF-X 38.7/175.4; XKV 5.7/141.3; T2T E2E 1257.6.

Sharer	Receiver	LCF-X	XKV	T2T
Qwen-0.6B	Qwen-0.6B	7.7/15.4	8.1 / 16.0	4.6/12.2
Qwen-0.6B	Gemma-1B	6.0/13.3	5.4/12.5	6.8 / 14.5
Qwen-0.6B	Llama-3B	16.0/26.3	19.2 / 29.9	9.3/19.5
Gemma-1B	Qwen-0.6B	8.4 / 16.6	8.1/15.8	3.4/10.8
Gemma-1B	Gemma-1B	4.6/11.9	5.8/13.1	6.2 / 13.4
Gemma-1B	Llama-3B	17.8/28.3	19.4 / 30.0	9.4/18.3

Table 8: MuSiQue EM/F1 scores (%). Higher is better; cell-wise best values are bold. Mean across the nine model pairs: LCF-X 10.00/18.47; XKV 11.07/19.59; T2T 7.99/16.92.

Sharer	Receiver	LCF-X		XKV		T2T
Sharer	Receiver	Comm.	E2E	Comm.	E2E	E2E
Qwen-0.6B	Qwen-0.6B	115	376	6	218	974
Qwen-0.6B	Gemma-1B	118	439	5	253	971
Qwen-0.6B	Llama-3B	116	331	6	201	1006
Gemma-1B	Qwen-0.6B	91	370	6	229	1236
Gemma-1B	Gemma-1B	84	427	5	264	1225

Table 9: MuSiQue latency in milliseconds per example. Communication is measured only for the latent modules (LCF-X fusor and XKV translator); T2T therefore reports end-to-end latency only. Lower is better; cell-wise minima at the displayed precision are bold. Means across the nine model pairs: LCF-X 107.8/375.0; XKV 5.7/235.1; T2T E2E 1196.7.

Sharer	Receiver	LCF-X	XKV	T2T
Qwen-0.6B	Qwen-0.6B	82.4	82.2	78.6
Qwen-0.6B	Gemma-1B	49.2	55.1	65.8
Qwen-0.6B	Llama-3B	92.0	93.3	86.7
Gemma-1B	Qwen-0.6B	81.3	82.4	84.6
Gemma-1B	Gemma-1B	44.8	53.9	55.7
Gemma-1B	Llama-3B	92.9	93.0	89.4

Table 10: QASC accuracy scores (%). Higher is better; cell-wise best values are bold. Mean across the nine model pairs: LCF-X 73.44; XKV 76.67; T2T 76.93.

Sharer	Receiver	LCF-X		XKV		T2T
Sharer	Receiver	Comm.	E2E	Comm.	E2E	E2E
Qwen-0.6B	Qwen-0.6B	42	156	6	119	328
Qwen-0.6B	Gemma-1B	42	177	5	136	291
Qwen-0.6B	Llama-3B	42	139	6	102	247
Gemma-1B	Qwen-0.6B	48	171	6	127	810
Gemma-1B	Gemma-1B	45	188	5	145	790

Table 11: QASC latency in milliseconds per example. Communication is measured only for the latent modules (LCF-X fusor and XKV translator); T2T therefore reports end-to-end latency only. Lower is better; cell-wise minima at the displayed precision are bold. Means across the nine model pairs: LCF-X 45.1/161.2; XKV 5.7/119.3; T2T E2E 650.6.

Sharer	Receiver	LCF-X	XKV	T2T
Qwen-0.6B	Qwen-0.6B	58.8	61.4	64.1
Qwen-0.6B	Gemma-1B	60.8	58.2	51.9
Qwen-0.6B	Llama-3B	72.9	73.6	73.4
Gemma-1B	Qwen-0.6B	59.3	62.0	65.9
Gemma-1B	Gemma-1B	58.2	58.8	57.6
Gemma-1B	Llama-3B	68.8	74.1	70.9

Table 12: StrategyQA accuracy scores (%). Higher is better; cell-wise best values are bold. Mean across the nine model pairs: LCF-X 61.17; XKV 65.01; T2T 63.91.

Sharer	Receiver	LCF-X		XKV		T2T
Sharer	Receiver	Comm.	E2E	Comm.	E2E	E2E
Qwen-0.6B	Qwen-0.6B	42	154	10	122	848
Qwen-0.6B	Gemma-1B	41	177	5	140	855
Qwen-0.6B	Llama-3B	42	139	6	109	807
Gemma-1B	Qwen-0.6B	48	170	6	126	1882
Gemma-1B	Gemma-1B	44	215	5	146	1884

Table 13: StrategyQA latency in milliseconds per example. Communication is measured only for the latent modules (LCF-X fusor and XKV translator); T2T therefore reports end-to-end latency only. Lower is better; cell-wise minima at the displayed precision are bold. Means across the nine model pairs: LCF-X 45.1/169.0; XKV 6.1/134.6; T2T E2E 1344.4.

Sharer	Receiver	LCF-X	XKV	T2T
Qwen-0.6B	Qwen-0.6B	29.1/40.6	29.5 / 40.7	16.6/27.1
Qwen-0.6B	Gemma-1B	23.3/34.0	23.8 / 34.7	22.4/34.0
Qwen-0.6B	Llama-3B	41.1/54.5	41.5 / 55.0	30.4/44.0
Gemma-1B	Qwen-0.6B	29.4/40.9	29.5 / 41.0	15.5/25.7
Gemma-1B	Gemma-1B	23.5/ 34.4	23.9 / 34.4	21.7/33.1
Gemma-1B	Llama-3B	40.2/53.5	41.5 / 55.1	30.4/44.5

Table 14: HotpotQA-bridge EM/F1 scores (%). Higher is better; cell-wise best values are bold. Mean across the nine model pairs: LCF-X 31.06/42.96; XKV 31.72/43.61; T2T 24.72/37.20.

Sharer	Receiver	LCF-X		XKV		T2T
Sharer	Receiver	Comm.	E2E	Comm.	E2E	E2E
Qwen-0.6B	Qwen-0.6B	64	259	6	201	995
Qwen-0.6B	Gemma-1B	65	295	5	240	1001
Qwen-0.6B	Llama-3B	65	224	6	169	999
Gemma-1B	Qwen-0.6B	62	266	7	213	1332
Gemma-1B	Gemma-1B	58	294	5	248	1372

Table 15: HotpotQA-bridge latency in milliseconds per example. Communication is measured only for the latent modules (LCF-X fusor and XKV translator); T2T therefore reports end-to-end latency only. Lower is better; cell-wise minima at the displayed precision are bold. Means across the nine model pairs: LCF-X 62.9/258.7; XKV 5.8/207.7; T2T E2E 1248.3.

Sharer	Receiver	Full	$-\mathrm{RP}$	$-\mathrm{RX}$	LCFP
Qwen-0.6B	Qwen-0.6B	82.2	81.6	80.8	81.7
Qwen-0.6B	Gemma-1B	55.1	53.8	49.6	51.9
Qwen-0.6B	Llama-3B	93.3	93.1	92.8	93.1
Gemma-1B	Qwen-0.6B	82.4	81.9	81.3	81.3
Gemma-1B	Gemma-1B	53.9	52.3	49.6	54.0
Gemma-1B	Llama-3B	93.0	93.6	93.2	93.5

Table 16: QASC accuracy scores (%). Higher is better; cell-wise best values are bold. Mean across the nine model pairs: Full 76.67;

-\mathrm{RP}

76.26;

-\mathrm{RX}

74.69; LCFP 75.84.

Sharer	Receiver	Full	$-\mathrm{RP}$	$-\mathrm{RX}$	LCFP
Qwen-0.6B	Qwen-0.6B	6/119	5/118	4 / 115	48/320
Qwen-0.6B	Gemma-1B	5/ 136	4/139	3 /138	31/167
Qwen-0.6B	Llama-3B	6/102	5/102	4 / 101	33/130
Gemma-1B	Qwen-0.6B	6/127	5/128	4 / 123	39/161
Gemma-1B	Gemma-1B	5/ 145	4/146	3 /147	36/178
Gemma-1B	Llama-3B	6/108	5/110	4 / 106	38/146

Table 17: QASC ablation latency in milliseconds per example. Entries are translator/total; lower is better and cell-wise minima are bold. Mean across the nine model pairs: Full 5.7/119.3;

-\mathrm{RP}

4.7/120.2;

-\mathrm{RX}

3.7/118.4; LCFP 37.4/171.3.

Sharer	Receiver	Full	$-\mathrm{RP}$	$-\mathrm{RX}$	LCFP
Qwen-0.6B	Qwen-0.6B	47.9/55.6	49.5 / 56.6	46.3/53.5	45.6/52.7
Qwen-0.6B	Gemma-1B	46.2/ 54.3	47.0 /54.1	45.5/53.7	46.6/53.9
Qwen-0.6B	Llama-3B	57.3 / 61.8	53.3/59.2	55.6/61.2	53.7/60.3
Gemma-1B	Qwen-0.6B	49.1 / 55.4	45.6/52.1	44.9/51.7	44.7/53.6
Gemma-1B	Gemma-1B	48.1 / 55.6	46.7/54.2	47.0/53.8	44.0/51.2
Gemma-1B	Llama-3B	54.0/58.7	52.8/58.1	57.5 / 62.9	52.0/58.3

Table 18: ROPES EM/F1 scores (%). Higher is better; cell-wise best values are bold. Mean across the nine model pairs: Full 50.48/56.99;

-\mathrm{RP}

49.38/56.17;

-\mathrm{RX}

49.79/56.54; LCFP 47.77/54.99.

Sharer	Receiver	Full	$-\mathrm{RP}$	$-\mathrm{RX}$	LCFP
Qwen-0.6B	Qwen-0.6B	6/ 139	5/ 139	4 /140	34/171
Qwen-0.6B	Gemma-1B	5/ 160	4/163	3 /167	33/194
Qwen-0.6B	Llama-3B	6/119	5/118	4 / 116	34/147
Gemma-1B	Qwen-0.6B	6/159	6/153	4 / 147	39/186
Gemma-1B	Gemma-1B	5/171	4/ 169	3 / 169	38/210
Gemma-1B	Llama-3B	6/ 126	5/127	4 /127	40/167

Table 19: ROPES ablation latency in milliseconds per example. Entries are translator/total; lower is better and cell-wise minima are bold. Mean across the nine model pairs: Full 5.7/141.3;

-\mathrm{RP}

4.8/144.6;

-\mathrm{RX}

3.7/141.1; LCFP 37.4/177.9.

Sharer	Receiver	Full	$-\mathrm{RP}$	$-\mathrm{RX}$	LCFP
Qwen-0.6B	Qwen-0.6B	61.4	63.2	61.7	63.2
Qwen-0.6B	Gemma-1B	58.2	58.2	57.3	59.1
Qwen-0.6B	Llama-3B	73.6	72.1	73.8	72.6
Gemma-1B	Qwen-0.6B	62.0	63.2	61.7	64.4
Gemma-1B	Gemma-1B	58.8	59.9	57.9	57.9
Gemma-1B	Llama-3B	74.1	73.0	77.1	73.6

Table 20: StrategyQA accuracy scores (%). Higher is better; cell-wise best values are bold. Mean across the nine model pairs: Full 65.01;

-\mathrm{RP}

64.86;

-\mathrm{RX}

64.93; LCFP 65.02.

Sharer	Receiver	Full	$-\mathrm{RP}$	$-\mathrm{RX}$	LCFP
Qwen-0.6B	Qwen-0.6B	10/122	5/ 118	4 /122	78/564
Qwen-0.6B	Gemma-1B	5/140	4/140	3 / 139	32/169
Qwen-0.6B	Llama-3B	6/109	5/ 103	4 /104	34/134
Gemma-1B	Qwen-0.6B	6/126	5/ 123	4 /128	38/161
Gemma-1B	Gemma-1B	5/ 146	4/148	3 /149	36/179
Gemma-1B	Llama-3B	6/112	6/116	4 / 108	43/152

Table 21: StrategyQA ablation latency in milliseconds per example. Entries are translator/total; lower is better and cell-wise minima are bold. Mean across the nine model pairs: Full 6.1/134.6;

-\mathrm{RP}

4.8/121.2;

-\mathrm{RX}

3.7/121.7; LCFP 42.4/201.9.

Method	Macro native score	Best datasets	Best/tied-best cells	Avg. rank
LCF-X	49.76	0/5	4/45	2.29
XKV	52.37	4/5	31/45	1.37
T2T	48.86	1/5	11/45	2.34

Table 22: Overall score comparison on five datasets (45 model-pair cells). The native score is F1 for generative QA and accuracy for classification. Tied best values at the displayed precision are counted for each tied method.

Method	Communication (ms)	E2E (ms)	E2E relative to XKV	Trainable params
LCF-X	59.9	227.9	1.36 $\times$	19.03M
XKV	5.8	167.6	1.00 $\times$	4.55M
T2T	–	1139.5	6.80 $\times$	0

Table 23: Overall efficiency on five datasets (45 model-pair cells). Communication is fusor-only latency for LCF-X and translator-only latency for XKV; it is not defined for T2T. XKV is 26.4% faster end-to-end than LCF-X, 6.8

\times

faster than T2T, and uses 76.1% fewer trainable parameters than LCF-X.

Dataset	$\Delta$ vs. LCF-X	$\Delta$ vs. T2T	Speedup vs. LCF-X	Speedup vs. T2T
ROPES	+4.20	+7.68	1.24 $\times$	8.90 $\times$
MuSiQue	+1.12	+2.67	1.59 $\times$	5.09 $\times$
QASC	+3.22	-0.27	1.35 $\times$	5.45 $\times$
StrategyQA	+3.84	+1.10	1.26 $\times$	9.99 $\times$
HotpotQA-bridge	+0.66	+6.41	1.25 $\times$	6.01 $\times$

Table 24: Per-dataset XKV improvements recomputed from the displayed detailed entries. Score deltas use F1 for generative QA and accuracy for classification; positive values favor XKV. Speedups are ratios of baseline E2E latency to XKV E2E latency.

Variant	$\Delta$ score	Avg. rank	Best/tied-best	Component (ms)	E2E (ms)	Params
Full	+0.00	1.87	13/27	5.8	131.7	4.55M
$-\mathrm{RP}$	-0.46	2.35	5/27	4.7	128.7	3.58M
$-\mathrm{RX}$	-0.83	3.02	5/27	3.7	127.1	3.43M
LCFP	-0.94	2.76	5/27	39.1	183.7	3.53M
$-\mathrm{RP}$ vs. Full	-0.46	–	–	-18.5%	-2.3%	-21.2%
$-\mathrm{RX}$ vs. Full	-0.83	–	–	-36.9%	-3.5%	-24.5%

Table 25: Ablation summary over 27 fully matched cells on QASC, ROPES, and StrategyQA, recomputed from the displayed detailed entries. Tied best values are counted for each tied variant.

Multi-Agent LLM Systems

Multi-agent large language model systems coordinate multiple AI agents to solve tasks through communication, specialization, shared tools, and memory.

See all 1,040 papers

May 19, 2026

Latent Cache Flow: Model-to-Model Communication Without Text

LLM agents today communicate via text, which incurs considerable latency and information loss due to the need to autoregressively decode the sharer model's state and encode at the receiver model. Recent work such as Cache-to-Cache (C2C; Fu et al., 2026) seeks to exchange KV caches by learning adapters that translate sharer KV matrices to the receiver model. However, the adapters are large and expensive to train, and translate individual tokens, which requires the target context to be identical. This is unsuitable for agent communication, where the LLMs have differing context. We introduce Latent Cache Flow (LCF). To address efficiency, we observe that keys and values can be jointly translated and compressed, reducing the adapter to about 4% of C2C's size. To address differing context, we design the adapter to transmit a summary of new information that the target model does not have. Our early experiments show that a pruned 13 MB LCF adapter can be more accurate than C2C at 956 MB in shared-context settings; for different contexts, LCF improves F1 by 7.5% and Exact Match by 23% while 8.5 times faster than text-based communication.

Maximillian Rossi, Prajwal Raghunath, Eugene Wu

Columbia University

KV Caching KV-Cache Compression Multi-Agent LLMs

Ask PDF

Sep 26, 2026

Prefill-Free Cross-Family KV Cache Transfer for Heterogeneous Multi-Agent LLMs

Recent multi-agent LLM systems increasingly combine heterogeneous models for specialized agent roles. However, text-based communication requires each receiver to prefill shared context already processed by the sender. Reusing the sender's key-value (KV) cache avoids this redundancy, but prefill-free transfer across model families must handle differences in tokenization, model depth, and KV representations. To address these issues, we propose HeteroFold, a prefill-free cross-family KV cache transfer method that keeps both the sender and receiver frozen. HeteroFold aligns model structures, maps the sender cache into the receiver space, and calibrates it to preserve receiver behavior. Across six transfer directions, HeteroFold achieves the best cache-transfer performance on all four long-context benchmarks and most short-context settings. It also matches text-based communication on the multi-agent benchmark. At 32K context length, Llama-3.1-8B

\rightarrow

Ministral-3-14B transfer is

10.7\times

faster than Native Prefill and

1.18

1.47\times

faster than the state-of-the-art prefill-free baselines, Dense Latent and KV Ridge. These results show that HeteroFold enables efficient cross-family KV reuse without receiver prefill.

Vincent-Daniel Yun, Woosang Lim, Haneul Yoo +3

University of Southern California, USA · Seoul National University, Republic of Korea · New York University, USA

Multi-Agent LLM Systems LLM Inference Acceleration Multi-LLM Collaboration

Ask PDF

Oct 12, 2026

Think Before You Speak: Enabling Training-Free Latent Communication Between Language Models

Large language models (LLMs) have demonstrated remarkable capabilities in solving complex tasks in coding, mathematical logic, and iterative reasoning, especially by coordinating effectively within multi-agent systems (MAS). However, language agents traditionally communicate via text exchange, which introduces a bottleneck in information transfer and adds prefill latency for downstream agents. Recent work proposes that directly sharing the key-value (KV) cache between language models can enable efficient latent collaboration, entirely bypassing standard text-based communication. To enable latent communication between heterogeneous models of different sizes, prior work introduces trained fusion adapters for each model pair, which limits practical deployment. We introduce CacheFuse, a training-free method which demonstrates that augmenting a downstream model with an upstream model's KV cache using linear interpolation consistently enables superior performance than the standalone downstream model can achieve. We systematically analyze CacheFuse across four reasoning and knowledge benchmarks and demonstrate up to 8.6 percentage points higher accuracy compared to single-model baselines and up to 12.2x TTFT speedup compared to text-based communication in multi-model inference.

Rishi Khare

Georgia Institute of Technology

Multi-LLM Collaboration LLM Inference Acceleration

Ask PDF

Abstract

Multi-Agent LLM Systems

Explore similar work

Latent Cache Flow: Model-to-Model Communication Without Text

Prefill-Free Cross-Family KV Cache Transfer for Heterogeneous Multi-Agent LLMs

Think Before You Speak: Enabling Training-Free Latent Communication Between Language Models