Multi-agent systems built from frozen language models communicate through text, which discards the internal state the sender already computed and makes the receiver pay to rebuild it. Work on internal channels avoids that cost, framing the problem as transfer: agent A sends information to agent B. Every agent pair that must communicate needs its own transfer bridge, and every agent added to a system needs a new bridge fitted against each previous agent. We reframe this problem as sharing: an agent publishes its internal state once into a pool, and every peer reads that pool through a single mechanism fitted to no sender in particular, so adding an agent adds a publisher and no new bridge. Whereas the carrier of the internal state and the fusion mechanism are usually chosen independently in the literature, our main finding is that these two choices are coupled; the deciding factor is where the carrier enters the receiver's computation. Consumability is a property of the receiving computation rather than of the representation. We find that the key-value (KV) cache must be augmented rather than swapped in: the receiver cross-attends over a peer's cache while keeping its own, and with only a small query-side adapter trained, matches the accuracy it gets from reading the source document. Activations invert this. They are best written straight into the receiver, with no bridge, no adapter and no training, which beats exposing them alongside the receiver's own, though that advantage rests on knowing where in the receiver the state belongs. The rule is augment the KV cache, replace the activation, established under one protocol with a shared control ladder. One adapter, trained once, also serves publishers it never trained on, and a pool of several at once, but only those whose internal state lies close to the receiver's own.
Figures & tables
Figure 1: The two carriers take opposite mechanisms. (a) The KV cache cannot be substituted, even at the fidelity annotated on the bar, and must be augmented instead. (b) Activations invert this: direct replacement works, training-free. Floors and native references are per carrier.
α=0.0
0.1
0.25
0.5
0.75
1.0
k=1
0.180
0.185
0.110
0.005
0.000
0.000
k=2
0.180
0.180
0.110
0.020
0.010
0.005
k=4
0.180
0.205
0.120
0.010
0.005
0.000
k=8
0.180
0.175
0.120
0.005
0.000
0.000
k=16
0.180
0.145
0.135
0.020
0.015
0.010
Table 1: Oracle substitution: the receiver’s key is interpolated toward the k -th nearest key the receiver itself produced elsewhere, so every candidate is on its own manifold. No neighbor rank recovers a graded tolerance. Qwen2.5-1.5B, n=200 , floor 0.18 , ceiling 0.665 .
Setting
Adapted
Shared
Control
Δ vs. raw
vs. ceiling
same model
none (raw)
0.555
0.185
—
−0.110
q
0.600
0.155
+0.045
−0.065
k,v
0.640
0.090
+0.085
−0.025
q,k,v
0.695
0.200
+0.140
+0.030
cross family
none (raw)
0.010
0.005
—
—
q
0.010
0.005
+0.000
—
Table 2: KV augmentation. The receiver keeps its own cache and cross-attends over the holder’s verbatim cache; only the receiver’s projections adapt. The control feeds a holder cache built on the answer-masked passage. Same-model, n=200 .
Inject layer
Shared fact
Cond. acc.
Span-copy
Mismatch ctrl
No-op ctrl
L=4
0.533
0.708
0.167
0.067
0.200
L=8
0.550
0.719
0.167
0.058
0.200
L=12
0.567
0.742
0.175
0.100
0.200
L=16
0.567
0.708
0.167
0.408
0.200
L=20
0.575
0.708
0.183
0.425
0.200
all of the above
0.517
0.685
0.142
0.067
0.200
Table 3: Activation replacement, training-free. The receiver’s hidden states at the fact region are overwritten with the holder’s. Qwen2.5-1.5B, n=120 ( condn=89 ), floor 0.20 , ceiling 0.742 . The mismatch control is clean at L≤12 but not at L≥16 (shaded reading in the text).
Figure 2: Carrier ordering across injection depth: activations dominate values, which dominate keys, and keys never leave the floor band. The clean operating band is shallower than the peak (Section 4.3 ).
Mechanism
KV cache ( K,V )
Activations ( h )
Substitution / replacement
0.000 at floor, dead
0.742 at ceiling
Augmentation
0.695 at ceiling
0.438 above floor, below replacement
Winning mechanism
augment
replace
Tolerable displacement
cos≈0.995
cos≈0.90
Consumption point
through the softmax
before the softmax
Cross-family
collapses
collapses
Table 4: The asymmetry. Best result per (carrier, mechanism) cell, each against its own probe’s anchors. Cache numbers are shared-fact accuracy at n=200 (floor 0.18 , ceiling 0.665 ); activation numbers are conditional accuracy at n=120 (floor 0.20 , ceiling 0.742 ). Cells are not comparable across columns in absolute terms, only relative to their anchors.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Multi-source KV augmentation over a pool of N holders, one of which read the answer-bearing passage. (a) Raw cross-attention collapses below the floor by N=4 ; the trained query-side bridge decays gracefully and remains above floor at N=8 . Dotted lines are the corresponding controls, which themselves lift above floor at large N . (b) Lift over the strongest control at each pool size: positive throughout, eroding with N .
sender
cosine
rel. diff.
raw lift
bridge shared
bridge lift
95% interval
self (seen)
—
—
+0.070
0.550
+0.310
[+0.219,+0.401]
instruct (held out)
0.9994
0.033
+0.070
0.550
+0.325
[+0.235,+0.415]
coder (held out)
0.9933
0.170
+0.000
0.130
+0.030
[−0.032,+0.092]
math (held out)
0.9810
0.433
+0.000
0.100
−0.005
[−0.064,+0.054]
Appendix
Table 5: Held-out senders. One rank-16 query-side bridge, trained on pools built from the receiver’s own weights only, evaluated against senders it never saw. Senders are Qwen2.5-1.5B checkpoints with the receiver’s geometry and a bit-identical tokeniser, ordered by the cosine between their cache and the receiver’s. We report relative Frobenius difference alongside cosine: the two rank the senders identically, so the ordering is not an artifact of one metric, though two metrics over three checkpoints cannot settle which is the right notion of compatibility. Lift is shared accuracy minus the stronger of the two controls; intervals are 95% normal-approximation intervals on that difference. Pool size N=4 , n=200 , floor 0.18 , native ceiling 0.665 . The self row reproduces the N=4 point of Figure 3 exactly.
Figure 4: Held-out senders. (a) The bridge, trained on the receiver’s own pools only, reaches the same accuracy on the nearest held-out sender as on the sender it trained on, and falls to its control on the two more distant senders. (b) Control-subtracted lift against the cosine between the sender’s cache and the receiver’s own. The effect is present at 0.9994 and absent by 0.9933 , which measures where the effect falls on the sender axis. Three checkpoints locate a transition, not a radius.
receiver
L
G
floor
identity
nearest genuine
Pythia-70M
6
1
0.000
0.025
0.000
GPT-2
12
1
0.033
0.233
0.000
Pythia-410M
24
1
0.067
0.375
0.000
Pythia-1.4B
24
1
0.092
0.517
0.000
Qwen2.5-0.5B
24
7
0.092
0.633
0.008
Qwen2.5-1.5B
28
6
0.200
0.742
0.000
Appendix
Table 6: The substitution bound across architectures. Each receiver is probed with its own bank of genuine keys and its own anchors. Identity hands the receiver its own exact keys and recovers that receiver’s ceiling in every case, which verifies the harness on each architecture. Nearest genuine substitutes the closest key the receiver itself produced elsewhere. Depth spans L=6 to L=36 and the grouped-query factor spans G=1 (multi-head) to G=8 . n=120 .
Figure 5: Why the two carriers differ. (a) The measured readout Jacobian gain is O(1) both on and off manifold, with a ratio of only 1.47× : the wall is not built by amplification, so no single layer and no stacking of layers blows up a tolerable perturbation. (b) Tolerable displacement by carrier: activations remain usable to cosine 0.90 , keys only to ≈0.995 , about 20× less headroom. Same receiver, same depth; the difference is the consumption point.
Figure 6: The boundaries. (a) Cross-family KV augmentation collapses at every rung of the adaptation ladder, while the same ladder within one model reaches the native ceiling. (b) On the activation carrier, the apparently strongest trained arm fails a mis-source control ( 0.300 against a floor of 0.19 ); the additive arm passes it ( 0.069 ) at a lower but genuine 0.454 .
Multi-agent communication studies how autonomous agents exchange information to coordinate decisions, share knowledge, and act together under constraints.