Share the Symbol, Not the Vector: Training-Free Fact Communication Across Language Models
Organizations: Nanyang Technological University · Yale University · University of Manchester
Abstract
Multi-agent systems allow language models to solve tasks collaboratively by sharing complementary information. These systems typically communicate through text, while recent approaches also explore exchanging internal model states. However, sharing these states across architectures often requires learned translators or assumptions about model compatibility. We introduce SYNC (Symbol-Indexed Neural Communication), a word-indexed interface that connects heterogeneous models without fine-tuning or a learned translator. From descriptions only it can see, the sender selects words from a shared list and sends their indices for each object, without seeing the questions. The receiver writes the described facts into its hidden states using its own word directions, vectors fitted once from its activations. We test selective use by swapping one attribute between two objects and requiring correct answers about both changed and unchanged attributes before and after the swap, four answers in total. We choose an early write layer on four development receivers and then fix it. With three different senders and nine newer receivers from five families, four-answer success on new scenes reaches 72.0-98.3%, up to 96 percentage points above middle-layer writing, and the two largest receivers match plain text. A task-free probe predicted in advance which depths each new receiver can use (45 of 45 cases). Without retuning, messages listing every fact keep 89.6-100% success under new layouts, and sender messages reach 23.9-64.0% on a landmark task with multi-token symbols, below text but from 0% without a message. These results show that shared word indices and receiver-local directions can support selective fact use across model families without additional training.
Figures & tables
| dictionary (log key) | kind | seconds | hours | log | how read |
|---|---|---|---|---|---|
| glm-4-9b | J-lens C=221 | 2560 | 0.71 | fit_glm-4-9b_inv2.log | integrated from per-batch s/batch |
| gpt-oss-20b | J-lens C=221 | 2304 | 0.64 | fit_gpt-oss-20b_inv2.log | integrated from per-batch s/batch |
| llama | J-lens C=221 | 1416 | 0.39 | fit_llama_inv2.log | integrated from per-batch s/batch |
| mistral-nemo | J-lens C=221 | 1800 | 0.50 | fit_mistral-nemo_inv2.log | integrated from per-batch s/batch |
| phi-4 | J-lens C=221 | 2824 | 0.78 | fit_phi-4_inv2.log | integrated from per-batch s/batch |
| qwen | J-lens C=221 | 1800 | 0.50 | fit_qwen_inv2.log | integrated from per-batch s/batch |
| sender | bank | k | dict | min_cand | source_dir | n | n_matched | fallback_rate | gis@4 grounded | gis@4 open | ev_cov grounded | disc.-token recall grounded | disc.-token recall open |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| llama-3.1-8b | ac-countries | 4 | acaug | 2 | m1_dev_k4post | 200 | 4.71 | 0.000 | 0.000 | 0.000 | 1.000 | 0.560 | 0.220 |
| llama-3.1-8b | ac-countries | 4 | acaug | 2 | m1_dev_mc2 | 200 | 4.71 | 0.000 | 0.000 | 0.000 | 1.000 | 0.350 | 0.045 |
| llama-3.1-8b | ac-countries | 4 | acaug | 8 | m1_dev | 200 | 4.71 | 0.960 | 0.000 | 0.000 | 0.251 | 0.205 | 0.045 |
| llama-3.1-8b | ac-countries | 4 | ladder20000 | 8 | m1_dev | 200 | 1.68 | 1.000 | 0.000 | 0.000 | 0.179 | 0.130 | 0.020 |
| llama-3.1-8b | ac-countries | 6 | acaug | 2 | m1_dev_k6 | 200 | 4.71 | 0.000 | – | – | 1.000 | 0.560 | 0.255 |
| llama-3.1-8b | ac-countries | 8 | acaug | 2 | m1_dev_k8 | 200 | 4.71 | 0.000 | – | – | 1.000 | 0.560 | 0.280 |
| model | role | eff. rank (of 221) | mean off-diag cos | p90 cos | max cos | distinct packets / MC episodes |
|---|---|---|---|---|---|---|
| llama-3.1-8b | sender | 19.7 | 0.492 | 0.606 | 0.876 | 267/300 |
| qwen3-8b-base | sender | 14.7 | 0.532 | 0.665 | 0.883 | 237/300 |
| qwen3.5-9b | sender | 11.7 | 0.570 | 0.715 | 0.902 | 27/300 |
| gemma-3-27b | sender | 6.0 | 0.588 | 0.879 | 0.985 | 1/300 |
| qwen3.5-27b | receiver | 16.1 | 0.521 | 0.661 | 0.879 | – |
| qwen3-32b | receiver | 24.7 | 0.468 | 0.573 | 0.852 | – |
| anchor | = replace random [95% CI] | clusters | entities | write | |
|---|---|---|---|---|---|
| qwen3-8b-base | 221 | +6.156 [+5.456,+6.907] | 200 | 48 | dose 1 |
| qwen3-8b-base | 1,000 | +6.168 [+5.464,+6.924] | 200 | 48 | dose 1 |
| qwen3-8b-base | 5,000 | +6.191 [+5.492,+6.942] | 200 | 48 | dose 1 |
| qwen3-8b-base | 20,000 | +6.205 [+5.505,+6.958] | 200 | 48 | dose 1 |
| llama-3.1-8b | 221 | +1.795 [+1.538,+2.058] | 200 | 48 | dose 1 |
| llama-3.1-8b | 1,000 | +1.790 [+1.532,+2.053] | 200 | 48 | dose 1 |
| anchor (sender) | extractor | n | gold-in-support@4 | |
|---|---|---|---|---|
| llama-3.1-8b | J-lens | 221 | 150 | 0.727 |
| llama-3.1-8b | J-lens | 1,000 | 150 | 0.620 |
| llama-3.1-8b | J-lens | 5,000 | 150 | 0.393 |
| llama-3.1-8b | probe | 221 | 300 | 0.463 |
| llama-3.1-8b | probe | 5,000 | 150 | 0.127 |
| llama-3.1-8b | probe | 20,000 | 150 | 0.047 |
| link | source | none | none_padded | inject_packet | inject_wrongset | inject_shuffled | inject_oracle | text_packet | text_concise | text_matched_text | text_matched_wire | text_oracle | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| L Q | 1 | jlens | 0.500 | 0.500 | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] | 0.645 | 0.566 | 0.531 | 0.500 | 0.997 |
| L Q | 2 | jlens | 0.500 | 0.500 | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] | 0.642 | 0.565 | 0.536 | 0.500 | 1.000 |
| L Q | 4 | jlens | 0.500 | 0.500 | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] | 0.650 | 0.583 | 0.542 | 0.497 | 1.000 |
| L Q | 8 | jlens | 0.500 | 0.500 | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] | 0.650 | 0.530 | 0.533 | 0.500 | 1.000 |
| L Q | 16 | jlens | 0.500 | 0.500 | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] | 0.598 | 0.525 | 0.512 | 0.500 | 1.000 |
| L Q | 1 | mix pilot | 0.500 | 0.500 | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] | 0.645 | 0.566 | 0.531 | 0.500 | 0.997 |
| link | source | dose | inject_packet | inject_oracle | inject_shuffled | |
|---|---|---|---|---|---|---|
| L Q | 1 | jlens | 0.5 | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] |
| L Q | 1 | jlens | 2 | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] |
| L Q | 1 | jlens | 4 | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] |
| L Q | 8 | jlens | 0.5 | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] |
| L Q | 8 | jlens | 2 | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] | +0.000 [+0.000,+0.000] |
| L Q | 8 | jlens | 4 | -0.003 [-0.008,+0.000] | -0.003 [-0.008,+0.000] | +0.000 [+0.000,+0.000] |
| dose policy | n | merged none [95% CI] | text_all none [95% CI] | merged wire bytes | text bytes | zero-packet | |
|---|---|---|---|---|---|---|---|
| 2 | fixed-norm 0.5 | 298 | -0.007 [-0.030,+0.017] | +0.359 [+0.305,+0.409] | 40.9 | 364.8 | exact |
| 2 | fixed-norm 1 | 298 | +0.010 [-0.017,+0.037] | +0.359 [+0.305,+0.409] | 40.9 | 364.8 | exact |
| 2 | fixed-norm 2 | 298 | +0.023 [-0.003,+0.050] | +0.359 [+0.305,+0.409] | 40.9 | 364.8 | exact |
| 2 | fixed-norm 4 | 298 | -0.003 [-0.034,+0.027] | +0.359 [+0.305,+0.409] | 40.9 | 364.8 | exact |
| 2 | fixed- | 298 | +0.023 [+0.000,+0.047] | +0.359 [+0.305,+0.409] | 40.9 | 364.8 | exact |
| 4 | fixed-norm 1 | 296 | +0.003 [-0.020,+0.027] | +0.291 [+0.230,+0.345] | 60.7 | 725.5 | exact |
| link | contrast | estimate [95% CI] | n | entities | NI |
|---|---|---|---|---|---|
| llama-3.1-8b qwen3-8b-base | PRIMARY Delta m-permX (gap) | +2.2364 [+1.6302,+2.7742] * | 861 | 47 | – |
| llama-3.1-8b qwen3-8b-base | m-none (gap) | +2.1997 [+1.5911,+2.7475] * | 900 | 47 | – |
| llama-3.1-8b qwen3-8b-base | m-none (acc) | +0.1600 [+0.1193,+0.1977] * | 900 | 47 | – |
| llama-3.1-8b qwen3-8b-base | oracle-none (acc) | +0.4311 [+0.4034,+0.4600] * | 900 | 47 | – |
| llama-3.1-8b qwen3-8b-base | m-textpkt (gap) | -0.4326 [-0.7789,-0.0610] * | 900 | 47 | – |
| llama-3.1-8b qwen3-8b-base | m-textpkt (acc, NI -0.05) | -0.0233 [-0.0530,+0.0069] n.s. | 900 | 47 | MISS |
| bank (test) | contrast | scope | acc [95% CI] | n items | entities | |
|---|---|---|---|---|---|---|
| ac-countries | packet donor’s packet | full | +0.0517 [+0.0216,+0.0823] | 0.0010 | 400 | 78 |
| ac-countries | packet donor’s packet | hard slice | +0.0515 [+0.0249,+0.0813] | 0.0001 | 298 | 74 |
| ac-countries | packet random addresses | full | +0.0483 [+0.0220,+0.0747] | 0.0001 | 400 | 78 |
| ac-countries | packet random addresses | hard slice | +0.0481 [+0.0226,+0.0756] | 0.0001 | 298 | 74 |
| ac-countries | packet no comm | full | +0.0533 [+0.0052,+0.1024] | 0.0342 | 400 | 78 |
| ac-countries | packet no comm | hard slice | +0.1779 [+0.1355,+0.2226] | 0.0001 | 298 | 74 |
| llama-3.1-8b | qwen3-8b-base | two-sender mean | zero-integrity | |
|---|---|---|---|---|
| 8 | +0.0155 [-0.0386,+0.0722] | -0.0134 [-0.0861,+0.0595] | +0.0010 | yes |
| 16* | +0.0473 [-0.0164,+0.1182] | -0.0023 [-0.0690,+0.0696] | +0.0225 | yes |
| 24 | +0.0484 [+0.0052,+0.0953] | -0.0178 [-0.0701,+0.0395] | +0.0153 | yes |
| 32 | +0.0089 [-0.0290,+0.0480] | -0.0114 [-0.0635,+0.0447] | -0.0012 | yes |
| 48 | +0.0104 [-0.0210,+0.0431] | -0.0074 [-0.0383,+0.0265] | +0.0015 | yes |
| link | benchmark | dim | [95% CI] | [95% CI] | |
|---|---|---|---|---|---|
| llama-3.1-8b glm-4-9b | ac-countries | same | -0.0008 [-0.0180,+0.0164] | -0.0033 [-0.0233,+0.0167] | |
| llama-3.1-8b glm-4-9b | ac-tipsheets | same | +0.0000 [-0.0143,+0.0144] | -0.0058 [-0.0246,+0.0111] | |
| llama-3.1-8b llama-3.1-8b | ac-countries | same | +0.0000 [-0.0143,+0.0137] | -0.0100 [-0.0301,+0.0093] | |
| llama-3.1-8b llama-3.1-8b | ac-tipsheets | same | +0.0017 [-0.0169,+0.0199] | -0.0008 [-0.0291,+0.0250] | |
| llama-3.1-8b qwen3-32b | ac-countries | cross | +0.0383 [+0.0102,+0.0687]* | +0.0017 [-0.0442,+0.0480] | primary |
| llama-3.1-8b qwen3-32b | ac-tipsheets | cross | -0.0108 [-0.0251,+0.0017] | +0.0167 [+0.0048,+0.0316] |
| [95% CI] | one-sided | CI excludes 0 | zero-write | |
|---|---|---|---|---|
| 8 | +0.1199 [-0.0196,+0.2156] | 0.0430 | no | exact |
| 16 | +0.1776 [+0.0247,+0.3079] | 0.0139 | yes | exact |
| 24 | +0.1906 [+0.0457,+0.3298] | 0.0057 | yes | exact |
| 32 | +0.1864 [+0.0298,+0.3359] | 0.0102 | yes | exact |
| 48 ∗ | +0.2390 [+0.0781,+0.3856] | 0.0019 | yes | exact |
| llama-3.1-8b [95% CI] | qwen3-8b-base [95% CI] | |
|---|---|---|
| 8 | +0.1199 [-0.0196,+0.2156] | +0.1459 [+0.0165,+0.2612] † |
| 16 | – | +0.1605 [+0.0060,+0.3052] |
| 24 | – | +0.1567 [+0.0182,+0.3014] |
| 32 | – | +0.1620 [-0.0211,+0.3586] ∗ |
| 48 | +0.2390 [+0.0781,+0.3856] | +0.0621 [-0.0911,+0.2417] |
| 56 | +0.2636 [+0.0955,+0.4116] † | – |
| sender (bank) | [95% CI] | one-sided | Holm | zero-write | |
|---|---|---|---|---|---|
| llama-3.1-8b (T3 relation) | 24 | -0.0888 [-0.2534,+0.0770] | 0.8548 | 1.000 | exact |
| llama-3.1-8b (T3 relation) | 48 | -0.0163 [-0.2029,+0.1611] | 0.5641 | 1.000 | exact |
| llama-3.1-8b (T3 relation) | 64 | -0.0450 [-0.2088,+0.1034] | 0.7078 | 1.000 | exact |
| llama-3.1-8b (T3 relation) | 96 | -0.0969 [-0.2170,+0.0199] | 0.9479 | 1.000 | exact |
| qwen3-32b (T2 attribute) | 48 | -0.1814 [-0.3133,-0.0688] | 0.9994 | – | exact |
| qwen3-32b (T2 attribute) | 64 | -0.1550 [-0.2769,-0.0562] | 0.9995 | – | exact |
| sender | (matched) | (donor) | gold-in-support | ||
|---|---|---|---|---|---|
| llama-3.1-8b | 48 | +0.2231 | -0.0159 | +0.2390 | 0.162 |
| llama-3.1-8b | 64 | +0.2129 | -0.0469 | +0.2597 | 0.162 |
| qwen3-8b-base | 48 | +0.1629 | +0.1009 | +0.0621 | 0.108 |
| qwen3-32b | 48 | -0.2276 | -0.0462 | -0.1814 | 0.218 |
| qwen3-32b | 64 | -0.1817 | -0.0267 | -0.1550 | 0.218 |
| leg | link | [95% CI] | one-sided | zero-write | |
|---|---|---|---|---|---|
| leg A (cross-family) | qwen3-32b glm-4-9b | 8 | +0.1632 [-0.0113,+0.3654] | 0.0337 | exact |
| leg A (cross-family) | qwen3-32b glm-4-9b | 16 | +0.2006 [-0.0291,+0.4698] | 0.0416 | exact |
| leg A (same-family, never pooled) | qwen3-32b qwen3-8b-base | 16 | +0.1373 [-0.0813,+0.3558] | 0.1109 | exact |
| leg A (same-family, never pooled) | qwen3-32b qwen3-8b-base | 32 | +0.0292 [-0.0680,+0.1283] | 0.2761 | exact |
| leg B (incumbent receiver) | qwen3-8b-base glm-4-9b | 8 ∗ | +0.3660 [+0.1936,+0.5272] | 0.0001 | exact |
| leg B (incumbent receiver) | qwen3-8b-base glm-4-9b | 16 ‡ | +0.3400 [+0.1886,+0.5696] | 0.0001 | exact |
| contrast | corrected fit | CMDV0902 (pre-fix fit) |
|---|---|---|
| H | +0.0342 [+0.0055,+0.0646] | +0.0383 [+0.0102,+0.0687] |
| D_cmdv | +0.0058 [-0.0410,+0.0518] | +0.0017 [-0.0442,+0.0480] |
| cmdv_vs_acgraft | +0.0092 [-0.0118,+0.0307] | +0.0000 [-0.0192,+0.0185] |
| link | [95% CI] | one-sided | zero-write | |||
|---|---|---|---|---|---|---|
| llama-3.1-8b glm-4-9b | 8 | +0.3962 [+0.1364,+0.6357] | 0.0021 | 495 | 11 | exact |
| llama-3.1-8b glm-4-9b | 16 ∗ | +0.4986 [+0.1538,+0.8153] | 0.0020 | 495 | 11 | exact |
| qwen3-8b-base glm-4-9b | 8 ∗ | +0.5856 [+0.3881,+0.8115] | 0.0001 | 433 | 11 | exact |
| qwen3-8b-base glm-4-9b | 16 | +0.5642 [+0.3149,+0.8595] | 0.0001 | 433 | 11 | exact |
| cell | gold in own slot | all ( ) | hit episodes ( ) | non-hit episodes ( ) | top concept word (share) |
|---|---|---|---|---|---|
| llama-3.1-8b qwen3-14b-base | 3/592 (0.5%) | +0.566 (551) | +0.542 (3) | +0.566 (548) | scarf (36.5%) |
| llama-3.1-8b qwen3-30b-a3b | 3/592 (0.5%) | +1.443 (551) | +3.458 (3) | +1.432 (548) | scarf (36.5%) |
| llama-3.1-8b qwen3-32b | 3/592 (0.5%) | +1.433 (551) | +3.354 (3) | +1.423 (548) | scarf (36.5%) |
| llama-3.1-8b qwen3.5-27b | 3/592 (0.5%) | +0.827 (551) | +1.208 (3) | +0.824 (548) | scarf (36.5%) |
| llama-3.1-8b qwen3.5-397b | 1/296 (0.3%) | +1.034 (261) | -0.438 (1) | +1.040 (260) | scarf (36.5%) |
| qwen3-8b-base qwen3.5-27b | 43/592 (7.3%) | +0.541 (488) | +0.423 (21) | +0.546 (467) | block (23.3%) |
| cell | LOO min mean (dropped) | LOO worst CI-lo (dropped) | leave-two-out worst CI-lo | every interval | |
|---|---|---|---|---|---|
| llama-3.1-8b qwen3-14b-base | 14 | +0.507 (belt) | +0.306 (belt) | +0.262 | yes |
| llama-3.1-8b qwen3-30b-a3b | 14 | +1.173 (belt) | +0.506 (belt) | +0.376 | yes |
| llama-3.1-8b qwen3-32b | 14 | +1.032 (fox) | +0.300 (fox) | +0.134 | yes |
| llama-3.1-8b qwen3.5-27b | 14 | +0.692 (belt) | +0.317 (belt) | +0.247 | yes |
| llama-3.1-8b qwen3.5-397b | 14 | +0.817 (belt) | +0.281 (belt) | +0.164 | yes |
| qwen3-8b-base qwen3.5-27b | 14 | +0.476 (pear) | +0.295 (cocoa) | +0.252 | yes |
| cell | within 5% | median | max rel. norm gap | ||
|---|---|---|---|---|---|
| llama-3.1-8b qwen3-14b-base | 551 | 167/551 (30%) | 519/551 (94%) | 48 | 0.129 |
| llama-3.1-8b qwen3-30b-a3b | 551 | 168/551 (30%) | 518/551 (94%) | 47 | 0.124 |
| llama-3.1-8b qwen3-32b | 551 | 167/551 (30%) | 508/551 (92%) | 47 | 0.128 |
| llama-3.1-8b qwen3.5-27b | 551 | 168/551 (30%) | 506/551 (92%) | 47 | 0.117 |
| llama-3.1-8b qwen3.5-397b | 261 | 64/261 (25%) | 249/261 (95%) | 63 | 0.104 |
| qwen3-8b-base qwen3.5-27b | 488 | 56/488 (11%) | 372/488 (76%) | 45 | 0.137 |
| pair | lineage | Jaccard median | observed mean | null mean | gap | null zero-fraction | exact-set (obs.) |
|---|---|---|---|---|---|---|---|
| llama-3.1-8b vs llama-3.1-8b-instruct | same-base | 0.33 | 0.310 | 0.139 | +0.171 | 0.37 | 0.003 |
| llama-3.1-70b vs llama-3.1-8b | same-family | 0.17 | 0.213 | 0.114 | +0.099 | 0.45 | 0.000 |
| llama-3.1-70b vs llama-3.1-8b-instruct | same-family | 0.33 | 0.246 | 0.117 | +0.129 | 0.42 | 0.000 |
| qwen3-14b-base vs qwen3-30b-a3b | same-family | 0.00 | 0.076 | 0.069 | +0.008 | 0.57 | 0.000 |
| qwen3-14b-base vs qwen3-32b | same-family | 0.00 | 0.005 | 0.002 | +0.003 | 0.99 | 0.000 |
| qwen3-14b-base vs qwen3-8b-base | same-family | 0.00 | 0.015 | 0.006 | +0.009 | 0.96 | 0.000 |
| cell | (donor) | none | MC | MS | PC | PS | text packet | text oracle | zero packet | |
|---|---|---|---|---|---|---|---|---|---|---|
| llama-3.1-8b qwen3-14b-base | 592 | 551 | 48.6 | 58.6 | 36.7 | 49.2 | 42.8 | 62.7 | 100.0 | 48.6 |
| llama-3.1-8b qwen3-30b-a3b | 592 | 551 | 49.2 | 57.6 | 41.9 | 51.0 | 47.4 | 67.1 | 100.0 | 49.2 |
| llama-3.1-8b qwen3-32b | 592 | 551 | 49.0 | 60.3 | 37.0 | 52.6 | 44.8 | 64.9 | 99.8 | 49.0 |
| llama-3.1-8b qwen3.5-27b | 592 | 551 | 48.3 | 62.3 | 34.5 | 53.2 | 46.3 | 71.5 | 100.0 | 48.3 |
| llama-3.1-8b qwen3.5-397b | 296 | 261 | 44.9 | 61.5 | 35.8 | 52.9 | 51.0 | 69.6 | 100.0 | 44.9 |
| qwen3-8b-base qwen3.5-27b | 592 | 488 | 48.3 | 56.2 | 39.7 | 46.1 | 49.2 | 76.5 | 100.0 | 48.3 |
| channel | base (qwen2.5-14b) | post (R1 distill) | instruct sibling | [95% CI] | class | [95% CI] | [cons. CI] |
|---|---|---|---|---|---|---|---|
| unembed | +0.321 | +0.402 | +0.365 | 1.253 [1.191,1.329] | preserved | 1.137 [1.098,1.180] | 1.159 [1.072,1.260] (vs ) |
| probe | +0.320 | +0.261 | +0.332 | 0.814 [0.738,0.889] | preserved | 1.038 [0.989,1.089] | 0.788 [0.694,0.876] (vs ) |
| cell | episodes | four-arm | gold-entity 95% CI | (Holm) | verdict | worst LOO CI-lower (entity) | ||
|---|---|---|---|---|---|---|---|---|
| llama-3.1-8b qwen3-14b-base | 596 | 536 | 42 | +1.345 | [+0.995,+1.739] | 0.000875 | closes | +0.929 (dolphin) |
| llama-3.1-8b qwen3-30b-a3b | 596 | 536 | 42 | +4.477 | [+3.240,+5.805] | 0.000875 | closes | +3.035 (mango) |
| llama-3.1-8b qwen3-32b | 596 | 536 | 42 | +4.212 | [+3.000,+5.502] | 0.000875 | closes | +2.818 (dolphin) |
| llama-3.1-8b qwen3.5-27b | 596 | 536 | 42 | +2.504 | [+1.648,+3.398] | 0.000875 | closes | +1.524 (dolphin) |
| llama-3.1-8b qwen3.5-397b | 596 | 536 | 42 | +3.674 | [+2.625,+4.751] | 0.000875 | closes | +2.472 (dolphin) |
| qwen3-8b-base qwen3.5-27b | 596 | 548 | 42 | +0.777 | [+0.462,+1.141] | 0.000875 | closes | +0.413 (potato) |
| write dose | injected coverage | free | 95% CI | forced |
|---|---|---|---|---|
| 16 (adjudicated) | 0.058 | not evaluable | – | +0.566 |
| 8 | 0.711 | +0.111 | [+0.058,+0.161] | +0.321 |
| 4 | 0.885 | +0.078 | [+0.028,+0.134] | +0.182 |
| receiver | dose | injected coverage | free | 95% CI | forced | CI excludes 0 |
|---|---|---|---|---|---|---|
| qwen3-30b-a3b | 0.169 | +0.0436 | [+0.0100,+0.0809] | +0.5719 | yes | |
| qwen3-30b-a3b | 0.211 | +0.0018 | [-0.0129,+0.0207] | +0.2754 | no | |
| qwen3-32b | 0.479 | +0.0871 | [+0.0192,+0.1574] | +1.0375 | yes | |
| qwen3-32b | 0.454 | +0.0254 | [-0.0017,+0.0519] | +0.3033 | no | |
| qwen3.5-27b | 0.063 | +0.0363 | [+0.0076,+0.0695] | +0.4974 | yes | |
| qwen3.5-27b | 0.040 | +0.0328 | [+0.0068,+0.0568] | +0.2590 | yes |
| cell | on gap_seq | gold-entity 95% CI | |
|---|---|---|---|
| llama-3.1-8b qwen3-14b-base | +0.571 | [+0.356,+0.795] | 0.000125 |
| llama-3.1-8b qwen3-30b-a3b | +1.443 | [+0.666,+2.340] | 0.000125 |
| llama-3.1-8b qwen3-32b | +1.479 | [+0.514,+2.582] | 0.001125 |
| llama-3.1-8b qwen3.5-27b | +0.814 | [+0.398,+1.271] | 0.000125 |
| qwen3-8b-base qwen3.5-27b | +0.540 | [+0.348,+0.707] | 0.000125 |
| llama-3.1-8b -> qwen3.5-397b | VOID | replay 4/50 | 1.0 |
| contrast | Llama-3.1-8B | 95% CI | Qwen3-8B-Base | 95% CI |
|---|---|---|---|---|
| write the gold entity’s coordinate | +0.682 | [+0.544,+0.831] | +1.211 | [+0.804,+1.664] |
| gold coordinate random entity’s | +0.649 | [+0.521,+0.784] | +1.166 | [+0.879,+1.466] |
| write the foil ’s coordinate | -0.759 | [-0.891,-0.622] | -1.253 | [-1.627,-0.837] |
| random entity’s coordinate (null) | +0.033 | [-0.041,+0.106] | +0.045 | [-0.174,+0.284] |
| extracted coefficients | +0.245 | [+0.172,+0.323] | +0.236 | [+0.077,+0.423] |
| bank | split | clusters | opposite dir. | same dir. | both | pairs | entities |
|---|---|---|---|---|---|---|---|
| t2_dev400 | dev | 400 | 76 | 95 | 12 | 9 | 2 |
| t2_test | test | 300 | 127 | 46 | 10 | 5 | 1 |
| t2_confirm | dev | 200 | 37 | 61 | 9 | 7 | 2 |
| t2_xfament0905 | calibration | 284 | 58 | 76 | 8 | 4 | 2 |
| t2_confirm0818 | dev | 300 | 42 | 36 | 4 | 2 | 2 |
| t2_construction | construction | 20 | 10 | 6 | 3 | 2 | 2 |
| direction | budget | arm | acc vs none [95% CI] | mean bytes billed | n | entities |
|---|---|---|---|---|---|---|
| L Q | 12 | text_budget | +0.022 [+0.006,+0.040] | 9.8 | 900 | 49 |
| L Q | 12 | text_codebook | +0.087 [+0.061,+0.115] | 7.6 | 900 | 49 |
| L Q | 12 | text_gen | +0.369 [+0.333,+0.408] | 8.3 | 900 | 49 |
| L Q | 16 | text_budget | +0.038 [+0.017,+0.060] | 12.2 | 900 | 49 |
| L Q | 16 | text_codebook | +0.130 [+0.092,+0.172] | 11.4 | 900 | 49 |
| L Q | 16 | text_gen | +0.377 [+0.342,+0.416] | 9.6 | 900 | 49 |
| cell | Holm | (random dict) | covers 0 | ||
|---|---|---|---|---|---|
| llama-3.1-8b -> qwen3-14b-base | +0.721 [+0.302,+1.192] | 0.00150 | +0.025 [-0.217,+0.256] | yes | +2.83 |
| llama-3.1-8b -> qwen3-30b-a3b | +2.272 [+0.508,+4.330] | 0.00700 | +0.098 [-0.384,+0.585] | yes | +15.93 |
| llama-3.1-8b -> qwen3-32b | +1.586 [+0.363,+3.060] | 0.00700 | +0.475 [+0.002,+0.959] | NO | +9.79 |
| llama-3.1-8b -> qwen3.5-27b | +1.265 [+0.594,+2.034] | 0.00087 | -0.037 [-0.111,+0.049] | yes | +6.60 |
| llama-3.1-8b -> qwen3.5-397b | +1.434 [+0.461,+2.496] | 0.00337 | +0.014 [-0.110,+0.150] | yes | +8.94 |
| qwen3-8b-base -> qwen3.5-27b | +0.399 [+0.194,+0.595] | 0.00087 | +0.041 [-0.065,+0.157] | yes | +6.60 |
| cell | 95% CI | Holm | permuted-dict slot effect [CI] | |
|---|---|---|---|---|
| llama-3.1-8b -> qwen3-14b-base | +0.9443 | [+0.4637,+1.4804] | 0.00088 | -0.1985 [-0.4217,+0.0199] |
| llama-3.1-8b -> qwen3-30b-a3b | +2.5362 | [+0.6961,+4.6977] | 0.00550 | -0.1666 [-0.6669,+0.3041] |
| llama-3.1-8b -> qwen3-32b | +2.7622 | [+0.8328,+4.9024] | 0.00550 | -0.7013 [-1.7029,+0.3107] |
| llama-3.1-8b -> qwen3.5-27b | +1.3538 | [+0.5862,+2.2652] | 0.00088 | -0.1258 [-0.3825,+0.1058] |
| llama-3.1-8b -> qwen3.5-397b | +2.0513 | [+0.7171,+3.4395] | 0.00313 | -0.6030 [-1.4241,+0.1234] |
| qwen3-8b-base -> qwen3.5-27b | +0.1432 | [-0.0862,+0.3924] | 0.12812 | +0.2963 [+0.1000,+0.5015] |
| cell | free [95% CI] | Holm | closes | forced [95% CI] | cov. none/inj. | oracle (margin) | MC none, free (pp) | |
|---|---|---|---|---|---|---|---|---|
| llama-3.1-8b -> qwen3-14b-base | 8 | +0.2146 [+0.1439,+0.2919] | 0.00050 | yes | +0.8001 [+0.5736,+1.0617] | 0.997/0.895 | 0.990 (+0.492) | +9.90 [+6.04,+13.80] |
| llama-3.1-8b -> qwen3.5-27b | 4 | +0.0000 [-0.0345,+0.0337] | 1.00000 | no | +0.9534 [+0.6157,+1.3258] | 0.983/0.732 | 1.000 (+0.508) | -11.74 [-15.50,-8.22] |
| qwen3-8b-base -> qwen3.5-27b | 4 | -0.0164 [-0.0391,+0.0038] | 1.00000 | no | +0.2392 [+0.1204,+0.3731] | 0.983/0.903 | 1.000 (+0.508) | -4.19 [-6.29,-2.27] |
| qwen3-8b-base -> glm-4-9b | 2 | +0.0493 [+0.0148,+0.0862] | 0.00825 | yes | +0.1649 [+0.1036,+0.2368] | 0.678/0.609 | 0.990 (+0.651) | -1.01 [-3.50,+1.51] |
| group | (linear) | (flex) | pairs null | Holm | (fixed support) | positive control | |
|---|---|---|---|---|---|---|---|
| G1 (17-sender bank) | +0.723 [+0.656,+0.773] | +0.724 | 28/28 | 0.0010 | +0.661 [+0.578,+0.718] | +0.063 [+0.034,+0.095] | 0/8 |
| G2 (lineage bank) | +0.740 [+0.694,+0.782] | +0.745 | 15/15 | 0.0010 | – | not measured | 0/6 |
| bank | cell | sign-flip (Holm ok) | joint gold+donor 95% CI | acc. DiD pp [95% CI] | MC none / MS none (pp) | within-tol. donors [CI] ( ) | ||
|---|---|---|---|---|---|---|---|---|
| CC1 | llama-3.1-8b qwen3-14b-base | +0.566 | 14 | 0.0001 (yes) | [+0.305,+0.854] | +15.8 [+8.5,+22.5] | +10.3 / -11.8 | +0.516 [+0.215,+0.849] (167) |
| CC1 | llama-3.1-8b qwen3-30b-a3b | +1.443 | 14 | 0.0009 (yes) | [+0.526,+2.578] | +13.2 [+3.7,+22.3] | +9.3 / -7.6 | +1.019 [-0.045,+2.234] (168) |
| CC1 | llama-3.1-8b qwen3-32b | +1.433 | 14 | 0.0054 (yes) | [+0.388,+2.642] | +15.4 [+4.1,+28.4] | +11.1 / -12.2 | +0.906 [+0.120,+1.829] (167) |
| CC1 | llama-3.1-8b qwen3.5-27b | +0.827 | 14 | 0.0004 (yes) | [+0.356,+1.423] | +21.1 [+13.3,+28.9] | +14.0 / -14.0 | +0.602 [+0.186,+1.125] (168) |
| CC1 | llama-3.1-8b qwen3.5-397b | +1.034 | 14 | 0.0035 (yes) | [+0.126,+1.983] | +22.6 [+10.4,+34.8] | +18.4 / -6.1 | +0.404 [-0.018,+0.923] (64) |
| CC1 | qwen3-8b-base qwen3.5-27b | +0.541 | 14 | 0.0005 (yes) | [+0.332,+0.735] | +21.5 [+10.5,+31.7] | +8.6 / -9.8 | +0.877 [+0.573,+1.214] (38) |
| bank | cell | attr. in gold slot | attr. anywhere | gold entity in own slot | any entity anywhere | , attr. hit [CI] ( ) | attr. no-hit [CI] ( ) | gold-slot hit [CI] ( ) | gold-slot no-hit [CI] ( ) |
|---|---|---|---|---|---|---|---|---|---|
| CC1 | llama-3.1-8b qwen3-14b-base | 0.0% | 0.0% | 0.5% | 14.9% | n too small (0) | +0.566 [+0.347,+0.796] (551) | n too small (3) | +0.566 [+0.346,+0.797] (548) |
| CC1 | llama-3.1-8b qwen3-30b-a3b | 0.0% | 0.0% | 0.5% | 14.9% | n too small (0) | +1.443 [+0.659,+2.348] (551) | n too small (3) | +1.432 [+0.641,+2.344] (548) |
| CC1 | llama-3.1-8b qwen3-32b | 0.0% | 0.0% | 0.5% | 14.9% | n too small (0) | +1.433 [+0.467,+2.519] (551) | n too small (3) | +1.423 [+0.450,+2.516] (548) |
| CC1 | llama-3.1-8b qwen3.5-27b | 0.0% | 0.0% | 0.5% | 14.9% | n too small (0) | +0.827 [+0.407,+1.300] (551) | n too small (3) | +0.824 [+0.401,+1.300] (548) |
| CC1 | llama-3.1-8b qwen3.5-397b | 0.0% | 0.0% | 0.4% | 13.8% | n too small (0) | +1.034 [+0.404,+1.737] (261) | n too small (1) | +1.040 [+0.409,+1.744] (260) |
| CC1 | qwen3-8b-base qwen3.5-27b | 0.0% | 0.0% | 4.3% | 6.4% | n too small (0) | +0.541 [+0.345,+0.712] (488) | +0.423 [-0.250,+0.759] (21) | +0.546 [+0.350,+0.720] (467) |
| cell | 95% CI | Holm | ablated-dict slot effect [CI] | |
|---|---|---|---|---|
| llama-3.1-8b -> qwen3-14b-base | +0.1426 | [-0.0676,+0.3858] | 0.70613 | +1.4174 [+1.1224,+1.7693] |
| llama-3.1-8b -> qwen3-30b-a3b | +0.2417 | [-0.2672,+0.8116] | 1.00000 | +4.7418 [+3.6972,+5.8301] |
| llama-3.1-8b -> qwen3-32b | +0.1618 | [-0.3405,+0.6858] | 1.00000 | +4.3060 [+3.2075,+5.4336] |
| llama-3.1-8b -> qwen3.5-27b | -0.0076 | [-0.2469,+0.2637] | 1.00000 | +2.6722 [+2.0740,+3.3250] |
| llama-3.1-8b -> qwen3.5-397b | -0.5159 | [-1.0941,+0.0749] | 1.00000 | +4.5019 [+3.5274,+5.5002] |
| qwen3-8b-base -> qwen3.5-27b | -0.1224 | [-0.2496,+0.0002] | 1.00000 | +0.8923 [+0.5532,+1.2751] |
| cell | episodes | four-arm | gold-entity 95% CI | (Holm) | verdict | ||
|---|---|---|---|---|---|---|---|
| llama-3.1-8b qwen3-14b-base | 596 | 536 | 42 | +0.800 | [+0.572,+1.059] | 0.000875 | closes |
| llama-3.1-8b qwen3-30b-a3b | 596 | 536 | 42 | +4.477 | [+3.240,+5.805] | 0.000875 | closes |
| llama-3.1-8b qwen3-32b | 596 | 536 | 42 | +2.231 | [+1.375,+3.178] | 0.000875 | closes |
| llama-3.1-8b qwen3.5-27b | 596 | 536 | 42 | +2.504 | [+1.648,+3.398] | 0.000875 | closes |
| llama-3.1-8b qwen3.5-397b | 596 | 536 | 42 | +1.676 | [+1.266,+2.087] | 0.000875 | closes |
| qwen3-8b-base qwen3.5-27b | 596 | 548 | 42 | +0.777 | [+0.462,+1.141] | 0.000875 | closes |
| bank / receiver | episodes/entities | linear [95% CI] (flex) | raw perm. | [95% CI] | [95% CI] | pos. ctrl | frozen |
|---|---|---|---|---|---|---|---|
| CC1 / qwen3.5-27b ( 8) | 592/14 | +0.344 [+0.233,+0.520] (+0.335) | 0.0005 | +0.372 [+0.244,+0.509] | -0.028 [-0.124,+0.109] | 2/2 | VOID |
| CC1 / qwen3.5-397b ( 4) | 296/14 | +0.214 [+0.134,+0.304] (+0.218) | 0.0005 | +0.281 [+0.146,+0.420] | -0.067 [-0.197,+0.067] | 2/2 | VOID |
| ENTDISJ / qwen3.5-27b ( 8) | 596/42 | +0.361 [+0.253,+0.479] (+0.348) | 0.0005 | +0.266 [+0.214,+0.327] | +0.094 [-0.029,+0.221] | 2/2 | VOID |
| ENTDISJ / qwen3.5-397b ( 4) | 596/42 | +0.155 [+0.041,+0.273] (+0.172) | 0.0005 | +0.185 [+0.141,+0.232] | -0.030 [-0.159,+0.095] | 2/2 | VOID |
| cell | stage | tokens | ms median [p5, p95] | of which in-hook solve (ms) | |
|---|---|---|---|---|---|
| binding: llama-3.1-8b -> qwen3-14b-base (\{}gamma 16, probe221) | sender: packet (forward + batch-1 solve, per lane lanes) | 253 | – | 71.9 (2 [10.1+25.8]) | – |
| sender: text_gen@B greedy decode (B median 22 B) | 125 | 11 new | 106.4 [85.7,133.1] | – | |
| receiver: none prefill | 128 | 100 | 18.0 [17.8,18.2] | 0 | |
| receiver: text_oracle prefill | 128 | 98 | 18.0 [17.9,18.2] | 0 | |
| receiver: text_packet prefill | 128 | 148 | 19.6 [19.1,23.2] | 0 | |
| receiver: MC prefill | 128 | 100 | 430.6 [387.7,473.6] | 409.3 |
| dose | cov. no message | cov. packet write | cov. text oracle | admitted? |
|---|---|---|---|---|
| 0.4865 (72/148) | 0.2095 (31/148) | 1.0000 (148/148) | no | |
| 0.4865 (72/148) | 0.0000 (0/148) | 1.0000 (148/148) | no |
| point | bytes pkt/txt | no comm. | packet | text | packet text [95% CI] | Holm | recv. prefill pkt/txt |
|---|---|---|---|---|---|---|---|
| 13/12 | 0.240 | 0.213 | 0.507 | -0.2933 [-0.4220,-0.1707] | 0.0006 | 12.1 s / 31 ms | |
| 16/14 | 0.240 | 0.220 | 0.527 | -0.3067 [-0.4274,-0.1989] | 0.0006 | 12.2 s / 31 ms | |
| 16/14 | 0.240 | 0.227 | 0.527 | -0.3000 [-0.4151,-0.1917] | 0.0006 | 12.2 s / 31 ms | |
| 16/14 | 0.240 | 0.253 | 0.527 | -0.2733 [-0.3882,-0.1636] | 0.0006 | 12.2 s / 31 ms | |
| 16/14 | 0.240 | 0.273 | 0.527 | -0.2533 [-0.3636,-0.1527] | 0.0006 | 12.3 s / 31 ms |
| sender family | episodes | candidates | both entity words | exactly one | words used | top-1 | null mean / p99 |
|---|---|---|---|---|---|---|---|
| llama-3.1-8b | 592 | 266 | 0 (0.000) | 88 (0.149) | 41 | 0.0000 | 0.00005 / 0.00169 |
| qwen3-8b-base | 592 | 266 | 0 (0.000) | 57 (0.096) | 22 | 0.0000 | 0.00000 / 0.00000 |
| arm | gap | what it is |
|---|---|---|
| none | +2.1250 | no message |
| MC | +2.6875 | the message assigned to the correct objects |
| MS | -4.2500 | the same message with object assignments exchanged |
| PC | +0.4375 | another example’s message |
| PS | -0.2500 | the replacement message with assignments exchanged |
| zero_packet | +2.1250 | zero vectors added through the same write operation |
| cell | pair units | FOLLOW | SLOT | verdict | ||
|---|---|---|---|---|---|---|
| llama-3.1-8b -> glm-4-9b 16 | 26 | +0.083 [-0.416,+0.582] | +0.414 [+0.004,+0.846] | +0.249 [-0.024,+0.504] | -0.166 [-0.555,+0.209] | INCONCLUSIVE |
| llama-3.1-8b -> qwen3-30b-a3b 8 | 26 | +1.329 [-0.800,+3.411] | +1.857 [+0.056,+3.852] | +1.593 [+0.784,+2.429] | -0.264 [-2.177,+1.516] | INCONCLUSIVE |
| llama-3.1-8b -> qwen3-32b 16 | 26 | +1.546 [-0.123,+3.218] | +1.818 [+0.145,+3.535] | +1.682 [+0.573,+2.761] | -0.136 [-1.430,+1.082] | INCONCLUSIVE |
| llama-3.1-8b -> qwen3.5-27b 8 | 26 | +0.788 [-0.055,+1.630] | +1.361 [+0.489,+2.326] | +1.075 [+0.550,+1.600] | -0.287 [-1.037,+0.379] | INCONCLUSIVE |
| llama-3.1-8b -> qwen3.5-397b 4 | 26 | +1.901 [+0.499,+3.223] | +1.668 [+0.189,+3.220] | +1.784 [+0.821,+2.697] | +0.117 [-1.013,+1.161] | FOLLOWS-QUERY |
| qwen3-8b-base -> qwen3.5-27b 8 | 27 | +0.470 [+0.006,+0.986] | +0.608 [+0.124,+1.127] | +0.539 [+0.140,+0.985] | -0.069 [-0.326,+0.194] | FOLLOWS-QUERY |
| cell | , one layer [95% CI] | (Holm) | , band (CC1) | retention | norm ratio | verdict | |
|---|---|---|---|---|---|---|---|
| llama-3.1-8b -> qwen3-14b-base | 551 | +0.439 [+0.27,+0.62] | 0.0009 | +0.566 [+0.35,+0.79] | 0.78 [0.67,0.90] | 0.73 | closes |
| llama-3.1-8b -> qwen3-30b-a3b | 551 | +1.247 [+0.62,+1.95] | 0.0009 | +1.443 [+0.67,+2.34] | 0.86 [0.77,1.04] | 0.79 | closes |
| llama-3.1-8b -> qwen3-32b | 551 | +1.344 [+0.41,+2.40] | 0.0019 | +1.433 [+0.48,+2.53] | 0.94 [0.76,1.13] | 0.63 | closes |
| llama-3.1-8b -> qwen3.5-27b | 551 | +0.811 [+0.39,+1.28] | 0.0009 | +0.827 [+0.40,+1.30] | 0.98 [0.80,1.20] | 0.65 | closes |
| llama-3.1-8b -> qwen3.5-397b | 261 | +1.064 [+0.45,+1.71] | 0.0009 | +1.034 [+0.40,+1.74] | 1.03 [0.68,1.79] | 0.35 | closes |
| qwen3-8b-base -> qwen3.5-27b | 488 | +0.662 [+0.43,+0.88] | 0.0009 | +0.541 [+0.35,+0.71] | 1.22 [1.03,1.44] | 0.56 | closes |
| cell | draws | worst-draw CI lower | (worst draw) | frozen draw’s CI lower | draw SD / SE | verdict |
|---|---|---|---|---|---|---|
| llama-3.1-8b -> qwen3-14b-base | 6 | +0.090 | 0.0075 | +0.465 | 0.76 | misses |
| llama-3.1-8b -> qwen3-30b-a3b | 6 | -0.479 | 0.0759 | +0.725 | 0.66 | misses |
| llama-3.1-8b -> qwen3-32b | 6 | -0.651 | 0.1065 | +0.862 | 0.76 | misses |
| llama-3.1-8b -> qwen3.5-27b | 6 | +0.133 | 0.0090 | +0.578 | 0.54 | misses |
| llama-3.1-8b -> qwen3.5-397b | 6 | -1.653 | 0.4913 | +0.750 | 1.29 | misses |
| qwen3-8b-base -> qwen3.5-27b | 6 | -0.135 | 0.3680 | -0.086 | 1.99 | misses |
| cell | [95% CI] | matched random slot effect | verdict | ||
|---|---|---|---|---|---|
| llama-3.1-8b -> qwen3-14b-base | 592 | +0.689 [+0.28,+1.15] | 0.0002 | +0.056 [-0.15,+0.29] | SURVIVES-CELL |
| llama-3.1-8b -> qwen3-30b-a3b | 592 | +2.238 [+0.51,+4.25] | 0.0032 | +0.132 [-0.30,+0.56] | SURVIVES-CELL |
| llama-3.1-8b -> qwen3-32b | 592 | +1.740 [+0.58,+3.12] | 0.0006 | +0.321 [+0.02,+0.64] | SURVIVES-CELL |
| llama-3.1-8b -> qwen3.5-27b | 592 | +1.263 [+0.59,+2.04] | 0.0001 | -0.035 [-0.08,+0.02] | SURVIVES-CELL |
| llama-3.1-8b -> qwen3.5-397b | 296 | +1.449 [+0.41,+2.54] | 0.0026 | -0.001 [-0.22,+0.23] | SURVIVES-CELL |
| qwen3-8b-base -> qwen3.5-27b | 592 | +0.352 [+0.11,+0.57] | 0.0025 | +0.088 [-0.06,+0.25] | SURVIVES-CELL |
| cell | unmatched realised | unmatched plan-side | rescale factor | matched fraction |
|---|---|---|---|---|
| llama-3.1-8b -> qwen3-14b-base | 1.161 | 0.844 | 0.939 | 1.0000 |
| llama-3.1-8b -> qwen3-30b-a3b | 1.071 | 0.846 | 1.049 | 1.0000 |
| llama-3.1-8b -> qwen3-32b | 1.058 | 0.851 | 1.081 | 1.0000 |
| llama-3.1-8b -> qwen3.5-27b | 1.139 | 0.848 | 0.937 | 1.0000 |
| llama-3.1-8b -> qwen3.5-397b | 0.551 | 0.837 | 3.131 | 1.0000 |
| qwen3-8b-base -> qwen3.5-27b | 0.825 | 0.680 | 1.516 | 1.0000 |
| control | cell | episodes | median ratio to MC | [min, max] |
|---|---|---|---|---|
| Gaussian rows | llama-3.1-8b -> qwen3-14b-base | 592 | 0.843 | [0.637, 1.008] |
| Gaussian rows | llama-3.1-8b -> qwen3-30b-a3b | 592 | 0.848 | [0.642, 1.009] |
| Gaussian rows | llama-3.1-8b -> qwen3-32b | 592 | 0.851 | [0.652, 0.998] |
| Gaussian rows | llama-3.1-8b -> qwen3.5-27b | 592 | 0.846 | [0.654, 1.004] |
| Gaussian rows | llama-3.1-8b -> qwen3.5-397b | 296 | 0.839 | [0.631, 1.000] |
| Gaussian rows | qwen3-8b-base -> qwen3.5-27b | 592 | 0.681 | [0.623, 0.758] |
| arm | share of total | band mean | |||
|---|---|---|---|---|---|
| the receiver’s own rows (MC) | 240.7 | 55.3 | 78.3 | 64% / 15% / 21% | 125.0 |
| Gaussian rows | 206.9 | 83.1 | 109.1 | 52% / 21% / 27% | 134.2 |
| ratio, Gaussian / own rows | 0.859 | 1.502 | 1.394 |
| cell | episodes | layers | median per layer | share of total |
|---|---|---|---|---|
| llama-3.1-8b -> qwen3-14b-base | 592 | 16/20/24 | 108.45 / 27.14 / 13.46 | 73% / 18% / 9% |
| llama-3.1-8b -> qwen3-30b-a3b | 592 | 19/24/29 | 10.72 / 1.97 / 0.91 | 79% / 15% / 7% |
| llama-3.1-8b -> qwen3-32b | 592 | 26/32/38 | 210.09 / 52.39 / 71.23 | 63% / 16% / 21% |
| llama-3.1-8b -> qwen3.5-27b | 592 | 26/32/38 | 52.96 / 16.15 / 11.76 | 65% / 20% / 15% |
| llama-3.1-8b -> qwen3.5-397b | 296 | 24/30/36 | 2.21 / 2.16 / 2.02 | 35% / 34% / 32% |
| qwen3-8b-base -> qwen3.5-27b | 592 | 26/32/38 | 44.40 / 19.36 / 15.94 | 56% / 24% / 20% |
| Gemma-4 sender | Qwen3.6 sender | gpt-oss sender | Middle | |||||
| Receiver | Size | Internal | Text | Internal | Text | Internal | Text | (Gemma-4) |
| Qwen3.5-2B | 2B | 81.2 | 86.7 | 79.3 | 83.8 | 77.5 | 81.5 | 4.7 |
| Qwen3.5-4B | 4B | 96.0 | 96.7 | 93.4 | 94.3 | 89.9 | 88.0 | 3.1 |
| Qwen3.5-9B | 9B | 93.4 | 94.7 | 93.6 | 96.2 | 86.6 | 88.5 | 4.9 |
| Qwen3.5-27B | 27B | 98.3 | 98.6 | 96.4 | 96.7 | 88.8 | 89.1 | 77.8 |
| Qwen3.6-27B | 27B | 97.7 | 98.2 | 96.0 | 96.0 | 90.6 | 89.5 | 82.5 |
| Relative | Qwen3.5 | Qwen3.6 | Granite-4.1 | OLMo-3 | gpt-oss | Mistral-3.2 | |||
|---|---|---|---|---|---|---|---|---|---|
| depth | 2B | 4B | 9B | 27B | 27B | 8B | 7B | 20B | 24B |
| 0.2 | 89.6 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 89.6 | 99.0 |
| 0.35 | 35.4 | 7.3 | 34.4 | 100.0 | 100.0 | 99.0 | 20.8 | 26.0 | 10.4 |
| 0.5 | 0.0 | 0.0 | 0.0 | 18.8 | 85.4 | 21.9 | 2.1 | 2.1 | 0.0 |
| 0.65 | 2.1 | 0.0 | 3.1 | 0.0 | 11.5 | 0.0 | 2.1 | 1.0 | 0.0 |
| 0.8 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| New layouts (complete facts) | Countries (Gemma-4 sender) | |||||||
| Receiver | Standard | Slots | Template | Three obj. | Internal | Middle | Reass. | Text |
| Qwen3-32B | 100.0 | 100.0 | 98.6 | 99.3 | 46.5 | 18.7 | 1.7 | 92.9 |
| Qwen3.5-27B | 100.0 | 100.0 | 100.0 | 100.0 | 64.0 | 5.8 | 5.0 | 95.7 |
| Qwen3.6-27B | 100.0 | 100.0 | 100.0 | 100.0 | 47.3 | 12.3 | 3.9 | 97.2 |
| Mistral-3.2-24B | 97.9 | 97.2 | 89.6 | 95.1 | 23.9 | 0.0 | 0.6 | 87.1 |
| Qwen3.5-9B | 93.8 | 99.3 | 93.8 | 95.1 | ||||
Appendix figures & tables38 assets
Supplementary material from the paper’s appendix.
Appendix
| Development gate (96 scenes) | Confirmation messages (576 scenes) | ||||||
| Sender | Parsed | Recall | Complete | Parsed | Recall | Complete obj. | Bytes |
| Gemma-4 | 100.0 | 100.0 | 100.0 | 100.0 | 98.2 | 96.4 | 21.9 |
| Qwen3.6 | 100.0 | 100.0 | 100.0 | 100.0 | 97.8 | 95.6 | 22.5 |
| gpt-oss | 99.5 | 95.4 | 90.1 | 98.9 | 94.1 | 89.2 | 20.3 |
| Receiver | Internal [95% CI] | Text | Middle | Reass. J/valid | Valid | Int. text [95% CI] | Verdict |
| Gemma-4 sender | |||||||
| Qwen3.5-2B | 81.2 [76.7, 85.4] | 86.7 | 4.7 | 2.5 / 99.4 | 99.9 | 5.6 [-9.1, -2.3] | text better |
| Qwen3.5-4B | 96.0 [93.9, 97.9] | 96.7 | 3.1 | 1.8 / 97.8 | 99.8 | 0.7 [-3.7, 2.1] | unresolved |
| Qwen3.5-9B | 93.4 [89.7, 96.5] | 94.7 | 4.9 | 2.9 / 93.9 | 99.3 | 1.4 [-3.1, 0.4] | unresolved |
| Qwen3.5-27B | 98.3 [96.2, 99.8] | 98.6 | 77.8 | 2.2 / 86.3 | 99.5 | 0.3 [-1.6, 0.5] | similar |
| Qwen3.6-27B | 97.7 [96.0, 99.1] | 98.2 | 82.5 | 3.0 / 93.9 | 99.9 | 0.5 [-1.2, 0.0] | similar |
| Receiver | Internal | Native states | Text | Full evidence | Internal text [95% CI] |
|---|---|---|---|---|---|
| Qwen3.5-2B | 86.3 | 85.9 | 89.6 | 97.2 | 3.3 [-5.6, -1.0] |
| Qwen3.5-4B | 99.0 | 99.7 | 98.4 | 100.0 | +0.5 [-0.7, 1.7] |
| Qwen3.5-9B | 96.2 | 99.8 | 100.0 | 100.0 | 3.8 [-7.0, -1.2] |
| Qwen3.5-27B | 100.0 | 100.0 | 100.0 | 100.0 | +0.0 [0.0, 0.0] |
| Qwen3.6-27B | 100.0 | 100.0 | 100.0 | 100.0 | +0.0 [0.0, 0.0] |
| Granite-4.1-8B | 98.1 | 97.6 | 100.0 | 99.8 | 1.9 [-3.1, -0.9] |
| Receiver | 0.2 | 0.35 | 0.5 | 0.65 | 0.8 |
|---|---|---|---|---|---|
| Development receivers (threshold fitted here) | |||||
| Qwen3-32B | 1.00 / 100.0 | 1.00 / 100.0 | 1.00 / 96.9 | 0.40 / 1.0 | 0.08 / 0.0 |
| Qwen3-14B-Base | 1.00 / 100.0 | 0.98 / 87.5 | 0.04 / 6.2 | 0.01 / 1.0 | 0.03 / 0.0 |
| GLM-4-9B | 0.89 / 88.5 | 0.68 / 8.3 | 0.09 / 0.0 | 0.12 / 0.0 | 0.12 / 0.0 |
| Gemma-3-27B-IT | 1.00 / 100.0 | 1.00 / 85.4 | 0.09 / 0.0 | 0.04 / 0.0 | 0.05 / 0.0 |
| Held-out receivers (predictions committed before these outcomes) | |||||
| Receiver | Layout | Internal [95% CI] | Valid | Standard [95% CI] | Text | Reass. | Verdict |
|---|---|---|---|---|---|---|---|
| Qwen3-32B | Standard | 100.0 [100.0, 100.0] | 100.0 | 99.3 | 1.4 | reference | |
| Slots | 100.0 [100.0, 100.0] | 100.0 | +0.0 [0.0, 0.0] | 100.0 | 1.4 | transfers | |
| Template | 98.6 [96.5, 100.0] | 100.0 | 1.4 [-3.5, 0.0] | 99.3 | 0.0 | transfers | |
| Three obj. | 99.3 [97.9, 100.0] | 100.0 | 0.7 [-2.1, 0.0] | 99.3 | 0.0 | transfers | |
| Qwen3.5-27B | Standard | 100.0 [100.0, 100.0] | 100.0 | 100.0 | 2.1 | reference | |
| Slots | 100.0 [100.0, 100.0] | 100.0 | +0.0 [0.0, 0.0] | 100.0 | 3.5 | transfers |
| Receiver | Internal [95% CI] | Valid | Middle | Reass. | Text | Evidence | Int. reass. [95% CI] | Int. text [95% CI] |
|---|---|---|---|---|---|---|---|---|
| Qwen3-32B | 46.5 [38.9, 56.7] | 97.9 | 18.7 | 1.7 | 92.9 | 97.5 | +44.8 [37.1, 54.6] | 46.4 [-54.8, -35.8] |
| Qwen3.5-27B | 64.0 [56.3, 76.0] | 100.0 | 5.8 | 5.0 | 95.7 | 96.3 | +59.0 [51.4, 69.3] | 31.7 [-39.6, -20.7] |
| Qwen3.6-27B | 47.3 [39.7, 58.0] | 100.0 | 12.3 | 3.9 | 97.2 | 99.4 | +43.4 [35.5, 53.1] | 49.9 [-57.5, -40.7] |
| Mistral-3.2-24B | 23.9 [15.1, 31.5] | 97.5 | 0.0 | 0.6 | 87.1 | 100.0 | +23.3 [14.7, 30.8] | 63.2 [-73.2, -52.4] |
| Weight of the competing value | ||||||
|---|---|---|---|---|---|---|
| Receiver | Route | 0 | 64 | 128 | 191 | 255 |
| Qwen3-32B | Internal | 100.0 | 99.3 | 72.9 | 4.9 | 0.0 |
| Text | 99.3 | 31.2 | 19.4 | 18.1 | 0.0 | |
| Qwen3.5-27B | Internal | 100.0 | 100.0 | 95.1 | 50.7 | 0.0 |
| Text | 100.0 | 64.6 | 41.0 | 38.9 | 0.0 | |
| gpt-oss-20b | Internal | 84.0 | 76.4 | 63.2 | 34.0 | 0.0 |
| Receiver | Middle | Early [95% CI] | Text | Reassigned Success / Valid |
|---|---|---|---|---|
| Qwen3-32B | 94.4 | 97.0 [93.2, 99.5] | 97.0 | 1.2 / 62.2 |
| Qwen3-14B-Base | 31.0 | 97.7 [95.1, 99.8] | 97.7 | 2.3 / 99.2 |
| GLM-4-9B | 0.5 | 74.3 [66.9, 81.3] | 87.8 | 0.6 / 49.8 |
| Gemma-3-27B-IT | 30.5 | 97.7 [94.8, 100.0] | 97.9 | 3.0 / 89.8 |
| Receiver | Directions | Native states | Difference [95% CI] | Text |
|---|---|---|---|---|
| Qwen3-32B | 99.8 | 100.0 | -0.2 [-0.5, +0.0] | 99.8 |
| Qwen3-14B-Base | 99.5 | 99.8 | -0.3 [-0.9, +0.0] | 100.0 |
| GLM-4-9B | 76.5 | 75.6 | +0.9 [-1.2, +3.2] | 93.0 |
| Gemma-3-27B-IT | 100.0 | 100.0 | +0.0 [+0.0, +0.0] | 100.0 |
| Receiver | Twelve contexts | One fitting context | Embedding (= text) | ||
|---|---|---|---|---|---|
| First | Middle | Last | |||
| Qwen3-32B | 98.8 | 94.6 | 98.8 | 98.6 | 99.3 |
| Qwen3-14B-Base | 99.1 | 97.2 | 99.8 | 99.0 | 100.0 |
| GLM-4-9B | 76.4 | 8.9 | 75.5 | 76.7 | 92.7 |
| Gemma-3-27B-IT | 100.0 | 99.8 | 100.0 | 100.0 | 100.0 |
| Early write | Cut at 0.35 | Cut at 0.5 | Cut at 0.65 | |||||
|---|---|---|---|---|---|---|---|---|
| Receiver | Source | (0.2) | Write | Restore | Write | Restore | Write | Restore |
| Qwen3-14B-Base | Fitted | 100.0 | 87.5 | 97.9 | 6.2 | 59.4 | 1.0 | 0.0 |
| Native | 100.0 | 96.9 | 97.9 | 22.9 | 52.1 | 0.0 | 0.0 | |
| GLM-4-9B | Fitted | 88.5 | 8.3 | 72.9 | 0.0 | 0.0 | 0.0 | 0.0 |
| Native | 90.6 | 66.7 | 75.0 | 0.0 | 0.0 | 0.0 | 0.0 | |
| Gemma-3-27B-IT | Fitted | 100.0 | 85.4 | 97.9 | 0.0 | 2.1 | 0.0 | 0.0 |
| Depth | Fitted | Scene-matched | Native | Text | Matched fitted | Native matched | Text native |
|---|---|---|---|---|---|---|---|
| 0.2 | 88.5 | 87.5 | 90.6 | 99.0 | 1.0 [-5.2, 3.1] | +3.1 [-2.1, 8.3] | +8.3 [1.0, 17.7] |
| 0.35 | 8.3 | 40.6 | 66.7 | 99.0 | +32.3 [24.0, 39.6] | +26.0 [11.5, 39.6] | +32.3 [20.8, 45.8] |
| 0.5 | 0.0 | 0.0 | 0.0 | 99.0 | +0.0 [0.0, 0.0] | +0.0 [0.0, 0.0] | +99.0 [96.9, 100.0] |
| Indices | Symbols |
|---|---|
| 0–11 | accordion, ambulance, axe, bakery, banana, barn, bed, beer, beetle, belt, bench, bicycle |
| 12–23 | blender, boat, boot, bowl, broccoli, broth, buffalo, bus, butterfly, cabbage, cabinet, camel |
| 24–35 | car, carpet, carrot, celery, cherry, chicken, church, cider, clam, clamp, coat, cocoa |
| 36–47 | coconut, cod, copper, cotton, cow, crab, crow, cucumber, curtain, deer, desk, dolphin |
| 48–59 | drawer, drill, elephant, espresso, factory, flea, flute, fork, fox, fridge, garlic, glass |
| 60–71 | glove, goat, gold, goose, grape, greenhouse, hat, hawk, helicopter, hoe, hospital, hotel |
| Objects | Attribute | Values |
|---|---|---|
| Painted blocks | Color | green, orange, red, yellow |
| Material | gold, iron, silver, steel | |
| Candy | Color | green, orange, red, yellow |
| Taste | bitter, salty, sour, sweet | |
| Coats | Fibre | cotton, leather, silk, wool |
| Buttons | copper, glass, plastic, wooden |
| World | Object | Described facts | Transmitted indices (decoded here) |
|---|---|---|---|
| 0 | BANQ | green, gold | 62 (gold), 160 (green) |
| CIRV | red, steel | 133 (steel), 166 (red) | |
| 1 | BANQ | red, gold | 62 (gold), 166 (red) |
| CIRV | green, steel | 133 (steel), 160 (green) |
| Unit | In confirmation | Also in earlier banks |
|---|---|---|
| Object names | 24 | 0 |
| Full scenes | 576 | 0 |
| Evidence templates | 12 | 0 |
| Single-object fact tuples | 48 | 48 |
| Two-object combinations | 72 | 14 |
| Paired-value cells | 36 | 7 |
| Receiver | Delivery | Joint | Accuracy | Valid | Changed | Unchanged |
|---|---|---|---|---|---|---|
| Qwen3-32B | Early write | 97.0 | 99.1 | 99.6 | 97.9 | 98.8 |
| Same packet as text | 97.0 | 99.1 | 99.4 | 97.9 | 98.8 | |
| Reassignment 1 | 1.9 | 31.7 | 59.5 | 8.6 | 20.3 | |
| Reassignment 2 | 0.9 | 31.3 | 62.5 | 5.0 | 20.1 | |
| Reassignment 3 | 0.7 | 32.5 | 64.5 | 5.1 | 22.1 | |
| Qwen3-14B-Base | Early write | 97.7 | 99.3 | 100.0 | 98.1 | 99.3 |
| Receiver | Contrast | Gain (pp) | 36-family interval | 12-template interval |
|---|---|---|---|---|
| Qwen3-32B | Early minus middle | 2.6 | [1.2, 4.2] | [1.6, 3.7] |
| Early minus reassigned | 95.9 | [92.1, 98.4] | [90.8, 98.9] | |
| Early minus text | 0.0 | [-0.5, 0.5] | [-0.5, 0.5] | |
| Qwen3-14B-Base | Early minus middle | 66.7 | [63.1, 70.2] | [63.0, 70.6] |
| Early minus reassigned | 95.4 | [92.8, 97.6] | [90.9, 98.1] | |
| Early minus text | 0.0 | [-0.7, 0.9] | [-0.5, 0.5] |
| Receiver | Middle write | Early write | Native states | Packet text | Evidence text |
|---|---|---|---|---|---|
| Qwen3-32B | 97.7 | 99.8 | 100.0 | 99.8 | 99.5 |
| Qwen3-14B-Base | 27.1 | 99.5 | 99.8 | 100.0 | 100.0 |
| GLM-4-9B | 0.0 | 76.5 | 75.6 | 93.0 | 99.5 |
| Gemma-3-27B-IT | 26.7 | 100.0 | 100.0 | 100.0 | 100.0 |
| Receiver | Condition | Joint | Accuracy | Valid | Changed | Unchanged |
|---|---|---|---|---|---|---|
| Qwen3-32B | No message | 0.0 | 47.3 | 95.5 | 0.0 | 46.7 |
| Identity write | 0.0 | 47.3 | 95.5 | 0.0 | 46.7 | |
| Middle write | 94.4 | 98.4 | 99.0 | 96.9 | 97.0 | |
| Other-world packet | 0.0 | 49.9 | 99.6 | 0.0 | 98.8 | |
| Qwen3-14B-Base | No message | 0.0 | 50.8 | 100.0 | 0.0 | 51.6 |
| Identity write | 0.0 | 50.8 | 100.0 | 0.0 | 51.6 |
| Receiver | Block | Edit norm | Residual norm | Edit/residual |
|---|---|---|---|---|
| Qwen3-32B | 13 | 317.6 | 320.7 | 0.99 |
| Qwen3-14B-Base | 8 | 167.8 | 161.2 | 1.04 |
| GLM-4-9B | 8 | 50.6 | 44.0 | 1.15 |
| Gemma-3-27B-IT | 12 | 2186.0 | 2490.3 | 0.88 |
| Fitted directions | Native states | |||
|---|---|---|---|---|
| Receiver | Raw | Native-sized | Raw | Fitted-sized |
| Qwen3-32B | 100.0 | 100.0 | 100.0 | 100.0 |
| Qwen3-14B-Base | 100.0 | 100.0 | 100.0 | 100.0 |
| GLM-4-9B | 88.5 | 87.5 | 90.6 | 89.6 |
| Gemma-3-27B-IT | 100.0 | 100.0 | 100.0 | 100.0 |
| Native-state write depth | Joint success (%) | Valid answers (%) |
|---|---|---|
| Middle band | 43.8 | 100.0 |
| 0.20 | 100.0 | 100.0 |
| 0.35 | 96.9 | 100.0 |
| 0.50 | 0.0 | 100.0 |
| 0.65 | 0.0 | 100.0 |
| 0.80 | 0.0 | 99.5 |
| Receiver | Automatic write [95% CI] | Same-message text | Complete-fact write |
|---|---|---|---|
| Qwen3-32B | 86.9 [78.0, 94.6] | 86.9 | 97.3 |
| Gemma-3-27B-IT | 91.1 [83.3, 97.9] | 91.1 | 100.0 |
| Objects | Messages | Fact recall (%) | Complete objects (%) | Mean bytes |
|---|---|---|---|---|
| Two | 1152 | 4570/4608 (99.2) | 2269/2304 (98.5) | 22.6 |
| Four | 384 | 2968/3072 (96.6) | 1432/1536 (93.2) | 36.0 |
| Objects | Receiver | Both | Write only | Text only | Neither |
|---|---|---|---|---|---|
| Two | Qwen3-32B | 553 | 2 | 2 | 15 |
| Two | Qwen3-14B-Base | 557 | 2 | 2 | 11 |
| Two | GLM-4-9B | 409 | 16 | 93 | 54 |
| Two | Gemma-3-27B-IT | 559 | 0 | 1 | 12 |
| Four | Qwen3-32B | 162 | 1 | 1 | 24 |
| Four | Gemma-3-27B-IT | 171 | 0 | 0 | 17 |
| Receiver | Objects | Write count | Write (%) | Text (%) | Write valid (%) |
|---|---|---|---|---|---|
| Qwen3-32B | Blocks | 191/191 | 100.0 | 100.0 | 100.0 |
| Candy | 189/190 | 99.5 | 99.5 | 99.9 | |
| Coats | 175/191 | 91.6 | 91.6 | 98.8 | |
| Qwen3-14B-Base | Blocks | 191/191 | 100.0 | 100.0 | 100.0 |
| Candy | 190/190 | 100.0 | 100.0 | 100.0 | |
| Coats | 178/191 | 93.2 | 93.2 | 99.9 |
| Early write (ms) | Text (ms) | |||
|---|---|---|---|---|
| Receiver | Median | p90 | Median | p90 |
| Qwen3-32B | 1301 | 1456 | 1018 | 1177 |
| Qwen3-14B-Base | 640 | 717 | 515 | 591 |
| GLM-4-9B | 651 | 736 | 545 | 656 |
| Gemma-3-27B-IT | 1300 | 1406 | 1051 | 1163 |
| Receiver | Internal | Text | Full evidence | Reassigned Success / Valid |
|---|---|---|---|---|
| Qwen3-32B | 95.3 | 94.1 | 98.6 | 0.0 / 97.3 |
| Qwen3-14B-Base | 95.7 | 96.9 | 100.0 | 0.0 / 99.9 |
| GLM-4-9B | 41.8 | 70.3 | 87.7 | 0.0 / 62.2 |
| Gemma-3-27B-IT | 94.4 | 97.7 | 99.7 | 0.1 / 76.8 |
| Receiver | Message | Joint count | Accuracy | Valid | Text | ||
|---|---|---|---|---|---|---|---|
| Qwen3-32B | Automatic | 549/576 | 98.8 | 99.5 | 96.4 | 99.0 | 94.1 |
| Complete facts | 558/576 | 99.2 | 99.6 | 97.6 | 99.3 | 97.7 | |
| Qwen3-14B-Base | Automatic | 551/576 | 98.9 | 100.0 | 97.2 | 98.4 | 96.9 |
| Complete facts | 563/576 | 99.4 | 100.0 | 98.4 | 99.3 | 98.4 | |
| GLM-4-9B | Automatic | 241/576 | 80.1 | 99.1 | 66.5 | 68.9 | 70.3 |
| Complete facts | 257/576 | 80.6 | 99.9 | 67.0 | 70.1 | 89.2 |
| Receiver | Internal text [95% CI] | Internal reassigned [95% CI] | Valid answers (%) Draws 1 / 2 / 3 |
|---|---|---|---|
| Qwen3-32B | 1.2 [-0.5, 3.0] | 95.3 [92.9, 97.6] | 94.4/99.0/98.4 |
| Qwen3-14B-Base | -1.2 [-2.8, 0.2] | 95.7 [93.4, 97.6] | 99.8/100.0/100.0 |
| GLM-4-9B | -28.5 [-35.4, -20.8] | 41.8 [36.1, 47.6] | 57.1/63.4/66.1 |
| Gemma-3-27B-IT | -3.3 [-5.2, -1.6] | 94.4 [92.1, 96.4] | 88.6/56.9/84.9 |
| Receiver | Message | Twelve | First | Middle | Last | Text | Embedding |
|---|---|---|---|---|---|---|---|
| Qwen3-32B | Automatic | 95.3 | 87.2 | 95.1 | 95.3 | 94.8 | 94.8 |
| Complete facts | 98.8 | 94.6 | 98.8 | 98.6 | 99.3 | 99.3 | |
| Qwen3-14B-Base | Automatic | 99.0 | 97.0 | 99.1 | 99.0 | 99.3 | 99.3 |
| Complete facts | 99.1 | 97.2 | 99.8 | 99.0 | 100.0 | 100.0 | |
| GLM-4-9B | Automatic | 80.6 | 9.9 | 82.8 | 83.5 | 87.2 | 88.9 |
| Complete facts | 76.4 | 8.9 | 75.5 | 76.7 | 92.7 | 93.2 |
| Receiver | Message | Twelve mean single [95% CI] | Twelve embedding [95% CI] |
|---|---|---|---|
| Qwen3-32B | Automatic | 2.8 [1.0, 5.1] | 0.5 [-0.7, 1.9] |
| Complete facts | 1.4 [0.6, 2.4] | -0.5 [-1.2, 0.0] | |
| Qwen3-14B-Base | Automatic | 0.6 [0.0, 1.2] | -0.3 [-1.2, 0.3] |
| Complete facts | 0.5 [-0.8, 1.6] | -0.9 [-2.1, 0.0] | |
| GLM-4-9B | Automatic | 21.8 [17.1, 26.2] | -8.3 [-12.0, -4.9] |
| Complete facts | 22.7 [17.3, 27.7] | -16.8 [-27.1, -7.3] |
| Receiver | Internal | Message text | Full evidence | Internal validity | |
|---|---|---|---|---|---|
| Qwen3-14B-Base | 1 | 0.1 | 24.3 | 100.0 | 99.2 |
| Qwen3-32B | 1 | 0.3 | 11.8 | 100.0 | 90.1 |
| GLM-4-9B | 0 | 0.0 | 7.9 | 99.0 | 96.8 |
| Qwen3.5-27B | 0 † | 0.0 | 17.8 | 100.0 | 60.5 |
| Candy / blocks | Coats | |||||
|---|---|---|---|---|---|---|
| Message source | Text | Old | Targeted | Text | Old | Targeted |
| Original sparse | 4.2 | 1.0 | 1.0 | 5.2 | 1.0 | 7.3 |
| Centered sparse | 70.8 | 0.0 | 38.5 | 8.3 | 1.0 | 13.5 |
| Prompted selection | 100.0 | 0.0 | 95.8 | 100.0 | 0.0 | 77.1 |
| Complete facts | 100.0 | 2.1 | 99.0 | 100.0 | 1.0 | 87.5 |
| Neutral + direction | Coefficient update | ||||
|---|---|---|---|---|---|
| Receiver | Each layer | Midpoint | Rescaled mid | Each layer | Midpoint |
| Qwen3-32B | 99.0 | 54.2 | 61.5 | 0.0 | 0.0 |
| Qwen3-14B-Base | 13.5 | 13.5 | 15.6 | 0.0 | 0.0 |
| GLM-4-9B | 6.2 | 8.3 | 8.3 | 0.0 | 0.0 |
| Receiver | Symbol contrast (%) | J-lens (%) | J-lens minus contrast (pp) |
|---|---|---|---|
| Qwen3-32B | 97.9 | 2.1 | -95.8 [-99.0, -92.7] |
| Qwen3-14B-Base | 10.4 | 1.0 | -9.4 [-15.6, -4.2] |
| GLM-4-9B | 0.0 | 1.0 | 1.0 [0.0, 3.1] |
| Llama-3.1-8B-Instruct | 0.0 | 0.0 | 0.0 [0.0, 0.0] |
Multi-LLM Collaboration
Multi-LLM collaboration studies how large language models share work, exchange information, and combine outputs to improve capability, reliability, or cost.
See all 85 papers