Multi-module neural systems often expose every module to the full input. We test whether a slot-selective evidence-masking regime -- restricting each module to its own evidence span -- changes which solutions gradient-based training discovers. Four-cell societies share one frozen pretrained language model and one low-rank adapter, communicating only through two model-width continuous vectors in a fixed relay. On a prospectively sealed natural-language function-composition task, we train ten matched restricted/global pairs identical except for the attention mask. Restricted-visibility societies outperform their globally visible twins by at least 20 percentage points at both depths in 9 of 10 pairs, with median paired advantages of 0.7648 and 0.6050. Cutting communication reduces every restricted society to chance, and in a post hoc collision-stratified analysis the depth-three advantage remains 0.558 on programs whose complete affine map never appeared in training. In six post hoc-selected restricted societies, packet interventions on correctly answered held-out episodes are consistent with approximately value-indexed relay states; the sole high-performing global model also requires communication, but its same-value packets are not interchangeable across episodes. Thus restricted visibility is not necessary for composition. Under the tested seeds, streams, task world, and training budget, the masking regime strongly shifted which solutions training discovered: a post hoc mask crossover finds both arms mask-native. Because the restricted mask both blocks foreign evidence and implicitly identifies each cell's assigned slot, attribution to evidence visibility alone awaits a role-marked control. The preregistered battery nevertheless formally fails because restricted-arm median depth-three accuracy is 0.6988, below the 0.70 floor; an earlier qualification cohort yielded 0/10 complete passes.
Figures & tables
Figure 1: The society architecture and the treatment. Four cells share one frozen backbone and one rank-8 LoRA. Cell 1 publishes P1 from its assigned start-value span without incoming mail; cells 2–4 read P1 – P3 and write P2 – P4 (each packet: two 896-dimensional vectors, the only inter-cell pathway). Each cell receives an identical layout containing the shared question (orange) plus four evidence slots. In the restricted arm a cell’s attention mask exposes only its own evidence slot (teal) and the question, leaving foreign slots (hatched) with exactly zero direct attention influence; in the global arm all four slots are readable. The attention mask is the sole implementation-level difference between twins; it jointly changes foreign-evidence access, role disambiguation, and active-context load (§ 7 ).
Preregistered component
Result
P−G≥0.20 at both depths
9/10 pairs — pass
Median Δ2≥0.25
0.7648 — pass
Median Δ3≥0.25
0.6050 — pass
Restricted-arm all-cut communication
10/10 at 1/17 — pass
No restricted run below 0.40
pass
Restricted median depth two ≥0.70
0.8339 — pass
Table 1: Preregistered gate decomposition. Thresholds were frozen before training; the complete conjunction determines the preregistered verdict.
init
order
P 2
P 3
G 2
G 3
Δ2
Δ3
P all-cut
G all-cut
200
900
0.8358
0.7578
0.0723
0.0770
+0.7635
+0.6808
0.0588
0.0600
200
950
0.8775
0.6713
0.0980
0.0787
+0.7795
+0.5926
0.0588
0.0701
201
901
0.7966
0.6581
0.1103
0.0917
+0.6863
+0.5664
0.0588
0.0679
201
951
0.9926
0.9338
0.0797
0.0650
+0.9129
+0.8688
0.0588
0.0674
202
902
0.8321
0.7054
0.0870
0.0880
+0.7451
+0.6174
0.0588
0.0588
202
952
0.8211
0.6824
0.0551
0.0958
+0.7660
+0.5866
0.0588
0.0512
Table 2: All ten pairwise results. Held-out accuracy at depths two and three for the restricted (P) and global (G) twin of each pair, paired differences, and all-packets-cut accuracy at depth three (chance =1/17≈0.0588 ).
Figure 2: The paired masking-regime effect. (a) Final held-out accuracies for all ten twin pairs; vertical grey segments connect each pair at depth three. (b) Paired advantages Δ2 and Δ3 per pair with medians (dashed) and the preregistered 0.20 per-pair threshold (dotted). Pair 204/954 is the globally visible exception discussed in § 5.2 .
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
arm
init
order
ℓ0
ℓ1
depth 2
depth 3
all-cut
best cell
batch shuffle
train3
G
200
900
1.000
0.9914
0.0723
0.0770
0.0600
0.0600
0.0691
1.00
G
200
950
1.000
0.9816
0.0980
0.0787
0.0701
0.0701
0.0735
0.78
G
201
901
1.000
0.9755
0.1103
0.0917
0.0679
0.0679
0.0809
0.42
G
201
951
1.000
0.9743
0.0797
0.0650
0.0674
0.0674
0.0681
0.41
G
202
902
1.000
0.9265
0.0870
0.0880
0.0588
0.0588
0.0814
0.43
G
202
952
1.000
0.9534
0.0551
0.0958
0.0512
0.0512
0.0880
0.99
Appendix
Table 3: Per-model evaluation diagnostics, all twenty finals. ℓ0 : identity accuracy; ℓ1 : single-operation (held-out phrasing) accuracy; all-cut: all packets severed (depth three); best cell: best single-cell-only accuracy; batch shuffle: all-interface batch packet shuffle (committed packets rolled across the evaluation batch at all three interfaces simultaneously — a broad diagnostic distinct from the audit’s interface-specific cross-example shuffle, Table 5 ); train3: depth-three training-bank exact accuracy on the first 150 training programs. Depth-two training-bank exact accuracy is 1.000 for every model and is omitted.
init
order
P novel
G novel
Δ novel
P redundant
G redundant
200
900
0.7104
0.0667
+0.6437
0.7710
0.0798
200
950
0.5905
0.0532
+0.5373
0.6937
0.0857
201
901
0.6131
0.0939
+0.5192
0.6705
0.0911
201
951
0.9129
0.0690
+0.8439
0.9396
0.0638
202
902
0.6369
0.0792
+0.5577
0.7243
0.0904
202
952
restricted checkpoint archive-pending (disclosed)
Appendix
Table 4: Post hoc collision-stratified depth-three scores per pair. Map-novel: composite affine map absent from training (13 of 60 programs; 884 scored episodes per checkpoint); map-redundant: map present (47 programs; 3,196 episodes). At depth two, 3 of 12 programs are map-novel. One restricted checkpoint (init 202, order 952) is archive-pending and disclosed.
model
n
base
same-value
counterfactual
span-cf
del/noise
shuffle
all gates
G:m200:o950
12
0.067
0.00–0.50
0.00–0.33
0.25–0.50
0.250
0.25–0.33
no
G:m204:o954
155
0.861
0.12–0.25
0.15–0.26
0.70–0.91
0.071
0.06–0.11
no
P:m200:o950
114
0.633
0.96–1.00
0.91–0.98
0.86–0.88
0.105
0.04–0.08
no
P:m201:o951
160
0.889
0.99–1.00
0.98–1.00
0.97–0.98
0.062
0.05–0.07
yes
P:m203:o903
96
0.533
0.96–1.00
0.74–0.90
0.73–0.82
0.073
0.06–0.07
no
P:m203:o953
125
0.694
0.94–1.00
0.86–0.92
0.81–0.86
0.096
0.08–0.11
no
Appendix
Table 5: Packet-audit summary, eight audited models. Ranges span the three relay interfaces (cell1 → 2, 2 → 3, 3 → 4). Each audit begins with 180 held-out episodes and conditions intervention analyses on the subset the model answers correctly; the resulting per-model denominators n are shown in the table (the n=12 global control’s estimates are correspondingly imprecise), and exact values are recorded in the released audit JSONs. Base: intact accuracy on the audited episode pool. Del/noise: worst (highest) of packet deletion and approximately norm-matched noise. Shuffle: cross-example packet shuffle (its preregistered gate accepts either ≤0.11 or ≥ 30 points below intact accuracy). Span-cf: counterfactual evidence-span rewrite control. Gate marks are the preregistered absolute audit gates.
Figure 3: Training trajectories (restricted and global arms), moved to the appendix per the main-figure policy: depth-three held-out probe accuracy over training. Curves are shown for 16 societies with complete per-500-step logs and 2 with truncated logs; 2 trajectories have no surviving probe log, lost to provider interruptions. Final-checkpoint evaluations are available for all 20 societies, while 19 final checkpoint files are currently recoverable.
Restricting what a module can read may improve what a system learns to compute. We test this in a preregistered confirmation with sixty four-cell systems sharing a frozen language-model backbone and communicating through learned continuous packets. Five conditions vary evidence masking, ownership markers, and replacement of foreign evidence with neutral filler (task-irrelevant text of the same token length), across six initialization clusters, each with two data orders, on one fresh task world. With markers available in both regimes, masking improved accuracy on held-out two- and three-operation compositions by median paired differences of 0.846 and 0.859; all twelve pairs cleared the required margins, and the full preregistered behavioral criterion passed. The unmarked replication also passed. No marked global-visibility (G+) system passed the marker-following check, so the effect of usable role information remains unresolved. The filler condition yielded seven full generalizers, but its decomposition criteria were inconclusive. Packet interventions in all eighteen audited masked systems followed the predicted intermediate-value changes on eligible cases; these finite, success-conditioned audits do not establish mediation. The results confirm a large advantage of the tested masking regime, while leaving its finer attribution and generality open. Protocols, results, and checkpoints are public.
In shared-genome language-model societies, restricted evidence visibility favors reusable, value-indexed latent packet interfaces, whereas the sole high-performing globally visible model in the parent study learned an episode-entangled code. This companion study asks whether independently trained societies share one packet language, where strict zero-shot transfer fails, and whether inherited interface state helps or harms later learning. First, a leakage-controlled causal interoperability audit over all 30 ordered pairs of six independently trained restricted societies -- under sealed held-out structure and a preregistered raw/orthogonal/linear/nonlinear alignment ladder -- shows the six semantically similar interfaces do not form one raw language: one same-initialization pair is exactly interoperable in both directions, a second shows asymmetric partial compatibility, and all 26 cross-initialization directions fail every frozen alignment rung. Second, within the tested decomposition and a single sealed source formulation, a source-span control localizes strict zero-shot failure to interpretation and execution of the new operator instructions. Third, in a matched adaptation factorial, the globally trained communication interface acts as a severe negative-transfer prior: reinitializing only the packet reader, writer, and mouth raises final depth-three accuracy from 0.169 to 0.857. Fourth, across two restricted checkpoints and two independently frozen target streams each, inherited interfaces never exceeded fresh-interface controls by the preregistered 0.10 margin. All primary conclusions are bounded to a near-transfer 17-state setting; the negative-transfer factorial concerns one globally visible parent-cohort checkpoint, while an appendix adds a post hoc tagged-global twin case study.
Large Language Models fail at implicit multi-hop reasoning: a model answers "When was X born?" and "Who is Y's closest friend?" correctly but fails on "When was Y's closest friend born?" in a single forward pass, even when both facts are perfectly memorized and individually retrievable. We study this failure in a controlled natural language setting with a strict separation between individuals exposed to compositional contexts during pretraining and those that never appear in any such context. We confirm that compositional failure persists even at 97% 1-hop accuracy, establishing the gap as a pretraining failure rather than a knowledge absence. We propose and test nine data-centric augmentation formats and find that compositional pretraining transfers to unseen questions for exposed individuals, but never to individuals absent from compositional pretraining, suggesting that exposure to compositional contexts during pretraining is a necessary condition for implicit multi-hop reasoning. Code is available at https://github.com/ykrmm/composition-bound .
Yannis Karmim, Luis Marti, Djamé Seddah +1
Inria, Paris, France · Inria Chile, Las Condes, Chile · Dept. of Computer Science, Univ. de Chile +1