We present a memory-augmented transformer in which attention serves simultaneously as a retrieval, consolidation, and write-back operator. The core update, A⊤AVW, re-grounds retrieved values into persistent memory slots via the Gram matrix A⊤A, providing a principled tripartite projection: observation space → latent memory → supervised transformation. We partition the memory into lateralized left and right banks coupled through a sign-controlled cross-talk matrix Ws, and show that the sign of this coupling is decisive for specialization. Excitatory cross-talk (s=+1) causes bank-dominance collapse: one bank monopolises all inputs and Pct→0.5, despite lowering task loss. Inhibitory cross-talk (s=−1), motivated by the net inhibitory effect of callosal projections in human cortex, actively suppresses contralateral bank activation and achieves saturated specialization (Dsep=±1.00, Pct≈0). On a controlled symbolic benchmark combining an episodic bijection cipher (requiring associative recall) with a strict arithmetic progression (requiring rule extraction), the inhibitory model reduces cipher-domain loss by 124× over the baseline while matching it on the arithmetic domain, confirming that persistent lateralized memory is necessary for episodic recall but not for rule-based prediction.
Figures & tables
Figure 1: The tripartite projection sequence mapping latent values to the observation space and back, before applying the supervised transformation.
Figure 2: Bidirectional cross-talk pathways for lateralized memory (inhibitory mode, s=−1 , proposed). The dashed red lines show the paths where one bank’s values reach the contralateral update. Under inhibitory cross-talk, these paths carry a negative coefficient: the right bank’s values Vr suppress the left bank’s update (and vice versa), sharpening bank separation rather than sharing representations. Setting s=+1 reverses these signs to excitatory; setting the cross-weights to zero gives the split-brain baseline.
Model
Accuracy
Loss
Datasetl
Datasetr
Datasetlr
Datasetl
Datasetr
Datasetlr
Standard Transformer
100.0%
100.0%
93.8%
0.0747
0.0002
0.1692
Attention-Coupled Lateral (Ours)
100.0%
100.0%
94.4%
0.0006
0.0002
0.1452
Table 1: Next-token prediction accuracy and cross-entropy loss on homogeneous and interleaved symbolic sequences. The lateralized memory prevents the performance collapse seen in the baseline on mixed tasks.
Figure 3: Training convergence of the Attention-Coupled Lateral model over 50 epochs. Panel 1 (task loss) : cross-entropy drops from 1.12 to 0.03. Panel 2 (total loss) : routing auxiliary term drives total loss slightly negative once routing saturates. Panel 3 ( Dsep ) : Separation Degree converges after brief instabilities at epochs 15 and 33. Panel 4 ( Pct ) : Cross-Talk Penalty collapses to ≈0 within 4 epochs and stays there, confirming stable lateralized routing throughout training.
Metric
Datasetl
Datasetr
Datasetlr
Left Separation Degree Dsep(L)
+ 1.00
− 1.00
− 0.42
Right Separation Degree Dsep(R)
− 1.00
+ 1.00
+ 0.42
Cross-Talk Penalty Pct
0.00
0.00
0.03
Table 2: Lateralization metrics demonstrating the emergence of specialized memory banks. Higher Dsep indicates strong routing to the correct hemisphere, while low Pct indicates minimal inappropriate attention cross-talk.
Cross-talk Mode
Dsep(L)∣l↑
Dsep(L)∣r↓
Pct↓
Acc lr↑
Loss lr↓
None (split-brain)
+ 1.00
− 1.00
0.00
94.4%
0.1456
Excitatory ( s=+1 )
− 0.82
− 0.93
0.46
95.8%
0.1017
Inhibitory ( s=−1 , ours)
+ 1.00
− 1.00
0.03
94.4%
0.1454
Table 3: Cross-talk mode ablation on the mixed dataset Datasetlr , evaluated with Dsep(L) (measured on Datasetl and Datasetr separately), Pct (on Datasetlr ), and accuracy/loss on Datasetlr . All conditions share the same architecture, training procedure, and routing auxiliary loss.