cs.CLOct 11, 2026

Binding Phoneme Identity to Position in Neural Models and the Human Brain

Authors: Nafiseh Momeni, Malo Renaudin, Itzhak Fried, Yair Lakretz

Organizations: Université Paris Cité · École Normale Supérieure – PSL · University of California, Los Angeles

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

A fundamental challenge in cognitive science and neural computation is the binding problem. Compositional generalization can be achieved by factorizing representations into roles, the structural slots of a representation (e.g., serial position or syntactic function), and fillers, the contents occupying them (e.g., a phoneme or a word), which can be recombined freely to represent novel combinations. How neural systems implement role--filler binding, however, remains unknown. One of its simplest cases is phoneme sequencing, where phoneme identity (filler) must be bound to position (role) so that /gum/ and /mug/ are represented differently. A proposal from NLP, the onion hypothesis' (Csordas et al., 2024), holds that the direction of the neural population state encodes token identity, while its magnitude encodes position. It was formulated for random token sequences; we extend it to natural language and test it in neural models and the human brain. We first evaluated it in an encoder-decoder trained to repeat English words, using representational geometry analyses and causal interventions. Magnitude alone proves incomplete: position is encoded by magnitude together with a systematic rotation of the state direction, a scheme we formalize as the spiral hypothesis'. We then analyzed intracranial recordings from the auditory cortex of 20 participants listening to short sentences. Across superior temporal gyrus electrodes, the change in population response between consecutive phonemes decreases in magnitude as position advances, and its direction is aligned within consonants and within vowels but near orthogonal across the two classes, supporting the magnitude coding shared by both accounts. These results provide a geometric account of how neural systems bind phoneme identity to position, and show how neural-model hypotheses can yield testable predictions about human neural computation.

Figures & tables

Figure 1: Hypotheses for How a Recurrent State Binds Phoneme Identity to Position. (A) The binding problem for sequences. Two words made of the same phonemes in a different order, /gum/ and /mug/, must map to different states, so the representation cannot depend on phoneme identity alone (e.g., a bag-of-phoneme model). (B) Two hypotheses for how position is encoded. Under the onion hypothesis (left) identity is carried by the direction of the representation and position by its magnitude, so a repeated phoneme keeps its direction across positions while its magnitude shrinks. Under the spiral hypothesis (right) identity is again carried by direction, but position is carried jointly by magnitude and rotation. (C) A recurrent network trained to repeat words serves as a hypothesis generator. Its positional representation can be both measured and causally manipulated, and the predictions it yields are then tested in human auditory-cortex recordings.
Figure 2: Delta states and the intervention. (A) How the contribution Δzi\Delta z_{i} of the phoneme at position ii is computed from consecutive encoder states. (B) Each phoneme adds an increment Δzi\Delta z_{i} , and the intervention maps the increment of the old phoneme /N/ to that of the new phoneme /S/, turning ANT into AST .
Figure 3: Hypothesis-Free Analysis of the Geometry of Phoneme-Sequence Encoding. Phoneme-position encoding: (A) Norm of the contextualized phoneme embedding Δz\Delta z as a function of phoneme position. In accordance with the onion hypothesis, the norm is largest at the first position and decreases for later positions in the word. Phoneme-identity encoding: The onion hypothesis holds that identity is carried by the direction of Δz\Delta z . (B) Distribution of cosine similarities between Δz\Delta z vectors, split by within-class (C–C, V–V) and across-class (C–V) pairs, where C and V denote consonants and vowels; medians dashed. Within-class pairs are positively aligned, while across-class pairs are close to orthogonal, so direction carries identity information. (C) Mean cosine similarity between contextualized embeddings of the same phoneme at each pair of positions, averaged over the phoneme inventory. If direction were preserved across positions, as the onion hypothesis requires, all entries would be close to one. Instead, similarity varies systematically with position, indicating that the identity direction changes as position advances and motivating the spiral refinement.
Figure 4: The Spiral Modulation Outperforms the Onion Modulation. (A) Mean intervention accuracy by method, with identity embeddings either learned or fixed. The spiral method outperforms the onion method in both settings. (B) Intervention accuracy by position. The two methods are comparable over the first few positions, but the onion method degrades sharply from position 4 onward while the spiral method declines only gradually, indicating that the rotation matters most where magnitude decay alone has compressed the representation. (C) PCA of the learned modulation ϕ\phi for the spiral method, in polar coordinates, one point per position (colour). The radius decays roughly exponentially with position, while the angle turns monotonically.
 Subspace baselineScale based
TargetDAS, k=1k=1DAS, k=2k=2OnionSpiral
Pseudoword0.000 ±\pm 0.0000.173 ±\pm 0.0160.631 ±\pm 0.0080.723 ±\pm 0.006
Real word0.102 ±\pm 0.0050.227 ±\pm 0.0050.901 ±\pm 0.0040.927 ±\pm 0.003
Table 1: Causal interventions show that the spiral model outperforms the onion and subspace models. Intervention accuracy by method and target, mean ±\pm standard deviation over four cross-validation seeds (Section 2.2 ).
Figure 5: Phoneme representation in the human superior temporal gyrus. (A) STG electrodes on the MNI template. Red marks the 59 of 108 that respond to the speech stimuli and are used in all analyses. (B) Norm of phoneme representations against its within-word position, shading is SEM. (C) Cosine similarity between pairs of phoneme representations, split by consonant and vowel pair type. (D) AUC for the same contrasts, the probability that a within-class pair is more similar than a between-class pair ( 0.50.5 is chance, dashed). Grey band is the 95% interval of the label-permutation null (1000 shuffles), ∗* marks p<0.001p<0.001 . (E) Same as (B) but recomputed separately within each of four surprisal quartiles, showing that the positional gradient is not carried by phoneme predictability.
Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Table 7
CVN-gramAccuracy
✗✗0.722 ±\pm 0.009
✓✗0.743 ±\pm 0.011
✗20.772 ±\pm 0.005
✓20.790 ±\pm 0.006
✗30.866 ±\pm 0.005
✓30.866 ±\pm 0.010
Appendix
Table 4: Effect of each phonotactic filter and their combinations, for the spiral model with trained mean- Δz\Delta z embeddings. CV enforces the consonant-vowel patterns of real words and N-gram rejects unattested bigrams (2) or trigrams (3).
MethodBaseSourceTarget
AdditiveP L AE N T—P L IH N T
DAS, k=1k=1P L AE N TK R IH S PP L IH N T
DAS, k=2k=2P L AE N TF L IH N CHP L IH N T
Appendix
Table 5: Data for replacing the phoneme at position p=3p=3 of plant with /IH/. Bold marks the edit, underlining marks the window on which the source must agree with the target, which widens with kk . Every other source position is resampled. The additive method needs no source sequence.
Figure 6: (A, B) Intervention accuracy by position for each method, including the unconstrained low-rank-8 method. The spiral method is competitive with the unconstrained method using far fewer parameters. (C) Mean ∥Δz∥\|\Delta z\| by position for the trained repeat model and three randomly initialised encoders of the same architecture, each curve normalised to its own position-1 value (untrained weights sit at an arbitrary scale, so only the shape is comparable). The trained profile decays; the untrained ones are flat. The decay in Figure 3 A is learned, is not a generic consequence of LSTM dynamics, but a learned modulation that carries position. (D) Learned modulation vectors ϕ\phi for the main methods in PCA space. The rotation is present in the unconstrained and spiral methods and absent from the onion method. (E) Consonants and vowels separate in PCA space. (F) Earlier positions are more separated while later positions converge toward the center, which explains why fixed embeddings underperform at early positions.
 Rotation
ScaleNoneRotary
Onion0.650.62
Exponential decay0.640.72
Appendix
Table 6: Intervention accuracy for each combination of scale parameterization and rotation, with the counterfactual as target. The rotation improves the monotonically decaying scale and degrades the onion scale.
 Trained emb.Fixed emb.
MethodRepeatAll filt.RepeatAll filt.
Low-rank-80.7800.8980.6780.807
Per-position0.7730.8780.6790.800
Low-rank-40.7360.8890.6630.802
Spiral0.7290.8660.6320.770
Low-rank-20.7230.8650.6600.788
Appendix
Table 7: All parameterizations on the source-to-counterfactual set, with trained and fixed mean- Δz\Delta z embeddings, under the repeat-only filter and under all filters.
 mean Δz\Delta zrepetition emb.
StateTrainedFixedFixedTrained
z=[h;c]z=[h;c]0.8990.8070.7390.893
cc (cell)0.8880.7680.7370.896
hh (hidden)0.0000.0000.0000.000
Appendix
Table 8: Intervention accuracy by the state edited, on the source-to-counterfactual set with all filters, the rank-8 method, across embedding settings. When the intervention edits cc , the decoder receives the source sequence’s hh together with the intervened c′c^{\prime} , and the reverse holds for editing hh .
More in this topic

Neural Encoding

Neural encoding studies how information is represented in biological neural activity and how models map stimuli or signals into neural codes.

See all 91 papers

Explore similar work

CardsList
Sep 29, 2026

Interpreting and Evaluating Dynamic-Rate Speech Codec Boundaries

Dynamic-frame-rate neural speech codecs replace a uniform frame grid with variable-duration tokens, making boundary placement part of the representation itself. Yet it is unclear what these boundaries encode and whether interpretable boundaries are also useful for neural speech reconstruction. This work combines boundary interpretation and controlled reconstruction analysis by comparing predicted boundaries with linguistic and acoustic references. We find that boundary meaning depends on the underlying speech representation. For the ASR-oriented SenseVoice and Whisper encoders, shallow layers emphasize phonetic, voicing, and acoustic transitions, whereas deeper layers shift toward syllable and subword structure. In a comparison of six dynamic-frame-rate algorithms and a uniform (fixed-frame-rate) baseline, higher-level linguistic alignment is associated with lower pooling distortion and better reconstruction from semantic tokens. Frame-rate-matched boundary tests make the distinction concrete: a syllable-derived partition improves over both Uniform and Similarity, while a phoneme-derived partition does not improve over Uniform. We infer that for semantically rich speech representations, useful codec boundaries are best understood as allocation decisions organized around the syllable scale.

Han Wang, Jiaqi Li, Yingda Shen +2

The Chinese University of Hong Kong, Shenzhen · Zhejiang University · Amphion Technology Co., Ltd.

Neural Audio CodecsAudio TokenizationDiscrete Speech Representation Learning
AskPDF
Mar 9, 2026

Structure from rank: Rank-order coding as a bridge from sequence to structure

Understanding how structured sequence information can be represented and generalized in neural systems is key to modeling the transition from acoustic input to emergent structure. In this study, we propose a rank-order based neural network inspired by the STG-LIFG-PMC pathway, modeling the bottom-up transition from acoustic input to abstract rank representation and the top-down generation from that representation to motor execution. Building on previous work in rank coding, we first demonstrate that this model efficiently compresses input while retaining the capacity to reconstruct full utterances from partial cues, revealing emergent structure-sensitive generation process that reflects context-general representations of sensorimotor states, which are later shaped into context-specific motor plans during speech planning. We then show that the network exhibits global-level novelty detection similar to the P3B novelty wave, replicating the global-sequence-sensitive mechanism. As a supplement, we also compare the model's behavior under local (index-level) and global (rank-level) perturbations, revealing robustness to superficial variation and sensitivity to abstract structural violation, key features associated with hierarchical generalization. These results suggest that rank-order coding not only serves as a compact encoding scheme but also captures hierarchical structure in acoustic sequences.

Xiaodan Chen, Alexandre Pitti, Mathias Quoy +1

ETIS, CY Cergy-Paris Université - ENSEA – CNRS, UMR 8051, 2 Av. Adolphe Chauvin, Pontoise, 95300, Ile de France, France · A*Star, 1 Fusionopolis Way, #20-10 Connexis North Tower, Singapore, 138632, Singapore · IPAL (International Research Laboratory on Artificial Intelligence), 1 Fusionopolis Way, #21-01 Connexis (South Tower), Singapore, 138632, Singapore

Speech ProductionSequence ModelingHierarchical Representation Learning
AskPDF
Sep 13, 2026

Formal Properties of Language as Constraints on Neural Dynamics

What must a neural system be capable of to implement language? Current research annotates stimuli with linguistic variables and tests which electrodes, voxels, or language-model layers predict neural activity. Yet predictive success leaves mechanisms under-constrained. Here, we show that algebraic properties of language specify invariants that mechanisms must preserve: non-associative hierarchical grouping, commutativity, recursive closure, access to substructures, and structured workspace transitions. We term this the Neural Admissibility Program (NAP). Syntactic structure building is analyzed algebraically, with candidate mechanisms offered for each requirement: content-addressable workspace memory, graph-structured transient dynamics scheduling structure-building operations (e.g. stable heteroclinic channels), and a phase-coupled sealing operation recording grouping. Simulations show that a corrected Marcolli-Berwick entropy-optimized binding gate preserves grouping only within a narrow commitment band. As an alternative, we propose a novel neural binding operation we term 'Meld': two constituent populations converge through shared synapses, integrate sublinearly, and saturate. Meld is, to our knowledge, the closest neurally plausible composition law to syntactic Merge. It preserves every NAP invariant, uses known cortical operations, and recovers hierarchical structure at every tested depth and temperature. It predicts that effective population dimensionality separates alternative bracketings and that the composite depends on constituent disagreement. Importantly, the laws decoding bracketing most accurately are a priori inadmissible, showing that decoding accuracy alone cannot adjudicate between mechanisms. By specifying how neural dynamics can remain faithful to linguistic structure, the NAP changes the criterion by which neural implementations of cognition are evaluated.

Elliot Murphy

Vivian L. Smith Department of Neurosurgery, McGovern Medical School, UTHealth, Houston, TX, USA · Texas Institute for Restorative Neurotechnologies, UTHealth, Houston, TX, USA

Neural EncodingComputational Psycholinguistics
AskPDF