Binding Phoneme Identity to Position in Neural Models and the Human Brain

Nafiseh Momeni; Malo Renaudin; Itzhak Fried; Yair Lakretz

cs.CLOct 11, 2026

Binding Phoneme Identity to Position in Neural Models and the Human Brain

Authors: Nafiseh Momeni, Malo Renaudin, Itzhak Fried, Yair Lakretz

Organizations: Université Paris Cité · École Normale Supérieure – PSL · University of California, Los Angeles

Abstract

A fundamental challenge in cognitive science and neural computation is the binding problem. Compositional generalization can be achieved by factorizing representations into roles, the structural slots of a representation (e.g., serial position or syntactic function), and fillers, the contents occupying them (e.g., a phoneme or a word), which can be recombined freely to represent novel combinations. How neural systems implement role--filler binding, however, remains unknown. One of its simplest cases is phoneme sequencing, where phoneme identity (filler) must be bound to position (role) so that /gum/ and /mug/ are represented differently. A proposal from NLP, the onion hypothesis' (Csordas et al., 2024), holds that the direction of the neural population state encodes token identity, while its magnitude encodes position. It was formulated for random token sequences; we extend it to natural language and test it in neural models and the human brain. We first evaluated it in an encoder-decoder trained to repeat English words, using representational geometry analyses and causal interventions. Magnitude alone proves incomplete: position is encoded by magnitude together with a systematic rotation of the state direction, a scheme we formalize as the spiral hypothesis'. We then analyzed intracranial recordings from the auditory cortex of 20 participants listening to short sentences. Across superior temporal gyrus electrodes, the change in population response between consecutive phonemes decreases in magnitude as position advances, and its direction is aligned within consonants and within vowels but near orthogonal across the two classes, supporting the magnitude coding shared by both accounts. These results provide a geometric account of how neural systems bind phoneme identity to position, and show how neural-model hypotheses can yield testable predictions about human neural computation.

Figures & tables

Figure 1: Hypotheses for How a Recurrent State Binds Phoneme Identity to Position. (A) The binding problem for sequences. Two words made of the same phonemes in a different order, /gum/ and /mug/, must map to different states, so the representation cannot depend on phoneme identity alone (e.g., a bag-of-phoneme model). (B) Two hypotheses for how position is encoded. Under the onion hypothesis (left) identity is carried by the direction of the representation and position by its magnitude, so a repeated phoneme keeps its direction across positions while its magnitude shrinks. Under the spiral hypothesis (right) identity is again carried by direction, but position is carried jointly by magnitude and rotation. (C) A recurrent network trained to repeat words serves as a hypothesis generator. Its positional representation can be both measured and causally manipulated, and the predictions it yields are then tested in human auditory-cortex recordings.

Figure 2: Delta states and the intervention. (A) How the contribution $\Delta z_{i}$ of the phoneme at position $i$ is computed from consecutive encoder states. (B) Each phoneme adds an increment $\Delta z_{i}$ , and the intervention maps the increment of the old phoneme /N/ to that of the new phoneme /S/, turning ANT into AST .

Figure 3: Hypothesis-Free Analysis of the Geometry of Phoneme-Sequence Encoding. Phoneme-position encoding: (A) Norm of the contextualized phoneme embedding $\Delta z$ as a function of phoneme position. In accordance with the onion hypothesis, the norm is largest at the first position and decreases for later positions in the word. Phoneme-identity encoding: The onion hypothesis holds that identity is carried by the direction of $\Delta z$ . (B) Distribution of cosine similarities between $\Delta z$ vectors, split by within-class (C–C, V–V) and across-class (C–V) pairs, where C and V denote consonants and vowels; medians dashed. Within-class pairs are positively aligned, while across-class pairs are close to orthogonal, so direction carries identity information. (C) Mean cosine similarity between contextualized embeddings of the same phoneme at each pair of positions, averaged over the phoneme inventory. If direction were preserved across positions, as the onion hypothesis requires, all entries would be close to one. Instead, similarity varies systematically with position, indicating that the identity direction changes as position advances and motivating the spiral refinement.

Figure 4: The Spiral Modulation Outperforms the Onion Modulation. (A) Mean intervention accuracy by method, with identity embeddings either learned or fixed. The spiral method outperforms the onion method in both settings. (B) Intervention accuracy by position. The two methods are comparable over the first few positions, but the onion method degrades sharply from position 4 onward while the spiral method declines only gradually, indicating that the rotation matters most where magnitude decay alone has compressed the representation. (C) PCA of the learned modulation $\phi$ for the spiral method, in polar coordinates, one point per position (colour). The radius decays roughly exponentially with position, while the angle turns monotonically.

	Subspace baseline		Scale based
Target	DAS, $k=1$	DAS, $k=2$	Onion	Spiral
Pseudoword	0.000 $\pm$ 0.000	0.173 $\pm$ 0.016	0.631 $\pm$ 0.008	0.723 $\pm$ 0.006
Real word	0.102 $\pm$ 0.005	0.227 $\pm$ 0.005	0.901 $\pm$ 0.004	0.927 $\pm$ 0.003

Table 1: Causal interventions show that the spiral model outperforms the onion and subspace models. Intervention accuracy by method and target, mean

\pm

standard deviation over four cross-validation seeds (Section 2.2 ).

Figure 5: Phoneme representation in the human superior temporal gyrus. (A) STG electrodes on the MNI template. Red marks the 59 of 108 that respond to the speech stimuli and are used in all analyses. (B) Norm of phoneme representations against its within-word position, shading is SEM. (C) Cosine similarity between pairs of phoneme representations, split by consonant and vowel pair type. (D) AUC for the same contrasts, the probability that a within-class pair is more similar than a between-class pair ( $0.5$ is chance, dashed). Grey band is the 95% interval of the label-permutation null (1000 shuffles), $*$ marks $p<0.001$ . (E) Same as (B) but recomputed separately within each of four surprisal quartiles, showing that the positional gradient is not carried by phoneme predictability.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Table 7

CV	N-gram	Accuracy
✗	✗	0.722 $\pm$ 0.009
✓	✗	0.743 $\pm$ 0.011
✗	2	0.772 $\pm$ 0.005
✓	2	0.790 $\pm$ 0.006
✗	3	0.866 $\pm$ 0.005
✓	3	0.866 $\pm$ 0.010

Appendix

Table 4: Effect of each phonotactic filter and their combinations, for the spiral model with trained mean-

\Delta z

embeddings. CV enforces the consonant-vowel patterns of real words and N-gram rejects unattested bigrams (2) or trigrams (3).

Method	Base	Source	Target
Additive	P L AE N T	—	P L IH N T
DAS, $k=1$	P L AE N T	K R IH S P	P L IH N T
DAS, $k=2$	P L AE N T	F L IH N CH	P L IH N T

Appendix

Table 5: Data for replacing the phoneme at position

p=3

of plant with /IH/. Bold marks the edit, underlining marks the window on which the source must agree with the target, which widens with

k

. Every other source position is resampled. The additive method needs no source sequence.

Figure 6: (A, B) Intervention accuracy by position for each method, including the unconstrained low-rank-8 method. The spiral method is competitive with the unconstrained method using far fewer parameters. (C) Mean

\|\Delta z\|

by position for the trained repeat model and three randomly initialised encoders of the same architecture, each curve normalised to its own position-1 value (untrained weights sit at an arbitrary scale, so only the shape is comparable). The trained profile decays; the untrained ones are flat. The decay in Figure 3 A is learned, is not a generic consequence of LSTM dynamics, but a learned modulation that carries position. (D) Learned modulation vectors

\phi

for the main methods in PCA space. The rotation is present in the unconstrained and spiral methods and absent from the onion method. (E) Consonants and vowels separate in PCA space. (F) Earlier positions are more separated while later positions converge toward the center, which explains why fixed embeddings underperform at early positions.

	Rotation
Scale	None	Rotary
Onion	0.65	0.62
Exponential decay	0.64	0.72

Appendix

Table 6: Intervention accuracy for each combination of scale parameterization and rotation, with the counterfactual as target. The rotation improves the monotonically decaying scale and degrades the onion scale.

	Trained emb.		Fixed emb.
Method	Repeat	All filt.	Repeat	All filt.
Low-rank-8	0.780	0.898	0.678	0.807
Per-position	0.773	0.878	0.679	0.800
Low-rank-4	0.736	0.889	0.663	0.802
Spiral	0.729	0.866	0.632	0.770
Low-rank-2	0.723	0.865	0.660	0.788

Appendix

Table 7: All parameterizations on the source-to-counterfactual set, with trained and fixed mean-

\Delta z

embeddings, under the repeat-only filter and under all filters.

	mean $\Delta z$		repetition emb.
State	Trained	Fixed	Fixed	Trained
$z=[h;c]$	0.899	0.807	0.739	0.893
$c$ (cell)	0.888	0.768	0.737	0.896
$h$ (hidden)	0.000	0.000	0.000	0.000

Appendix

Table 8: Intervention accuracy by the state edited, on the source-to-counterfactual set with all filters, the rank-8 method, across embedding settings. When the intervention edits

c

, the decoder receives the source sequence’s

h

together with the intervened

c^{\prime}

, and the reverse holds for editing

h

Neural Encoding

Neural encoding studies how information is represented in biological neural activity and how models map stimuli or signals into neural codes.

See all 91 papers

Interpreting and Evaluating Dynamic-Rate Speech Codec Boundaries
Sep 29, 2026Han Wang, Jiaqi Li, Yingda Shen +2Neural Audio Codecs Audio Tokenization
The Chinese University of Hong Kong, Shenzhen · Zhejiang University · Amphion Technology Co., Ltd.
Structure from rank: Rank-order coding as a bridge from sequence to structure
Mar 9, 2026Xiaodan Chen, Alexandre Pitti, Mathias Quoy +1Speech Production Sequence Modeling
ETIS, CY Cergy-Paris Université - ENSEA – CNRS, UMR 8051, 2 Av. Adolphe Chauvin, Pontoise, 95300, Ile de France, France · A*Star, 1 Fusionopolis Way, #20-10 Connexis North Tower, Singapore, 138632, Singapore · IPAL (International Research Laboratory on Artiﬁcial Intelligence), 1 Fusionopolis Way, #21-01 Connexis (South Tower), Singapore, 138632, Singapore
Formal Properties of Language as Constraints on Neural Dynamics
Sep 13, 2026Elliot MurphyNeural Encoding Computational Psycholinguistics
Vivian L. Smith Department of Neurosurgery, McGovern Medical School, UTHealth, Houston, TX, USA · Texas Institute for Restorative Neurotechnologies, UTHealth, Houston, TX, USA

Sep 29, 2026J/K move · Enter open · S save

Interpreting and Evaluating Dynamic-Rate Speech Codec Boundaries

Han Wang, Jiaqi Li, Yingda Shen, Yuxiang Wang+1

The Chinese University of Hong Kong, Shenzhen · Zhejiang University · Amphion Technology Co., Ltd.

Neural Audio Codecs Audio Tokenization Discrete Speech Representation Learning

Open paper AskPDF arXiv

Dynamic-frame-rate neural speech codecs replace a uniform frame grid with variable-duration tokens, making boundary placement part of the representation itself. Yet it is unclear what these boundaries encode and whether interpretable boundaries are also useful for neural speech reconstruction. This work combines boundary interpretation and controlled reconstruction analysis by comparing predicted boundaries with linguistic and acoustic references. We find that boundary meaning depends on the underlying speech representation. For the ASR-oriented SenseVoice and Whisper encoders, shallow layers emphasize phonetic, voicing, and acoustic transitions, whereas deeper layers shift toward syllable and subword structure. In a comparison of six dynamic-frame-rate algorithms and a uniform (fixed-frame-rate) baseline, higher-level linguistic alignment is associated with lower pooling distortion and better reconstruction from semantic tokens. Frame-rate-matched boundary tests make the distinction concrete: a syllable-derived partition improves over both Uniform and Similarity, while a phoneme-derived partition does not improve over Uniform. We infer that for semantically rich speech representations, useful codec boundaries are best understood as allocation decisions organized around the syllable scale.

Abstract

Neural Encoding

Explore similar work

Interpreting and Evaluating Dynamic-Rate Speech Codec Boundaries

Structure from rank: Rank-order coding as a bridge from sequence to structure

Formal Properties of Language as Constraints on Neural Dynamics