Organizations: ETIS, CY Cergy-Paris Université - ENSEA – CNRS, UMR 8051, 2 Av. Adolphe Chauvin, Pontoise, 95300, Ile de France, France · A*Star, 1 Fusionopolis Way, #20-10 Connexis North Tower, Singapore, 138632, Singapore · IPAL (International Research Laboratory on Artificial Intelligence), 1 Fusionopolis Way, #21-01 Connexis (South Tower), Singapore, 138632, Singapore
Understanding how structured sequence information can be represented and generalized in neural systems is key to modeling the transition from acoustic input to emergent structure. In this study, we propose a rank-order based neural network inspired by the STG-LIFG-PMC pathway, modeling the bottom-up transition from acoustic input to abstract rank representation and the top-down generation from that representation to motor execution. Building on previous work in rank coding, we first demonstrate that this model efficiently compresses input while retaining the capacity to reconstruct full utterances from partial cues, revealing emergent structure-sensitive generation process that reflects context-general representations of sensorimotor states, which are later shaped into context-specific motor plans during speech planning. We then show that the network exhibits global-level novelty detection similar to the P3B novelty wave, replicating the global-sequence-sensitive mechanism. As a supplement, we also compare the model's behavior under local (index-level) and global (rank-level) perturbations, revealing robustness to superficial variation and sensitivity to abstract structural violation, key features associated with hierarchical generalization. These results suggest that rank-order coding not only serves as a compact encoding scheme but also captures hierarchical structure in acoustic sequences.
We investigated the relationship among neural representations of vocalized, mimed, and imagined speech recorded using publicly available stereotactic EEG recordings. Most prior studies have focused on decoding speech responses within each condition separately. Here, instead, we explore how responses across conditions relate by training linear spectrogram reconstruction models for each condition and evaluate their generalization across conditions. We demonstrate that linear decoders trained on one condition generally transfer successfully to others, implying shared speech representations. This commonality was assessed with stimulus-level discriminability by performing a rank-based analysis demonstrating preservation of stimulus-specific structure in both within- and across-conditions. Finally, we compared linear reconstructions to those from a nonlinear neural network. While both exhibited cross-condition transfer, linear models achieve superior stimulus-level discriminability.
Maryam Maghsoudi, Rupesh Chillale, Shihab A. Shamma
University of Maryland - College Park, United States
A fundamental challenge in cognitive science and neural computation is the binding problem. Compositional generalization can be achieved by factorizing representations into roles, the structural slots of a representation (e.g., serial position or syntactic function), and fillers, the contents occupying them (e.g., a phoneme or a word), which can be recombined freely to represent novel combinations. How neural systems implement role--filler binding, however, remains unknown. One of its simplest cases is phoneme sequencing, where phoneme identity (filler) must be bound to position (role) so that /gum/ and /mug/ are represented differently. A proposal from NLP, the onion hypothesis' (Csordas et al., 2024), holds that the direction of the neural population state encodes token identity, while its magnitude encodes position. It was formulated for random token sequences; we extend it to natural language and test it in neural models and the human brain. We first evaluated it in an encoder-decoder trained to repeat English words, using representational geometry analyses and causal interventions. Magnitude alone proves incomplete: position is encoded by magnitude together with a systematic rotation of the state direction, a scheme we formalize as the spiral hypothesis'. We then analyzed intracranial recordings from the auditory cortex of 20 participants listening to short sentences. Across superior temporal gyrus electrodes, the change in population response between consecutive phonemes decreases in magnitude as position advances, and its direction is aligned within consonants and within vowels but near orthogonal across the two classes, supporting the magnitude coding shared by both accounts. These results provide a geometric account of how neural systems bind phoneme identity to position, and show how neural-model hypotheses can yield testable predictions about human neural computation.
Nafiseh Momeni, Malo Renaudin, Itzhak Fried +1
Université Paris Cité · École Normale Supérieure – PSL · University of California, Los Angeles
Sperm-whale codas are usually described by the number and timing of their clicks. These measures have revealed patterns in whale communication, but leave open how click sound contributes beyond timing and whether the order of whole codas carries additional structure. We investigate these questions through two tiers of acoustic organization, first within codas and then across coda sequences. Frozen pretrained audio encoders provide a framework for measuring acoustic variation beyond timing. We use them to identify recurring forms independently for clicks and whole codas in 1,483 codas, then validate their differences through direct waveform measurements. Within codas, click sound provides information beyond timing alone, while timing remains informative when click composition is held fixed. At the level of complete codas, we also find recurring acoustic forms whose differences are not fully captured by conventional timing-based types. With these forms identified, we examine how ordering contributes at each tier. Within codas, click-form order adds little predictive information beyond composition. Across codas, knowing the last two codas improves prediction over knowing only the last one. Together, these findings support two tiers with different organizing rules: acoustic composition and rhythmic timing within codas, and predictive relationships among whole codas across recorded sequences.
Mudit Sinha, Sanika Chavan
Independent Researcher, San Bruno, California 94066, United States