Convergent Representations of Linguistic Constructions in Human and Artificial Neural Systems
Authors: Pegah Ramezani, Thomas Kinfe, Andreas Maier, Achim Schilling, Patrick Krauss
Organizations: Department of English and American Studies, University Erlangen-Nuremberg, Germany · Pattern Recognition Lab, University Erlangen-Nuremberg, Germany · Neuromodulation and Neuroprosthetics, University Hospital Mannheim, University Heidelberg, Germany · BGU Ludwigshafen, Germany · Neuroscience Lab, University Hospital Erlangen, Germany
Understanding how the brain processes linguistic constructions is a central challenge in cognitive neuroscience and linguistics. Recent computational studies show that artificial neural language models spontaneously develop differentiated representations of Argument Structure Constructions (ASCs), generating predictions about when and how construction-level information emerges during processing. The present study tests these predictions in human neural activity using electroencephalography (EEG). Ten native English speakers listened to 200 synthetically generated sentences across four construction types (transitive, ditransitive, caused-motion, resultative) while neural responses were recorded. Analyses using time-frequency methods, feature extraction, and machine learning classification revealed construction-specific neural signatures emerging primarily at sentence-final positions, where argument structure becomes fully disambiguated, and most prominently in the alpha band. Pairwise classification showed reliable differentiation, especially between ditransitive and resultative constructions, while other pairs overlapped. Crucially, the temporal emergence and similarity structure of these effects mirror patterns in recurrent and transformer-based language models, where constructional representations arise during integrative processing stages. These findings support the view that linguistic constructions are neurally encoded as distinct form-meaning mappings, in line with Construction Grammar, and suggest convergence between biological and artificial systems on similar representational solutions. More broadly, this convergence is consistent with the idea that learning systems discover stable regions within an underlying representational landscape - recently termed a Platonic representational space - that constrains the emergence of efficient linguistic abstractions.
Figures & tables
Constructions
Structure
Example
Transitive
Subject + Verb + Object
The baker baked a cake.
Ditransitive
Subject + Verb + Object1 + Object2
The teacher gave students homework.
Caused-Motion
Subject + Verb + Object + Path
The cat chased the mouse into the garden.
Resultative
Subject + Verb + Object + State
The chef cut the cake into slices.
Table 1: Name, structure, and example of each construction
Figure 1: Token Duration Variability Across Syntactic Roles. This figure presents boxplots of the durations of subject, verb, and object tokens in the stimulus set. Each boxplot displays the median, interquartile range, whiskers, and outliers, showing that all three syntactic roles exhibit substantial variability in word length. Object tokens show the widest distribution, while verbs are generally shorter. The visualization illustrates the challenge posed by duration differences in natural-language EEG experiments, which motivated the use of maximum-length epoch alignment for ERP analyses and length-independent feature extraction for statistical analyses.
Figure 2: ERP Waveforms Time-Locked to Sentence Onset Across All Channels. This figure shows the averaged event-related potentials from 63 EEG channels, aligned to the onset of each sentence. Each colored trace represents one electrode, illustrating the spatial diversity of responses across the scalp. A prominent positive deflection around approximately 200 ms is visible across many channels, corresponding to the P200 component associated with early perceptual and lexical processing. The waveform morphology indicates that the dataset contains robust time-locked neural responses suitable for later time–frequency analyses across construction types.
Figure 3: Variation in Sentence Length Across Constructions. This figure presents boxplots of sentence durations for the four construction types used in the experiment: Cause_Motion, Resultative, Ditransitive, and Transitive. Each boxplot shows the median, interquartile range, whiskers, and outliers, illustrating that sentence length varies substantially both within and across constructions. The Transitive sentences tend to be shorter on average, while the other three constructions cluster around longer median durations. This variability motivates standardizing epoch length in EEG analyses, ensuring that neural responses can be aligned and compared despite differences in stimulus duration.
Figure 4: Time-frequency representations for each construction type after removing the common average across constructions. The plots show the average Morlet wavelet power (2–45 Hz) for the four construction types (Cause_Motion, Resultative, Ditransitive, Transitive). Each subplot visualizes power over time and frequency with a shared color scale. Subtracting the common average emphasizes where oscillatory dynamics diverge between constructions, thereby making construction-specific spectral patterns visible across corresponding sentence positions.
Figure 5: Significant EEG Feature Counts Across Construction Pairs and Syntactic Roles. This figure displays a heatmap summarizing the number of EEG features that showed significant differences (p < 0.05, Benjamini–Hochberg-corrected) for each construction pair and each syntactic role: subject, verb, and object. Lighter colors indicate fewer significant features, while darker colors correspond to more distinguishing features. The subject position shows almost no significant differences across all construction pairs, suggesting that early sentence information does not permit construction discrimination. Verb-aligned epochs exhibit only sparse effects, consistent with verbs being less construction-specific. In contrast, object-aligned epochs show substantially more significant features, particularly for the Ditransitive–Resultative pair, indicating that construction-specific neural signatures emerge primarily when listeners have accumulated sufficient contextual information later in the sentence.
Figure 6: Significant EEG Feature Counts by Frequency Band. This figure shows the number of EEG features within each frequency band that exhibited significant differences (p < 0.05, Benjamini–Hochberg corrected) across construction pairs. The alpha band displays the highest number of discriminative features, followed by beta and delta, indicating that these bands were most responsive to constructional variation. Theta and gamma show markedly fewer significant effects. These counts should not be interpreted as exclusive involvement of specific bands in construction processing, but rather as evidence that certain spectral ranges were more sensitive to construction differences. The broader oscillatory patterns underlying these effects are visible in the time–frequency analyses.
Figure 7: Significant Feature Counts for Broadband Signal Measures. This figure shows the number of broadband EEG features that exhibited significant differences (p < 0.05, Benjamini–Hochberg-corrected) across construction pairs for each feature type. Kurtosis and peak amplitude yielded the largest number of distinguishing features, indicating strong sensitivity to construction-related variability. Mean amplitude and standard deviation showed moderate effects, while zero-crossing rate revealed only minimal differences. Together, these results highlight which broadband features were most effective in capturing construction-specific neural patterns.
Figure 8: Classification Performance for Each Construction Pair. This figure shows the mean accuracy, F1-score, and recall obtained from pairwise SVM classification across subjects for all construction pairs. Each bar group represents one pair of constructions, and the three colors indicate the different evaluation metrics. Pairs such as Ditransitive vs. Resultative achieve the highest performance, indicating stronger discriminability in the underlying EEG patterns. In contrast, pairs like Caused-Motion vs. Resultative and Ditransitive vs. Transitive show lower performance, consistent with the statistical analyses suggesting limited separability. The results demonstrate that while some constructions can be reliably distinguished from one another, others exhibit overlapping neural signatures.
Label pair
Mean
Median
W
p
Rank-biserial r
Caused Motion vs Ditransitive
0.1718
0.1700
66.0
0.000488
1.0000
Caused Motion vs Transitive
0.1218
0.1300
66.0
0.000488
1.0000
Caused Motion vs Resultative
-0.0491
-0.0300
17.0
0.926270
-0.4848
Ditransitive vs Transitive
-0.0309
0.0000
18.0
0.833560
-0.3455
Ditransitive vs Resultative
0.2055
0.2000
66.0
0.000488
1.0000
Transitive vs Resultative
0.1427
0.1400
66.0
0.000488
1.0000
Table 2: Wilcoxon signed-rank test results (accuracy minus chance).
Label pair
Observed acc.
Perm. acc.
Perm. p
Caused Motion vs Ditransitive
0.673
0.454
0.0136
Caused Motion vs Transitive
0.625
0.455
0.0386
Caused Motion vs Resultative
0.454
0.451
0.4980
Ditransitive vs Transitive
0.471
0.456
0.4565
Ditransitive vs Resultative
0.705
0.456
0.0037
Transitive vs Resultative
0.642
0.455
0.0181
Table 3: Grand average decoding results across subjects ( n=11 ) for each pairwise construction contrast. Reported are the mean observed leave-one-out accuracy, the mean permutation null accuracy (1000 label shuffles), and the mean empirical permutation p -value (averaged across subjects).
Figure 9: Permutation test null distribution for Subject 06 (Caused Motion vs Transitive). The histogram shows decoding accuracies obtained from 1000 label permutations (null distribution). The solid red line marks the observed accuracy (0.650), and the dashed blue line marks the mean null accuracy (0.445). The permutation p value (p = 0.004) is the proportion of permuted accuracies greater than or equal to the observed accuracy.
Figure 10: Mean binary classification performance across subjects for each pair of constructions for two senarios, one with standard CV and the other one with Leave verb out..
Language-model representations provide structured, high-dimensional annotations of naturalistic language stimuli and can serve as informative neural predictors during comprehension. We analyzed locked derived data from Brain Treebank, MEG-MASC, and Podcast ECoG with eight frozen language models, blocked encoding models, and matched temporal, nuisance, and representation-capacity controls. Positive held-out prediction and gains over low-level baselines were widespread in source-level summaries. Across Brain Treebank and Podcast ECoG, 67 of 432 evaluable rows met a controlled predictive-only criterion, and model-side feature ablations changed prediction scores in most evaluable source rows. Brain-derived, timing-linked, acoustic, and implanted-signal controls confirmed component-level sensitivity of the analysis pipeline. These findings show that language-model-derived quantities can annotate neural activity during natural speech and text comprehension. Participant-level matched-control advantages were localized rather than uniform, response-profile and feature-specificity contrasts bounded representational or computational interpretations, and complete co-indexed integrated interpretation will require future jointly indexed coverage. Together, the analyses identify language-model features as useful neural predictors and separate predictive usefulness from claims about shared neural organization or language-processing computations.
Xiao Jia
School of Artificial Intelligence The Chinese University of Hong Kong, Shenzhen
A central goal of cognitive neuroscience is to characterize the features that are represented by human language cortex. Artificial language models (LMs) have emerged as a powerful tool to address this challenge, but studies relating biological and artificial representations are often criticized as relating one black box to another. The present work introduces Augmented Sparse Encoding Models, an encoding framework that replaces dense LM hidden states with hierarchically-organized sparse autoencoder (SAE) features, while explicitly including surprisal as a predictor. Using this approach, we (i) produce interpretations of neural responses and (ii) test whether model-brain alignment reflects primary or idiosyncratic variation in LM representations. Using a high-field 7T fMRI dataset of eight participants listening to 200 linguistically diverse sentences, we first validate our modeling framework by recovering previous interpretations of voxel populations tuned to processing difficulty and meaning abstractness. We then interpret a previously-uncharacterized (but reliable) voxel population and find that it is tuned to people-related content. Next, we show that the fronto-temporal human language network is predicted by a common set of features across its constituent regions, but find that frontal regions are relatively well-explained by surprisal alone, even in the absence of LM-based features. Finally, we show that brain responses during language processing are not merely predictable from an arbitrary set of LM features. Rather, brain responses are best explained by the features that tend to capture the most general information encoded in LM representations, suggesting a nontrivial correspondence between brain and LM language representation.
Michael A. Lepori, Kendrick Kay, Greta Tuckute
Dept. of Computer Science Brown University · Dept. of Radiology University of Minnesota · Kempner Institute Harvard University
Understanding how Large Language Models (LLMs) encode linguistic structures remains a fundamental challenge in interpretability research. While diagnostic classifiers (or "probes") are widely used for this task, they face significant methodological criticism: training auxiliary classifiers introduces capacity confounds and calibration issues, often making it difficult to distinguish the model's intrinsic representations from the probe's ability to learn the task. To address these limitations, we introduce a probe-free framework for localizing linguistic selectivity at the individual neuron level. Leveraging the controlled contrasts of linguistic minimal pairs, we propose a Neuron Separability Index (NSI), a metric that directly quantifies how reliably single neurons differentiate grammatical from ungrammatical constructions without parameter updates. Applying NSI across 68 linguistic paradigms and seven checkpoints reveals three main patterns: 1) raw separability reaches near-peak levels earlier for morphological and syntactic distinctions than for syntax-semantics interface and conceptual distinctions. 2) after permutation normalization, single-unit selectivity is sparse, weak, and narrowly tuned: only a small fraction of units are sensitive to an average paradigm, and strongly selective "grandmother neurons" are rare. 3) whole-vector linear separability, single-neuron selectivity, and behavioral competence are largely dissociated, and targeted ablations further separate activation selectivity from causal reliance.
Linyang He, Nima Mesgarani
Zuckerman Mind Brain Behavior Institute, Columbia University