Organizations: Department of Computer Science, The University of Manchester, Manchester, UK. · State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, CAS, Beijing, China. · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China. · Department of Linguistics and Translation, City University of Hong Kong, Hong Kong, China. · Department of Language Science and Technology, The Hong Kong Polytechnic University, Hong Kong, China. · Cognitive Science Department, Johns Hopkins University, Baltimore, MD, USA.
How the brain supports language across different languages is a basic question in neuroscience and a useful test for multilingual artificial intelligence. Neuroimaging has identified language-responsive brain regions across languages, but it cannot by itself show whether the underlying processing is shared or language-specific. Here we use six multilingual large language models (LLMs) as controllable systems and create targeted ``computational lesions'' by zeroing small parameter sets that are important across languages or especially important for one language. We then compare intact and lesioned models in predicting functional magnetic resonance imaging (fMRI) responses during 100 minutes of naturalistic story listening in native English, Chinese and French (112 participants). Lesioning a compact shared core reduces whole-brain encoding correlation by 60.32% relative to intact models, whereas language-specific lesions preserve cross-language separation in embedding space but selectively weaken brain predictivity for the matched native language. These results support a shared backbone with embedded specializations and provide a causal framework for studying multilingual brain-model alignment.
Figures & tables
Figure 1: Overview of the multilingual neural encoding framework with computationally lesioned large language models (a), Multilingual neural encoding pipeline. Native speakers of English (EN), Chinese (CN), and French (FR) listened to audiobook narratives while undergoing fMRI scanning. Textual stimuli were processed by large language models (LLMs) to extract contextualized embeddings from the final hidden layer, which were then used in voxel-wise encoding models to account for neural responses during native-language comprehension. (b), Multilingual LLMs and lesion conditions. For each model family, we evaluated an intact multilingual LLM together with computationally lesioned variants, including a core-language–ablated model and language-specific–ablated models corresponding to English, Chinese, and French. These models were used to dissociate shared versus language-specific contributions to neural encoding. (c), Gradient-guided computational lesioning. Core and language-specific parameter subsets within each multilingual LLM were identified using gradient-based importance estimates derived from language-specific full-parameter fine-tuning. During backpropagation, parameter gradients were accumulated and combined with their corresponding weights (gradient × weight) to quantify functional importance. Parameters ranking in the top 1% of importance within each subset were then selectively ablated (zeroed) to induce targeted computational lesions, yielding core-lesioned and language-specific-lesioned models for downstream neural encoding analyses.
Figure 2: Neural encoding accuracy across multilingual large language models a–c, Cross-linguistically robust cortical encoding maps across multilingual language models. Cortical maps of voxel-wise encoding correlation for six large language models across English (EN), Chinese (CN), and French (FR) native listeners. For each model, we first computed voxel-wise neural encoding accuracy as the Pearson correlation between predicted and observed BOLD time series, averaged across all participants within each language. Voxels exceeding a significance threshold of p<0.01 were retained, and the intersection of significant voxels across the three languages was computed. The resulting shared voxel mask was then averaged across languages to generate a cross-linguistically robust cortical encoding map for each model. d, Region-of-interest neural encoding performance across languages and models. Model-wise neural encoding performance within functionally defined language regions of interest. Using the language network parcellation defined by Fedorenko and colleagues, each scatter point represents the mean encoding correlation of a single model within a given region and language, while bar plots indicate the across-model average performance. Across inferior frontal, middle frontal, anterior temporal, mid-temporal, and posterior temporal ROIs, all six models exhibited reliable predictive power, with broadly comparable performance across English, French, and Chinese. These results demonstrate that despite architectural and training differences, multilingual LLMs capture cortical representations that generalize across languages and align with established human language networks.
Figure 3: Disrupting core language regions in multilingual LLMs alters internal representations and reduces neural predictivity. (a), Cross-linguistic representational shift after core language lesion. UMAP visualizations of final-layer token embeddings from Qwen2.5–7B show distinct representational clusters for Chinese, English, and French in the intact model (light colors). After ablating the core language region (uniform 1% parameter removal across layers and components), the embeddings of all three languages become compressed and partially overlapping (solid colors), indicating a systematic loss of representational structure. (b), Parameter localization of the core language lesion. Heatmaps illustrate the disrupted parameters within the final transformer layer (layer 27), highlighting the distribution of ablated weights across both self-attention (key and value projections) and MLP (up- and down-projection) components. (c), Impaired linguistic competence following core language disruption. Across four probing tasks (sentence length, tree depth, top constituents, and coordination inversion), performance decreases consistently in all three languages, demonstrating that the ablation selectively disrupts features closely tied to syntactic and hierarchical linguistic structure. (d), Neural encoding consequences of core language lesion across languages. Voxel-wise neural encoding analyses reveal substantial decreases in predictive performance when using embeddings from core-language–ablated LLMs. Bar plots show cross-participant mean encoding correlations for intact models (blue), random ablated model (purple) and core-language-region ablated models (orange) across six LLMs, with grey dots representing individual participants. Across English, French, and Chinese, intact models consistently outperform their ablated counterparts. Below each bar chart, whole-brain t-maps visualize voxel-wise differences between intact and ablated model predictions, with larger t-values indicating stronger reductions in neural predictivity following ablation. Reduced encoding performance is most prominent within bilateral temporal cortices and inferior frontal regions, demonstrating that the ablated parameters correspond to neural computations that are critical for supporting native language processing across languages.
Figure 4: Language-specific lesions selectively disrupt language-matched cortical encoding while preserving cross-language representational structure. a, Language-specific representational separation is preserved but reorganized after lesioning. UMAP visualizations of final-layer token embeddings across English, French, and Chinese show well-separated clusters in the intact model (light colors). Following lesioning of each language-specific parameter subset, embeddings from the corresponding language (solid colors) shift to reorganized manifolds without collapsing onto other languages. This indicates that language-specific circuits support within-language representational structure rather than global cross-language separability. b, Distribution of language-specific parameters across transformer components. Heatmaps from the final transformer layer (layer 27) illustrate the spatial distribution of lesioned parameters for English-, French-, and Chinese-specific models. Language-specific parameters appear concentrated in structured rows within self-attention projections, whereas MLP components exhibit more spatially diffuse but systematic patterns, consistent with complementary roles in information routing versus representational transformation. c, Language-matched neural encoding disruption following language-specific lesioning. Voxel-wise paired t-tests comparing intact and lesioned models (Mistral-8B) were projected onto the cortical surface. Lesioning English-specific parameters selectively reduced neural predictivity in English participants, with analogous matched-language effects observed for French and Chinese. Disruptions localize to canonical language-selective regions, including anterior and posterior temporal cortex and inferior frontal gyrus. d, Convergent encoding impairment across models. For each language, T-maps from language-matched lesions were averaged across all six models and summarized using a Language Processing Index. Group-level cortical maps reveal robust, left-lateralized reductions in neural predictivity, demonstrating that language-specific parameter subsets disproportionately support alignment with native-language brain responses.
Model
Family
Params
Layers
Dim
Attention
Tokenizer
Ctx.
LLaMA2-7B
LLaMA2
7B
32
4096
MHA
SentencePiece
4k
LLaMA2-13B
LLaMA2
13B
40
5120
MHA
SentencePiece
4k
Qwen2.5-7B
Qwen2.5
7B
32
4096
GQA
BPE
128k
Qwen2.5-14B
Qwen2.5
14B
48
5120
GQA
BPE
128k
Mistral-8B
Mistral
8B
40
4096
GQA
SentencePiece
8k
Mistral-12B
Mistral
12B
40
5120
GQA
SentencePiece
8k
Table 1: Architectural comparison of multilingual LLMs used in this study. All models adopt decoder-only Transformer architectures but differ in scale(7-14B), attention design(MHA vs. GQA), tokenizer construction …, and multilingual training emphasis. Architectural specifications are compiled from official technical reports and model documentation 23 (23) ; 24 (24) ; 25 (25) ; 26 (26) ; 27 (27) ; 28 (28) .
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure A1: Cortical encoding t-maps comparing intact and core-language-ablated models across multilingual participants. Voxel-wise t-maps from six large language models are shown for intact and core-language-ablated conditions across English-, Chinese-, and French-native participant groups. Intact models exhibit robust encoding throughout the distributed language network, whereas core ablation produces widespread attenuation of encoding strength, particularly in superior temporal and inferior frontal cortices. Lesion effects are spatially convergent across participant groups. Maps are thresholded at FDR<0.01 (voxel-wise, cortical mask).
Brain-language model alignment is often interpreted as evidence that transformer models implement computations similar to those of the human brain. This assumes that neural predictivity reflects internal computational properties of large language models (LLMs), such as hierarchical contextual processing, predictive coding, or representational compression. An alternative possibility is that brain scores primarily reflect stable lexical-semantic correspondences shared by language models and the brain. Here we tested these interpretations using whole-brain encoding models across Mandarin, English, and French. Across all three languages, transformer representations significantly predicted activity in a distributed network spanning classical language regions, transmodal cortical systems, and subcortical structures. These spatial patterns showed substantial cross-linguistic overlap and remained remarkably stable across layers, providing little evidence that model depth systematically maps onto cortical processing hierarchies. Likewise, contextual transformer embeddings did not consistently outperform static lexical embeddings, despite providing some unique predictive variance. Finally, neither surprisal nor intrinsic dimensionality reproduced the layer-wise profile of brain scores, arguing against prediction and information compression as primary explanations for brain-LLM alignment. Together, these findings suggest that brain-LLM alignment is more robust across languages, transformer depth, and model architectures than previously appreciated, but less informative about shared computational mechanisms. Our results are more consistent with neural predictivity reflecting stable representational structure preserved across model transformations than with a one-to-one correspondence between their underlying computations.
Ni Yang, Rui He, Philipp Homan +3
Grammar and Cognition Lab, Department of Translation & Language Sciences, Universitat Pompeu Fabra, Barcelona, Spain. · Department of Adult Psychiatry and Psychotherapy, University of Zurich, Zurich, Switzerland. · Neuroscience Center Zurich, University of Zurich and ETH Zurich, Zurich, Switzerland. +3
Brain-LLM alignment is well established in English, yet the brain's language network is neuroanatomically universal across languages. Does alignment also generalize cross-linguistically, and what governs the variation? We test this using fMRI data from 112 participants across English, Chinese, and French (the Le Petit Prince corpus) and seven LLMs spanning English-dominant, Chinese-dominant, and multilingual architectures. Our central finding is that training-language dominance, not an inherent property of English, drives the alignment pattern: a Chinese-dominant model (Baichuan2-7B), architecture-matched to LLaMA-2-7B, reverses the gradient entirely, aligning best with Chinese brains and worst with English. Beyond training dominance, formal typological distance independently covaries with alignment degradation, syntax-associated brain regions (IFG) show 2.3× steeper typological gradients than lexico-semantic regions (PTL), and tokenization fertility accounts for ∼60% of a cross-linguistic shift in optimal encoding layer. These results reveal that the apparent "English advantage" in brain-LLM alignment is an artifact of training data composition, while the remaining variation reflects genuine typological structure concentrated in syntactic processing.
Dongxin Guo, Jikun Wu, Siu Ming Yiu
The University of Hong Kong Hong Kong, China · Stellaris AI Limited Hong Kong, China
Brain alignment of large language models (LLMs) offers a way to investigate representational similarities between LLMs and the human brain during language processing. While previous studies have mainly focused on passive language comprehension, little is known about such alignment during dialogue, in which speakers engage in both language production and comprehension. In this study, using fMRI data collected during dialogue, we examine brain alignment during language production and comprehension across LLM layers and brain regions. We assess its association with the intrinsic dimension (ID) of LLM representations and the contributions of several interpretable linguistic and communicative features. Our results reveal distinct patterns of brain alignment between production and comprehension, as well as strong associations with ID in brain regions associated with each processing mode. Furthermore, we show that the contributions of linguistic and communicative features also vary across processing modes, LLM layers, and brain regions.