SCRIBE: Scalable Contextual Representations of Interpretable Brain-Grounded Semantics

Camila Kolling; Omer Moussa; Emin Çelik; Mariya Toneva

Name	Type	Modality	Submodality	Description
Vision	Sensory	Vision	General	Something that you can easily see
Bright	Sensory	Vision	Intensity	Visually light or bright
Dark	Sensory	Vision	Intensity	Visually dark
Color	Sensory	Vision	Surface	Has a characteristic or defining color
Pattern	Sensory	Vision	Surface	Has a characteristic visual texture or surface pattern
Large	Sensory	Vision	Size	Large in size

Appendix

Table 3: Overview of the semantic properties from the feature space of Binder et al. (2016) , grouped by modality and cognitive domain. Each concept is represented as a vector over these interpretable semantic features. The table lists the

68

properties that are queried individually: the three bipolar properties Temperature , Texture , and Weight of the original

65

-dimensional space are each split into their two poles (Hot/Cold, Smooth/Rough, Light/Heavy), and the two resulting scores are recombined by taking their maximum, yielding the final

65

properties (Appendix B.2.1 ).

Figure 4: Distribution of semantic feature scores derived from human ratings in the Binder et al. (2016) dataset (composed of

535

words), shown for each of the

65

semantic properties.

Component	Variant	Modification
Baseline	Base prompt	Prompt used throughout the main experiments (Appendix A.2.1 ).
Role	Remove expert role	Remove the instruction: “You are an expert in rating to what degree words are associated with semantic properties.”
Task	Property name only	Provide only the property name instead of its relation and definition, e.g., “associated with Motion?”
Task	Property name + definition	Provide the property name followed by its relation and definition, e.g., “associated with Motion (showing a lot of visually observable movement)?”
Task	Relation: as	Replace “associated with” by “as”, e.g., “To what degree is the word {word} as {property relation} {property explanation}?”
Task	Relation: characterized by	Replace “associated with” by “characterized by”, e.g., “To what degree is the word {word} characterized by {property relation} {property explanation}?”

Appendix

Table 4: Prompt variants evaluated in the prompt ablation study. Each variant modifies a single component of the base prompt while leaving all other components unchanged.

Model	Ablation	MSE	Pearson Corr.
llama3.1-8b-instruct	Base	1.693	0.674
	No role	2.615	0.580
	Property name only	2.684	0.498
	Property name + definition	1.867	0.639
	Relation “as”	1.717	0.663
	Relation “characterized by”	1.700	0.659

Appendix

Table 5: Prompt ablation results across individual models and their ensemble on the

535

words of Binder et al. (2016) . Pearson correlation and MSE are computed per word across the

65

properties and averaged over words. The base prompt is the main prompt; its values differ slightly from Table 1 because of non-deterministic GPU inference across runs.

Example	Context size	Context
1	$k=3$	of those people
	$k=5$	I’m one of those people
	$k=10$	raised with a religion and I’m one of those people
	$k=50$	so a lot of people are raised with a religion and I’m one of those people
2	$k=3$	the word pretty
	$k=5$	extension of the word pretty

Appendix

Table 6: Examples of the context provided to SCRIBE for different context window sizes. The target word is shown in bold.

Figure 5: Distribution of (concept, property) pairs across discretized baseline bins under two binning strategies. (a) Quantile-based thresholds (

0.34

,

1.5

) yield more uniform distributions across bins; however, the threshold for the high bin is low relative to the full rating scale (

0

–

6

), leading to less interpretable groupings. (b) Manual thresholds aligned with the original rating scale (

[0,2)

,

[2,4)

,

[4,6]

) produce more interpretable assignments of low, mid, and high association levels across properties. These observations motivate the use of fixed thresholds in our synthetic dataset construction.

Property	Concept	Low-context sentence	High-context sentence
Angry	thunder	The thunderstorm was a beautiful display of nature.	The loud thunder made me furious.
Motion	bus	The bus was parked at the station.	The bus sped down the highway.
Social	dinner	I ate dinner alone at home.	We had dinner together with friends at the restaurant.
Pleasant	fireworks	The loud fireworks startled the baby.	Watching fireworks on the 4th of July was a joyful experience.
Fast	bus	The bus moved slowly through traffic.	The bus sped down the highway.
Motion	activist	The activist sat quietly in the corner.	The activist marched through the streets.

Appendix

Table 7: Examples of controlled contextual manipulations for selected property-concept pairs. For each pair, the low-context sentence is designed to reduce the target semantic property, while the high-context sentence increases it, demonstrating how context systematically modulates semantic feature annotations.

Region	Labels
Angular gyrus (AG)	PFm, PGs, PGi, TPOJ2, TPOJ3
Lateral temporal cortex (LTC)	STSda, STSva, STGa, TE1a, TE2a, TGv, TGd, A5, STSdp, STSvp, PSL, STV, TPOJ1
Inferior frontal gyrus (IFG)	44, 45, IFJa, IFSp
Middle frontal gyrus (MFG)	55b
Primary auditory cortex (A1)	A1
Early auditory regions	A1, PBelt, MBelt, LBelt, RI, A4

Appendix

Table 8: Brain regions and corresponding ROI labels.

Figure 6: Distribution of mean squared error (MSE) for out-of-context semantic annotation across models. For each model, we show the distribution of the MSE values computed on the true concept–property assignments (left) and under a permutation baseline obtained by randomly shuffling concept–property pairs (right). Across all models, the true MSE distribution is substantially lower than the permutation baseline, indicating that LLM-based annotations capture meaningful semantic structure beyond chance. Results are shown for individual models and the ensemble.

Figure 7: Distribution of Pearson correlation between LLM-generated semantic annotations and human ratings across models. For each model, we show the distribution of correlation values computed on the true concept–property assignments (left) and under a permutation baseline obtained by randomly shuffling concept–property pairs (right). Across all models, the true correlation distribution is substantially higher than the permutation baseline, indicating that LLM-based annotations capture meaningful semantic structure beyond chance. Results are shown for individual models and the ensemble.

	Round 1		Round 2
	Version 1	Version 2	Version 1	Version 2
Survey responses	270	276	256	258
Completed and matched to Prolific	249	255	250	250
After removing duplicates	245	255	250	250
Failed attention check(s)	– †	– †	25	29
Mean response time $<7$ s	2	13	4	5

Appendix

Table 9: Participant flow per data-collection round and questionnaire version. Each version covers half of the concept–property pairs.

Annotator	Concept	Average scores			Wilcoxon $p$ -value (two-sided)			Wilcoxon $p$ -value (one-sided)
	Bin	$\mathbf{L_{T}}$	O	$\mathbf{H_{T}}$	$\mathbf{L_{T}}$ vs O	$\mathbf{H_{T}}$ vs O	$\mathbf{H_{T}}$ vs $\mathbf{L_{T}}$	$\mathbf{L_{T}<O}$	$\mathbf{H_{T}>O}$	$\mathbf{H_{T}>L_{T}}$
Human	All O	2.67	3.23	4.23	$<.001$	$<.001$	$<.001$	$<.001$	$<.001$	$<.001$
	L O	1.73	1.78	3.36	$0.666$	$<.001$	$<.001$	$0.333$	$<.001$	$<.001$
	M O	2.75	3.43	4.33	$<.001$	$<.001$	$<.001$	$<.001$	$<.001$	$<.001$
	H O	3.52	4.47	4.98	$<.001$	$<.001$	$<.001$	$<.001$	$<.001$	$<.001$
Ensemble	All O	1.58	2.18	3.54	$<.001$	$<.001$	$<.001$	$<.001$	$<.001$	$<.001$

Appendix

Table 10: Contextual modulation of semantic feature scores for the human-evaluated subset. We report mean scores for human judgments and the SCRIBE ensemble across the original (

O

), low-target (

L_{\mathrm{T}}

), and high-target (

H_{\mathrm{T}}

) conditions, together with two- and one-sided Wilcoxon signed-rank tests. Results are shown across all concept–property pairs (

\mathrm{All}_{\mathrm{O}}

) and stratified by original out-of-context association strength (

L_{\mathrm{O}}

,

M_{\mathrm{O}}

,

H_{\mathrm{O}}

). Human means are computed from all ratings of each item. For pairs with low baseline association (

L_{\mathrm{O}}

), low-target contexts do not further decrease human ratings, consistent with a floor effect.

Model	Average scores			Wilcoxon $p$ -value (two-sided)			Wilcoxon $p$ -value (one-sided)
	$\mathbf{L_{T}}$	O	$\mathbf{H_{T}}$	$\mathbf{L_{T}}$ vs O	$\mathbf{H_{T}}$ vs O	$\mathbf{H_{T}}$ vs $\mathbf{L_{T}}$	$\mathbf{L_{T}<O}$	$\mathbf{H_{T}>O}$	$\mathbf{H_{T}>L_{T}}$
Human	2.67	3.23	4.23	$<.001$	$<.001$	$<.001$	$<.001$	$<.001$	$<.001$
Llama3.1-70B	1.71	2.53	4.09	$<.001$	$<.001$	$<.001$	$<.001$	$<.001$	$<.001$
Llama3.1-8B	1.77	2.27	3.22	$<.001$	$<.001$	$<.001$	$<.001$	$<.001$	$<.001$
Mistral-7B	1.39	1.60	3.50	$0.026$	$<.001$	$<.001$	$0.013$	$<.001$	$<.001$
Qwen2.5-7B	1.43	2.32	3.36	$<.001$	$<.001$	$<.001$	$<.001$	$<.001$	$<.001$

Appendix

Table 11: Comparison of low-target (

L_{T}

), original/no-context (O), and high-target (

H_{T}

) scores assigned by human annotators, individual LLMs, and LLM ensembles to the evaluated subset of the synthetic dataset. O denotes each annotator’s own out-of-context score. All annotators exhibit the intended ordering

L_{T}<O<H_{T}

, with significant differences in all three expected directions (

L_{T}<O

,

H_{T}>O

, and

H_{T}>L_{T}

; all

p<.05

, one-sided Wilcoxon signed-rank tests over concept–property pairs). Two-sided tests are also reported for completeness. The ensemble excluding Llama3.1-8B provides a control excluding the model used to generate the synthetic sentences.

Condition	$\mathrm{ICC}(1,1)$	$\mathrm{ICC}(1,8)$ [95% CI]	Ceiling	$F$	Raters for 0.90
$O$	0.40	0.84 [0.82, 0.87]	0.92	6.39	14
$L_{\mathrm{T}}$	0.27	0.74 [0.70, 0.78]	0.86	3.91	25
$H_{\mathrm{T}}$	0.24	0.72 [0.66, 0.76]	0.85	3.55	29

Appendix

Table 12: Reliability of human judgments per condition (mean over

20

balanced subsamples of

n=8

ratings per item;

95\%

bootstrap CI over items). The last column reports the number of raters needed for a reliability of

0.90

(Spearman–Brown).

Figure 8: Reliability of the mean human rating as a function of the number of raters per item. Lines show the Spearman–Brown prediction from

\mathrm{ICC}(1,1)

(shaded:

\pm

SD over

20

subsamples). Markers show empirical estimates from random subsets of raters: circles,

\mathrm{ICC}(1,n)

computed on

n

rater columns; crosses, the correlation between the means of two disjoint groups of

n

raters (error bars: SD over subsamples). The dotted line marks

n=8

, the number of raters used in the main analyses; dashed grey lines mark reliabilities of

0.7

,

0.8

and

0.9

.

	$r$ /ceiling $\uparrow$			MSE $\downarrow$
Model	$O$	$L_{\mathrm{T}}$	$H_{\mathrm{T}}$	$O$	$L_{\mathrm{T}}$	$H_{\mathrm{T}}$
Llama3.1-70B	0.79	0.76	0.64	2.98	3.97	3.30
Llama3.1-8B	0.70	0.68	0.66	3.49	2.77	4.27
Mistral-7B	0.57	0.65	0.67	5.82	4.41	3.69
Qwen2.5-7B	0.76	0.70	0.63	2.35	3.41	2.59
Ensemble	0.80	0.81	0.75	2.59	2.64	2.31

Appendix

Table 13: Alignment of individual models and the ensemble with human ratings (mean over

20

balanced subsamples of

8

ratings per item). Ceilings:

0.92

(

O

),

0.86

(

L_{\mathrm{T}}

),

0.85

(

H_{\mathrm{T}}

).

	$O$	$L_{\mathrm{T}}$	$H_{\mathrm{T}}$
$m=1$ rater	0.54	0.39	0.37
$m=2$ raters	0.65	0.50	0.47
$m=3$ raters	0.70	0.56	0.52
$m=4$ raters	0.73	0.59	0.56
SCRIBE ensemble	0.68	0.62	0.56

Appendix

Table 14: Correlation with the mean of

4

held-out raters, for the mean of

m

other raters and for the SCRIBE ensemble (

50

random splits

\times

20

subsamples).

Compared	Cond.	MSE [95% CI]	Chance	Floor	Offset $\bar{y}-\bar{h}$	Offset term	Scale term	Corr. term
SCRIBE	$O$	2.59 [2.22, 2.98]	6.53	0.36	$-1.04$	1.09	0.07	1.43
SCRIBE	$L_{\mathrm{T}}$	2.64 [2.31, 3.00]	5.71	0.45	$-1.09$	1.19	0.11	1.35
SCRIBE	$H_{\mathrm{T}}$	2.31 [1.91, 2.74]	4.80	0.35	$-0.68$	0.47	0.41	1.43
Binder norms	$O$	1.45 [1.22, 1.69]	5.54	0.36	$-0.38$	0.15	0.07	1.24

Appendix

Table 15: MSE between SCRIBE (or the Binder norms) and the mean of

8

human ratings, with

95\%

bootstrap CIs, the permutation baseline, the MSE floor, and the decomposition into offset, scale and correlation terms.

Model	Concept	Average scores			Wilcoxon $p$ -value (two-sided)			Wilcoxon $p$ -value (one-sided)
	Bin	L T	O	H T	L T vs O	H T vs O	H T vs L T	L T $<$ O	H T $>$ O	H T $>$ L T
Llama3.1-8B	All O	1.86	2.16	3.04	$<.001$	$<.001$	$<.001$	$<.001$	$<.001$	$<.001$
	L O	0.67	0.62	1.39	$0.168$	$<.001$	$<.001$	$0.916$	$<.001$	$<.001$
	M O	1.86	2.06	3.08	$0.009$	$<.001$	$<.001$	$0.005$	$<.001$	$<.001$
	H O	3.06	3.81	4.64	$<.001$	$<.001$	$<.001$	$<.001$	$<.001$	$<.001$
Mistral-7B	All O	1.37	1.55	3.11	$<.001$	$<.001$	$<.001$	$<.001$	$<.001$	$<.001$

Appendix

Table 16: Per-model results for the controlled contextual manipulation experiment (see Section 4.2 and Appendix A.3 ) using all the contextual generated sentences. We report average semantic scores and Wilcoxon signed-rank tests for each individual model and their ensemble.

O

denotes each model’s own out-of-context score. For all models and the ensemble, semantic feature values shift in the intended direction (

H_{\mathrm{T}}>O>L_{\mathrm{T}}

) across all pairs and within the mid and high bins. For pairs with low baseline association (

L_{\mathrm{O}}

), llama3.1-8b and mistral-7b do not lower scores further in the low-target condition, mirroring the floor effect observed in human judgments (Table 10 ); the ensemble shows significant effects in all bins.

Word type	Context Size	S1	S2	S3	S5	S6	S7	S8	Avg $\pm$ STE
Nouns	out-of-context	0.1372	0.1122	0.1190	0.1066	0.0933	0.0896	0.0931	0.1073 $\pm$ 0.0065
	window/k3	0.1420	0.1237	0.1301	0.1217	0.0789	0.0881	0.0807	0.1093 $\pm$ 0.0098
	window/k5	0.1442	0.1202	0.1330	0.1129	0.1051	0.0873	0.0884	0.1130 $\pm$ 0.0081 ∗
	window/k10	0.1621	0.1362	0.1384	0.1200	0.1101	0.0869	0.0858	0.1199 $\pm$ 0.0106 ∗
	window/k50	0.1760	0.1485	0.1207	0.1381	0.1116	0.0831	0.0928	0.1244 $\pm$ 0.0123
	up to concept	0.1566	0.1191	0.1001	0.1048	0.0732	0.0777	0.0681	0.1000 $\pm$ 0.0118

Appendix

Table 17: Brain encoding correlations across context sizes using llama3.1-8b-instruct . We compare out-of-context annotations, fixed-size context windows (

k\in\{3,5,10,50\}

), and an up to concept condition including all preceding stimulus context. Results are reported for three POS subsets: Nouns , Nouns, Verbs, Adjectives , and All Semantic POS . Values correspond to noise-ceiling-normalized correlations. Bold indicates the best context within each POS group, and underlining indicates the overall best configuration.

\ast

denotes significant improvements over the out-of-context baseline (one-sided Wilcoxon signed-rank test,

p<0.05

).

Word type	Context Size	S1	S2	S3	S5	S6	S7	S8	Avg $\pm$ STE
Nouns	out-of-context	0.1368	0.0917	0.1012	0.0848	0.0821	0.0871	0.0693	0.0933 $\pm$ 0.0081
	window/k3	0.1449	0.1248	0.1000	0.0938	0.0445	0.0939	0.0721	0.0963 $\pm$ 0.0124
	window/k5	0.1506	0.1337	0.1156	0.0975	0.0794	0.1048	0.0898	0.1102 $\pm$ 0.0095 ∗
	window/k10	0.1729	0.1521	0.1340	0.1300	0.0897	0.1103	0.0944	0.1262 $\pm$ 0.0115 ∗
	window/k50	0.1622	0.1441	0.1248	0.1206	0.0871	0.0975	0.0880	0.1177 $\pm$ 0.0109 ∗
	up to concept	0.1318	0.1171	0.1016	0.1051	0.0790	0.0832	0.0720	0.0985 $\pm$ 0.0082

Appendix

Table 18: Brain encoding correlations across context sizes using mistral-7b-instruct . We compare out-of-context annotations, fixed-size context windows (

k\in\{3,5,10,50\}

), and an up to concept condition including all preceding stimulus context. Results are reported for three POS subsets: Nouns , Nouns, Verbs, Adjectives , and All Semantic POS . Values correspond to noise-ceiling-normalized correlations. Bold indicates the best context within each POS group, and underlining indicates the overall best configuration.

\ast

denotes significant improvements over the out-of-context baseline (one-sided Wilcoxon signed-rank test,

p<0.05

).

Word type	Context Size	S1	S2	S3	S5	S6	S7	S8	Avg $\pm$ STE
Nouns	out-of-context	0.1420	0.1083	0.1209	0.1181	0.0919	0.0883	0.0811	0.1072 $\pm$ 0.0082
	window/k3	0.1537	0.1455	0.1360	0.1236	0.0880	0.0857	0.0906	0.1176 $\pm$ 0.0110 ∗
	window/k5	0.1618	0.1482	0.1321	0.1304	0.1061	0.0927	0.0917	0.1233 $\pm$ 0.0103 ∗
	window/k10	0.1809	0.1581	0.1449	0.1396	0.1097	0.0958	0.1036	0.1332 $\pm$ 0.0119 ∗
	window/k50	0.1823	0.1363	0.1358	0.1358	0.0987	0.0878	0.0883	0.1236 $\pm$ 0.0130 ∗
	up to concept	0.1606	0.1162	0.1415	0.1211	0.0917	0.0856	0.0876	0.1149 $\pm$ 0.0109 ∗

Appendix

Table 19: Brain encoding correlations across context sizes using qwen2.5-7b-instruct . We compare out-of-context annotations, fixed-size context windows (

k\in\{3,5,10,50\}

), and an up to concept condition including all preceding stimulus context. Results are reported for three POS subsets: Nouns , Nouns, Verbs, Adjectives , and All Semantic POS . Values correspond to noise-ceiling-normalized correlations. Bold indicates the best context within each POS group, and underlining indicates the overall best configuration.

\ast

denotes significant improvements over the out-of-context baseline (one-sided Wilcoxon signed-rank test,

p<0.05

).

Word type	Context Size	S1	S2	S3	S5	S6	S7	S8	Avg $\pm$ STE
Nouns	out-of-context	0.1347	0.1239	0.1135	0.1104	0.0942	0.0943	0.0872	0.1083 $\pm$ 0.0066
	window/k3	0.1454	0.1430	0.1136	0.1143	0.0832	0.0971	0.0838	0.1115 $\pm$ 0.0097
	window/k5	0.1502	0.1382	0.1241	0.1247	0.0791	0.1026	0.0926	0.1160 $\pm$ 0.0096
	window/k10	0.1711	0.1519	0.1295	0.1300	0.0927	0.1094	0.0961	0.1258 $\pm$ 0.0109 ∗
	window/k50	0.1789	0.1372	0.1069	0.1370	0.0686	0.0948	0.0947	0.1169 $\pm$ 0.0139
	up to concept	0.1509	0.1383	0.1080	0.1200	0.0524	0.0905	0.1028	0.1090 $\pm$ 0.0123

Appendix

Table 20: Brain encoding correlations across context sizes using llama3.1-70b-instruct . We compare out-of-context annotations, fixed-size context windows (

k\in\{3,5,10,50\}

), and an up to concept condition including all preceding stimulus context. Results are reported for three POS subsets: Nouns , Nouns, Verbs, Adjectives , and All Semantic POS . Values correspond to noise-ceiling-normalized correlations. Bold indicates the best context within each POS group, and underlining indicates the overall best configuration.

\ast

denotes significant improvements over the out-of-context baseline (one-sided Wilcoxon signed-rank test,

p<0.05

).

Figure 9: Voxel-wise differences in brain-prediction accuracy between contextual and out-of-context SCRIBE representations (

\mathrm{SCRIBE}_{C}-\mathrm{SCRIBE}_{O}

) across subjects. Red regions indicate greater prediction accuracy for

\mathrm{SCRIBE}_{C}

, whereas blue regions indicate greater accuracy for

\mathrm{SCRIBE}_{O}

. Contextualization-related improvements are predominantly observed in language-related cortical regions, with stronger effects in the right hemisphere.

Figure 10: Unique variance explained by

\mathrm{SCRIBE}_{C}

and QAEmb across cortical regions. Bars show the mean unique explained variance across subjects, with error bars indicating the standard error of the mean; individual dots represent subjects.

p

-values correspond to one-sided paired Wilcoxon signed-rank tests testing whether the unique variance explained by

\mathrm{SCRIBE}_{C}

is greater than that explained by QAEmb. Asterisks indicate significant differences (

p<0.05

).

Region	Method	S1	S2	S3	S5	S6	S7	S8	Avg $\pm$ STE
Whole Brain	$\mathrm{SCRIBE}_{\mathrm{C}}$	0.2284	0.2263	0.1714	0.2023	0.1800	0.1519	0.1258	0.1837 $\pm$ 0.0144
	QAEmb	0.2343	0.2115	0.1626	0.1788	0.1554	0.1544	0.1477	0.1778 $\pm$ 0.0125
	QAEmb + $\mathrm{SCRIBE}_{\mathrm{C}}$	0.2579	0.2426	0.1895	0.2140	0.1926	0.1684	0.1504	0.2022 $\pm$ 0.0146 ∗
Late Language	$\mathrm{SCRIBE}_{\mathrm{C}}$	0.1673	0.1990	0.2112	0.1348	0.1371	0.1191	0.0536	0.1460 $\pm$ 0.0201
	QAEmb	0.1715	0.1943	0.1932	0.1136	0.1214	0.1153	0.0615	0.1387 $\pm$ 0.0186
	QAEmb + $\mathrm{SCRIBE}_{\mathrm{C}}$	0.1880	0.2145	0.2253	0.1395	0.1450	0.1285	0.0632	0.1577 $\pm$ 0.0213 ∗

Appendix

Table 21: Noise-ceiling-normalized brain encoding correlations for SCRIBE (

\mathrm{SCRIBE}_{C}

), QAEmb, and their combined representation, evaluated across the whole brain and semantic language ROIs. Values are reported for individual subjects and as mean

\pm

standard error across subjects.

\ast

denotes that QAEmb +

\mathrm{SCRIBE}_{\mathrm{C}}

significantly outperforms both individual representations (one-sided Wilcoxon signed-rank tests,

p<0.05

).

Figure 11: Voxel-wise unique variance explained by SCRIBE and QAEmb across subjects. Variance partitioning was performed directly between the two representations, without including the syntactic control features. Red regions indicate greater unique variance explained by SCRIBE, whereas blue regions indicate greater unique variance explained by QAEmb. Only voxels with a noise ceiling above

0.4

are shown. Across subjects, the maps predominantly show greater unique variance for SCRIBE, particularly in language-related cortical areas, although this pattern is less pronounced in Subject 8.

Region	Method	S1	S2	S3	S5	S6	S7	S8	Avg $\pm$ STE
Whole Brain	$\mathrm{SCRIBE}_{\mathrm{C}}$ (65)	0.2284	0.2263	0.1714	0.2023	0.1800	0.1519	0.1258	0.1837 $\pm$ 0.0144
Whole Brain	$\mathrm{SCRIBE}_{\mathrm{C}}$ (33)	0.2275	0.2101	0.1848	0.2046	0.1897	0.1504	0.1340	0.1859 $\pm$ 0.0126
Late Language	$\mathrm{SCRIBE}_{\mathrm{C}}$ (65)	0.1673	0.1990	0.2112	0.1348	0.1371	0.1191	0.0536	0.1460 $\pm$ 0.0201
Late Language	$\mathrm{SCRIBE}_{\mathrm{C}}$ (33)	0.1660	0.1897	0.2124	0.1351	0.1396	0.1178	0.0561	0.1452 $\pm$ 0.0197
LTC	$\mathrm{SCRIBE}_{\mathrm{C}}$ (65)	0.1991	0.2046	0.1770	0.1609	0.1819	0.1155	0.0377	0.1538 $\pm$ 0.0223
LTC	$\mathrm{SCRIBE}_{\mathrm{C}}$ (33)	0.1938	0.1974	0.1762	0.1615	0.1814	0.1146	0.0402	0.1522 $\pm$ 0.0216

Appendix

Table 22: Noise-ceiling-normalized brain encoding correlations for

\mathrm{SCRIBE}_{\mathrm{C}}

before and after feature selection, evaluated across the whole brain and semantic language ROIs. Feature selection reduces the representation from 65 to 33 semantic dimensions. Values are reported for individual subjects and as mean

\pm

standard error across subjects. At the whole-brain level, performance does not significantly differ after feature selection (two-sided Wilcoxon signed-rank test).

Figure 12: Variance partitioning of

\mathrm{SCRIBE}_{C}

and QAEmb with syntactic features across language-related ROIs. (a) Brain-predictive variance shared with the syntactic feature space. (b) Unique brain-predictive variance retained by each representation after accounting for the syntactic feature space. Bars show means across subjects and error bars indicate the standard error of the mean. Bold ROI labels and asterisks indicate significant paired differences between

\mathrm{SCRIBE}_{C}

and QAEmb (one-sided paired Wilcoxon signed-rank test,

p<0.05

).

Figure 13: Permutation-based significance testing of the unique neural variance explained by

\mathrm{SCRIBE}_{\mathrm{C}}

after accounting for syntactic features. Histograms show permutation-based null distributions, and dashed lines indicate the observed unique variance.

Figure 14: Brain-encoding performance of Llama2-7b,

\mathrm{QAEmb}

, and

\mathrm{SCRIBE}_{C}

. (a) Whole-brain encoding performance. (b) Encoding performance across the Late Language, lateral temporal cortex (LTC), IFG/MFG, and Angular Gyrus ROIs. Bars show mean noise-ceiling-normalized correlation across subjects, individual points represent subjects, and error bars indicate the standard error of the mean. Asterisks indicate significant differences between

\mathrm{SCRIBE}_{C}

and the corresponding baseline (

p<0.05

; one-sided Wilcoxon signed-rank test).

Figure 15: Shared brain-predictive variance with syntactic features for contextual Llama2-7B and SCRIBE (

\mathrm{SCRIBE}_{\mathrm{C}}

). Bars show the mean shared variance across subjects and error bars indicate the standard error of the mean. SCRIBE shares significantly less variance with the syntactic feature space than Llama2-7B in IFG/MFG (one-sided paired Wilcoxon signed-rank test,

p<0.05

), indicated by the bold ROI label and asterisk.

Figure 16: Permutation-based significance testing of the unique neural variance explained by

\mathrm{SCRIBE}_{\mathrm{C}}

in variance partitioning with Llama2-7b. Histograms show permutation-based null distributions, and dashed lines indicate the observed unique variance.

Model	Pearson $\uparrow$	$p_{r}$	MSE $\downarrow$	$p_{\text{MSE}}$
Mistral 7B Instruct	0.61	$<0.001$	2.16	$<0.001$
Qwen 2.5 7B Instruct	0.69	$<0.001$	1.32	$<0.001$
LLaMA 3.1 8B Instruct	0.67	$<0.001$	1.70	$<0.001$
LLaMA 3.1 70B Instruct	0.74	$<0.001$	1.53	$<0.001$
Ensemble	0.77	$<0.001$	1.07	$<0.001$

SCRIBE: Scalable Contextual Representations of Interpretable Brain-Grounded Semantics

Abstract

Figures & tables

Appendix

Distributional Semantics

Decomposing Brain-LLM Alignment with Interpretable Semantic Representations

Contextual Semantic Relevance Tracks fMRI BOLD Responses During Naturalistic Speech Comprehension

Brain Score Tracks Shared Properties of Languages: Evidence from Many Natural Languages and Structured Sequences

(a) Contextual modulation
Annotator	$\mathbf{L_{T}}$	O	$\mathbf{H_{T}}$	$\mathbf{L_{T}<O}$	$\mathbf{H_{T}>O}$	$\mathbf{H_{T}>L_{T}}$
Human	2.67	3.23	4.23	$<.001$	$<.001$	$<.001$
Ensemble	1.58	2.18	3.54	$<.001$	$<.001$	$<.001$

Abstract

Distributional Semantics

Explore similar work

Decomposing Brain-LLM Alignment with Interpretable Semantic Representations

Contextual Semantic Relevance Tracks fMRI BOLD Responses During Naturalistic Speech Comprehension

Brain Score Tracks Shared Properties of Languages: Evidence from Many Natural Languages and Structured Sequences