The production of meaning in the processing of natural language
Authors: Christopher J. Agostino, Quan Le Thien, Nayan D'Souza, Louis van der Elst
Organizations: NPC Worldwide, Bloomington, IN, USA · Department of Physics, Indiana University, Bloomington, IN, USA · Department of Linguistics, Indiana University, Bloomington, IN, USA · Imperial College London, London, UK
Understanding the fundamental mechanisms governing the production of meaning in the processing of natural language is critical for designing safe, thoughtful, engaging, and empowering human-agent interactions. If meaning is constituted rather than retrieved, then the search for context-independent features or circuits in the pursuit of mechanistic interpretability may be fundamentally limited. Experiments in cognitive science and social psychology have demonstrated that human semantic processing exhibits contextuality more consistent with quantum logical mechanisms than classical Boolean theories, and recent works have found similar results in large language models---in particular, clear violations of the Bell inequality in experiments of contextuality during interpretation of ambiguous expressions. In this work, we explore the CHSH ∣S∣ parameter---the metric associated with the inequality---across the inference parameter space of models spanning four orders of magnitude in scale and cross-reference our findings with MMLU, hallucination rate, and nonsense detection benchmarks. We find that the interquartile range of the ∣S∣ distribution is completely orthogonal to all external benchmarks, while overall violation rate shows weak anticorrelation with all three benchmarks. We investigate how ∣S∣ varies with sampling parameters and word order, and discuss the information-theoretic constraints that genuine contextuality imposes on prompt injection defenses and its human analogue, whereby careful construction and maintenance of social contextuality can be carried out at scale, shaping the space of possible interpretations before any particular one is reached. We consider the implications for mechanistic interpretability and how genuine contextuality sets an information-theoretic bound on the decomposability of semantic processing.
Figures & tables
Figure 1: Per-grid-point S distributions for each model. Pink bands: IQR. Vertical lines: mean (red), median (black), classical bound ∣S∣=2 (dashed blue).
Figure 2: S distribution statistics versus external benchmarks. Left: IQR of S . Right: violation rate. Rows: MMLU, hallucination rate [ 31 ] , BullshitBench pushback rate [ 11 ] . Red lines: linear trends; green bands: ±2σ . The IQR—the most discriminating statistic—shows no relationship to any benchmark.
Model
n
σ
γ1
κ
IQR
Viol.%
Claude Haiku 4.5 †
10
0.43
− 1.27
+ 0.27
0.35
10.0
Claude Sonnet 4.6 †
10
0.62
− 0.97
+ 1.49
0.55
40.0
Cogito 3B
18
0.47
− 0.12
+ 0.47
0.40
55.6
DeepSeek-V3
155
0.37
+ 0.68
+ 4.99
0.40
40.0
DeepSeek-V4 Flash
30
0.36
+ 0.28
+ 3.13
0.20
23.3
Gemini 2.5 Flash
119
0.37
− 1.32
+ 3.80
0.29
29.4
Table 1: Distribution statistics of per-grid-point S values for each model (25 models with sufficient data). Each grid point is a unique (word pair, word order, τ , p , k ) configuration. All models mode near the classical bound ( S≈2.0 ) but differ in spread ( σ ), skewness ( γ1 ), and excess kurtosis ( κ ). IQR = interquartile range. Viol.% = fraction of grid points with ∣S∣>2 . † Tested at default parameters only (no grid sweep) due to API restrictions.
Figure 3: Word-order effects on individual interpretation choices. Each point plots P(meaning A) for one word under one interpretive lens and parameter configuration, comparing original versus flipped word order. Points on the diagonal indicate order-invariant interpretation; off-diagonal points indicate non-commutative dependence on presentation order. Marginal densities (top, right) show the bimodal character of interpretation probabilities. Bottom: distribution of ∣ΔP(A)∣ across all paired conditions.
Model
Word pair
Sorig
Sflip
ΔS
(Populated from figures/agg_commutativity.csv )
Table 2: Per-model, per-word-pair commutativity analysis. ΔS=Sorig−Sflip measures the effect of reversing word order on the CHSH parameter.
Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.
We reply to the comments by M. Sienicki and K. Sienicki (arXiv:2512.07881) and by K. Sienicki (arXiv:2601.06104) on our work on quantum-mechanical statistics in human language (arXiv:2407.14924) and on quantum structure in AI-generated language (arXiv:2511.21731). We thank the authors for their careful reading and address what we consider to be the main points of criticism: the exploratory nature of the protocol used in the experiments with large language models; the role of marginal-law violations, and of the Contextuality-by-Default criterion, in the identification of entanglement; the limited diagnostic value of a Bose-Einstein fit taken in isolation; the meaning of assigning the lowest energy levels to the most frequent words; and the relation between the vector spaces used by LLMs and quantum state spaces. We also correct a typographical error in Table 3 of arXiv:2511.21731, which does not affect the reported CHSH value.
Massimiliano Sassoli de Bianchi, Roberto Leporini
Center Leo Apostel for Interdisciplinary Studies, Vrije Universiteit Brussel (VUB), Pleinlaan 2, 1050 Brussels, Belgium · Department of Economics, University of Bergamo, via dei Caniana 2, Bergamo, 24127, Italy
Understanding how the human brain constructs meaning from real-world language remains a fundamental challenge, as semantics depend on context that varies in natural settings. To enable the study of real-world linguistic meaning in the brain, we introduce SCRIBE, a framework grounded in a neuroscience-informed semantic space that leverages an ensemble of open-source large language models to generate interpretable, word-level semantic representations of arbitrary text. In a behavioral study with 921 participants, we show that human semantic feature judgments vary systematically with context, and that SCRIBE captures these shifts, reaching 75-81% of the estimated human noise ceiling. Using brain activity recorded while participants listen to naturalistic stories, we further demonstrate that SCRIBE predicts neural responses in language-selective brain regions, significantly outperforming non-contextual semantic representations. Variance partitioning reveals that SCRIBE explains unique brain-predictive information beyond syntactic features, suggesting that its representations capture aspects of linguistic meaning not accounted for by the measured syntactic features. Together, these findings establish SCRIBE as a scalable and interpretable foundation for studying natural language semantics in the brain.
Camila Kolling, Omer Moussa, Emin Çelik +1
Max Planck Institute for Software Systems (MPI-SWS)
Quantum language models have shown competitive performance on sequential tasks, yet whether trained quantum circuits exploit genuinely quantum resources -- or merely embed classical computation in quantum hardware -- remains unknown. Prior work has evaluated these models through endpoint metrics alone, without examining the memory strategies they actually learn internally. We introduce the first mechanistic interpretability study of quantum language models, combining causal gate ablation, entanglement tracking, and density-matrix interchange interventions on a controlled long-range dependency task. We find that single-qubit models are exactly classically simulable and converge to the same geometric strategy as matched classical baselines, while two-qubit models with entangling gates learn a representationally distinct strategy that encodes context in inter-qubit entanglement -- confirmed by three independent causal tests (p < 0.0001, d = 0.89). On real quantum hardware, only the classical geometric strategy survives device noise; the entanglement strategy degrades to chance. These findings open mechanistic interpretability as a tool for the science of quantum language models and reveal a noise-expressivity tradeoff governing which learned strategies survive deployment.