Entanglement as Memory: Mechanistic Interpretability of Quantum Language Models

Nathan Roll

quant-phMar 27, 2026

Entanglement as Memory: Mechanistic Interpretability of Quantum Language Models

Authors: Nathan Roll

Organizations: Department of Linguistics, Stanford University

Abstract

Quantum language models have shown competitive performance on sequential tasks, yet whether trained quantum circuits exploit genuinely quantum resources -- or merely embed classical computation in quantum hardware -- remains unknown. Prior work has evaluated these models through endpoint metrics alone, without examining the memory strategies they actually learn internally. We introduce the first mechanistic interpretability study of quantum language models, combining causal gate ablation, entanglement tracking, and density-matrix interchange interventions on a controlled long-range dependency task. We find that single-qubit models are exactly classically simulable and converge to the same geometric strategy as matched classical baselines, while two-qubit models with entangling gates learn a representationally distinct strategy that encodes context in inter-qubit entanglement -- confirmed by three independent causal tests (p < 0.0001, d = 0.89). On real quantum hardware, only the classical geometric strategy survives device noise; the entanglement strategy degrades to chance. These findings open mechanistic interpretability as a tool for the science of quantum language models and reveal a noise-expressivity tradeoff governing which learned strategies survive deployment.

Figures & tables

Figure 1: Hemisphere separation distinguishes successful from failing quantum memory. Bloch sphere trajectories for 10-distractor sequences. Left: Shared-parameter model (MinimalQLM). Contexts A and B overlap after distractor tokens, causing misclassification (76% accuracy). Right: Decoupled model (DecoupledQLM). Contexts remain in opposite hemispheres throughout, achieving 100% generalization. A classical SO(3) baseline produces identical trajectories, confirming this mechanism is not inherently quantum.

Token	Encoding ( $x$ )	Role
$A$	$+\pi/3$	Context $\to$ predict 0
$B$	$-\pi/3$	Context $\to$ predict 1
$D$	$0.0$	Distractor

Table 1: Parity-Switch Grammar token encodings. The task reduces long-range dependency to its minimal form: maintain context identity across arbitrarily many distractors.

Model	Type	Params	Distinguishing property
MinimalQLM	Quantum (1Q)	3	Shared context/distractor gate
DecoupledQLM	Quantum (1Q)	6	Separate context/distractor gates
TwoQubitQLM	Quantum (2Q)	24	CNOT entangling gate
ClassicalSO(3)	Classical	3/6	SO(3) rotation matrices
ClassicalRNN4	Classical	24	tanh-RNN, 4D hidden state

Table 2: Model zoo spanning the classical–quantum boundary. Parameter counts are exactly matched between quantum and classical counterparts to control for model capacity.

Model	Sim. Acc.	HW Acc.	$\Delta_{z}$	Mechanism
MinimalQLM (1Q shared)	76%	–	0.36	Periodic recurrence
DecoupledQLM (1Q)	100%	100%	1.36	Z-preservation
ClassicalSO(3) (matched)	100%	–	1.36	Z-preservation
2Q + CNOT (30 seeds)	96.3 $\pm$ 9.1%	39 $\pm$ 7%	–	Entanglement
2Q + SWAP (10 seeds)	97.1 $\pm$ 5.7%	–	–	Z-preservation
2Q no CNOT (10 seeds)	100 $\pm$ 0%	41 $\pm$ 5%	–	Z-preservation

Table 3: Entanglement-based memory emerges exclusively with CNOT gates; the task is trivially classical. HW = IBM hardware (ibm_fez / ibm_marrakesh, 156 qubits, 1000 shots/circuit).

\Delta_{z}

= Bloch Z-coordinate separation (max 2.0). 2Q+CNOT results: 30 seeds

\times

3 restarts. The classical RNN solves the task with Adam but fails under SPSA, confirming that the SPSA comparison is optimizer-confounded. The SWAP control (non-entangling inter-qubit coupling) achieves high accuracy via Z-preservation, isolating entanglement as the resource enabling the distinct strategy.

Figure 2: Z-coordinate preservation is robust across 100 distractors. Bloch-sphere Z-coordinate for Context A and Context B across 0–100 distractors (DecoupledQLM). Hemisphere separation grows slightly with length because distractor rotations push states toward the poles. The shared model loses separation after $\sim$ 5 distractors.

Figure 3: Entanglement entropy diverges by context, providing a representationally distinct memory channel. Von Neumann entanglement entropy $S(\rho_{0})$ per timestep for the trained two-qubit model with CNOT gates on 10-distractor sequences. Context A and B traces diverge after the context token (timestep 0) and maintain distinct entanglement levels across all distractors. Ablating the CNOT eliminates this divergence, collapsing both traces to zero entropy; the model then reverts to Z-preservation.

Context	Distractors	$P(\|1\rangle)$	Correct?
A	0	0.24	Yes
B	0	0.98	Yes
A	5	0.06	Yes
B	5	0.89	Yes
A	10	0.25	Yes
B	10	0.98	Yes

Table 4: The geometric strategy executes perfectly on quantum hardware. Single-qubit hardware results on ibm_fez (1000 shots per circuit). All predictions are correct with wide margins from the 0.5 decision boundary, confirming that the Z-preservation mechanism is fully noise-robust at current device fidelities.

Figure 4: A sharp phase transition separates viable from collapsed geometric memory. Distractor rotation $\theta_{2}^{\mathrm{dist}}$ sweeps from 0 to $\pi$ (10 distractors). (a) Classification accuracy: robust up to $\theta_{2}^{\mathrm{dist}}\approx 0.83$ rad, then drops to chance; recovery at $\pi$ is period-2 recurrence. (b) Z-coordinate variance across timesteps; lower variance indicates more stable memory. (c) Heatmap of Bloch Z-coordinate per timestep (columns) at each $\theta_{2}^{\mathrm{dist}}$ (rows). Blue = northern hemisphere, red = southern.

Model / Mechanism	$p=0$	$p=0.06$	$p=0.12$
1Q Decoupled / Z-Preservation	100%	100%	50%
2Q no CNOT / Z-Preservation	100%	100%	100%
2Q + CNOT / Entanglement	100%	50%	50%

Table 5: Entanglement-based memory is the least noise-robust strategy. Simulated depolarizing noise at rate

p

per gate. Entanglement degrades to chance at

p=0.06

; Z-preservation strategies survive until

p=0.12

or beyond.

Long-Range Dependency Modeling

See all 27 papers

Trainable Quantum-Native Language Models: Dissipative Memory, Capacity Limits, and Hardware Validation
Oct 8, 2026Deniz Yunus GöğüşLanguage Modeling Quantum Machine Learning
Watching Quantum Models Think: Hilbert-Space Interpretability in Quantum Transformer Blocks
Sep 19, 2026Diego Iacopetta, Andrea GaspariniQuantum Neural Networks Transformer Interpretability
Query Machines B.V. Delft, The Netherlands
Separating quantum circuits from classical LLMs
Aug 4, 2026Srinivasan Arunachalam, Arkopal Dutt, Hari Krovi +1Decoder-Only Language Models Diffusion Language Models
IBM Research

Oct 8, 2026J/K move · Enter open · S save

Trainable Quantum-Native Language Models: Dissipative Memory, Capacity Limits, and Hardware Validation

Deniz Yunus Göğüş

Language Modeling Quantum Machine Learning Byte-Level Language Model Memory-Augmented Language Models

Open paper AskPDF arXiv

We study byte-level autoregressive language models whose hidden state is a small quantum register, whose transitions are learned token-conditioned quantum channels, and whose next-byte probabilities come directly from Born-rule measurements without a learned classical vocabulary projection or softmax. In the fixed-width recurrent setting, unitary dynamics cannot contract or reset stored distinctions; experimentally, unitary recurrence loses to its retrained one-token control in every tested variant. Introducing learned dissipation, then token-conditioned dissipative gating, yields useful memory beyond one token. The gated model improves test negative log-likelihood by 0.20 +/- 0.03 nats/byte over retrained parameter-matched one-token controls at context length 12, and by 0.29-0.33 at longer contexts, but never beats retrained two-token controls. A key-value recall benchmark shows a width-dependent threshold: one stored pair meets the pre-registered accuracy bar at 4-5 memory qubits, two pairs at 6-7, and three pairs remain unsolved. This is consistent with, but does not establish, a capacity explanation for the short effective memory horizon. As a secondary result, one-qubit models reach lower stationary memory entropy than a heuristically optimized classical predictive frontier at matched error in 9/9 seeds. On IBM Heron hardware, computational-basis memory-register distributions show signal against a uniform null in 12/12 circuits across prefixes of up to 12 tokens, next-bit probabilities pass pre-registered accuracy bars at four-token prefixes, and bytes are generated autoregressively by sampling physically measured bit probabilities. The tested models are classically simulable and substantially worse than classical n-gram baselines. We claim no computational quantum advantage.

Abstract

Long-Range Dependency Modeling

Explore similar work

Trainable Quantum-Native Language Models: Dissipative Memory, Capacity Limits, and Hardware Validation

Watching Quantum Models Think: Hilbert-Space Interpretability in Quantum Transformer Blocks

Separating quantum circuits from classical LLMs