Reading Why: Born-Measured Attention with Exact Word Evidence on Light and Qubits

Teerawat Tongloy; Jiraporn Pongsopa; Rangsan Jomtarak

quant-phNov 3, 2026

Reading Why: Born-Measured Attention with Exact Word Evidence on Light and Qubits

Authors: Teerawat Tongloy, Jiraporn Pongsopa, Rangsan Jomtarak

Organizations: School of Integrated Innovative Technology, King Mongkut’s Institute of Technology Ladkrabang, Bangkok, Thailand · Faculty of Science and Technology, Suan Dusit University, Bangkok, Thailand

Abstract

When a language model answers, we can usually only estimate why. We build a transformer in which every step is a Born measurement - attention $|\langle q|k\rangle|^4$ normalised over each row, a measured product in place of the feed-forward nonlinearity, and a Born readout. With its denominators held at their values for the input, each class probability is a polynomial in the word amplitudes, so its integrated-gradient evidence is computed by a finite quadrature with no quadrature error: the words' contributions sum to the class probability above its no-information share, to $10^{-11}$ or better. On e-SNLI natural language inference, with about 98k parameters outside the embedding, the model reaches $76.98 \pm 0.17$ % test accuracy, level with a softmax transformer of the same size ( $76.62 \pm 0.36$ %). Deleting the word its evidence ranks first changes the answer in 45% of test pairs, out of the 53% any single deletion can change, against 32% for the softmax model explained the same way and 40% by its attention x gradient. Attention can look more like the words annotators highlighted, yet accounts for the answer less well: deleting the word it ranks first changes the Born Transformer's answer in 32% of pairs. The same weights compile exactly to optoelectronic optics and to qubits, and a coherent attention layer becomes one circuit equal to the model (852 CNOTs on 23 qubits). On IBM's ibm_kingston, reading only the attention overlaps by Hadamard tests, the device gave the model's answers on two short test pairs and kept or changed them under deletion as the model does; the evidence itself did not yet survive the device, and we work out the shots and calibration that measuring it on hardware would need.

Figures & tables

Figure 1: Overview, drawn from the trained model’s own data for the test pair “men play hockey . / aliens play” (contradiction). (a) On light: the modulator holds one field per word (phase as colour); in each of two blocks Born attention weighs the words by $|\langle q_{i}|k_{j}\rangle|^{4}$ read by interference, and the measured feed-forward reads $\mathrm{Re}[(Wh)\overline{(Vh)}]$ by homodyne detection; the camera’s three regions give the Born probabilities, and each word’s Born evidence (BEW; red for the answer, blue against it; dots: the annotators’ words) sums to $P_{g}-b_{g}$ exactly. (b) On IBM’s ibm_kingston, with the same weights: a word’s query, 16 complex numbers computed classically, is loaded as the amplitudes of four qubits, and every overlap is a Hadamard test (2 $\times$ 272 circuits); the measured overlaps follow the ideal ones ( $r=0.992$ ), the answer survives (contradiction, 0.988 against 0.983 exact), and deleting “aliens”, the word the evidence ranks first, turns the device’s answer to entailment.

Figure 2: Three models step by step, bottom to top, in the box colours of the original transformer figure: a transformer encoder [ 7 ] , the Born Transformer drawn as optics (a beam through a modulator, interferometers with balanced detectors, beam splitters, a lens and a camera; the same weights run on qubits) and the coherent attention layer drawn as its circuit, wires running upwards (one circuit, one measurement). Bands hold the same stage; detector symbols mark the measurements.

Figure 3: The task. One premise of the e-SNLI test set with its three hypotheses, one per label; highlighted are the words the first annotator marked, quoted beneath each is their written explanation. Right: test accuracy of simple baselines and of the two transformers of the same size (three seeds); a bag of words or the hypothesis alone reaches about 65 %, so the rest requires relating the sentences. Below: the dataset in numbers.

model	(%)	(%)	median / largest	(random)	(random)	with annotators
	test	dev	gap	AOPC	flip@1	agreement
Born Transformer	$76.98\pm 0.17$	77.30	$6{\times}10^{-13}$ / $8{\times}10^{-12}$	0.618 (0.125)	0.45 (0.06)	$0.510\pm 0.005$
Softmax	$76.62\pm 0.36$	76.36	$0.04$ / 47	0.321 (0.059)	0.31 (0.05)	$0.468\pm 0.003$
Poly-sim-born	$78.98\pm 0.31$	78.89	$10^{-14}$ / $5{\times}10^{-12}$	0.501 (0.065)	0.45 (0.06)	$0.527\pm 0.011$

Table 1: e-SNLI, 20 epochs, three seeds: accuracy and agreement as mean

\pm

sample standard deviation, the other columns as means (dev on seed means). Gap: how far the word evidence is from summing to the explained score, the worst seed’s median and the largest over the 1,000 pairs (the Born models’ gaps are in probability, the softmax model’s in units of its logit, which it explains by integrated gradients with 256 nodes, not exact for it). AOPC and flip@1 are measured on the predicted class, with a random order of deletion in brackets. Agreement: share of the evidence’s top-

k

words that annotators highlighted (chance 0.354).

model	ranking	flip@1	of ceiling	flip@5, lowest	agreement
Born Transformer	evidence (BEW)	0.45 [0.43, 0.48]	84 %	0.009	0.51 [0.50, 0.52]
	attention	0.32	59 %	0.077	0.50
	attention $\times$ gradient	0.37	70 %	0.064	0.53
	gradient $\times$ input	0.41	78 %	0.062	0.46
	leave-one-out	0.53 [0.51, 0.56]	100 %	0.006	0.49
Softmax	IG, logit, embeddings	0.31	56 %	0.103	0.47

Table 2: Every ranking on the same 1,000 test pairs, pooled over three seeds per pair; 95 % bootstrap intervals over pairs in brackets (the others are as wide,

\pm 0.02

for flip@1 and

\pm 0.01

for agreement; resampling the seeds as well widens them to about

\pm 0.03

and

\pm 0.015

, and of the comparisons in the text only Born evidence against Born attention on agreement,

+0.01

(

-0.02

+0.03

), is not resolved). flip@1: deleting the top word changes the answer; of ceiling: flip@1 over the share of pairs that some single deletion changes; flip@5, lowest: deleting the five ranked last changes it; agreement: share of the top-

k

that annotators highlighted (chance 0.354; on correctly answered pairs only, every value rises by up to 0.03 and the comparisons in the text hold). The softmax model’s gated integrated gradients use 4,096 nodes (median gap

5\times 10^{-8}

in the worst seed, largest 0.045). Attention rollout [ 13 ] gives agreement 0.52, 0.55 and 0.42.

attention	feed-forward	test (%)	gap (median)	flip@1	flip@5, lowest	agreement
$(q{\cdot}k)^{2}$ / row sum	bilinear	$78.98\pm 0.31$	$10^{-14}$	0.45	0.02	$0.53\pm 0.01$
$(q{\cdot}k)^{2}$ / row sum	GELU	$77.13\pm 1.42$	$4\times 10^{-3}$	0.27	0.11	$0.46\pm 0.02$
softmax	bilinear	$65.65\pm 0.44$	not complete	0.15	0.25	$0.37\pm 0.01$
softmax	GELU	$65.47\pm 0.25$	not complete	0.17	0.23	$0.39\pm 0.01$

Table 3: Poly-sim-born (first row) with its attention, its feed-forward or both replaced; same size, training and test pairs as Table 1 , three seeds. Evidence: exact quadrature for the polynomial model, integrated gradients with 256 nodes otherwise. flip@1: deleting the word ranked first changes the answer (random 0.05–0.06); flip@5, lowest: deleting the five ranked last does (random 0.17–0.20). Agreement: chance 0.354.

Figure 4: Born evidence per word (BEW) for four e-SNLI test pairs: the first right answer of each class in test order and the first wrong one. Each word is shaded by its evidence for the model’s answer (red for, blue against; scaled within the pair); dots mark the words annotators highlighted; bars give the Born probabilities. In the wrong case the evidence shows why: “calf” in both sentences pulls towards entailment, and “ropes”, the word that should decide, pushes against it but too weakly.

Figure 5: One test pair through the compiled optics, stage by stage: the modulator’s fields (phase as hue), each block’s attention and field after attention, the homodyne readings of the measured feed-forward and the field after it (with the share of the encoded light left), and the camera with its three class regions; below, the answer, the exact evidence of each word and the attention each word receives in the last block.

coherent attention layer	params	test (%)	CNOTs	qubits	post-selection
dense maps	82k	73.14	136,727	27	$10^{-9}$ – $10^{-7}$
brickwall readout	18k	72.41	17,452	25	$10^{-8}$ – $10^{-6}$
every map brickwall (3 seeds)	3.7k	$67.04\pm 0.12$	852	23	$10^{-5}$ – $10^{-4}$

Table 4: The coherent attention layer as one circuit (8 slots,

p=3

; 10 epochs). The circuit equals the trained model to

\leq 3.3\times 10^{-16}

(GPU statevector, two test pairs). Post-selection with the copies prepared apart, for the two pairs. The dense model reaches

74.67\pm 0.18\,\%

at 20 epochs over three seeds.

Figure 6: The coherent attention layer as one circuit, left to right: three copies of the sentence prepared, the query–key brickwall

U_{M}

, a Bell projection that leaves

c_{j,h}=\langle x_{j}|m_{h}\rangle

as an amplitude, the slots paired and summed, the value–readout brickwall

U

, and one measurement of seven qubits. 23 qubits, 852 CNOTs; the circuit equals the trained model.

run	word deleted, and why	answer	device	exact	95 % interval
pair 3763	–	contradiction	0.988	0.983	0.978–0.991
	“aliens”, evidence first	entailment	0.573	0.497	0.494–0.652
	“play”, evidence nearest zero	contradiction	0.985	0.986	0.976–0.987
pair 3765	–	entailment	0.587	0.596	0.501–0.668
	“hockey”, attention first	entailment	0.819	0.768	0.741–0.867
	“men”, evidence first	entailment	0.573	0.497	0.494–0.652

Table 5: The runs on ibm_kingston: the probability of the answer, on the device and in the exact model, with the 95 % shot-noise interval of the device’s value; in every redraw the device’s answer was the same. Pair 3763 is “men play hockey . / aliens play”, pair 3765 “men play hockey . / men play”. Pair 3765 without “men” has the same words as pair 3763 without “aliens”, so the ledger reused those jobs. Jobs, circuits and quantum seconds: Table 10 .

Figure 7: The real-device run. Left: the test pair, attention read by Hadamard tests on ibm_kingston (2,000 shots), against the exact model, with 95 % intervals from redrawing each circuit’s shots. Top middle: the same pair with “aliens” deleted, the word the exact evidence ranks first; the device’s answer turns to entailment, as the model’s. Top right: “play” deleted instead, the word the evidence ranks nearest zero; the answer stays. Bottom: a second test pair (entailment), without “hockey”, the word attention ranks first (entailment rises), and without the hypothesis’s “men”, the word the evidence ranks first (the same words, and jobs, as top middle). Only the attention overlaps ran on the device.

slots (words $+$ reference)	12–16	17–32	33–54
test pairs	28	224	48
qubits	25	27	29
$P_{\mathrm{pass}}$ , median	$1.6\times 10^{-5}$	$6.8\times 10^{-6}$	$2.8\times 10^{-6}$
runs per accepted shot, $1/P_{\mathrm{pass}}$	$6.3\times 10^{4}$	$1.5\times 10^{5}$	$3.5\times 10^{5}$
amplification rounds	200	300	470
CNOTs per accepted shot, amplified	$3\times 10^{5}$	$5\times 10^{5}$	$8\times 10^{5}$

Table 6: The coherent attention layer’s post-selection, Eq. ( 9 ), on 300 test pairs (the brickwall model, 852 CNOTs per pass). Amplification rounds:

\pi/(4\sqrt{P_{\mathrm{pass}}})

, two passes each.

shots per circuit	100	300	1,000	2,000	$10^{4}$	$10^{5}$
answer agrees	0.92	0.96	0.977	0.990	0.997	0.997
$\|\Delta P\|$ , median	0.054	0.029	0.018	0.013	0.006	0.002
evidence: correlation				0.992	0.999	0.9999
evidence: top-3				0.87	0.96	0.98
visibility spread per circuit	0	0.02	0.05	0.1	0.2
answer agrees ( $10^{4}$ shots)	0.997	0.990	0.963	0.923	0.873

Table 7: The Born Transformer on qubits, simulated: attention overlaps from Hadamard tests with shot noise (and, in the lower rows, a visibility that varies by circuit, or a row-sum calibration that is off), everything else exact; 300 test pairs (evidence: 100). Correlation and top-3 agreement are against the exact evidence.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

	Born Transformer	softmax	Poly-sim-born	coherent attention layer
state	complex, 32 modes	real, $d=64$	real, $d=64$	3 copies of 5 feature qubits, a slot register of $p=3$ qubits each, 2 head qubits
blocks, heads	2, two of 16 modes	2, two	2, two	1, four (a 2-qubit register)
attention	$\|\langle q\|k\rangle\|^{4}$ over its row sum	softmax	$(q\cdot k)^{2}$ over its row sum	Bell projection $\langle x_{j}\|m_{h}\rangle$ , unnormalised
feed-forward	$\mathrm{Re}[(W_{1}h)\overline{(W_{2}h)}]$ , 170 channels, gain, $W_{3}$	GELU, width 256	bilinear, width 170	none
normalisation	none	LayerNorm	none	none
positions	additive fields, a phase per position	learned, additive	learned, additive	additive fields, a phase per position, a field per sentence

Appendix

Table 8: The models. Parameters: total and, in brackets, outside the word embedding. The Born Transformer also has a reference slot with a learned gain; its embeddings and position fields start at scale 1.

data	e-SNLI: 549,367 training, 9,842 dev, 9,824 test pairs; premise <sep> hypothesis, at most 64 tokens; vocabulary of 22,319 (words seen at least 3 times)
loss	cross-entropy of $P_{g}$ (Born models), of the logits (softmax, Poly-sim-born)
optimiser	Adam, learning rate $2\times 10^{-3}$ (chosen on dev for the Born and softmax models, and used untuned for the others), cosine decay, gradient norm clipped at 1
schedule	batch 64; 20 epochs (coherent layer: 10); word dropout 0.1; seeds 0–2; the last epoch is reported
precision	float32 for training, float64 for evidence and compilation
evidence	integrated gradients from the zero baseline with the denominators held; Gauss–Legendre with half the degree in nodes (Born Transformer: degree 648, 324 nodes)

Appendix

Table 9: Training and evidence, the same for every model unless stated.

job	id	circuits	quantum s
pilot: first 30 circuits of block 1, pair 3763	dbkn49hkdqps7392qgpg	30	18
pair 3763, block 1	dbknllpkdqps7392rnu0	272	147
pair 3763, block 2	dbknn91kdqps7392rra0	272	147
pair 3763 without “aliens”, block 1	dbkntu5gg49c73djaps0	196	106
pair 3763 without “aliens”, block 2	dbknvdt6p51s73cn2bj0	196	106
pair 3765, block 1	dbkrnt0nceqs73amh5g0	272	147

Appendix

Table 10: Jobs on ibm_kingston (Heron r2), Qiskit 2.6.1 and qiskit-ibm-runtime 0.51.0: SamplerV2, 2,000 shots, optimisation level 3, the best of 8 transpiler seeds, dynamical decoupling (XY4). Pair 3765 without “men” (the same words as pair 3763 without “aliens”) used those jobs. Circuits, calibrations and raw counts are in the released ledger.

hardware
GPU, CPU, memory	NVIDIA GeForce RTX 5050 Laptop GPU (8 GB); Intel Core i5-13450HX; 16 GB
quantum computer	IBM ibm_kingston (Heron r2) through Qiskit Runtime
software
Python 3.12.3
PyTorch 2.11 (CUDA 12.8) [ 39 ]	models, training, evidence, optical compile
NumPy 2.5, SciPy 1.18	arrays, data

Appendix

Table 11: Computational setup.

Faithfulness of Language Model Explanations

Faithfulness of language model explanations measures whether stated reasons reflect the evidence and processes driving model outputs.

See all 111 papers

Legible-by-Construction: Attention and End-to-End Transformers
Jul 5, 2026Mark OskinTransformer Interpretability Transformer Attention
School of Computer Science and Engineering University of Washington
Causal Localization of Evidence Effects in Transformers
Oct 27, 2026Jipeng Li, Pramod Khargonekar, Yanning ShenTransformer Interpretability Attention Head Analysis
A Quantum Roadmap for Softmax Attention: Exact Born-Rule Analogs for Softmax Attention on the Probability Simplex
Aug 11, 2026Eric A. F. Reinhardt, Adam J. HauserSoftmax Attention Quantum Machine Learning
Department of Physics and Astronomy University of Alabama, Tuscaloosa, AL 35487, USA

Jul 5, 2026J/K move · Enter open · S save

Legible-by-Construction: Attention and End-to-End Transformers

Mark Oskin

School of Computer Science and Engineering University of Washington

Transformer Interpretability Transformer Attention

Open paper AskPDF arXiv

A companion paper showed that a transformer's feed-forward layer can be rebuilt from explicit fuzzy set operations - intersection, set-difference, and a self-forgetting sequence quantifier - so its hidden units read as named logical operators at no cost to language-model quality. That left the other half of the transformer opaque. Here we carry the same idea into attention and join the two into one model. The mechanism is minimal: a head's value is passed through a sigmoid, so each value channel becomes a readable detector of whether a feature holds at a token. This adds no parameters and leaves the standard head otherwise untouched. A Boolean variant goes further, restructuring the value into an explicit within-token intersection and negation-capable set-difference. In both designs the output projection is left free, not tied to the vocabulary, which is the load-bearing decision: bounding what a head detects while leaving what it writes unconstrained yields selective detectors, whereas constraining the write does not. A bounded value is shaped into a readable detector by two selectivity pressures - one for sparse firing, one for decisive firing at the rails - and which a design wants is not universal. Across five specialized-attention designs at 125M parameters, 44 to 62 percent of value channels become crisp, contextually selective detectors, and their legibility rises with depth rather than crystallizing only on punctuation. Language-model quality is at parity with a conventional baseline. Finally, we couple the Boolean attention to the legible feed-forward layer and train an end-to-end legible-by-construction language model at benchmark parity: its feed-forward units are named set and quantifier operations throughout, and we can take a token it generates and read the named units that compose to produce it.

Abstract

Faithfulness of Language Model Explanations

Explore similar work

Legible-by-Construction: Attention and End-to-End Transformers

Causal Localization of Evidence Effects in Transformers

A Quantum Roadmap for Softmax Attention: Exact Born-Rule Analogs for Softmax Attention on the Probability Simplex