Which Question Is Your Attention Metric Answering? Attention Rows as Compositional Data

Marios Papamichalis; Regina Ruane

cs.CLAug 11, 2026

Which Question Is Your Attention Metric Answering? Attention Rows as Compositional Data

Authors: Marios Papamichalis, Regina Ruane

Organizations: Human Nature Lab, Yale University, New Haven, CT 06511 · Department of Statistics and Data Science, The Wharton School, University of Pennsylvania, 3733 Spruce Street, Philadelphia, PA 19104-6340

Abstract

Each row of a transformer's attention matrix is a probability distribution over tokens, and in trained models most of that probability lands on a single sink token, usually the first. Standard tools for comparing attention rows (cosine similarity, Jensen--Shannon divergence, Shannon entropy) therefore hinge on a choice papers rarely report: keep the sink, or drop it and renormalize. This choice can reverse conclusions. On ten pretrained models from five families, 17--47% of verdicts about which of two heads is more similar flip with the convention, and the most prominent structure in a standard BERT head-clustering pipeline is an artifact of it. The reason is that one-number summaries mix two questions: how much attention the sink takes, and how the rest is divided among the content tokens. Treating rows as compositional data separates them exactly: the Aitchison distance splits orthogonally into a sink term and a content term, entropy splits by an exact identity, and the content distance is characterized by invariances the transformer itself possesses. The separation matters in practice: most measured entropy collapse during training is the sink growing, not attention sharpening (30% of the drop at 70M parameters, 95% at 1B, 79% at 1.4B), and pruning heads with the wrong channel can inflate perplexity more than a hundredfold. We map where each convention is safe, test a frozen out-of-sample predictor (one confirmation, one abstention, one failure), and release code regenerating every number.

Figures & tables

Figure 1: The reversal in one construction (Theorem 1 ). Three attention rows over

\{\text{sink},a,b\}

p=(0.90,0.08,0.02)

q=(0.70,0.24,0.06)

r=(0.90,0.02,0.08)

. (a) With the sink column,

p

and

r

share the same sink mass while

q

differs. (b) Dropping the sink and renormalizing, a routine, usually undocumented preprocessing choice, makes

p^{\prime}

and

q^{\prime}

identical (both allocate content

4{:}1

a{:}b

) while

r^{\prime}

mirrors them. (c) Cosine, JS, and Euclidean distance all say

r

is the closer head with the sink and

q

without it: the scientific conclusion is an artifact of the convention. The Aitchison distance returns the same verdict in both views, and its content part

d_{A}^{\perp}

is numerically identical.

	with sink			sink dropped
metric	$d(p,q)$	$d(p,r)$	closer	$d(p^{\prime},q^{\prime})$	$d(p^{\prime},r^{\prime})$	closer
cosine	$0.9693$	$0.9956$	$r$	$1.0000$	$0.4706$	$q$	reversal
JS	$0.0324$	$0.0193$	$r$	$0.0000$	$0.1927$	$q$	reversal
Euclidean	$0.2592$	$0.0849$	$r$	$0.0000$	$0.8485$	$q$	reversal
$d_{A}$	$1.1022$	$1.9605$	$q$	$0.0000$	$1.9605$	$q$	stable

Table 1: Exact witness for Theorem 1 with

p=(0.90,0.08,0.02)

q=(0.70,0.24,0.06)

r=(0.90,0.02,0.08)

q

p

’s content ratio (

4{:}1

) and

r

p

’s sink mass with mirrored content. “Closer” means larger cosine similarity or smaller dissimilarity. All three classical verdicts reverse; the JS and Euclidean values also violate subcompositional dominance (

\mathrm{JS}(p^{\prime},r^{\prime})>\mathrm{JS}(p,r)

\lVert p^{\prime}-r^{\prime}\rVert>\lVert p-r\rVert

Figure 2: Entropy conflates sink strength with content spread;

C_{A}^{\perp}

separates them. Synthetic 12-layer models,

4{,}000

rows per layer. (a) Sink mass grows

0.06\to 0.85

with content log-ratio dispersion held fixed: mean Shannon entropy falls

2.71\to 0.82

nats (

-70\%

, a textbook “entropy collapse”) while

C_{A}^{\perp}

moves

48.1\to 48.0

(

-0.3\%

): the collapse is phantom. (b) Content genuinely collapses while the sink recedes

0.70\to 0.05

: entropy drifts

1.73\to 1.27

(

-27\%

, reads as mild) while

C_{A}^{\perp}

rises

10.0\to 70.4

(

7\times

): the real collapse is masked.

Figure 3: Calibrated reversal curves and ten measured models. Curves: Monte Carlo closer-head disagreement rates between sink conventions for cosine, JS, and Euclidean distance, computed before any pretrained model was run (solid

\alpha{=}0.1

, dashed

\alpha{=}1

). Stars: the ten models, placed at the sink mass of the signatures the distances are computed on; mean absolute deviation from the solid curves is

4.8

points (max

13

). The

d_{A}^{\perp}

rates are zero identically (Lemma 1 ), plotted as a code check.

Model	sink (row/sig.)	cosine	JS	Euclid.	Hell./FR	collapse disagr.
		ranking-reversal rate
GPT-2 (124M)	$0.68/0.78$	$36.1{\scriptstyle\pm 0.1}$ %	$35.0{\scriptstyle\pm 0.2}$ %	$38.1{\scriptstyle\pm 0.2}$ %	$34.4{\scriptstyle\pm 0.2}$ %	$27.3{\scriptstyle\pm 0.0}$ %
Llama-3.2-1B	$0.84/0.94$	$46.1{\scriptstyle\pm 0.2}$ %	$44.1{\scriptstyle\pm 0.2}$ %	$47.2{\scriptstyle\pm 0.3}$ %	$43.6{\scriptstyle\pm 0.2}$ %	$46.7{\scriptstyle\pm 0.0}$ %
Qwen2.5-1.5B	$0.68/0.82$	$43.9{\scriptstyle\pm 0.1}$ %	$42.5{\scriptstyle\pm 0.1}$ %	$44.3{\scriptstyle\pm 0.1}$ %	$42.3{\scriptstyle\pm 0.1}$ %	$29.6{\scriptstyle\pm 0.0}$ %
Llama-3.2-3B	$0.86/0.95$	$45.0{\scriptstyle\pm 0.3}$ %	$43.7{\scriptstyle\pm 0.3}$ %	$46.7{\scriptstyle\pm 0.3}$ %	$43.5{\scriptstyle\pm 0.3}$ %	$37.0{\scriptstyle\pm 0.0}$ %
ViT-B/16	$0.04/0.07$	$22.4{\scriptstyle\pm 0.1}$ %	$17.3{\scriptstyle\pm 0.1}$ %	$23.2{\scriptstyle\pm 0.1}$ %	$18.1{\scriptstyle\pm 0.1}$ %	$9.1{\scriptstyle\pm 0.0}$ %

Table 2: Measured convention-dependence on ten pretrained models (mean

\pm

sd over three seeds). Reversal

=

% of

50{,}000

head triples whose closer-head verdict flips between sink conventions (bit-identical across the

\varepsilon

sweep). Rates rise with signature sink mass (Figure 3 ). Hell./FR: Hellinger reversal; Fisher–Rao is identical since both are strictly monotone in the Bhattacharyya coefficient, so every ranking verdict coincides. Collapse disagr.: fraction of adjacent-layer transitions on which

H

and the content entropy

H(\pi)

move in opposite directions. Sink (row/sig.): arithmetic row-level mean and the sink mass of the Aitchison-mean signatures the distances use. The content-distance column is omitted: its rate is zero by construction (Lemma 1 ).

Figure 4: Measured layer-wise collapse decomposed. Mean Shannon entropy

H

(red) versus content entropy

H(\pi)

(green), sink mass dotted (seed 0); shaded bands mark adjacent-layer transitions on which the two disagree in sign. Where the sink strengthens with depth,

H

reports deepening collapse that the content channel does not show: Proposition 1 ’s conflation observed in pretrained models.

Figure 5: Prospectively specified downstream outcomes. (a) reversal tracks sink mass, which strengthens with scale within each family. (b) at Pythia-410M

H

collapses while

H(\pi)

barely moves; the share

1-\Delta H(\pi)/\Delta H

30/53/70/95/79\%

across 70M–1.4B, rising to

95\%

at 1B (

79\%

at 1.4B).

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: The reversal region is not thin. Fixing $p$ and $r$ of the witness and sweeping the third row $q$ over the interior of the simplex ( $20{,}000$ Dirichlet $(1,1,1)$ samples with all parts $>0.004$ ), the cosine verdict about which of $q,r$ is closer to $p$ differs between the with-sink and sink-dropped conventions on $77.9\%$ of positions (red).

Figure 7: A measured Llama-3.2-1B head triple realizing Theorem 1 . Found automatically by find_real_triple.py from the saved seed-0 signatures: with the sink, cosine, JS, and Euclidean distance all rank

r

closer to

p

; after dropping the sink and re-closing, all three rank

q

closer; the

d_{A}^{\perp}

verdict is identical in both pipelines.

	Perm.	Perturb. ( $\oplus$ )	Temp.	Sink	Dom.
Euclidean	✓	$\times$	$\times$	$\times$	$\times$
cosine	✓	$\times$	$\times$	$\times$	$\times$
JS	✓	$\times$	$\times$	$\times$	$\times$
Shannon $H$	✓	$\times$	$\times$	$\times$	n/a
$d_{A}$	✓	✓	✓ †	✓ ‡	✓
$\rho_{A}$ (ILR-cosine)	✓	$\times$	✓ †	n/a	n/a

Appendix

Table 4: Invariance matrix. Perm.

=

permutation invariance; Perturb.

=

invariance to a shared logit bias (Prop. 2 (i)); Temp.

=

global-temperature ranking preservation; Sink

=

coherent under keep-vs-drop of the sink column; Dom.

=

subcompositional dominance.

\dagger

d_{A}

1/\tau

-equivariant, so rankings are exact;

\rho_{A}

is invariant even to per-head temperatures.

\ddagger

d_{A}

decomposes exactly (Lemma 1 );

d_{A}^{\perp},C_{A}^{\perp}

are exactly sink-invariant. “n/a”: not applicable.

Model	$1-\cos$	JS	Euclid.
BERT-base	$0.024$	$0.066$	$0.030$
GPT-2 (124M)	$-0.014$	$0.022$	$-0.061$
Llama-3.2-1B	$-0.030$	$-0.082$	$-0.086$

Appendix

Table 5: Measured taxonomy stability across sink conventions (the

d_{A}^{\perp}

column is omitted: its partitions are identical across conventions by construction, Proposition 3 (i), with ARI

=1

verified in code) (mean ARI over

K\in\{4,6,8\}

, average linkage; reanalyze_clustering.py ). The

d_{A}^{\perp}

column is exact by Proposition 3 (i).

Model	Hell. rev.	FR rev.	share $<10^{-6}$	$\rho$	NN agr.	cross-ARI
GPT-2 (124M)	$0.35$	$0.34$	$0.046$	$0.957$	$0.73$	$0.40$
Llama-3.2-1B	$0.43$	$0.44$	$0.003$	$0.978$	$0.81$	$0.33$
Qwen2.5-1.5B	$0.43$	$0.42$	$0.000$	$0.993$	$0.86$	$0.63$
Llama-3.2-3B	$0.44$	$0.44$	$0.000$	$0.971$	$0.80$	$0.60$
ViT-B/16	$0.19$	$0.18$	$0.000$	$0.998$	$0.85$	$0.77$
Pythia-70M	$0.30$	$0.30$	$0.000$	$0.970$	$0.81$	$0.87$

Appendix

Table 6: Estimator and rival-geometry audit (seed-0 signatures). Hell./FR

=

closer-head reversal rate between sink conventions under Hellinger and Fisher–Rao. Share

=

fraction of squared

d_{A}^{\perp}

carried by coordinate pairs with smaller entry below

10^{-6}

. Last three columns compare dropped-row JS with

d_{A}^{\perp}

on the content channel: Spearman correlation of pairwise distances, nearest-neighbor agreement, and mean cross-ARI of the induced taxonomies over

K\in\{4,6,8\}

	arithmetic mean			Aitchison mean
Model	cosine	JS	Euclid.	cosine	JS	Euclid.
GPT-2 (124M)	$0.33$	$0.30$	$0.35$	$0.36$	$0.35$	$0.38$
Llama-3.2-1B	$0.40$	$0.37$	$0.41$	$0.46$	$0.44$	$0.47$
Qwen2.5-1.5B	$0.40$	$0.37$	$0.41$	$0.44$	$0.43$	$0.45$
Llama-3.2-3B	$0.44$	$0.41$	$0.45$	$0.45$	$0.44$	$0.47$
ViT-B/16	$0.21$	$0.18$	$0.22$	$0.22$	$0.18$	$0.23$

Appendix

Table 7: Aggregation robustness : with-sink versus sink-dropped reversal rates (seed 0) computed on arithmetic-mean signatures (the classical metrics’ native aggregation) and on Aitchison-mean signatures.

	temperature (JS ′ )		sink set flips
Model	flip (max)	set retention (min)	JS ′	$d_{A}^{\perp}$
GPT-2 (124M)	$0.066$	$0.69$	$0.064$	$0.039$
Llama-3.2-1B	$0.046$	$0.87$	$0.154$	$0.128$
Qwen2.5-1.5B	$0.027$	$0.96$	$0.086$	$0.075$
Llama-3.2-3B	$0.040$	$0.90$	$0.205$	$0.176$
ViT-B/16	$0.017$	$0.93$	$0.010$	$0.012$

Appendix

Table 8: Convention stakes (seed-0 signatures). Temperature: worst-case dropped-JS verdict flip rate and top-

20\%

prune-set retention over

\tau\in\{0.5,0.7,1.5,2\}

(the

d_{A}^{\perp}

values are exactly

0

and

1

on every model, verified to the bit, and are omitted). Sink set: closer-head verdict flip rate between the first-token and first-four-token sink definitions.

	JS			cosine	$d_{A}^{\perp}$
$K$	cross	SEP Jac. (keep)	SEP Jac. (drop)	cross	cross	SEP Jac.
$4$	$0.05$	$0.92$	$0.49$	$0.06$	$1$	$0.47$
$5$	$0.04$	$0.87$	$0.49$	$0.01$	$1$	$0.48$
$6$	$0.01$	$0.87$	$0.46$	$0.01$	$1$	$0.50$
$8$	$0.03$	$0.87$	$0.48$	$0.00$	$1$	$0.50$

Appendix

Table 9: Clark-style clustering on the full

128

-token support (BERT-base; sink set

\{

CLS, SEP

\}

). Cross

=

between-convention ARI; SEP Jac.

=

Jaccard of the SEP-head set with its best-matching cluster.

Model	sparsity	base	JS	$d_{A}^{\perp}$	$d_{A}$ (total)
GPT-2 (124M)	10%	$43.5{\scriptstyle\pm 0.4}$	$50.5{\scriptstyle\pm 0.5}$	$88.9{\scriptstyle\pm 0.9}$	$61.5{\scriptstyle\pm 0.7}$
	20%		$74.5{\scriptstyle\pm 0.7}$	$104.7{\scriptstyle\pm 0.7}$	$104.8{\scriptstyle\pm 0.8}$
	30%		$85.2{\scriptstyle\pm 1.0}$	$122.3{\scriptstyle\pm 1.1}$	$117.4{\scriptstyle\pm 1.6}$
Llama-3.2-1B	10%	$15.3{\scriptstyle\pm 0.1}$	$358.0{\scriptstyle\pm 31.1}$	$45.6{\scriptstyle\pm 2.9}$	$18.2{\scriptstyle\pm 0.4}$
	20%		$2094.1{\scriptstyle\pm 139.7}$	$63.6{\scriptstyle\pm 4.1}$	$2480.3{\scriptstyle\pm 210.5}$
	30%		$2961.7{\scriptstyle\pm 150.0}$	$79.8{\scriptstyle\pm 7.7}$	$51742.6{\scriptstyle\pm 6382.9}$

Appendix

Table 10: Full pruning sweep at

10/20/30\%

sparsity (mean

\pm

sd over three seeds). LMs: perplexity (lower better); ViT: top-1 agreement (higher better; base

=1

by definition). All rates in Table 3 are bit-identical across

\varepsilon\in\{10^{-5},10^{-6},10^{-7}\}

, as the fixed-support protocol predicts.

Model	total $d_{A}$	JS ′ -L1	random A/B
GPT-2 (124M)	$104.8{\scriptstyle\pm 0.8}$	$268.0{\scriptstyle\pm 3.1}$	$314$ / $171$
Llama-3.2-1B	$2480.3{\scriptstyle\pm 210.5}$	$2642.2{\scriptstyle\pm 242.6}$	$187$ / $296$
Qwen2.5-1.5B	$32.3{\scriptstyle\pm 0.2}$	$28.4{\scriptstyle\pm 0.5}$	$30$ / $35$
Llama-3.2-3B	$36.4{\scriptstyle\pm 0.6}$	$24.1{\scriptstyle\pm 0.5}$	$39$ / $309$
ViT-B/16	$0.573{\scriptstyle\pm 0.005}$	$0.562{\scriptstyle\pm 0.011}$	$0.520$ / $0.535$
Pythia-70M	$1484.9{\scriptstyle\pm 11.3}$	$2051.1{\scriptstyle\pm 27.0}$	$1556$ / $1085$

Appendix

Table 11: Additional pruning baselines at

20\%

sparsity (mean

\pm

sd over three resampling seeds; two random draws shown individually). Total

d_{A}

re-mixes the sink balance; JS ′ -L1 is dropped-row JS under the same-layer keep-one protocol; the

10/30\%

values are in the released JSONs.

Attention Head Analysis

Attention head analysis studies how individual Transformer heads route information, support computations, and affect model behavior.

See all 79 papers

A Unifying View of Attention Sinks: From Mechanisms to Architectural Interventions
Jun 6, 2026Lukas Fesser*, Mozes Jacobs*, Thomas Fel* +2Transformer Interpretability Vision Transformer
Kempner Institute Harvard University Cambridge, MA
Choosing the Number of Attention Heads from Intrinsic Dimension: A Small-Scale Study
Oct 14, 2026Arjun Boya, Charan Sai Bandaru, Masthanaiah Pillimetla +1Multi-Head Attention
Institute of Cognitive Science University of Osnabrück · Department of Social Sciences and Technology Technical University of Munich
Importance Scoring of Transformer Attention Heads in Learning Tabular Data
Aug 27, 2026Ahmad Jad Allah, Kazi F. Akhter, Md. Kamrozzaman Bhuiyan +1Attention Head Analysis Transformer Attention
Department of Computer Science Tennessee State University Nashville, TN, USA · Enosis Solutions Dhaka, Bangladesh · Department of Computer Science North Carolina Agricultural and Technical University Greensboro, NC, USA

Jun 6, 2026J/K move · Enter open · S save

A Unifying View of Attention Sinks: From Mechanisms to Architectural Interventions

Lukas Fesser*, Mozes Jacobs*, Thomas Fel*, Andy Keller+1

Kempner Institute Harvard University Cambridge, MA

Transformer Interpretability Vision Transformer Attention Sinks Multi-Head Attention Attention Head Specialization

Open paper AskPDF arXiv

When attention concentrates on a single token, a sink, what is the model actually computing? Attention sinks are ubiquitous in softmax transformers, yet this shared visual signature can hide fundamentally different algorithms. We show that visually similar sink patterns can reflect two distinct mechanisms: (i) adaptive nop, where a head suppresses its update by routing to a null token, and (ii) broadcast, where a sink aggregates and redistributes global information. Each mechanism leaves distinct traces (nop-sinks exhibit negligible value norms; broadcast sinks induce low-rank outputs), which we formalize on synthetic tasks and use to derive practical diagnostics. Applied to pretrained vision transformers, these diagnostics reveal that both mechanisms exist at scale: sinks transition from CLS in early layers to patches in deeper layers and concentrate in specialized heads. Causal interventions further connect these signatures to near-null suppression and shared residual contributions. We then use architectural interventions to show how these computations can be reorganized: gating eliminates detected nop-like sinks but increases broadcast-like sinks, registers relocate rather than remove sink computation, and our position-free global pathway provides an explicit route for shared communication that reduces the broadcast-like sinks induced by gating. On dense probes, combining gating with the global pathway gives the strongest results among the tested variants, despite retaining some broadcast-like sinks. Overall, we find that the same attention pattern can reflect two very different computations, and that effective intervention depends not only on identifying the computation, but also on providing architectural alternatives through which the model can reorganize it.

Abstract

Attention Head Analysis

Explore similar work

A Unifying View of Attention Sinks: From Mechanisms to Architectural Interventions

Choosing the Number of Attention Heads from Intrinsic Dimension: A Small-Scale Study

Importance Scoring of Transformer Attention Heads in Learning Tabular Data