Measuring Affinity between Attention-Head Weight Subspaces via the Projection Kernel
Organizations: Kyoto University · RIKEN AIP
Abstract
Understanding relationships between attention heads is essential for interpreting the internal structure of Transformers, yet existing metrics do not capture this structure well. We focus on the subspaces spanned by attention-head weight matrices and quantify head-to-head relationships using the Projection Kernel (PK), a principal-angle-based measure of subspace similarity. Experiments on GPT2-small, BLOOM-560M, and Qwen2-0.5B-Instruct show that PK reproduces known head-to-head interactions on the IOI task more clearly than prior metrics such as the Composition Score. We further introduce a framework to quantify the informativeness of PK distributions by comparing them with a reference distribution derived from random orthogonal subspaces. As an application, we analyze a directed graph constructed from PK and show that, in GPT2-small, L4H7 acts as a hub by functioning as an Identity Head.
Figures & tables
| Method | OQ | OK | OV | Avg. | |
|---|---|---|---|---|---|
| PR-AUC | PK | 0.446 | 0.451 | 0.289 | 0.395 |
| CS | 0.227 | 0.352 | 0.215 | 0.265 | |
| Simple-CS | 0.245 | 0.359 | 0.305 | 0.303 | |
| Linear CKA | 0.337 | 0.323 | 0.409 | 0.356 | |
| Proc. Sim. | 0.328 | 0.269 | 0.428 | 0.342 |
| Method | PR-AUC | ROC-AUC |
|---|---|---|
| PK | 0.047 | 0.809 |
| CS | 0.013 | 0.785 |
| Simple-CS | 0.016 | 0.792 |
| Linear CKA | 0.021 | 0.715 |
| Proc. Sim. | 0.038 | 0.787 |
| Top 1 | Top 2 | Top 3 | Top 4 | Top 5 | ||
|---|---|---|---|---|---|---|
| Inlet | OQ | L10H7 | L5H6 | L7H1 | L8H10 | L10H11 |
| OK | L5H1 | L7H7 | L5H5 | L11H3 | L11H8 | |
| OV | L4H7 | L11H3 | L9H3 | L5H2 | L11H8 | |
| Outlet | OQ | L7H6 | L1H1 | L4H7 | L8H10 | L5H2 |
| OK | L4H7 | L4H11 | L0H9 | L1H11 | L5H10 | |
| OV | L0H1 | L3H0 | L4H7 | L5H10 | L5H11 |
| BLOOM-560M | Qwen2-0.5B-Instruct | |||
|---|---|---|---|---|
| Method | PR-AUC | ROC-AUC | PR-AUC | ROC-AUC |
| PK | 0.027 | 0.866 | 0.032 | 0.781 |
| CS | 0.003 | 0.844 | 0.005 | 0.804 |
| Simple-CS | 0.015 | 0.869 | 0.004 | 0.794 |
| Linear CKA | 0.001 | 0.533 | 0.024 | 0.509 |
| Proc. Sim. | 0.001 | 0.568 | 0.024 | 0.557 |
Appendix figures & tables56 assets
Supplementary material from the paper’s appendix.
Appendix
| ID | Token | ID | Token | ID | Token |
|---|---|---|---|---|---|
| 188 | \x00 | 29372 | _guiActiveUn | 39177 | ItemThumbnailImage |
| 189 | \x01 | 30202 | _guiName | 39253 | _UCHIJ |
| 190 | \x02 | 30208 | _externalTo | 39446 | _SetFontSize |
| 191 | \x03 | 30209 | _unfocusedRange | 39749 | DeliveryDate |
| 192 | \x04 | 30210 | _guiActiveUnfocused | 39752 | quickShip |
| 193 | \x05 | 30211 | _guiIcon | 39753 | quickShipAvailable |
| Unembed. Norm | Query | Key | Value | Output | All | |
|---|---|---|---|---|---|---|
| Head Classes | Count | Heads |
|---|---|---|
| Duplicate Token Heads | 3 | L0H1, L0H5 ∗ , L3H0 |
| Previous Token Heads | 10 | L2H2, L2H5 ∗ , L2H9, L3H2 ∗ , L3H6 ∗ , L3H7 ∗ , L3H8 ∗ , L4H11, L5H6 ∗ , L6H8 ∗ |
| Induction Heads | 6 | L5H0 ∗ , L5H1 ∗ , L5H5, L6H9, L7H2 ∗ , L7H10 ∗ |
| Name Mover Heads | 3 | L9H6, L9H9, L10H0 |
| Negative Name Mover Heads | 2 | L10H7, L11H10 |
| Backup Name Mover Heads | 8 | L9H7, L10H1, L10H2, L10H6, L10H10, L11H2, L11H3, L11H9 |
| Heads | Edges | ||||||||
| Method | WP | #Total | #Annot. | #Unannot. | #Total | #Both Annot. | #Either Annot. | Figure | |
| Top10 | PK | 30 | 15 (50.0%) | 15 (50.0%) | 30 | 10 (33.3%) | 22 (73.3%) | Fig. 19 | |
| CS | 30 | 9 (30.0%) | 21 (70.0%) | 30 | 3 (10.0%) | 9 (30.0%) | Fig. 21 | ||
| Top20 | PK | 52 | 22 (42.3%) | 30 (57.7%) | 60 | 18 (30.0%) | 42 (70.0%) | Fig. 6 | |
| ✓ | 51 | 21 (41.2%) | 30 (58.8%) | 60 | 17 (28.3%) | 41 (68.3%) | Fig. 37 | ||
| CS | 46 | 14 (30.4%) | 32 (69.6%) | 60 | 3 (5.0%) | 20 (33.3%) | Fig. 22 | ||
| GPT2-small | BLOOM-560M | Qwen2-0.5B-Instruct | |
| # parameters | 124M | 560M | 494M |
| # layers | 12 | 24 | 24 |
| # query heads | 12 | 16 | 14 |
| # KV heads | 12 | 16 | 2 |
| Attention architecture | MHA | MHA | GQA |
| 768 | 1024 | 896 |
| Head Classes | Count | Heads |
|---|---|---|
| Duplicate Token Heads | 4 | L1H14 ∗ , L3H12 ∗ , L6H14 ∗ , L11H10 |
| Previous Token Heads | 10 | L4H0, L5H0, L6H0, L6H2, L7H0, L8H0 ∗ , L9H3 ∗ , L11H0 ∗ , L11H5 ∗ , L17H0 ∗ |
| Induction Heads | 10 | L7H12, L8H11, L8H13, L8H14 ∗ , L9H14 ∗ , L10H10, L10H12 ∗ , L10H13 ∗ , L13H12 ∗ , L18H13 ∗ |
| Name Mover Heads | 11 | L18H7, L18H8, L19H8, L19H11, L20H8, L20H9, L20H10, L20H11, L20H12, L21H9, L21H10 |
| Negative Name Mover Heads | 3 | L17H9, L19H15, L21H6 |
| S-Inhibition Heads | 6 | L12H6, L14H6, L16H5, L19H4, L21H2, L22H3 |
| Head Classes | Count | Heads |
|---|---|---|
| Duplicate Token Heads | 5 | L1H1 ∗ , L5H0, L5H1 ∗ , L7H7, L12H2 ∗ |
| Previous Token Heads | 10 | L0H5 ∗ , L0H7 ∗ , L1H5, L2H3 ∗ , L4H0, L5H5, L8H10 ∗ , L10H13 ∗ , L15H4 ∗ , L20H9 ∗ |
| Induction Heads | 9 | L9H8 ∗ , L9H9, L9H12, L11H9 ∗ , L11H11, L11H12 ∗ , L11H13, L16H1 ∗ , L16H6 ∗ |
| Name Mover Heads | 6 | L16H0, L20H11, L21H5, L21H6, L21H11, L21H12 |
| Negative Name Mover Heads | 3 | L20H7, L22H6, L23H12 |
| S-Inhibition Heads | 8 | L13H8, L14H8, L14H10, L15H7, L16H5, L17H6, L21H0, L22H8 |
| Weight preprocessing | Module | Original | Preprocessed | Shape |
|---|---|---|---|---|
| Layer normalization folding | ||||
| Weight after LN | ||||
| Bias after LN | ||||
| Centering writing weights | Weight before LN | |||
| Bias before LN | ||||
| Centering the unembedding matrix | Unembedding matrix |
Attention Head Analysis
Attention head analysis studies how individual Transformer heads route information, support computations, and affect model behavior.
See all 87 papersA Geometric Invariant of Attention Predicts Head Specialization from Weights Alone
Projection-Free Transformers via Gaussian Kernel Attention
nanochat framework, we implement causal masking and sliding-window constraints by masking and renormalizing the Gaussian kernel. At depth 20, a GKA model with the parameters and the total training FLOPs of a standard attention baseline trains stably, exhibits a near-zero train-validation gap, and demonstrates competitive behavior on standard benchmarks, albeit with higher bits-per-byte (BPB) at this compute scale. Overall, GKA provides a minimal, interpretable attention mechanism with an explicit locality scale, offering a dimension in the accuracy-efficiency trade-off for Transformer design.