cs.LGSep 27, 2026

Rethinking Contextualization by Reinterpreting Attention Head Channels

Authors: Hakaze Cho, Haolin Yang, Zhun Sun, Naoya Inoue, Benjamin Heinzerling, Kentaro Inui

Organizations: RIKEN · Tohoku University · New York University · JAIST · MBZUAI

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Contextualization, the core operation of language modeling, transmits information across words to build sentence-specific word representations. Prior works mainly study contextualization, focusing on individual words and attention heads as a growing discrete dictionary, lacking a global view of their general behavior. Therefore, we propose a general principle: Globally, we find and estimate that different words carry different amounts of information, and less-informative words tend to absorb more contextual information. Specifically, these low-information words do not absorb contextual words uniformly, and finer-grained selectivity enables more precise routing to promote information transmission between matched words. Moreover, to find what mechanism causes such processing, we reinterpret attention heads as channels gated by their singular vectors and find that: (1) these singular vectors point to the hidden states of more informative words, allowing such words to write their information to others more strongly to act as information sources, and vice versa; and (2) these singular vectors can be viewed equally as hidden state features, enabling automated interpretation of attention heads beyond prior heuristic head discovery, also embedding heads into a continuous space rather than treating them as discrete, independent dictionary entries.

Figures & tables

Figure 1: (A) Words initially carry different amounts of static information. (B) During contextualization, information tends to flow from information-rich to information-poor tokens with semantic selectivity. (C) We reinterpret each attention head as an information channel gated by singular vectors.
Figure 2: Methods to characterize contextualization. (A) Static information is measured by output divergence with the word to be tested input, against the blank background. (B) Contextualization magnitude is measured through hidden-state similarities. (C) Selective transmission is measured by output divergence against the background run and the activation-patched run from contexted inputs.
Figure 3: Static information across word categories. Distributions of word-level information gain for the 16 word categories in Llama 3.2-1B, dots are category means. Words with more specific semantics or lower frequency generally exhibit greater information gain.
Figure 4: Layer-wise absorption magnitude across word categories. Marker is layer-wise result, bar is mean across layers.
Figure 5: Correlation of static information with information absorption magnitude (negative) in Layer 9 of Llama 3.2-1B. All layer results are in Appendix E .
Figure 6: Contextualization selectivity on ( Left ) male and female names to “he” and “she”, ( Middle ) animate and inanimate nouns to “who” and “which”, ( Right ) plural and singular nouns to “is” and “are”. Upper : Examples on Llama 3.2-1B Layer 5; Lower : Statistics on all layers and similar inputs.
Figure 7: Attention scores (attention sink removed) of one head from Llama 3.2-1B (interpretation in Table 1 ). When we ablate the q2q_{2} and k2k_{2} features, the attention scores change as interpreted.
Attn. Head InterpretationSingl. ValueScore (F1)
Layer 2 Head 11−q1-q_{1}“references to specific years or numbers in historical and descriptive contexts”4.800.96
−k1-k_{1}“the substring ’ pre ’ and ’ post ’ in various legal and procedural contexts”1.00
−v1-v_{1}“phrases indicating the start , effective date , or duration of time-related events or actions”0.481.00
Layer 11 Head 25q2q_{2}“words related to celebrating cultural or religious festivals , particularly Christmas and similar events”5.050.96
k2k_{2}“ months holidays and seasonal events mentioned in the text”1.00
−v1-v_{1}“references to Christmas and related holiday concepts”0.760.93
Table 1: Examples of automatically extracted interpretations by AutoInterp of attention head singular vectors in Llama 3.2-1B. The F1 score of random prediction is 0.1430.143 .
Figure 8: Correlation: length of words’ hidden states projected to top q,k,vq,k,v and words’ information.
Figure 9: Histogram of AutoInterp F1 scores.
Figure 10: t-SNE of head embedding.
Figure 11: Correlation between head embedding distance and attention map.
Appendix figures & tables47 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 12: Experiment result of Fig. 8 on Llama 3-8B (only 20% of heads are calculated).
Figure 13: Experiment result of Fig. 11 on Llama 3-8B (only 20% of heads are calculated).
Figure 14: More attention score visualization like Fig. 7 on Llama 3.2-1B Layer 2 Head 11.
Figure 15: More attention score visualization like Fig. 7 on Llama 3.2-1B Layer 11 Head 25.
Figure 16: More attention score visualization like Fig. 7 on Llama 3.2-1B Layer 11 Head 25.
Figure 17: More attention score visualization like Fig. 7 on Llama 3.2-1B Layer 6 Head 29.
Figure 18: More attention score visualization like Fig. 7 on Llama 3.2-1B Layer 6 Head 29.
Figure 19: More attention score visualization like Fig. 7 on Llama 3-8B Layer 17 Head 1 with the interpretation of the first set of singular vector: q1q_{1} : “specific dates and mentions of events or locations like the Bahamas, court cases, and significant donations”, k1k_{1} : “years denoted in a specific format with examples like 2014 and 2009”, v1v_{1} : “geographic locations and their corresponding state abbreviations in text”.
Figure 21
Figure 22
Figure 23
Figure 24
Figure 25
Figure 26
Figure 27
Figure 28
Figure 29
Figure 30
Figure 31
Figure 32
Figure 33
Figure 34
Figure 35
Figure 50: Full layer correlation (absorption is a negative measurement) of Fig. 5 on Granite 4.1-30B.
Figure 51: Full layer correlation (absorption is a negative measurement) of Fig. 5 on Llama 2-13B.
Figure 52: Full layer correlation (absorption is a negative measurement) of Fig. 5 on Olmo 3-32B.
Figure 39
Figure 55: Full layer correlation (absorption is a negative measurement) of Fig. 5 on Qwen 3.6-27B.
Figure 41
Figure 42
Figure 43
Figure 44
Figure 45
Figure 46
Figure 47
Figure 48
Figure 49
Figure 50
Figure 76: Full layer correlation with token length 2 (absorption is a negative measurement) of Fig. 5 on Granite 4.1-30B.
Figure 77: Full layer correlation with token length 2 (absorption is a negative measurement) of Fig. 5 on Llama 2-13B.
Figure 78: Full layer correlation with token length 2 (absorption is a negative measurement) of Fig. 5 on Olmo 3-32B.
Figure 79: Full layer correlation with token length 2 (absorption is a negative measurement) of Fig. 5 on Qwen 3-14B.
Figure 80: Full layer correlation with token length 2 (absorption is a negative measurement) of Fig. 5 on Qwen 3-8B.
Figure 81: Full layer correlation with token length 2 (absorption is a negative measurement) of Fig. 5 on Qwen 3.6-27B.
CategoryCountAvg. Freq.Words
Famous People Name475.36Einstein, Shakespeare, Oprah, Picasso, Mandela, Cleopatra, Churchill, Tesla, Elvis, Marie Curie, Gandhi, Aristotle, Monroe, Beethoven, Newton, Amelia Earhart, Frida Kahlo, Rothschild, Darrow, Hemingway, Frost, Twain, Jobs, Job, Obama, J.K. Rowling, Rihanna, Zuckerberg, Mother Teresa, Winston Churchill, Bill Gates, Marlon Brando, Charlie Chaplin, John Lennon, Bruce Lee, Michael Jackson, Madonna, Coco Chanel, Nikola Tesla, Audre Lorde, George Washington, Agatha Christie, Nelson Mandela, Thomas Edison, Steve Jobs, Vincent van Gogh, Serena Williams
normal people name5211.80Alice, Bob, Charlie, Emma, David, Sarah, Michael, Jessica, Daniel, Laura, James, Lily, Andrew, Zoe, Matthew, Olivia, Ethan, Mia, Joshua, Ava, Ryan, Grace, Benjamin, Ella, Samuel, Chloe, Lucas, Scarlett, Jacob, Victoria, Henry, Natalie, Jonathan, Aria, Chris, Ruby, Adam, Hailey, William, Leah, Kevin, Julia, Jason, Alexa, Thomas, Claire, Jordan, Sophie, Nathan, Hazel, Isaac, Stella
famous location name450.22Eiffel Tower, Great Wall of China, Taj Mahal, Statue of Liberty, Stonehenge, Machu Picchu, Colosseum, Big Ben, Pyramids of Giza, Golden Gate Bridge, Christ the Redeemer, Sydney Opera House, Acropolis, Niagara Falls, Leaning Tower of Pisa, Burj Khalifa, Mount Fuji, Angkor Wat, Petra, Mount Rushmore, Alhambra, Buckingham Palace, Louvre Museum, Kremlin, Sagrada Família, Table Mountain, Galápagos Islands, Vatican City, Serengeti National Park, Great Barrier Reef, Chichen Itza, Uluru, Himalayas, Amalfi Coast, Venice Canals, Santorini, Yellowstone National Park, Arches National Park, Ha Long Bay, Bora Bora, Mount Kilimanjaro, Bali, Old Faithful, Iguazu Falls, Victoria Falls
normal location name5121.14Park, River, Hill, Street, Corner, Avenue, Square, Bridge, Lake, Forest, Trail, Mountain, Road, Path, Beach, Garden, Plaza, Meadow, Bay, Valley, Pond, Cliff, Harbor, Dock, Canyon, Oasis, Sanctuary, Campground, Crossroads, Highway, Dunes, Knoll, Summit, Glade, Orchard, Wetland, Lighthouse, Quarry, Fort, Estuary, Peninsula, Cove, Bungalow, Resort, Retreat, Ranch, Estate, Villa, Commons, Grove, Wilderness
frequent object noun5051.36apple, table, chair, computer, book, phone, pen, car, tree, house, bottle, cup, bag, clock, shoe, key, window, door, mirror, lamp, bed, wallet, magazine, guitar, camera, plate, fork, knife, spoon, basket, painting, flower, umbrella, toy, speaker, suitcase, pillow, shirt, jeans, watch, blanket, candle, recipe, map, remote, brush, comb, photo, couch, dish
hardly used object noun500.32abacus, bauble, carafe, dirndl, epergne, flaxen, gazetteer, hovel, ichthyology, jägermeiste, knickknack, locket, marzipan, nightgown, obelisk, parsol, quagmire, ragamuffin, sextant, tiffin, umbilicus, vial, whittle, xylophone, yarmulke, ziggurat, ankh, balustrade, colander, dalliance, effigy, filigree, goblet, harpsichord, incunabulum, jigsaw, kumquat, lichen, madrigal, nanotechnology, omphalos, pecuniary, quasar, recumbent, sconce, tambourine, ukulele, vorpal, wattle, xenophobia
Appendix
Table 2: Sampled words as the experimental datasets. Frequencies are on the scale of 10−610^{-6} .
No.QuestionOutput Candidates
1What grammatical gender (if any) is typically associated with this people-word in languages that mark gender?1) masculine; 2) feminine; 3) common/epicene; 4) no grammatical gender
2What age group does this word primarily denote?1) child; 2) adolescent; 3) adult; 4) elderly; 5) age-neutral
3Does this word denote a familial kinship relation? If so, which basic role?1) parent; 2) child; 3) sibling; 4) grandparent; 5) spouse/partner; 6) other kin; 7) not a kinship term
4What occupational sector or field does this word most directly refer to?1) healthcare; 2) education; 3) government/public service; 4) business/finance; 5) arts/entertainment; 6) trades/industry; 7) science/technology; 8) agriculture; 9) service/hospitality; 10) not an occupation
5Does this word primarily indicate a social or legal status? If so, which?1) citizen; 2) immigrant/resident; 3) refugee/asylum-seeker; 4) stateless; 5) legal/official title (e.g., holder of an office); 6) not a status term
6Is this word typically used as a formal title or honorific?1) honorific/title (e.g., Sir, Dr.); 2) professional title (e.g., Judge, Professor); 3) noble/royal title (e.g., King, Duchess); 4) informal/colloquial address; 5) not a title
Appendix
Table 3: Sampled queries and their output candidates.
No.Template
1During the morning meeting the team asked whether the project could [MASK] before the client presentation next Tuesday.
2The gardener noticed that the leaves on the oldest tree had turned a deep [MASK], signaling the change of season.
3When packing for the long trip she insisted that each suitcase contain at least one warm [MASK] for cold nights.
4He smiled and gave a brief [MASK] when the host introduced him to the guests at the dinner.
5If the software continues to [MASK] under heavy load we will need to optimize the database queries and caching.
6The little boy ran across the yard and shouted [MASK] as he chased after the colorful butterfly in sunlight.
Appendix
Table 4: All sampled templates. [MASK] is the position to be filled with words.

Explore similar work

CardsList
Sep 23, 2026cs.CL

Scaling Attention Head Analysis via Gradient-Based Attribution in Context-Aware Machine Translation

In this paper, we introduce a gradient-based head attribution strategy where the Token-level Max-Margin loss is backpropagated to the attention maps. This framework enables a large-scale causal analysis of attention heads, making it suitable for LLMs. We evaluate our method on the task of disambiguation in Context-aware Machine Translation, where we analyze 50 phenomena across 4 models and 4 language directions. We empirically show the alignment of our method with the effects of increasing the attention scores of token-to-token relations on three models and two language directions, ensuring the robustness of our method. Our analysis reveals the presence of the "general-purpose" attention heads that improve the model's performance when attending to different relations. We find that the average attention a head assigns to a relation does not necessarily relate to the model's performance, which suggests that the models developed redundancies during training in terms of the head functions.

Paweł Mąka, Yusuf Can Semerci, Jan Scholtes +1

Department of Advanced Computing Sciences Maastricht University

Token-Level EntropyDisambiguation
AskPDF
Sep 14, 2026cs.LG

Attention Mean Fields Predict Average Representation Dynamics and Reveal Context-Specific Computation

A language model's representation geometry is not predetermined; it evolves as the model runs. A faithful account of that geometry must capture that dynamic process, and so cannot be based solely on model-independent statistics such as co-occurrence. Here we introduce a mean-field analysis of attention. The average attention from one token to another defines a kernel that carries representations layer to layer and can be iterated through the network to model how the geometry is transformed. We condition this average two ways. Conditioned on a whole corpus, the kernel predicts the average-case evolution of representation geometry. Conditioned instead on a single context, it predicts the expected geometry for that context. A head's departure from that prediction, its \emph{mean-field deviation}, isolates the context-specific computation that the mean field misses. Under the corpus-conditional reading, the kernel yields an open-loop model: from the input embeddings and the frozen weights alone, we can iterate the kernel and the model's own MLPs over token representations, never consulting a measured deviation at any layer. The resulting prediction is highly accurate. In early training the model and its corpus mean field are indistinguishable. Replace every attention head with its mean field, and the substitution leaves the loss on real text unchanged. Around the onset of induction, the two diverge, and the gap widens as representations become contextualized. Under the context-conditional reading, deviation from the mean field is a task-agnostic measure of context-specific computation. The residual decomposes additively into unusual attention routing and contextualization of the transported values. Across controlled induction and few-shot settings, greater deviation tracks greater reliance on in-context information.

Micah Adler, John W. Byers, Mark Crovella

Department of Computer Science, Boston University · Faculty of Computing & Data Sciences, Boston University

Language ModelingMean-Field LimitRepresentation SpaceVariational Inference
AskPDF
Sep 8, 2026cs.CL

Do New Attention Mechanisms Actually Fix Attention Sinks at Million-Token Context?

Long context language models now advertise windows of one million tokens, but two habits limit how much of that window is used. Attention heads with nothing useful to read still spend their budget on the first token, which is called the attention sink, and where a fact sits in the context changes whether the model finds it. Gated attention cut first token attention from 46.7 percent to 4.8 percent at NeurIPS 2025, and Kimi K3 pairs that idea with Kimi Delta Attention and Attention Residuals behind a one million token window, eight times past the range where these diagnostics have been reported. This paper asks whether the fix survives that jump. We build SinkProbe, a suite that measures sink mass, massive activation, position resolved recall and the recency gap, and apply it to four small models that differ only in how they mix tokens and depth. Three results follow. The training objective produces the sink, not the architecture. Gating did not reproduce its published effect at our scale. Sink mass, activations and position bias moved independently. Code, data and the measurement protocol are released at https://github.com/sararizwan7/Attention-Mechanisms-in-1M-Context-Window

Sara Rizwan, Samaanah Abdus Salam

Shadan Women’s College of Engineering and Technology Hyderabad, Telangana, India

Efficient Long-Context Inference
AskPDF