cs.LGSep 28, 2026
SaveHow Language Models Differ in Redistributing Attention-Head Activity Under Serial Demand
Organizations: Carnegie Mellon University · University of California, San Diego · Qualtrics LLC
Abstract
The way a model distributes activity over each layer's attention heads offers a coarse view of how it routes information through depth; how this changes with the task is part of what a mechanistic account must explain. Holding prompt length fixed, we vary how many serial steps a task demands and measure, in every layer of 17 open-weight models, whether activity concentrates on a few heads or spreads across many as demand rises. Both occur: in most models, layers just before mid-depth concentrate activity and later layers spread it. Models differ in where and how strongly this happens. The Qwen2.5 base models from 0.5B to 7B, for example, spread less than the average model in every task and concentrate activity in parts of their second half, where Llama models from 1B to 8B and OLMo-2 spread; the contrast largely holds between Llama-3.1-70B and Qwen2.5-72B, which have the same number of layers and heads. These differences are reproducible, and post-trained models keep much of their base model's pattern. An ablation study suggests that, within a task, models whose activity is more concentrated on their top heads also depend more on those heads for the answer. Concentration and spreading across layers thus offer a new way to compare models, by how they route information through depth. Code is available at https://github.com/johnnyjli/serial-demand-heads.
Figures & tables
| Family | Query (wording w0) | written | Chance | |
|---|---|---|---|---|
| Pointer chasing | “Rules: 0 goes to 5. … Start at 4. Take steps. You end at” | 1–5 | yes | 0.10 |
| Modular arithmetic | “Rule: add 2 and keep the remainder after dividing by 7. Start at 3. Apply the rule times. You end at” | 1–6 | yes | 0.14 |
| Variable binding | “Assignments: w = z. v = n. … d = 6. … Therefore z =” | 1–6 | no | 0.33 |
| Relational lookup | “Chart: The boss of M is G. … The top boss above Z is” | 1–6 | no | 0.33 |
| Numeral control | pointer chasing on 7 states; written step count fixed at 7 | 1–6 | fixed | 0.14 |
Table 1: Task families. Prompt length is constant across within a family. Chance is one over the number of admissible answers.
| Lineage | Stages | Lineage | Stages | ||||
|---|---|---|---|---|---|---|---|
| Llama-3.2-1B | base | 16 | 32 | Qwen2.5-3B | base | 36 | 16 |
| Llama-3.2-3B | base, Instruct | 28 | 24 | Qwen2.5-7B | base, Instruct | 28 | 28 |
| Llama-3.1-8B | base, Instruct | 32 | 32 | Qwen2.5-14B | base | 48 | 40 |
| Qwen2.5-0.5B | base | 24 | 14 | OLMo-2-1124-7B | base, SFT, DPO, Instruct | 32 | 32 |
| Qwen2.5-1.5B | base | 28 | 12 | Mistral-7B-v0.1 | base | 32 | 32 |
| Mistral-Small-24B | base | 40 | 32 |
Table 2: Models: layers, query heads per layer. Some figures and tables shorten OLMo-2-1124-7B to OLMo-2-7B and Mistral-Small-24B to Mistral-24B.
Figure 1: In relational lookup, the back half of the Llama models and OLMo-2 spreads activity as grows; that of the Qwen2.5 models up to 7B shows no significant spreading. (a) Standardized slope of each layer’s top-10% head share on (blue: spreads; red: concentrates; clipped at ); triangles: , uncorrected. (b) Mean over back-half layers; gray: 95% range of its permutation null; filled: . Permutations shuffle within prompt bodies. Base models only. Eager attention may understate spreading in Qwen2.5-7B ( Appendix F ); Qwen2.5-14B (not shown) spreads like the Llama models ( , ).
Figure 2: Some models spread more, or less, than the others in every task. Left: each model’s profile averaged over the four tasks (blue: spreading; red: concentrating; dots: deciles where all four tasks have the same sign), and, per task (P, A, B, R: pointer chasing, arithmetic, binding, relational lookup), its back-half response minus that of the average model (blue: spreads more; red: less). Bold names and triangles: the same sign in all four tasks. Right: correlation between model signatures . Black squares: lineages.
Figure 3: Axes of difference. Left: model signatures on their first two principal components. Right: mean at 50–70% of depth, averaged over the four tasks, against accuracy at . Color: family; open markers: post-trained models, with arrows from their base model. L: Llama; Q: Qwen2.5; M: Mistral.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
| Family | Chance | 2 | 3 | 4 | 5 | 6 | |
|---|---|---|---|---|---|---|---|
| Pointer chasing | 0.10 | 0.57 | 0.16 | 0.09 | 0.07 | 0.06 | – |
| Modular arithmetic | 0.14 | 0.63 | 0.15 | 0.16 | 0.13 | 0.12 | 0.11 |
| Variable binding | 0.33 | 0.86 | 0.61 | 0.44 | 0.34 | 0.32 | 0.28 |
| Relational lookup | 0.33 | 0.76 | 0.45 | 0.25 | 0.27 | 0.23 | 0.27 |
| Numeral control | 0.14 | 0.41 | 0.30 | 0.17 | 0.15 | 0.14 | 0.03 |
Table 3: Mean strict accuracy over the 17 models (both wordings, all items; Qwen2.5-7B-Instruct under SDPA). An item counts as strictly correct when both the prediction restricted to admissible digits and the first generated digit are correct.
| Within each run | Main run | Replication |
|---|---|---|
| Model share of non-shared variance (chance) | 0.30 (0.18) | 0.29 (0.19) |
| permutation | 0.003 | |
| Split-half across tasks | 0.36 | 0.31 |
| permutation | 0.005 | |
| Same model on different tasks / different models on the same task | 0.32 / 0.30 | 0.26 / 0.17 |
| Lineages: Llama-3.1-8B, Qwen2.5-7B, OLMo-2-1124-7B | 0.61, 0.39, 0.69 | 0.80, 0.62, 0.50 |
Table 4: Main run and replication on the eight models run in both, with signatures centered on these models. Top: statistics within each run. Permutation uses 5,000 label shuffles for the model share and 2,000 for the split-half ; the lineage test is exact over all relabelings, and 0.002 is its smallest attainable . Same model / different models: mean correlation of full task profiles. Within a run, centering alone makes the mean correlation between different models’ signatures . Bottom: correlation of each model’s replication signature with its main-run signature, estimated from both wordings, from w0 alone and from w1 alone; the last column correlates each model’s main-run w0 and w1 signatures, which come from the same items. Across runs, centering makes a model’s correlations with the other models average about of its own . Pooled : one correlation over the signatures of all eight models. Every mean own and pooled in the bottom part has exact permutation .
| Feature | Main run | Replication |
|---|---|---|
| Back half spreads | 8 of 8 | 8 of 8 |
| Back half spreads more than the front half | 8 of 8 | 8 of 8 |
| Mean at 50–80% of depth, lineage-weighted | ||
| Mean at 40–50% of depth, lineage-weighted | ||
| 40–50% decile above the 50–60% decile | 8 of 8 | 6 of 8 |
| Consistently spreading layers | 33 | 14 (11 shared) |
Table 5: Features described in the main text, on the eight models run in both. The replication’s values are about half as large (see text), so the columns are best compared by sign, count of models and correlation. Intervals are 95% -intervals across the five lineages. Consistent layers: significant in the same direction in at least three tasks and in the opposite direction in none (as in Table 9 ); shared: also consistent in the main run. Post-training: change in the back-half mean over layers from the base to the Instruct model, averaged over the four tasks; the main-run Qwen2.5-7B pair mixes attention kernels ( with both models under eager attention). Across-run : correlation over the eight models, with exact permutation ; for the shared profile , over the ten deciles.
Figure 4: Signatures on fresh items against the main run. Correlation between each model’s signature in the replication (rows) and in the main run (columns, both wordings), with signatures centered on the eight models. Black squares: lineages; numbers: a model’s correlation with its own main-run signature (mean 0.80). The off-diagonal entries average , about of the diagonal mean, as centering implies. Llama-3.1-8B resembles its own and its Instruct model’s main-run signature equally (both 0.82). L: Llama; Q: Qwen2.5; M: Mistral.
| Llama-3.1-70B | Qwen2.5-72B | |||
|---|---|---|---|---|
| Task | back half ( ) | 80–90% | back half ( ) | 80–90% |
| Pointer chasing | ( ) | (0.003) | ||
| Modular arithmetic | ( ) | (0.50) | ||
| Variable binding | (0.002) | (0.07) | ||
| Relational lookup | ( ) | (0.80) | ||
Table 6: Response of the top-10% head share to in a matched pair (80 layers, 64 heads per layer; wording w0, SDPA). Back half: mean over layers at 50% of depth, with its two-sided permutation ; 80–90%: mean in that decile. Negative: spreads; positive: concentrates.
| Variant | Model share | Split-half | Lineage | Other | |
|---|---|---|---|---|---|
| Top-10% share (primary) | 0.34 (0.21) | 0.36 | 1.00 | 0.73 | |
| Top-25% share | 0.30 (0.20) | 0.31 | 0.84 | 0.64 | |
| Top-50% share | 0.29 (0.20) | 0.29 | 0.42 | 0.71 | |
| Effective number of heads | 0.31 (0.20) | 0.34 | 0.84 | 0.70 | |
| Share of the strongest head | 0.36 (0.23) | 0.36 | 0.79 | 0.65 | |
| Attention-sink heads removed | 0.34 (0.21) | 0.36 | 0.95 | 0.78 |
Table 7: Model signatures under alternative statistics, controls and subsets. Model share: share of the non-shared variance explained by model identity, with its label-permutation chance level in parentheses (permutation in every row). Split-half: correlation of signatures estimated from two tasks with those from the other two. : correlation of the signatures with the primary ones; for the SDPA variant, with those under eager attention on the same 14 models, and for the no-exclusion variant, with primary signatures computed the same way, from per wording averaged over the two wordings. Lineage / other: mean signature correlation within and between lineages.
| Variant | Pointer | Arithmetic | Binding | Relational | Mean | |
| Primary | 0.75 | 0.36 | 0.69 | 0.56 | 0.59 | |
| Share minus its floor | 0.77 | 0.40 | 0.76 | 0.64 | 0.64 | |
| Partial on heads per layer | 0.67 | 0.31 | 0.63 | 0.47 | 0.52 | |
| Models with (14) | 0.71 (12) | 0.19 | 0.61 | 0.42 | 0.48 | |
| Base models (11) | 0.64 (7) | 0.55 | 0.64 | 0.67 | 0.63 | |
| Cells with 15 correct items | 0.76 (8) | 0.43 (13) | 0.69 | 0.56 | 0.61 |
Table 8: Association between a model’s top-10% share and its excess drop under alternative choices. Per task: Spearman across models (in parentheses, the number of models where it differs from the row’s total, which is 17 unless the row gives another; pointer chasing has 13 in the primary analysis). Mean: average over tasks, with a one-sided permutation that shuffles models within each task (10,000 permutations).
Figure 5: Models with more concentrated activity depend more on their most active heads. The top 10% of heads per layer at 40–80% of depth are mean-ablated at the answer position. Top: each model’s top-10% share in those layers against its excess drop (loss of answer margin, as a fraction of clean, from the top heads minus that from as many random mid-ranked heads; correctly answered items, depth-balanced), per task, with Spearman across models. Open markers: post-trained models. Bottom left: drops from the top heads (filled) and from random heads (open), averaged over tasks, with models ordered by share. Bottom right: the same when each 20% depth band is ablated on its own; gray, each task; black, their mean. L: Llama; Q: Qwen2.5; O: OLMo-2; M: Mistral; -I: Instruct.
| Model | Concentrate | Spread |
|---|---|---|
| Llama-3.2-1B | – | 4 (.27), 10 (.67) |
| Llama-3.2-3B | 12 (.44), 27 (1.0) | 19 (.70) |
| Llama-3.2-3B-Instruct | – | 16 (.59), 18 (.67) |
| Llama-3.1-8B | 15 (.48), 31 (1.0) | 16 (.52), 17 (.55), 24 (.77), 26 (.84) |
| Llama-3.1-8B-Instruct | – | 2, 13, 16, 17, 19, 24, 27, 30 (.06–.97) |
| Qwen2.5-0.5B | – | 23 (1.0) |
Table 9: Layers significant ( ) in the same direction in at least three of the four tasks and significant in the opposite direction in none (0-based layer index; relative depth in parentheses). Over all models, 56 layers spread consistently against 39.5 expected when each task’s pattern is shifted to a random depth ( ), and 20 concentrate against 11.0 ( ).
Figure 6: A majority of the Llama models’ layers spread through most of the back half; in the Qwen2.5 models, only in parts of it. At each relative depth, the fraction of layers whose top-10% head share falls with ( ), among the layers within of that depth in the four serial tasks (pooled). Llama-3.1-70B and Qwen2.5-72B have the same number of layers and heads (data of Appendix D ); Llama-3.1-8B and Qwen2.5-7B come from the main run. The fraction exceeds one half at 100% and 87% of back-half positions in Llama-3.1-70B and 8B, against 28% and 43% in Qwen2.5-72B and 7B. Among the models not drawn, it does so at 73% and 75% in Llama-3.2-1B and 3B, at 0–26% in Qwen2.5-0.5B, 1.5B and 3B, and at 100% in Qwen2.5-14B, which spreads like the Llama models.
Figure 7: Each model’s depth profile, per task. Standardized slope of the top-10% head share on , by relative depth (negative: the layer spreads activity as grows; positive: it concentrates). Gray: each of the four tasks; color: the model’s mean over tasks; dashed: mean of all 17 models. : mean correlation between the model’s four task profiles. Rows are families; post-trained models carry the suffix -Inst, -SFT or -DPO.
Figure 8: Layer-by-task maps for every model (top-10% head share). Rows: tasks; columns: layers; blue layers spread ( ), red layers concentrate. Triangles: layers significant in the same direction in at least three tasks and significant in the opposite direction in none. Vertical line: half depth. Orange titles: post-trained models.
Figure 9: Back-half response of base and post-trained models. Base models (filled) and their post-trained versions (open): mean over back-half layers in the four serial tasks (left) and in the numeral control (right); negative values mean spreading. The three open OLMo-2 markers are, from left to right, SFT, DPO and Instruct. Dashed: the Qwen2.5-7B pair, whose base model was run with eager attention and its Instruct model with SDPA.
Figure 10: Accuracy on fresh items. Accuracy by serial depth in the replication ( Appendix C ) for the eight rerun models, counting the answer each model writes. Top: the model answers directly, as in the main run. Bottom: the worked examples and the model write out the intermediate states before the answer. Color: family; dashed: post-trained models; gray line: chance, one over the number of admissible answers (the worked examples rule out some answers; Appendix C ).