Large language models often reason beyond surface tokens, but the internal stage at which token-level information becomes abstract relational structure remains unclear. We investigate this question by analyzing how attention heads and layers transform information during autoregressive reasoning. Across mathematical and symbolic reasoning tasks, we observe a consistent layer-wise division of labor: outer layers mainly preserve and route input-related features, whereas middle layers reorganize them into more transferable rule-level representations. This interpretation is supported by representation geometry: middle-layer states occupy lower-dimensional manifolds and show stronger alignment across disjoint vocabularies that instantiate the same symbolic rules. It is further supported by causal interventions: removing middle-layer components identified by our interaction-based criterion produces substantially larger downstream changes and accuracy drops than removing components from other regions or at random. Together, these results suggest that abstract reasoning is not uniformly distributed across transformer layers, but is preferentially formed in a middle-layer computation stage that converts token-level information into reusable relational structure.
Figures & tables
Figure 1: Layer-wise localization of composition- and preservation-oriented components in Qwen3-8B-Base. (a) Attention heads are ranked by the difference between interaction-based composition and preservation scores. Warmer colors indicate heads that more strongly combine information across components, whereas cooler colors indicate heads that more strongly preserve information. The highest concentration of composition-oriented heads appears in middle layers. (b) Cosine similarity of residual updates across layers. Outer layers tend to reinforce existing directions in the residual stream, whereas middle layers introduce more orthogonal or suppressive updates, consistent with a stronger role in representation transformation.
Figure 2: Layer-wise composition and preservation scores under different interaction settings. (a) Scores computed from the MLP contribution alone. In this setting, the layer-wise pattern shows a gradual transition, with preservation-oriented signals appearing more prominently in earlier layers and composition-oriented signals becoming stronger toward later intermediate layers. (b) Scores computed from the combined MLP and attention contributions. Adding attention interactions sharpens the middle-layer composition pattern, with the strongest composition-oriented region appearing around the intermediate-to-late middle layers. Together, these results suggest that the composition-preservation organization depends on the type of sublayer interaction being measured, while still supporting the broader observation that representation transformation is concentrated away from the outermost layers.
Figure 3: Low-dimensional rule-level representations in middle layers. (a) Intrinsic dimension of hidden representations across layers. The estimated dimension decreases in the middle computation stage, suggesting a more compact representation geometry. (b) Cross-vocabulary CKA similarity between representations generated from disjoint vocabularies that instantiate the same symbolic rules. Higher similarity indicates stronger vocabulary-invariant rule alignment. (c) Cross-vocabulary rule-probe accuracy using residual updates Δl=hl+1−hl . Middle layers achieve higher transfer accuracy, indicating that rule information is more recoverable from their updates. (d) Visible token-identity probe accuracy. Surface token identity becomes less recoverable in the same depth range, suggesting a shift from lexical information toward relational structure.
Figure 4: Layer-level interventions for measuring downstream dependence. (a) Full layer ablation skips source layer s during both prompt processing and decoding. (b) Prefill-only ablation skips source layer s only during prompt processing and measures its delayed influence during later decoding. (c) Circuit localization removes the contribution of source layer s from the input of a later layer l over all positions. (d) Future-position circuit localization applies the same removal only to generated-token positions. Across these settings, stronger disturbances indicate that later computation depends more strongly on the information written by the intervened source layer.
Figure 5: Cumulative attention-head ablation according to the composition–preservation orientation score in Gemma3-12B-IT. Removing composition-oriented heads produces the largest accuracy degradation, random ablation leads to a moderate drop, and preservation-oriented head ablation has the weakest effect. This ordered trend suggests that components identified by the interaction-based criterion have stronger functional influence on reasoning performance.
Model
Layer pattern
Rule geometry
Ablation trend
Qwen3-4B
✓
✓
✓
Qwen3-8B
✓
✓
✓
Qwen3-14B
✓
✓
✓
Llama-3.1-8B
✓
✓
✓
Gemma3-12B-IT
✓
✓
✓
Table 1: Cross-model summary. A check mark indicates that the same qualitative pattern is observed under the corresponding analysis. Full per-model results are reported in Appendix E .
Appendix figures & tables26 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Role in paper
Dataset
Qwen3-4B-Base
Cross-model validation
GSM8K
Qwen3-8B-Base
Main analysis
GSM8K
Qwen3-14B-Base
Cross-model validation
GSM8K
Llama-3.1-8B
Cross-family validation
GSM8K
Gemma3-12B-IT
Head-level ablation / validation
GSM8K
Appendix
Table 2: Models used for the main and supplementary analyses.
Figure 6: Layer-wise localization in Qwen3-4B-Base.
Figure 7: Layer-wise localization in Qwen3-14B-Base.
Figure 8: Layer-wise localization in Llama3.1-8B-Base.
Figure 9: Layer-wise localization in Gemma3-12B-Base.
Figure 10: Layer-wise interaction profiles in Qwen3-4B-Base.
Figure 11: Layer-wise interaction profiles in Qwen3-14B-Base.
Figure 12: Layer-wise interaction profiles in Llama3.1-8B-Base.
Figure 13: Layer-wise interaction profiles in Gemma3-12B-Base.
Figure 14: Rule-level representation analysis in Qwen3-4B-Base.
Figure 15: Rule-level representation analysis in Qwen3-14B-Base.
Figure 16: Rule-level representation analysis in Llama3.1-8B-Base.
Figure 17: Rule-level representation analysis in Gemma3-12B-Instruct.
Figure 18: Layer-level intervention results in Qwen3-4B-Base.
Figure 19: Layer-level intervention results in Qwen3-14B-Base.
Figure 20: Layer-level intervention results in Llama3.1-8B-Base.
Figure 21: Layer-level intervention results in Gemma3-12B-Instruct.
Figure 22: Head-level cumulative ablation in Qwen3-4B-Base.
Figure 23: Head-level cumulative ablation in Qwen3-8B-Base.
Figure 24: Head-level cumulative ablation in Qwen3-14B-Base.
Figure 25: Head-level cumulative ablation in Llama3.1-8B-Base.
Figure 26: Additional network-level analysis for Qwen3-4B-Base.
Figure 27: Additional network-level analysis for Qwen3-8B-Base.
Figure 28: Additional network-level analysis for Qwen3-14B-Base.
Figure 29: Additional network-level analysis for Llama3.1-8B-Base.
Figure 30: Additional network-level analysis for Gemma3-12B-Instruct.
Reasoning in large language models unfolds through diverse functional operations, such as problem formulation, goal decomposition, and deduction. Although these operations are explicitly distinguished in text, little is known about how they are geometrically organized in representation spaces. To this end, we investigate whether distinct reasoning operations exhibit corresponding geometric structure in hidden representations. We find that operations are separable in held-out representations, with separability peaking in middle layers, and verify that this structure is not explained by lexical or positional confounds. Across layers, token-wise operation-alignment becomes more distributed over spans, while identical surface tokens are represented differently depending on the operation of its surrounding chunk. Attention-masking interventions further show that operation-aligned representations at chunk onset depend on preceding reasoning context. Consequently, our work demonstrates that language models maintain representational correspondence between linguistic reasoning expressions and their internal geometric structures. Code and project materials are available at https://github.com/naver-ai/beneath-cot.
Transformer reasoning is limited by autoregressive decoding, which repeat edly compresses rich hidden computation through token space and makes it difficult for intermediate reasoning states to persist across time. We in troduce Transformers with Temporal Middle-Layer Recurrence (T2MLR), a transformers-based latent reasoning architecture that fuses a cached middle layer representation from the previous token directly into an earlier layer of the current token position, enabling abstract intermediate computation to persist across decoding steps with little inference overhead. Across natural-language pretraining and multi-hop reasoning finetuning, T2MLR consistently outperforms data- and parameter-matched Transformer base lines. Moreover, applying recurrence to only a localized middle-layer block (as little as 20% of the network) often outperforms full-layer recurrence. Im portantly, T2MLR does not require pretraining from scratch: retrofitting the recurrent pathway into an existing pretrained 1.7B Transformer and briefly finetuning substantially improves math reasoning, lowering the barrier to practical adoption. These results suggest that effective latent reasoning in Transformers does not require looping over all layers as in previous works, but can instead emerge more strongly from targeted middle-layer recurrence.
Ziyang Cai, Xingyu Zhu, Yihe Dong +2
Princeton Language and Intelligence, Department of Computer Science Princeton University
Recent studies have shown that Large Language Models (LLMs) can achieve strong reasoning performance by incorporating functional symbolic representations that abstractly describe graph traversal algorithms and step-by-step reasoning in few-shot learning settings. However, it remains unclear how LLMs genuinely understand the abstract meaning of each reasoning step and the overall algorithm from only a limited number of demonstrations. This work aims to localize the attention heads responsible for individual reasoning steps and characterize the types of information transferred among them. We first align constituent reasoning steps with their corresponding token logits under a symbolic-aided Chain-of-Thought (CoT) prompting framework. Our analysis shows that token positions that steer the reasoning process are associated with low confidence scores caused by constraints on satisfying reasoning behavior patterns in demonstrations. We then adopt causal mediation analysis techniques to identify the attention heads responsible for these patterns. In addition, our findings indicate that LLMs retrieve factual and rule-based information for individual sub-reasoning tasks through specialized attention heads (approximately 3% total heads), whereas higher layers predominantly facilitate information integration and the emergence of global reasoning strategies (e.g., graph traversal algorithms) that coordinate multiple intermediate reasoning steps to solve the overall task.
Phuong Minh Nguyen, Tien Huu Dang, Naoya Inoue
Japan Advanced Institute of Science and Technology