Reasoning in large language models unfolds through diverse functional operations, such as problem formulation, goal decomposition, and deduction. Although these operations are explicitly distinguished in text, little is known about how they are geometrically organized in representation spaces. To this end, we investigate whether distinct reasoning operations exhibit corresponding geometric structure in hidden representations. We find that operations are separable in held-out representations, with separability peaking in middle layers, and verify that this structure is not explained by lexical or positional confounds. Across layers, token-wise operation-alignment becomes more distributed over spans, while identical surface tokens are represented differently depending on the operation of its surrounding chunk. Attention-masking interventions further show that operation-aligned representations at chunk onset depend on preceding reasoning context. Consequently, our work demonstrates that language models maintain representational correspondence between linguistic reasoning expressions and their internal geometric structures. Code and project materials are available at https://github.com/naver-ai/beneath-cot.
Figures & tables
Figure 1: Overview of reasoning-vector analysis. Given an input problem and the model-generated reasoning trace, we probe hidden representations with reasoning operation vectors corresponding to different reasoning operations. The left panel shows token-level reasoning operation-alignment scores over the generated text, while the right panel projects span representations onto Decomposition and Recall reasoning-vector axes. Regions with strong reasoning operation-specific signals appear along the corresponding directions.
Problem-solving stage
Reasoning Operation
Example
Understanding the problem
Extraction Direct mapping
The problem says that two numbers add up to 10. x is greater than 3 →x>3
Planning the solution
Decomposition
Get the area of Heptagon → Divide it into triangles → Solve each triangle → Combine results
Area of a circle is needed → Recall A=πr2→ apply to problem x is greater than 3 and x is an integer →x≥4x+y=10→y=10−x10−3=7
Looking back and final answer
Final answer
Therefore, the values are x=3 and y=7 .
Table 1: Overview of main reasoning operations for our analyses. These eight frequent operation types are used as span labels in our representation analysis.
Figure 3: Layer-wise separability of reasoning operations across language models in middle token position. (A) Peak-layer one-vs-rest AUROC for each reasoning operation in each model, with 95% confidence intervals. (B) AUROC averaged across reasoning operations at different model depths (embedding, early, middle, and late layers), along with a random baseline. Across models and reasoning operations, separability remains high and is strongest in the middle layers.
Figure 4: Visualization of reasoning-operation clusters in the learned LDA space. For each target operation, held-out span representations from Qwen3-8B are projected onto a two-dimensional plane whose x -axis is the corresponding reasoning operation vector dc and whose y -axis is the first principal component of the residual representations. Points are colored by their annotated reasoning operation labels.
Figure 5: Quantitative analysis on intra-span LDA score variances. We investigate the LDA score variances within each span across layers using Qwen3-8B. Early-layer separability is often sparse and token-local, while mid–late-layer separability is distributed across the tokens. We further generalize the results for Qwen2.5-7B and Gemma4-31B in the Appendix.
Figure 6: Identical tokens acquire reasoning-dependent representations. For pairs of reasoning operations, we collect common surface tokens that appear in both operation contexts and visualize their layer-wise scores along the two corresponding reasoning operation vectors in Qwen3-8B . Each point is an occurrence of a shared token and is colored by its annotated reasoning label.
Model
Position-only
Text-only
Hidden(Ours)
Qwen3-8B
0.718/0.279
0.849/0.549
0.937/0.742
Qwen2.5-7B
0.708/0.269
0.854/0.562
0.895/0.646
Gemma4-31B
0.751/0.311
0.802/0.466
0.899/0.641
Table 2: Macro AUROC/AUPRC of position-only logistic regression, the stronger of bag-of-words and TF–IDF logistic regression, and mean-pooled hidden-state probes. Position and lexical content are informative, but hidden-state probes achieve higher aggregate performance across all three models.
Figure 7: Preceding context shapes operation representations. Masking attention to preceding context reduces target reasoning operation-alignment score, indicating that subsequent reasoning operations depend on prior reasoning context.
Appendix figures & tables26 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A: Layer-by-token heatmaps of reasoning-operation alignment scores. We visualize token-level alignment scores with the corresponding reasoning-operation vector across layers for representative reasoning traces. Each heatmap shows how strongly each generated token aligns with a target reasoning operation vector at each layer. The examples illustrate that operation-specific scores are often localized or weak in early layers, become more coherent over contiguous token spans in middle layers, and may become less sharply localized in later layers. This supports the view that reasoning-operation representations emerge through contextual processing rather than being attached only to isolated cue tokens.
Figure B: Additional layer-by-token heatmaps of reasoning-operation alignment scores. We provide additional representative examples for other reasoning-operation types. Consistent with Figure A , the alignment scores become more structured across contiguous generated tokens in middle layers, while early-layer scores are more sparse and token-local. These qualitative patterns are consistent with the span-level separability and intra-span variance analyses.
Figure C: Quantitative analysis on intra-span LDA score variances across additional models. We report the intra-span LDA score variances across layers for Qwen2.5-7B and Gemma4-31B. Consistent with Qwen3-8B, variance is high in early layers and decreases toward middle layers, indicating a shift from token-local cues to operation signals distributed across the span.
Reasoning operation
Pr(δonset<0)
Mean δonset
Median δonset
One-sided p -value
Extraction
0.596
−3.418
−1.0
7.16×10−15
Direct Mapping
0.391
−3.793
0.0
1.90×10−6
Decomposition
0.531
−2.344
−1.0
0.0173
Recall
0.406
−1.048
0.0
7.11×10−6
Deduction
0.441
−2.441
0.0
0.0181
Algebraic Manipulation
0.601
−4.017
−1.0
1.23×10−5
Appendix
Table A: Exploratory onset-to-onset temporal-ordering analysis. For each operation, δonset=tlate−tearly , where the two onsets are detected using the same causal three-token rolling-mean procedure and regime-specific thresholds. Negative values indicate that the sustained late-layer signal is detected before the sustained early-layer signal under this criterion. The fraction column reports Pr(δonset<0) . The reported p -values are from uncorrected one-sided Wilcoxon signed-rank tests for a negative location shift. A significant p -value does not necessarily indicate that negative onset differences occur in a majority of examples.
Figure D: Additional attention-masking intervention results. We report context-intervention results beyond the pre-token setting shown in the main text. In the pre-chunk setting, we mask attention from the first token of a target reasoning operation chunk to the entire immediately preceding annotated reasoning operation chunk. In the random-control setting, we mask a randomly selected preceding block with a comparable length. Across settings, blocking preceding context generally reduces the target reasoning operation-alignment score, indicating that subsequent reasoning-operation representations depend on prior context. The reduction is stronger for structured preceding-context masking than for the random control, supporting the view that relevant preceding reasoning context contributes to the formation of the next operation representation.
Operation
AUROC-M
AUROC-Mid
AUPRC-M
AUPRC-Mid
Extraction
0.959
0.903
0.880
0.711
Direct Mapping
0.936
0.890
0.821
0.721
Decomposition
0.987
0.943
0.852
0.659
Recall
0.974
0.918
0.857
0.785
Deduction
0.869
0.766
0.652
0.470
Algebraic Manipulation
0.985
0.926
0.907
0.739
Appendix
Table B: Llama-3-8B replication. M and Mid denote mean-pooled and middle-token span representations.
GPQA-Diamond
MATH-500
Operation
AUROC-M
AUROC-Mid
AUPRC-M
AUPRC-Mid
AUROC-M
AUROC-Mid
AUPRC-M
AUPRC-Mid
Extraction
0.983
0.967
0.925
0.861
0.986
0.983
0.983
0.960
Direct Mapping
0.907
0.835
0.699
0.506
0.902
0.837
0.732
0.455
Decomposition
0.951
0.871
0.859
0.650
0.970
0.881
0.862
0.615
Recall
0.903
0.844
0.675
0.538
0.954
0.902
0.845
0.707
Deduction
0.902
0.846
0.585
0.427
0.921
0.850
0.646
0.518
Appendix
Table C: Test-only transfer of Qwen3-8B probes to GPQA-Diamond and MATH-500 without task-specific probe retraining.
Operation
Error AUROC
Non-error AUROC
pAUROC
Error AUPRC
Non-error AUPRC
pAUPRC
(a) Middle-token representation
Recall
0.946
0.949
0.446
0.862
0.857
0.695
Deduction
0.867
0.906
<0.001
0.592
0.755
<0.001
Algebraic Manipulation
0.897
0.915
<0.001
0.632
0.630
0.607
Arithmetic Computation
0.926
0.957
<0.001
0.784
0.866
<0.001
Final Answer
0.964
0.960
0.773
0.924
0.932
0.406
Appendix
Table D: Transfer of reasoning-operation probes trained exclusively on correct reasoning traces to factual-error and operation-matched non-error spans from incorrect Qwen3-8B reasoning traces. Results are averaged across layers 10–30. The error and non-error groups contain equal numbers of spans for each operation. Reported p -values are from one-sided paired Wilcoxon signed-rank tests of whether separability is higher for non-error spans than for factual-error spans. Values smaller than 0.001 are reported as <0.001 .
Mean pooling
Middle token
Model
8 PCs
32 PCs
128 PCs
8 PCs
32 PCs
128 PCs
Qwen3-8B
0.908/0.605
0.934/0.699
0.938/0.716
0.807/0.434
0.861/0.549
0.873/0.583
Qwen2.5-7B
0.886/0.622
0.909/0.683
0.914/0.700
0.794/0.439
0.849/0.563
0.862/0.601
Gemma4-31B
0.846/0.479
0.867/0.538
0.872/0.552
0.744/0.336
0.783/0.399
0.793/0.424
Appendix
Table E: Macro AUROC/AUPRC of PCA-only evaluation. Operation separability remains above chance without supervised LDA across all models, component counts, and span representations.
Stage
Reasoning operation
Qwen2.5-7B
Qwen3-8B
Gemma4-31B
Stage 1: Understanding the Problem
Extraction
590
1,035
682
Direct-mapping
624
913
1,289
Reframing
146
390
393
Abstraction
8
35
36
State-space-definition
190
315
259
Stage 2: Planning the Solution
Pattern-matching
225
322
389
Appendix
Table F: Train + Test occurrence counts for each reasoning operation and model.
Model
Position
Hidden
Δ
Qwen3-8B
0.718/0.279
0.937/0.742
+0.218/+0.463
Qwen2.5-7B
0.708/0.269
0.895/0.646
+0.187/+0.377
Gemma4-31B
0.751/0.311
0.899/0.641
+0.148/+0.330
Appendix
Table G: Macro AUROC/AUPRC of position-only and mean-pooled hidden-state predictors. Δ denotes hidden-state minus position-only performance.
Position
Labels
AUROC/AUPRC
Chance AUPRC
[0.0,0.2)
7
0.973/0.898
0.143
[0.2,0.4)
5
0.930/0.808
0.200
[0.4,0.6)
5
0.947/0.836
0.200
[0.6,0.8)
5
0.916/0.769
0.200
[0.8,1.0]
5
0.927/0.798
0.200
Appendix
Table H: Position-stratified Qwen3-8B evaluation using mean-pooled representations. Macro averages include operations with sufficient examples in each interval.
Model
Text
Hidden
Δ
Qwen3-8B
0.849/0.549
0.937/0.742
+0.088/+0.193
Qwen2.5-7B
0.854/0.562
0.895/0.646
+0.041/+0.084
Gemma4-31B
0.802/0.466
0.899/0.641
+0.097/+0.175
Appendix
Table I: Macro AUROC/AUPRC of the stronger text-only baseline among TF-IDF and BoW and the mean-pooled hidden-state probe.
Operation
Median Δ
Δ>0
95% CI
Extraction
4.68
87.5%
[3.13,6.45]
Direct Mapping
0.57
75.0%
[0.03,1.53]
Decomposition
3.91
100.0%
[3.63,4.41]
Recall
1.05
70.0%
[−0.46,2.79]
Deduction
0.61
76.9%
[0.13,1.41]
Algebraic Manipulation
0.68
60.7%
[−0.26,1.27]
Appendix
Table J: Lexically matched pair analysis. Positive effects indicate that the frozen probe is more strongly aligned with operation identity than with broad lexical similarity. Confidence intervals are computed across anchor-level effects.
Qwen3-8B
Qwen2.5-7B
Operation
AUROC
AUPRC
AUROC
AUPRC
Extraction
0.997
0.992
0.957
0.904
Direct Mapping
0.888
0.811
0.753
0.499
Decomposition
0.899
0.806
0.873
0.838
Recall
0.923
0.893
0.878
0.664
Deduction
0.835
0.653
0.862
0.744
Appendix
Table K: Frozen-probe performance on held-out spans containing vocabulary associated with competing operation labels.
Operation
Chance AUPRC
Mean pooling
Middle token
AUROC
AUPRC
AUROC/AUPRC
Direct Mapping
0.204
0.994
0.979
0.925/0.808
Deduction
0.184
0.866
0.701
0.833/0.574
Algebraic Manipulation
0.204
0.912
0.755
0.811/0.498
Arithmetic Computation
0.204
0.816
0.510
0.780/0.458
Final Answer
0.204
0.999
0.998
0.969/0.924
Appendix
Table L: Evaluation on spans with 50–75% digit or mathematical-token density. The theoretical chance AUROC is 0.500 for all rows.
Reasoning operation
Mean AUROC [95% CI]
Mean AUPRC [95% CI]
Middle AUROC [95% CI]
Middle AUPRC [95% CI]
Qwen3-8B (layers 10–30)
Extraction
0.998 [0.995, 1.000]
0.988 [0.975, 0.998]
0.995 [0.991, 0.998]
0.970 [0.943, 0.989]
Direct mapping
0.947 [0.921, 0.969]
0.802 [0.724, 0.873]
0.875 [0.835, 0.911]
0.587 [0.490, 0.683]
Decomposition
0.980 [0.961, 0.994]
0.895 [0.824, 0.954]
0.926 [0.890, 0.956]
0.636 [0.514, 0.750]
Recall
0.965 [0.939, 0.986]
0.847 [0.761, 0.925]
0.917 [0.875, 0.952]
0.725 [0.640, 0.809]
Deduction
0.927 [0.898, 0.953]
0.648 [0.545, 0.753]
0.883 [0.848, 0.916]
0.531 [0.437, 0.640]
Appendix
Table M: Bootstrap confidence intervals for reasoning-operation separability. We report layer-averaged AUROC and AUPRC with 95% confidence intervals for mean-pooled and middle-token representations. Intervals are computed from 5,000 stratified bootstrap replicates.
Category
Metric
Result
Samples
Validation spans
84
Human agreement
Unanimous
50/84 (59.5%)
Majority label
81/84 (96.4%)
Fleiss’ κ
0.666
Reference labels
Determined
81/84 (96.4%)
Not determined
3/84 (3.6%)
Appendix
Table N: Human validation of GPT-5 operation-span annotations. Each span was independently labeled by three of seven human annotators.
Figure E: quantitative analyses on separability of reasoning operations in representation spaces at position middel. AUROC and AUPRC of one-vs-rest operation classifiers are shown across layers for Qwen3-8B , Qwen2.5-7B , and Gemma4-31B using middle-token span representations. Solid lines denote true reasoning-operation labels, while dashed lines denote random-label and random-position baselines. Across models, reasoning operation-alignment scores remain substantially above the baselines and peak in middle layers.
Figure F: Quantitative analyses on separability of reasoning operations in representation spaces at position mean. AUROC and AUPRC of one-vs-rest operation classifiers are shown across layers for Qwen3-8B , Qwen2.5-7B , and Gemma4-31B . Solid lines denote true reasoning-operation labels, while dashed lines denote random-label and random-position baselines. Across models, performance remains above the baselines and typically peaks in middle layers.
Figure G: Quantitative analyses on separability of reasoning operations in representation spaces at position first. AUROC and AUPRC of one-vs-rest operation classifiers are shown across layers for Qwen3-8B , Qwen2.5-7B , and Gemma4-31B . Solid lines denote true reasoning-operation labels, while dashed lines denote random-label and random-position baselines. Across models, performance remains above the baselines and typically peaks in middle layers.
Figure H: Quantitative analyses on separability of reasoning operations in representation spaces at position last. AUROC and AUPRC of one-vs-rest operation classifiers are shown across layers for Qwen3-8B , Qwen2.5-7B , and Gemma4-31B . Solid lines denote true reasoning-operation labels, while dashed lines denote random-label and random-position baselines. Across models, performance remains above the baselines and typically peaks in middle layers.
Problem-solving stage
Level 1
Level 2
Level 3
Reasoning operation
Description / Example
Stage 1 Understanding the Problem
Extraction
–
–
Extraction
Directly retrieving given information from the problem statement without modification. Example: “Two numbers add up to 10, and both are non-negative.”
Symbolization
Direct-mapping
–
Direct-mapping
Directly converting a statement into a formal representation without changing its structure. Example: The sum of two numbers is 10 →x+y=10 .
Reframing
–
Reframing
Reconstructing the problem in a different representational system. Example: Geometry problem → coordinate geometry.
Abstraction
–
Abstraction
Removing unnecessary details and retaining only the essential structure. Example: 3 red balls and 2 blue balls →(3,2) .
State-space-definition
–
–
State-space-definition
Defining possible states, constraints, and the solution space. Example: x≥0 , y≥0 , x+y=10 .
Stage 2 Planning the Solution
Pattern-matching
–
–
Pattern-matching
Mapping the current problem to a known problem type, template, or strategy. Example: Flip a coin three times → probability/combinatorics problem.
Appendix
Table O: Reasoning operation schema (Stages 1–2).
Problem-solving stage
Level 1
Level 2
Level 3
Reasoning operation
Description / Example
Stage 3 Carrying Out the Plan
Retrieval
Recall
–
Recall
Retrieving relevant definitions, rules, concepts, or formulas. Example: Need area of a circle →A=πr2 .
Matching
–
Matching
Determining whether retrieved knowledge applies to the current situation. Example: Apply A=πr2 when the radius is given.
Inference-flow
Chaining
Deduction
Deduction
Deriving a specific conclusion from a general rule. Example: General theorem → apply to the given case.
Induction
Induction
Inferring a general rule from observed patterns. Example: Several terms follow the same difference → infer arithmetic pattern.
Analogy
Analogy
Reasoning from a structurally similar case. Example: Solve a new problem using a similar solved template.
Branching
–
Branching
Splitting into cases and analyzing each separately. Example: x2=4→x=2 or x=−2 .
Appendix
Table P: Reasoning operation schema (Stage 3).
Problem-solving stage
Level 1
Level 2
Level 3
Reasoning operation
Description / Example
Stage 4 Looking Back and Final Answer
Strategy-validation
–
–
Strategy-validation
Verifying whether the chosen strategy or assumptions were valid. Example: Check whether differences are constant before assuming an arithmetic sequence.
Error-detection
–
–
Error-detection
Identifying mistakes in reasoning, calculation, or logical steps. Example: 10−3=6→ incorrect arithmetic; should be 7.
Dimensional-analysis
–
–
Dimensional-analysis
Checking whether units or dimensions are consistent with the required quantity. Example: Speed = distance/time → meters/second.
Extreme-case-testing
–
–
Extreme-case-testing
Testing whether the solution holds under extreme or boundary conditions. Example: For f(x)=x/(x+1) , test x=0 and x→∞ .
Final-answer
–
–
Final-answer
Clearly presenting the final result in the required format. Example: Therefore, x=3 and y=7 .
Appendix
Table Q: Reasoning operation schema (Stage 4).
Item
Instructions given to annotators
Task objective
Determine the role played by the highlighted text in the reasoning process. Annotators should identify the function of the highlighted span within the overall solution, rather than solve the problem themselves or verify whether the final answer is correct.
Question
The original problem to which the model-generated reasoning responds.
Response with highlighted span
The model-generated reasoning trace. Only the highlighted portion ( full span ) is the annotation target. The surrounding response is provided solely to understand why the highlighted span appears at that point in the reasoning process.
shortened_span_text
A shorter phrase extracted from the full span when the full span is long. When the full span is already short, the shortened span is identical to the full span.
Answer columns
For each example, annotators provide three judgments: (1) the reasoning-operation label that best characterizes the full span; (2) their confidence in that label; and (3) whether the shortened_span_text preserves the main reasoning operation expressed by the full span.
Disabled cells
Black cells are intentionally disabled and should be left untouched; annotators should not type or make any selection in these cells.
Appendix
Table R: Summary of the human annotation procedure and interface. The instructions were originally provided to annotators in Korean.
Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.
Chain-of-thought traces from large reasoning models can span tens of thousands of tokens, yet we lack a vocabulary for describing their internal structure. Previous methods developed to analyze chain-of-thought traces are either too rigid or not expressive enough, failing to capture features across domains and models. To remedy this, we develop ReasonOps, an unsupervised, expressive method for annotating chain-of-thought traces, providing succinct universal operators. Using ReasonOps, we analyze 44,662 traces from 12 thinking LLMs spanning 6 families across 8 reasoning benchmarks and discover that they share a common compositional structure: 7 recurring reasoning operators -- discourse-level moves such as backtracking, inferring, and hypothesizing -- that emerge from unsupervised clustering of sentence-initial 3-token pivots. These operators appear across every model family and benchmark domain, confirmed by three independent LLM judges who classify held-out samples at 70 -76% accuracy. We analyze the structure of operators on easy vs. hard problems, revealing that reflective operators are more helpful on hard problems and harm performance on easy problems. Operator sequences are highly model-identifying: a classifier trained on operator distributions alone recovers the source model with macro-AUC, revealing that each model family has a distinctive reasoning fingerprint. Structural operator features predict within-problem answer correctness well above baselines. Classifiers built on these operators reach WP-AUC and on AIME specifically. ReasonOps further enables early quality estimation well before the trace completes: we predict at WP-AUC for only 50% of the trace. The ReasonOps pipeline is unsupervised and annotation-free, enabling deep insights into LLM reasoning traces as well as strong downstream results on model identification and correctness prediction.
Chain-of-Thought (CoT) prompting significantly improves reasoning in Large Language Models, yet the temporal dynamics of the underlying representation geometry remain poorly understood. We investigate these dynamics by applying Manifold Capacity Theory (MCT) to two compositional reasoning tasks: a controlled Boolean logic tree that supports deep mechanistic analysis, and a natural-language eligibility task in which the model has to extract attributes from prose, compare them to thresholds, and compose the local decisions through a fixed evaluation tree. MCT lets us quantify the linear separability of latent representations without the confounding factors of probe training. On both tasks, and across several open-weight models, reasoning manifests as a transient geometric pulse: concept manifolds are untangled into linearly separable subspaces immediately prior to computation and rapidly compressed thereafter. This behavior diverges from standard linear probe accuracy, which remains high long after computation, suggesting a fundamental distinction between information that is merely retrievable and information that is geometrically prepared for processing. We interpret this phenomenon as Dynamic Manifold Management, a mechanism where the model dynamically modulates representational capacity to optimize the bandwidth of the residual stream throughout the reasoning chain.
Chanwoo Chun, Alexandre Polo, SueYeon Chung
Department of Physics, Harvard University · Center for Data Science, New York University · Kempner Institute, Harvard University +1
Recent studies have shown that Large Language Models (LLMs) can achieve strong reasoning performance by incorporating functional symbolic representations that abstractly describe graph traversal algorithms and step-by-step reasoning in few-shot learning settings. However, it remains unclear how LLMs genuinely understand the abstract meaning of each reasoning step and the overall algorithm from only a limited number of demonstrations. This work aims to localize the attention heads responsible for individual reasoning steps and characterize the types of information transferred among them. We first align constituent reasoning steps with their corresponding token logits under a symbolic-aided Chain-of-Thought (CoT) prompting framework. Our analysis shows that token positions that steer the reasoning process are associated with low confidence scores caused by constraints on satisfying reasoning behavior patterns in demonstrations. We then adopt causal mediation analysis techniques to identify the attention heads responsible for these patterns. In addition, our findings indicate that LLMs retrieve factual and rule-based information for individual sub-reasoning tasks through specialized attention heads (approximately 3% total heads), whereas higher layers predominantly facilitate information integration and the emergence of global reasoning strategies (e.g., graph traversal algorithms) that coordinate multiple intermediate reasoning steps to solve the overall task.
Phuong Minh Nguyen, Tien Huu Dang, Naoya Inoue
Japan Advanced Institute of Science and Technology