Organizations: Department of Physics, Harvard University · Center for Data Science, New York University · Kempner Institute, Harvard University · Center for Computational Neuroscience, Flatiron Institute
Chain-of-Thought (CoT) prompting significantly improves reasoning in Large Language Models, yet the temporal dynamics of the underlying representation geometry remain poorly understood. We investigate these dynamics by applying Manifold Capacity Theory (MCT) to two compositional reasoning tasks: a controlled Boolean logic tree that supports deep mechanistic analysis, and a natural-language eligibility task in which the model has to extract attributes from prose, compare them to thresholds, and compose the local decisions through a fixed evaluation tree. MCT lets us quantify the linear separability of latent representations without the confounding factors of probe training. On both tasks, and across several open-weight models, reasoning manifests as a transient geometric pulse: concept manifolds are untangled into linearly separable subspaces immediately prior to computation and rapidly compressed thereafter. This behavior diverges from standard linear probe accuracy, which remains high long after computation, suggesting a fundamental distinction between information that is merely retrievable and information that is geometrically prepared for processing. We interpret this phenomenon as Dynamic Manifold Management, a mechanism where the model dynamically modulates representational capacity to optimize the bandwidth of the residual stream throughout the reasoning chain.
Figures & tables
Figure 1: Manifold Untangling during Chain-of-Thought. Before any token is generated (left) the latent representations of tasks corresponding to answers A and B are entangled and hard to distinguish resulting in low capacity. As the model generates the CoT, it progressively untangles these representations. By the final token (right), the two manifolds are linearly separable and capacity is high.
Figure 2: a. Example of a Boolean logic tree with height h=2 . Left : The corresponding text input provided to the model, where internal nodes are labeled [01] through [03]. Right : Schematic of the tree structure. b. Accuracy on Boolean logic trees of varying height. Performance with CoT (purple) remains near-perfect ( >98% ), while standard prompting (No CoT, orange) degrades significantly as tree depth increases ( 59% for h=5 ). The dashed line marks the random baseline (50%).
Figure 3: Dynamic Modulation of Manifold Geometry during CoT. a. Manifold capacity ( α ) tracked across the Chain-of-Thought sequence (layer 20, tree height h=4 ; height-5 traces in Figure S.17 ). Lines are colored by node ID. Capacity for a specific node peaks sharply at two distinct moments: first when the node is intrinsically computed, and second when it is processed by its parent. b. Detailed analysis of Node 11 (yellow) and its children, Node 5 (blue) and Node 6 (turquoise). Insets visualize the latent geometry at key timesteps. High capacity peaks correspond to linearly separable manifolds (clear decision boundaries). Notably, representations become entangled (low capacity) in the interim periods between the initial solution (solve) and the subsequent retrieval (recall). c. Intrinsic dimensionality estimates of the latent embeddings using Two-Nearest Neighbors (TwoNN) and Participation Ratio (PR).
Figure 4: Aligned manifold capacity dynamics on both tasks. Data is aligned to the moment a node is computed (left column: a , c ) or when it is recalled by its parent (right column: b , d ). ( a , b ) Boolean task (Ministral, layer 20): node-specific capacity shows a sharp, transient peak at the moment of computation and again at recall, dropping quickly to baseline in between. ( c , d ) Eligibility task (Ministral, layer 20): the same qualitative pulse appears on the natural-language task. Vertical dashed lines mark the positions of structural anchors within each phase. The corresponding capacity vs. trained-probe accuracy comparison on the Boolean task is shown in Appendix F .
Figure 5: Spatiotemporal heatmap of manifold capacity change. The x-axis represents token positions relative to the computation step ( t=0 ). ( During Computation, t=0,1 ) A capacity increase initiates in the middle layers at t=0 , culminating in a sharp peak within the deep layers at the token preceding the result ( t=+2 ). ( Post-Computation ) For t>+2 , the early-to-mid layers retain elevated capacity as they retain the immediate context; however, this capacity decreases in the middle layers as the model reorganizes its representation to shift focus to the next node.
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
Task
Model
Prompt
Formatting
All nodes
Final
Perfect
Boolean
Qwen2.5-14B-Instruct
strict-format
256/256
242/256
248/256
242/256
Boolean
gpt-oss-20b
strict-format
192/256
170/256
200/256
153/256
Boolean
Ministral 3 8B Reasoning
standard CoT
256/256
251/256
254/256
251/256
Eligibility
Ministral 3 8B Reasoning
standard
256/256
256/256
256/256
256/256
Eligibility
Ministral 3 8B Reasoning
silenced group-logic
256/256
256/256
256/256
256/256
Eligibility
Ministral 3 8B Reasoning
unstructured
256/256
197/256
213/256
171/256
Appendix
Table S.1: Full-run behavioral accuracy ( n=256 examples per row) for the runs used in the reported analyses. “All nodes” means every reported node, criterion, or group label is correct. “Perfect” additionally requires strict formatting for the Boolean task, and for unstructured eligibility also requires the final-answer and violated-criteria lines to be correct.
Figure S.1 : Boolean task on Qwen2.5-14B-Instruct (layer 26). Left : full capacity traces over the CoT, colored by node ID. Right : solve- and recall-aligned capacity. The pulse and recall pattern is preserved.
Figure S.2 : Boolean task on gpt-oss-20b (layer 13). Same conventions as Figure S.1 .
Figure S.3 : Eligibility task, standard prompt, Ministral (layer 20): full traces. Capacity for each node ID across the full reasoning trace. The aligned solve and recall views are shown in panels c and d of Figure 4 .
Figure S.4 : Eligibility task, silenced group-logic prompt, Ministral (layer 20). The * Logic: lines for G1 , G2 , and Final are masked, so the parent step cannot explicitly mention the child labels. The recall-time capacity rise of the children persists, supporting the conclusion in Section 5.5 .
Figure S.5 : Eligibility task, unstructured prompt, Ministral (layer 20). The model writes free natural-language reasoning lines with only lightweight ID prefixes ( C1: , …, Final: ). Anchors are within-line word-end positions. The pulse and recall pattern is preserved despite the absence of the rigid Markdown template.
Figure S.6 : Eligibility task, unstructured prompt, Qwen2.5-7B-Instruct (layer 20). Same conventions as Figure S.5 . The pulse and recall pattern is also present on this model.
Figure S.7 : Manifold capacity vs. hard-margin SVM probe accuracy (Ministral, Boolean task, layer 20). (a, b) Capacity, aligned to solve and recall (matching panels a and b of Figure 4 ). (c, d) Hard-margin SVM probe test accuracy at the same anchors: while accuracy also peaks at computation, it exhibits a sustained plateau and decays much more slowly than capacity. A Logistic Regression probe shows the same qualitative behavior ( Figure S.9 ).
Figure S.8 : Spatiotemporal heatmap of Logistic Regression probe test accuracy. (Layer 20) The x-axis represents token positions relative to the computation step ( t=0 ). ( During Computation, t=0,1 ) The test accuracy increase initiates in the middle layers at t=0 , culminating in a value close to 1 across layers at the token preceding the result ( t=+2 ). ( Post-Computation ) For t>+2 , the early-to-mid layers retain close to 1 accuracy as they retain the immediate context; the accuracy decreases in the middle layers of the model but not very markedly. Only at the subsequent node ( t>+5 ) is the decrease more pronounced.
Figure S.9 : Spatiotemporal heatmap of Test Accuracy. Left : Hard-Margin Support Vector Machine. Right : Logistic Probe. The x-axis represents token positions relative to the computation step ( t=0 ). ( During Computation, t=0,1 ) The test accuracy increase initiates in the middle layers at t=0 , culminating in a value close to 1 across layers at the token preceding the result ( t=+2 ). ( Post-Computation ) For t>+2 , the early-to-mid layers retain close to 1 accuracy as they retain the immediate context; the accuracy decreases in the middle layers of the model but not very markedly. Only at the subsequent node ( t>+5 ) is the decrease more pronounced.
Silent CoT example response excerpt
…
Node [19]
* Logic: ‘[05] and [06]‘
* Result: ‘True‘
Node [20]
* Logic: ‘[07] xor [08]‘
Appendix
Table S.2: Excerpt of a Silent CoT response: child-node truth values are not explicitly written in the parent’s Logic line.
Figure S.10 : Comparison of Capacity dynamics between normal and silenced CoT. (Left) Data is aligned to the moment a node is computed. There is no statistically significant difference between the two prompts, which is expected as the only difference in the prompts is in the reference to the children. (Right) Data is aligned to the moment a node is recalled by its parent. There is no significant difference between the two prompts, which confirms the linear separability of the children is functionally necessary to compute the parent’s value and does not merely reflect that the children are being recalled.
Masked-logic CoT example response excerpt
…
Node [19]
* Logic: ‘***************‘
* Result: ‘True‘
Node [20]
* Logic: ‘***************‘
Appendix
Table S.3: Excerpt of a masked-logic response: the entire Logic expression is replaced by 15 asterisks of equal token length.
Figure S.11 : The masked logic case. Capacity values for each node grouped by the logic tree depth. Left: the node capacity traces aligned at the solve phases of the corresponding nodes. Right: the node capacity traces aligned at the recall phases of the parent nodes (the main focus of this figure).
Figure S.12 : The masked logic case. Left: node capacity at its solve phase vs. corresponding node accuracy. Right: child node capacity at its recall phase vs. its parent node accuracy.
Figure S.13 : Attention score heatmap between graph nodes for the masked logic case. The x-axis represents the target node currently being solved; the y-axis represents the source nodes in the context. The diagonal structure reflects the model attending to recent context, while off-diagonal points represent the retrieval of specific dependencies.
Figure S.14 : Cosine Similarity of Hyperplane Directions between Nodes when They are Solved or Recalled. Cosine similarity between the max-margin hyperplanes computed for nodes’ ground-truth answers at the * Logic: tokens (for the top row; * Result: tokens for the bottom row) in their solve phase (for the left column; recall phase for the middle column). For the recall-phase similarity, we also show the same plot with the left and right child nodes grouped, revealing the block structure (the right column).
Figure S.15 : Cosine Similarity of Each Node when It is Solved and Recalled. Cosine similarity between the max-margin hyperplanes computed for a single node’s ground truth answer when it is being solved (three structural tokens) and recalled (another three structural tokens).
Figure S.16 : Correlation between Attention and Manifold Capacity (Boolean task, Ministral). a. Attention score heatmap between graph nodes. The x-axis represents the target node currently being solved; the y-axis represents the source nodes in the context. The diagonal structure reflects the model attending to recent context, while off-diagonal points represent the retrieval of specific dependencies. b. Scatter plot of attention score versus manifold capacity of the source node at the moment of attention. We observe a strong positive correlation ( r=0.723 ), specifically for direct children nodes (blue), indicating that the model preferentially attends to previous information when its representation is linearly separable.
Figure S.17 : Dynamic Modulation of Manifold Geometry during CoT. a. Manifold capacity ( α ) tracked across the Chain-of-Thought sequence (layer 20, tree height h=4 ). Lines are colored by node ID. Capacity for a specific node peaks sharply at two distinct moments: first when the node is intrinsically computed, and second when it is processed by its parent. b. Detailed analysis of Node 11 (yellow) and its children, Node 5 (blue) and Node 6 (turquoise). High capacity peaks correspond to linearly separable manifolds (clear decision boundaries). Notably, representations become entangled (low capacity) in the interim periods between the initial solution (solve) and the subsequent retrieval (recall). c, d. Same views for a tree of height h=5 .
Figure S.18 : Evolution of Linear Probe Test Accuracy during CoT. a. Logistic Probe Test Accuracy across the Chain-of-Thought sequence (layer 20, tree height h=5 ). Lines are colored by node ID. b. Detailed analysis of Node 11 (yellow) and its children, Node 5 (blue) and Node 6 (turquoise). Accuracy peaks when each node is being solved and when it is being recalled. However, unlike manifold capacity (see panels c and d of Figure S.17 ), accuracy plateaus for several nodes. c, d. Same views for hard-margin SVM probe test accuracy.
Chain-of-Thought (CoT) prompting often improves the reasoning performance of large language models (LLMs), but the internal signal that triggers this behavior remains poorly understood. Leveraging the sparse features captured by Sparse Autoencoders (SAEs), we propose a systematic framework to analyze and intervene on the internal representations of LLMs, identifying a small set of latent features that are linked to reasoning behavior and can be causally tested through targeted intervention. Across multiple model families and reasoning benchmarks, we show that steering one or a small number of reasoning-related latent features can substantially induce reasoning behavior without explicit CoT prompting, achieving accuracy comparable to CoT. We further show that the identified features are not tied to particular wording patterns or verbosity, and confirm their causal role in reasoning through suppression experiments that impair performance even under CoT prompting. These results suggest that CoT prompting activates specific latent features to trigger reasoning, and that targeted intervention on these features offers an alternative pathway to elicit efficient reasoning behavior without explicit CoT prompting. Code is available at https://github.com/Zhenghao-He/LatentCoT.
Chain-of-thought (CoT) reasoning is the dominant paradigm for inference-time scaling in language models, yet the causal influence of individual steps on the final answer remains poorly understood. In this work, we use answer logits at the end of each reasoning step to estimate each step's causal importance to the final answer and intermediate guesses, shedding light on the answer formation process of several reasoning model families. Across diverse tasks, we find that reasoning typically crosses a commitment boundary, a sharp transition from transient intermediate guesses to a stable, high-confidence answer. This transition often happens in a single step, well before the model's reasoning block ends, and is followed by epiphenomenal CoT steps that leave the final answer probability unaltered. Using attention probes, we show that answer-formation stages can be linearly decoded from the activations of intermediate reasoning steps with high accuracy, showing robust generalization to unseen reasoning tasks. We leverage this property for early-exiting reasoning blocks at the commitment boundary location, reducing the length of CoTs up to 55% with negligible impact on model performance.
Daniel Scalena, Sara Candussio, Luca Bortolussi +3
CLCG, University of Groningen · University of Milano-Bicocca · University of Trieste +1
Reasoning in large language models unfolds through diverse functional operations, such as problem formulation, goal decomposition, and deduction. Although these operations are explicitly distinguished in text, little is known about how they are geometrically organized in representation spaces. To this end, we investigate whether distinct reasoning operations exhibit corresponding geometric structure in hidden representations. We find that operations are separable in held-out representations, with separability peaking in middle layers, and verify that this structure is not explained by lexical or positional confounds. Across layers, token-wise operation-alignment becomes more distributed over spans, while identical surface tokens are represented differently depending on the operation of its surrounding chunk. Attention-masking interventions further show that operation-aligned representations at chunk onset depend on preceding reasoning context. Consequently, our work demonstrates that language models maintain representational correspondence between linguistic reasoning expressions and their internal geometric structures. Code and project materials are available at https://github.com/naver-ai/beneath-cot.