Existing research indicates that the output of Chain-of-Thought (CoT) is significantly affected by input perturbations. Although many methods aim to mitigate such impact by optimizing prompts, a theoretical explanation of how these perturbations influence CoT outputs remains an open area of research. This gap limits our in-depth understanding of how input perturbations propagate during the reasoning process and hinders further improvements in prompt optimization methods. Therefore, in this paper, we theoretically analyze the effect of input perturbations on the fluctuation of CoT outputs. We first derive an upper bound for input perturbations under the condition that the output fluctuation is within an acceptable range, based on which we prove that: (i) This upper bound is positively correlated with the number of reasoning steps in the CoT; (ii) Even an infinitely long reasoning process cannot eliminate the impact of input perturbations. We then apply these conclusions to the Linear Self-Attention (LSA) model, which can be viewed as a simplified version of the Transformer. For the LSA model, we prove that the upper bound for input perturbation is negatively correlated with the norms of the input embedding and hidden state vectors. To validate this theoretical analysis, we conduct experiments on three mainstream datasets and four mainstream models. The experimental results align with our theoretical analysis, empirically demonstrating the correctness of our findings.
Figures & tables
Finding
Evidence
Experiment
More reasoning steps can reduce the effect of input perturbations.
Theorem 1
§ 4.3
The effect of input perturbations cannot be entirely eliminated by continuously increasing the number of CoT reasoning steps.
Equation 4
§ 4.3
CoT robustness is negatively correlated with the norms of the input embedding and hidden state vectors.
Theorem 2
§ 4.4
Table 1: The main findings and corresponding evidence and experiment of this paper.
Model
MATH
MMLU-Pro
GPQA
EM
OF
EM
OF
EM
OF
Llama2-7b
14.2±5.0
0.475
11.2±5.7
0.622
17.5±4.7
0.509
Llama3.1-8b
45.8±7.2
0.366
41.0±10.7
0.350
26.6±5.7
0.467
Llama-R1-8b
64.8±3.0
0.158
44.8±8.3
0.292
28.5±2.9
0.371
Qwen3-8b
77.2±1.6
0.097
46.9±5.2
0.162
37.3±1.9
0.214
Table 2: Average exact match (EM) and output fluctuation (OF) of different models using various prompts on different datasets. The highest EM and lowest OF under each setting is marked in bold .
Figure 1: The output fluctuation across input perturbation on all datasets and models. Each point denotes one question, where X-axis denotes the input perturbation as the average distance of embedding vectors from their mean vector, and Y-axis denotes OF. The curves at X and Y axes illustrate the data distribution. The Pearson correlation coefficient is 0.619 .
Figure 4Figure 5
Model
Method
MATH
MMLU-Pro
GPQA
Llama3.1-8b
-
46.8
45.7
23.7
TextGrad
45.2
47.4
27.6
OPRO
44.6
47.1
27.1
CFPO
47.0
48.1
27.6
Ours
47.2
49.0
32.3
Qwen3-8b
-
77.4
42.3
37.1
Table 3: EM on each model and dataset using different prompt optimization methods. “-” denotes using the single base prompt directly. The best result under each setting is marked in bold .
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Prompt of MATH
Solve the following math problem efficiently and clearly.
Regardless of the approach, always conclude with:
Therefore, the final answer is: boxedanswer. I hope it is correct.
Appendix
Table 4: The prompt used in this paper.
Dataset
Llama2-7b
Llama3.1-8b
Llama-R1-8b
Qwen3-8b
MATH
14
29
18
13
MMLU-Pro
20
29
20
16
GPQA
11
20
16
12
Appendix
Table 5: The total number of generated prompts using TextGrad, OPRO, and CFPO under each setting.
Model
MATH
MMLU-Pro
GPQA
Llama2-7b
0.662
0.892
0.671
Llama3.1-8b
0.476
0.218
0.014
Llama-R1-8b
0.879
0.896
0.871
Qwen3-8b
0.754
0.744
0.015
Appendix
Table 6: The fitted γ on different models and datasets.
Model
Amazon
FinQA
ToolE
EM
OF
EM
OF
EM
OF
Llama2-7b
17.4±11.9
0.711
5.9±3.7
0.479
27.1±17.6
0.383
Llama3.1-8b
61.1±35.6
0.271
37.6±6.1
0.377
49.8±18.0
0.365
Llama-R1-8b
60.3±39.8
0.242
45.7±7.4
0.276
51.5±4.9
0.162
Qwen3-8b
61.1±18.4
0.201
54.9±5.9
0.246
56.0±6.6
0.121
Appendix
Table 7: The performance on Amazon Rview (Amazon), FinQA, and ToolE.
Model
MATH
MMLU-Pro
GPQA
SV
OF
SV
OF
SV
OF
Llama2-7b
0.851
0.475
0.706
0.622
0.760
0.509
Llama3.1-8b
0.777
0.366
0.631
0.350
0.667
0.467
Llama-R1-8b
0.669
0.158
0.601
0.292
0.575
0.371
Qwen3-8b
0.653
0.097
0.579
0.162
0.554
0.214
Appendix
Table 8: The output fluctuation using different metrics.
Model
MATH
MMLU-Pro
GPQA
EM
OF
EM
OF
EM
OF
Llama2-7b
12.2
0.653
13.8
0.578
15.9
0.523
Llama3.1-8b
48.9
0.375
35.1
0.347
28.1
0.470
Llama-R1-8b
65.4
0.147
40.2
0.303
30.0
0.404
Qwen3-8b
77.2
0.097
46.9
0.162
37.3
0.214
Appendix
Table 9: The average EM and OF of different models and datasets. For the certain dataset, the prompts of each model are all same with Qwen3-8b.
Model
Scale
MATH
MMLU-Pro
GPQA
EM
OF
EM
OF
EM
OF
Llama3.1
8b
45.8±7.2
0.366
41.0±10.7
0.350
26.6±5.7
0.467
70b
56.0±12.8
0.284
63.0±14.4
0.186
42.6±9.7
0.232
Qwen3
8b
77.2±1.6
0.097
46.9±5.2
0.162
37.3±1.9
0.214
34b
80.8±4.0
0.075
67.8±5.1
0.104
43.6±3.2
0.177
Appendix
Table 10: The performance of Llama3.1 and Qwen3 on each dataset with different model scales.
Figure 6: EM and OF on MATH cross CoT steps with different models.
Chain-of-thought (CoT) prompting assumes that generated reasoning reflects a model's internal computation. We show this assumption is wrong in a specific, measurable way: models internally detect their own reasoning errors but outwardly express confidence in them. A linear probe on hidden states predicts trace correctness with 0.95 AUROC -- from the very first reasoning step (0.79) -- while verbalized confidence for wrong traces is 4.55/5, nearly identical to correct ones (4.87/5). A text-surface classifier achieves only 0.59 on the same data, confirming a 0.20-point gap invisible in the generated text. This hidden error awareness holds across three model families (Qwen, Llama, Phi), 1.5B-72B parameters, and RL-trained reasoning models (DeepSeek-R1, 0.852 AUROC). The natural question is whether this signal can fix the errors it detects. It cannot. Four interventions -- activation steering, probe-guided best-of-N, self-correction, and activation patching -- all fail; patching destroys output coherence entirely. The signal is diagnostic, not causal: a readout of computation quality, not a lever to redirect it. This delineates a boundary for mechanistic interpretability: error representations during reasoning are fundamentally different from the factual knowledge representations that prior work has successfully edited.
Aojie Yuan, Zhiyuan Julian Su, Haiyue Zhang +2
University of Southern California, Los Angeles, CA, USA.
Chain-of-Thought (CoT) prompting substantially improves the sample efficiency of transformers, reducing the complexity of tasks like parity learning from exponential to polynomial in the input length. However, generating explicit reasoning steps at inference is computationally expensive. Implicit Chain-of-Thought (ICoT) has emerged as a promising empirical remedy that trains models to internalize intermediate steps within their hidden states, but its theoretical foundations remain poorly understood. We give the first theoretical analysis of ICoT, proving that an L-layer transformer trained under our proposed Log-ICoT curriculum learns k-parity with poly(n) samples and L=log2k training stages. This matches the sample efficiency of explicit CoT while eliminating its inference overhead, and extends prior one-layer parity guarantees to multi-layer architectures. Compared to standard ICoT, which removes thinking tokens one at a time, Log-ICoT removes them in geometric chunks, reducing the number of stages from linear in k to logarithmic. Experiments on multi-layer transformers confirm the theory and visualize how reasoning is progressively absorbed into deeper layers.
Chain-of-thought (CoT) reasoning is the dominant paradigm for inference-time scaling in language models, yet the causal influence of individual steps on the final answer remains poorly understood. In this work, we use answer logits at the end of each reasoning step to estimate each step's causal importance to the final answer and intermediate guesses, shedding light on the answer formation process of several reasoning model families. Across diverse tasks, we find that reasoning typically crosses a commitment boundary, a sharp transition from transient intermediate guesses to a stable, high-confidence answer. This transition often happens in a single step, well before the model's reasoning block ends, and is followed by epiphenomenal CoT steps that leave the final answer probability unaltered. Using attention probes, we show that answer-formation stages can be linearly decoded from the activations of intermediate reasoning steps with high accuracy, showing robust generalization to unseen reasoning tasks. We leverage this property for early-exiting reasoning blocks at the commitment boundary location, reducing the length of CoTs up to 55% with negligible impact on model performance.
Daniel Scalena, Sara Candussio, Luca Bortolussi +3
CLCG, University of Groningen · University of Milano-Bicocca · University of Trieste +1