Reasoning Fails Where Step Flow Breaks

Xiaoyu Xu; Yulan Pan; Xiaosong Yuan; Zhihong Shen; Minghao Su; Yuanhao Su; Xiaofeng Zhang

cs.AIApr 8, 2026

Reasoning Fails Where Step Flow Breaks

Authors: Xiaoyu Xu, Yulan Pan, Xiaosong Yuan, Zhihong Shen, Minghao Su, Yuanhao Su, Xiaofeng Zhang

Organizations: Shanghai Jiao Tong University · Fuzhou University · Jilin University

Abstract

Large reasoning models (LRMs) that generate long chains of thought now perform well on multi-step math, science, and coding tasks. However, their behavior is still unstable and hard to interpret, and existing analysis tools struggle with such long, structured reasoning traces. We introduce Step-Saliency, which pools attention--gradient scores into step-to-step maps along the question--thinking--summary trajectory. Across several models, Step-Saliency reveals two recurring information-flow failures: Shallow Lock-in, where shallow layers over-focus on the current step and barely use earlier context, and Deep Decay, where deep layers gradually lose saliency on the thinking segment and the summary increasingly attends to itself and the last few steps. Motivated by these patterns, we propose StepFlow, a saliency-inspired test-time intervention that adjusts shallow saliency patterns measured by Step-Saliency via Odds-Equal Bridge and adds a small step-level residual in deep layers via Step Momentum Injection. StepFlow improves accuracy on math, science, and coding tasks across multiple LRMs without retraining, indicating that repairing information flow can recover part of their missing reasoning performance.

Figures & tables

Figure 1: From token- to step-level saliency. Token-level saliency maps are dense and noisy; Step-Saliency pools them into question/thinking/summary blocks. Correct traces show smooth question $\rightarrow$ thinking $\rightarrow$ summary flow, while errors exhibit shallow lock-in and weak thinking $\rightarrow$ summary links.

Figure 2: Step-Saliency patterns for shallow vs. deep layers and correct vs. error traces. Top: depth-collapsed step $\to$ step saliency maps; darker red indicates stronger influence between steps. Bottom: schematic diagrams summarizing the observed patterns—red arrows denote narrow, local flow ( Shallow Lock-in and Deep Decay in error traces), while blue arrows denote broad, long-range flow in correct traces.

Figure 3: Layer-wise saliency intensities across three models. (a) GPT-OSS-20B: thinking and summary self-intensities for correct vs. error traces. (b) R1-Distill-32B and QwQ-32B show the same pattern: stronger shallow lock-in and summary self-reinforcement in error traces.

Method	Math / Science					Code
	AIME24	AIME25	AMC23	MATH-500	GPQA-D	LiveCodeBench
DeepSeek-R1-Distill-Qwen 7B	54.0	39.2	82.5	92.8	49.1	37.6
+ Budget Forcing (S1)	56.2	39.5	84.0	93.1	51.3	39.8
+ Hint-Infer (Round1)	57.5	43.3	85.2	93.0	56.6	45.2
+ Plan-and-Solve (PS+)	58.8	42.6	86.0	93.3	52.8	42.0
+ Attn-Interv.	56.5	41.0	84.5	93.0	51.8	40.5

Table 1: Accuracy (%) on six benchmarks (columns) across multiple backbones.

Figure 4: Effect of StepFlow on Step-Saliency (representative error trace). See Appendix E for the full response.

Figure 5: Effect of StepFlow on AIME25 and GPQA-D across three backbones. Top:

\Delta

Step-Saliency (

I_{T}

I_{S}

; § 3.3 ) over error cases. Bottom: Accuracy gain.

Method	AIME25	GPQA D	LiveCodeBench
Baseline	62.0	65.2	70.0
+ OEB only (shallow)	64.5	66.7	74.5
+ SMI only (deep)	64.0	67.2	75.0
+ OEB + SMI (StepFlow)	69.2	70.3	79.5

Table 2: Ablation of StepFlow components on GPT-OSS-20B medium (Accuracy, %).

Model	Easy	Medium	Hard
GPT-OSS-20B	90.1	70.2	37.8
+StepFlow	93.5	84.0	52.0

Table 3: Accuracy (%) on LiveCodeBench by difficulty for GPT-OSS-20B medium with and without StepFlow.

AIME25
Active layers (OEB / SMI)	R1-Distill-32B	GPT-OSS-20B	QwQ-32B
Baseline (no StepFlow)	54.9	39.1	40.0
Bottom / top 1/4 of layers	66.7	46.5	48.0
Bottom / top 1/3 of layers	64.0	44.0	45.2
Bottom / top 1/2 of layers	62.3	42.0	43.0
GPQA D

Table 4: Effect of StepFlow layer coverage (OEB on bottom-

k

layers, SMI on top-

k

layers) on three LRMs (Accuracy, %).

Method	Compute	Acc. (%)
Baseline	1.0 $\times$	63.8
Longer gen	${\sim}$ 1.35 $\times$	65.0
StepFlow	${\sim}$ 1.35 $\times$	70.6
SC ( $k$ =2)	${\sim}$ 2.0 $\times$	66.5
StepFlow+SC ( $k$ =2)	${\sim}$ 2.7 $\times$	73.5
SC ( $k$ =4)	${\sim}$ 4.0 $\times$	68.5

Table 5: Compute-normalized comparison on AIME 24/25 (averaged accuracy, %) for R1-Distill-32B.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Perturbation	Level	R1-32B	GPT-OSS	QwQ-32B
Shift	$\pm 1$ tok	64.2	69.8	63.0
	$\pm 3$ tok	64.0	69.5	62.7
Dropout	25%	63.3	69.2	62.1
Insertion	25%	63.1	68.9	62.0
Dropout+Ins	50% each	62.8	67.5	60.5
Random unif	—	62.5	66.8	59.5

Appendix

Table 6: Effect of boundary perturbations on GPQA-D accuracy (%).

Figure 6: Accuracy vs. $\tau_{\max}$ on AIME24 and GPQA-D. Dashed lines show baseline accuracy; stars mark the chosen $\tau^{\star}$ .

Model / Benchmark	$\alpha_{\text{low}}$	$\alpha^{\star}$	$\alpha_{\text{high}}$	Range
R1-Distill-32B / AIME24	0.03	0.06	0.09	74.9–75.5
R1-Distill-32B / GPQA-D	0.03	0.06	0.09	64.3–64.9
GPT-OSS / AIME24	0.03	0.06	0.09	65.4–66.0
GPT-OSS / GPQA-D	0.03	0.06	0.09	67.5–68.1
QwQ-32B / AIME24	0.03	0.06	0.09	56.9–57.5
QwQ-32B / GPQA-D	0.03	0.06	0.09	62.6–63.2

Appendix

Table 7: Sensitivity of the SMI residual scale

\alpha

on AIME24 and GPQA-D. For each model/benchmark pair, The last column reports the corresponding accuracy interval.

Model	OEB layers	SMI layers	$\tau_{\max}$	$\alpha$
R1-Distill-7B	bottom 1/4	top 1/4	0.15	0.06
R1-Distill-14B	bottom 1/4	top 1/4	0.15	0.06
R1-Distill-32B	bottom 1/4	top 1/4	0.15	0.06
GPT-OSS-20B	bottom 1/4	top 1/4	0.15	0.06
QwQ-32B	bottom 1/4	top 1/4	0.15	0.06

Appendix

Table 8: StepFlow reproducibility summary.

Model	Baseline s/token	+StepFlow s/token
GPT-OSS-20B	0.036	0.047
R1-Distill-32B	0.023	0.032
QwQ-32B	0.068	0.090

Appendix

Table 9: Decoding efficiency on GPQA-D by model.

Model	Method	Compute	Acc. (%)
R1-Distill-32B	Baseline	1.0 $\times$	63.8
	Longer gen	${\sim}$ 1.35 $\times$	65.0
	StepFlow	${\sim}$ 1.35 $\times$	70.6
	SC ( $k$ =2)	${\sim}$ 2.0 $\times$	66.5
	StepFlow+SC ( $k$ =2)	${\sim}$ 2.7 $\times$	73.5
	SC ( $k$ =4)	${\sim}$ 4.0 $\times$	68.5

Appendix

Table 10: Compute-normalized comparison on AIME 24/25 (averaged accuracy, %) across three backbones.

Dataset	Baseline CI	StepFlow CI
AIME25	[58.0, 66.0]	[65.2, 73.2]
GPQA-D	[63.0, 67.4]	[68.1, 72.5]
LiveCodeBench	[65.5, 74.5]	[75.5, 83.5]

Appendix

Table 11: Bootstrap 95% CI for accuracy on GPT-OSS-20B medium.

Error Type	R1-32B (%)	GPT-OSS (%)
Arithmetic carry-forward	38	30
Premise forgetting	34	42
Step-skip	18	14
Conceptual	10	14

Appendix

Table 12: Error taxonomy of StepFlow corrections on AIME 24/25.

LLM Reasoning

Large language model reasoning studies how models solve multi-step problems, verify intermediate steps, and use computation or tools reliably.

See all 926 papers

Unrolled Flow Models for Reasoning
Oct 7, 2026Faissal Izermine, Hanru Bai, Oscar Davis +1Flow-Based Generative Modeling Flow Matching
Max Planck Institute for Intelligent Systems · ELLIS Institute Tübingen · ETH Zurich +3
ReasoningFlow: Discourse Structures for Understanding LLM Reasoning Traces
Jun 3, 2026Jinu Lee, Shivam Agarwal, Amruta Parulekar +3LLM Reasoning with Graphs LLM Auditing
The Grainger College of Engineering University of Illinois Urbana-Champaign
Interpreting Reasoning of Large Language Models via Partial Information Decomposition
Sep 30, 2026Barproda Halder, Qiuyi Zhang, Sanghamitra DuttaLLM Interpretability Efficient Language Model Reasoning
University of Maryland, College Park · Elorian AI

Oct 7, 2026J/K move · Enter open · S save

Unrolled Flow Models for Reasoning

Faissal Izermine, Hanru Bai, Oscar Davis, T. Konstantin Rusch

Max Planck Institute for Intelligent Systems · ELLIS Institute Tübingen · ETH Zurich+3

Flow-Based Generative Modeling Flow Matching Continuous Latent Reasoning

Open paper AskPDF arXiv

Flow matching enables language generation in few steps, but whether additional integration steps improve reasoning remains unclear. We prove that a flow parameterized by a two-layer Transformer can solve graph reachability, with the required number of integration steps increasing with the target's distance from the root. Yet, standard flow language models can fail to benefit from additional steps on reasoning tasks. We attribute this limitation to objectives that supervise each time point independently, without explicitly training successive steps to build on one another. To address this, we instead train through the model's own latent rollout over a randomly sampled subinterval of [0, 1], decoding only at the endpoint. On ProsQA, this raises accuracy to 97% and enables performance to improve with additional integration steps. For the longer rollouts required by reasoning tasks such as Sudoku and Maze, retracting the latent state onto a sphere stabilizes the dynamics and yields substantial gains over baselines with more than three times as many parameters. Sampling multiple rollouts further improves performance when paired with a parameter-free selection score, although reliable selection remains challenging for longer answers. Together, these results establish a theoretical basis for reasoning with flows and show how rollout training, stable latent dynamics, and rollout selection help realize this capacity in practice.

Abstract

LLM Reasoning

Explore similar work

Unrolled Flow Models for Reasoning

ReasoningFlow: Discourse Structures for Understanding LLM Reasoning Traces

Interpreting Reasoning of Large Language Models via Partial Information Decomposition