When Can LLM Reasoning Reliably Reuse Intermediate Results?

Deyuan Liu; Zecheng Wang; Xuanyi Liu; Jungang Li; Yihao Hu; Mingjie Tang; Bryan Dai; Jie Fu

cs.AIOct 20, 2026

When Can LLM Reasoning Reliably Reuse Intermediate Results?

Authors: Deyuan Liu, Zecheng Wang, Xuanyi Liu, Jungang Li, Yihao Hu, Mingjie Tang, Bryan Dai, Jie Fu

Organizations: Westlake University · Wechat AI · Peking University · HKUST ·

Abstract

Sampling complete solutions gives large language models more chances to solve a problem, but each attempt starts again. For tasks with several subproblems, all required answers must be correct in the same attempt. Retaining intermediate results can avoid repeating correct work, but errors in those results may affect later attempts. We study when this reuse is reliable and how much computation it requires. For local retries that stop when no candidate passes within the retry budget, we give conditions for correct completion and bound the average number of attempts. We also show how checking cost and the chance of passing a check affect total computation. Experiments across eight reasoning models examine composite mathematics and arithmetic. Across four DeepSeek models, average success on the second subproblem is 36.2 percent without first-subproblem context, 57.2 percent with its answer, and 58.3 percent with its reasoning and answer. These scores measure second-subproblem coverage and include continuations made when selection fails. Arithmetic experiments show that similar first-attempt accuracies can lead to very different coverage after repeated attempts. Code is available at https://github.com/SempraETY/Intermediate-State-Reuse.

Figures & tables

Figure 1: Complete-solution sampling and intermediate-state reuse. Left: Independent sampling requires a successful complete run. The analyzed reuse loop preserves earlier accepted work, retries rejected candidates, and stops when a step’s budget is exhausted. Passing a check does not guarantee correctness. Right: An illustrative two-subproblem candidate-selection example, with SC denoting self-consistency, $M$ the first-subproblem candidate budget, and $B$ the continuation cap. Fractions count correct candidates among those shown; the example is not an experimental measurement or the single-state loop used in the proof.

Table 2

Strategy	$A_{2}^{\mathrm{ret}}$	Change
no_context	36.2%	0%
answer_only	57.2%	+58%
full_context	58.3%	+61%

Table 3: Context for the Second Subproblem. Four-DeepSeek means at the same continuation budget.

A_{2}^{\mathrm{ret}}

counts second-subproblem success, including fallbacks. Change is the relative percentage increase over no_context . Strategies supply no first-subproblem context, its answer only, or its reasoning and answer.

Model

M{=}4,B{=}2

M{=}8,B{=}2

M{=}8,B{=}4

M{=}16,B{=}8

Succ.

Tokens

Eff.

Succ.

Tokens

Eff.

Succ.

Tokens

Eff.

Succ.

Tokens

Eff.

DeepSeek-R1-1.5B

53.6%

25.1K

2.13

54.0%

46.8K

1.15

\uparrow

2.0 55.6%

50.0K

1.11

\uparrow

3.2 56.8%

100.6K

0.56

DeepSeek-R1-7B

61.2%

29.2K

2.09

61.2%

52.8K

1.16

\uparrow

.4 61.6%

57.4K

1.07

61.2%

115.0K

0.53

DeepSeek-R1-8B

41.2%

59.6K

0.69

44.8%

100.7K

0.44

\uparrow

10.0 51.2%

116.7K

0.44

\uparrow

12.4 53.6%

233.0K

0.23

DeepSeek-R1-14B

59.6%

32.6K

1.83

60.0%

57.7K

1.04

\uparrow

.8 60.4%

63.7K

0.95

\uparrow

1.6 61.2%

128.1K

0.48

Qwen3-4B

47.2%

58.7K

0.80

48.0%

100.3K

0.48

\uparrow

6.4 53.6%

117.6K

0.46

53.2%

234.7K

0.23

Table 4: Budget Allocation Sweep.

M

is the first-subproblem candidate budget and

B

the cap retained for continuation. Succ. is second-subproblem success (%), including fallbacks, regardless of first-answer correctness. Tokens reports recorded counts; Eff. divides Succ. by Tokens in thousands. Average-row efficiencies divide mean success by mean tokens; they are not means of model-level efficiencies. Superscripts give percentage-point differences from

(4,2)

. Orange marks the reference configuration

(8,4)

, not an efficiency optimum; blue marks configurations with higher success and a larger measured token cost.

Figure 2: Unsolved Fraction after 32 Attempts. Each cell reports $\hat{E}(32)$ , the fraction of 50 problems unsolved after 32 attempts; $n$ is operand digit count. Panels show Base 10 addition/multiplication and Base 7 addition/multiplication. All Base 10 addition problems are solved; Base 7 multiplication has the largest mean unsolved fraction. For R1-7B, the latter rises from 0.06 at $n=3$ to 0.74 at $n=4$ . GPT-OSS-120B shows nonmonotonic behavior at low digit counts under the recorded sampling protocol.

Figure 3: Gain from Additional Attempts. Each point is a combination of model, base, operation, and digit count. The gain is Pass@32 minus Pass@1; the dashed line is its maximum

1-\mathrm{Pass}@1

. Large observed gains occur around Pass@1 values of 0.05–0.35 in these evaluated arithmetic conditions.

Figure 4: Agent Score Prediction at 12 Hours. Each point is a task–model pair. Predictions use its scores at 2 and 4 hours and parameter estimates from other tasks in the same category, excluding the target task. The retry curve models diminishing gains from repeated attempts; the comparison curve is sigmoid in log time (Appendix F.1 ). All 252 pairs are plotted; median relative errors are 10.7% and 10.9%, respectively, over 248 pairs with positive observed scores. Dashed and dotted lines mark perfect prediction and

\pm 20\%

bands.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Meaning
Base reasoning quantities
$\bm{\theta}$ , $p_{\bm{\theta}}$	model parameters and induced generation distribution
$x$ , $x_{1}\oplus x_{2}$	task and composite task
$z_{1:N}$ , $z_{t}$	generated text sequence and text generated at one step
$s_{t}$ , $y$ , $a^{\star}(x)$	intermediate state, extracted final answer, and target answer
$N$ , $n$	number of steps in the chosen decomposition and digit count

Appendix

Table 5: Core notation used in the paper. Proof, fit, and metric symbols are defined where they appear.

Figure 5: Accuracy and Generated Tokens by Digit Count. Top: first-attempt accuracy from Figure 8 . Bottom: mean generated tokens per attempt from Figure 15 . Both rows plot the reported heatmap cells under the protocol in Appendix C.3 ; colors identify models.

Figure 6: Unsolved Fraction in Base 7 and Base 10. Each curve reports

\hat{E}(32)

by digit count for one model and operation, comparing the two bases under the stated sampling protocol.

Figure 7: Empirical Unsolved-Fraction Curves. Rows are models; columns are Base 10 addition, Base 10 multiplication, Base 7 addition, and Base 7 multiplication. Colors encode

n=1,\ldots,10

. Solid curves show

\hat{E}(R)

and dashed gray curves show fitted two-component models. The panels retain both zero and nonzero unsolved fractions within 32 attempts, including GPT-OSS-120B’s nonmonotonic Base 7 multiplication results.

Figure 8: First-Attempt Accuracy. For Base 7 multiplication, R1-14B ( $n=6$ ) has Pass@1=0.14 and $\hat{E}(32)=0.16$ (84% solved within 32 attempts); R1-1.5B ( $n=7$ ) has Pass@1=0 and $\hat{E}(32)=1$ .

Figure 9: Unsolved Fraction at Different Attempt Budgets. Rows correspond to

R\in\{2,4,8,16,32\}

and columns to the four base–operation combinations. Curves show

\hat{E}(R)

against digit count; colors identify models. The reported mean absolute gap between

R=16

and

R=32

is 0.031 across 320 conditions, with a maximum of 0.32; every checkpoint uses the same nested attempt pool.

Figure 10: Two-Component Fits for Base 7. Solid curves show empirical

\hat{E}(R)

; dashed curves show nonlinear least-squares fits of

\pi_{s}\varepsilon_{\mathrm{good}}^{R}+(1-\pi_{s})\varepsilon_{\mathrm{bad}}^{R}

n\in\{3,5,7,9\}

. The slowly decaying component’s weight

1-\pi_{s}

is compared with the unsolved fraction in Appendix E.4 .

Figure 11: Two-Component Fits for Base 10. The layout, checkpoints, and curve styles follow Figure 10 , with Base 10 addition and multiplication shown in the two panels.

Figure 12: Gain from Additional Attempts by Digit Count. Top: addition; bottom: multiplication. Panels identify models, the horizontal axis is digit count, colors encode

R\in\{4,8,16,32\}

, and line styles distinguish bases. The plotted quantity is Pass@

R

minus Pass@1; its gap to

1-\mathrm{Pass}@1

equals

\hat{E}(R)

Figure 13: Unsolved Fraction for Eight-Digit Multiplication. Each model panel shows

\hat{E}(R)

for Base 7 (solid green) and Base 10 (dashed blue) at

n=8

. Markers identify

R\in\{2,4,8,16,32\}

. The curves retain the full observed range from 1 to 32 attempts for both numeral bases.

Figure 14: Pass@1 and Pass@32 by Digit Count. Top: addition; bottom: multiplication. Panels identify models, colors distinguish Base 7 and Base 10, and markers distinguish Pass@1 from Pass@32. The vertical separation at each digit count is the gain from additional attempts.

Figure 15: Average Generated Tokens per Attempt. Cells show mean output length for each combination of model, base, operation, and digit count under a 10,240-token generation cap. The displayed range is 115–10,240 tokens; averages are 7,391 for Base 7 multiplication and 2,702 for Base 10 addition.

Time	Retry Curve (Eq. 105 )			Sigmoid (log time) (Eq. 106 )
Time	Median	${<}10\%$	${<}20\%$	Median	${<}10\%$	${<}20\%$
6 hours	3.8%	71%	87%	3.9%	67%	81%
8 hours	6.5%	61%	78%	5.8%	58%	73%
10 hours	8.9%	53%	69%	8.8%	52%	66%
12 hours	10.7%	48%	65%	10.9%	48%	62%

Appendix

Table 6: Agent Score Prediction from 2- and 4-Hour Observations. Both curves use priors from other tasks as specified in Appendix F.2 . Median relative errors and fractions below 10% and 20% use 248 pairs with positive observed scores at each prediction time. All 252 pairs receive predictions; four pairs with zero observed scores are excluded from relative errors at each prediction horizon.

LLM Reasoning

Large language model reasoning studies how models solve multi-step problems, verify intermediate steps, and use computation or tools reliably.

See all 931 papers

It's the Problem, Not the Path: Budget and Difficulty Confounds in LLM Reasoning Trajectories
Sep 3, 2026Yigit Utku BulutLLM Evaluation Language Model Probing
Johannes Kepler University Linz
Beyond Repeated Sampling: Learning Search Policies for LLM Reasoning
Sep 22, 2026Ismail Labiad, Matthieu Kowalski, Marc Schoenauer +2RL for Language Model Reasoning LLM Reasoning
Meta FAIR · Université Paris-Saclay, LISN, Inria, CNRS · NYU Courant Institute and CDS
How Much Thinking is Enough? Quantifying and Understanding Redundancy in LLM Reasoning
Apr 21, 2026Zhiyuan Zhai, Xinkai You, Wenjing Yan +1LLM Inference Efficiency RL for Language Model Reasoning
Fudan University · The Chinese University of Hong Kong

Sep 3, 2026J/K move · Enter open · S save

It's the Problem, Not the Path: Budget and Difficulty Confounds in LLM Reasoning Trajectories

Yigit Utku Bulut

Johannes Kepler University Linz

LLM Evaluation Language Model Probing LLM Reasoning

Open paper AskPDF arXiv

Reasoning traces of large language models are widely read as containing "breakthrough" moments and early-legible fates. Both readings rest on measurements missing a counterfactual control at the level of the claim; we supply both controls. First, a restart-controlled truncation probe separates when a solution fits the continuation budget from when a prefix carries value that fresh computation cannot buy, comparing per-anchor continuation solve rates against from-scratch restart curves at matched total generated-token budget. Applied to 178 problem-model cells (89 MATH problems x two small open models, an outcome-blind but difficulty-targeted cohort), exactly 1 of 178 cells survives as prefix-limited; restart dose-response separates a compute-starved model from a capability-limited one; and wherever the matched budget lies inside the restart grid, continuing the model's own prefix beats restarting (9 of 9) -- predominantly compute compression rather than expanded reachability. Second, a pre-registered, difficulty-controlled test finds no detectable outcome information in early-window internal signals beyond a problem-difficulty baseline, and two generation-free analyses of public corpora show why this control is needed: a trace-blind difficulty proxy reaches AUROC 0.873 on 192K DeepSeek-R1 generations -- inside the published probe range -- and a closely matched reconstruction of the closest published early-window positive recovers a comparable pooled result (0.849) while within problem it is statistically indistinguishable from chance at all ten anchors (0.496 at t=4); a post-hoc within-targeted probe finds only a small average residual, concentrated in three low-failure problems. High pooled probe AUROCs cannot by themselves establish within-attempt information; a question-only baseline or within-problem evaluation is required.

Abstract

LLM Reasoning

Explore similar work

It's the Problem, Not the Path: Budget and Difficulty Confounds in LLM Reasoning Trajectories

Beyond Repeated Sampling: Learning Search Policies for LLM Reasoning

How Much Thinking is Enough? Quantifying and Understanding Redundancy in LLM Reasoning