Characterizing a Configuration Where Inference-Time PRM-Pruned Fragment Grafting Is Inert: Evidence from Three Reasoning LMs

Khawaja Murad ul Hassan; Mehran Ebrahimi

cs.AISep 3, 2026

Characterizing a Configuration Where Inference-Time PRM-Pruned Fragment Grafting Is Inert: Evidence from Three Reasoning LMs

Authors: Khawaja Murad ul Hassan, Mehran Ebrahimi

Organizations: QLU.ai, Islamabad · Faculty of Science, Ontario Tech University Oshawa, ON, Canada

Abstract

Diversity collapse in parallel chain-of-thought has motivated inference-time interventions built on a natural design: when a process reward model (PRM) prunes a chain, its high-PRM prefix is extracted and grafted verbatim as an in-context demonstration into a still-decoding sibling. We isolate this mechanism, PRM-Pruned Fragment Grafting (PPFG), as the most cost-minimal operationalization of cross-trajectory step-level transfer, and test it at the operating point where prior fragment-grafting work reports gains only under additional compensating ingredients. On Qwen2.5-7B-Instruct with Math-Shepherd on full MATH500 (n=500, three seeds), PPFG in both stagnation- and random-targeting variants is statistically indistinguishable from an independent parallel-CoT baseline on every measured axis. We characterize why: a four-bucket classification of 322 stagnation-rule injection events shows only 14% targeted a genuinely struggling chain; the rest landed on chains that had already succeeded, were near completion, or sat on a flat PRM plateau, states a rescue graft cannot change. No compound-gate refinement jointly achieves well-targeted firing and adequate density, and a random control matches the same parity at 2.4x the firing rate, so the inertness is not heuristic-specific. The finding replicates across three base LMs, six benchmarks, a second PRM, and a compatibility-gate sweep; two-one-sided-tests analysis promotes the parity to positive equivalence on all twelve Qwen/LLaMA cells. A per-event spot-check finds injected chains prune at 2.75x the matched-step rate, but a surviving-sibling counterfactual finds no population-level compensation. A hindsight oracle bounds any per-problem gain from choosing PPFG over independent at +0.13 pp. We contribute an equivalence-testing template for establishing inference-time mechanism nulls, with every claim scoped to its tested operating point.

Figures & tables

Method	Pass@1	Pass@2	Pass@4	Pass@8	Pass@16	mode rate $\downarrow$	distinct-4 $\uparrow$	tpp
independent ( $N{=}8$ ) ‡	$0.592\pm.007$	$0.683\pm.006$	$0.742\pm.005$	$0.783\pm.006$	—	$0.788\pm.009$	$0.535\pm.003$	4030
SMC ( $N{=}8$ )	$\mathbf{0.634\pm.003}$	$\mathbf{0.687\pm.007}$	$0.722\pm.010$	$0.745\pm.008$	—	$0.835\pm.008$	$0.260\pm.010$	4143
BoN ( $N{=}16$ , 2 $\times$ compute)	$0.632\pm.001$	$0.706\pm.002$	$\mathbf{0.757\pm.003}$	$0.793\pm.006$	$\mathbf{0.817\pm.007}$	$0.769\pm.005$	$0.440\pm.002$	8251
BoN-selector over indep ( $N{=}8$ argmax-PRM, tie-averaged) †	$\mathbf{0.654\pm.007}$	$0.681\pm.007$	$0.741\pm.006$	$0.782\pm.007$	—	$0.789\pm.006$	$0.535\pm.003$	4055
PPFG-stag ( $N{=}8$ ) ‡	$0.591\pm.008$	$0.682\pm.007$	$0.742\pm.007$	$0.783\pm.011$	—	$0.789\pm.010$	$0.535\pm.003$	4025
PPFG-random ( $N{=}8$ ) ‡	$0.591\pm.007$	$0.682\pm.006$	$0.743\pm.005$	$0.786\pm.007$	—	$0.788\pm.010$	$0.535\pm.002$	4019

Table 1: Pass@

k

, mode rate, and distinct-4 on full MATH500 (

n=500

), Qwen2.5-7B-Instruct, mean

\pm

std. All numbers computed with the canonical scoring path (correctness

=

task.is_correct on bare math_verify , no normalized-string-match prefilter; answers always re-extracted from steps, never read from a cached field; see § 5.2 ); PPFG-vs-baseline parity is preserved because all methods shift together under the canonical comparator. † BoN-selector at

N{=}8

is not a distinct inference-time procedure; it applies argmax-PRM selection (tie-averaged: a tie’s expected accuracy is the mean correctness over the tied chains, replacing an earlier no-tie-handling convention biased to chain index 0) to the independent

N{=}8

trajectories. It shares per-chain compute, mode rate, distinct-4, and tpp with the independent row by construction. We report it for compute-frontier visualization (Fig. 4 ) only. ‡ 5 seeds (

\{42,\ldots,46\}

) for these rows; the corresponding 3-seed means agree to three decimals. Other rows (SMC, BoN

N{=}16

, BoN-selector) at 3 seeds (

\{42,43,44\}

); the BoN-selector row inherits the 3-seed independent values by construction.

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark (PRM)	indep $P{@}1$	stag $P{@}1$	$\Delta P{@}1$	$\Delta$ mode rate
AIME 2024+2025 (MS)	0.0417	0.0417	$+0.00$	$+1.60$
NuminaMath (MS)	0.4116	0.4116	$+0.00$	$+1.02$
GPQA-Diamond (MS)	0.2363	0.2353	$-0.11$	$-0.01$
GPQA-Diamond (VP)	0.1109	0.1115	$+0.06$	$-0.86$
MMLU-Pro L/B/P (MS)	0.3940	0.3937	$-0.03$	$-0.27$
MMLU-Pro L/B/P (VP)	0.3247	0.3242	$-0.05$	$-0.10$

Appendix

Table 2: Replication summary across four further benchmarks (Qwen2.5-7B-Instruct, 3 seeds, indep vs PPFG-stag; full per-benchmark tables below). All

\Delta

values within MDE.

Method	Pass@1	Pass@8	Mode rate
independent	0.0417	0.1111	0.1880
PPFG-stag	0.0417	0.1111	0.2040
PPFG-random	0.0417	0.1056	0.2137
$\Delta$ (stag $-$ indep)	$+0.00$	$+0.00$	$+1.60$
$\Delta$ (random $-$ indep)	$+0.00$	$-0.55$	$+2.57$

Appendix

Table 3: AIME 2024+2025 (

n{=}120

, 3 seeds, Qwen2.5-7B-Instruct). MDE

\pm 2.4

pp (Pass@1) /

\pm 4.7

pp (mode rate). The identical

0.0417

across methods reflects

5

problems

\times\,3

seeds

=15

problem-seed instances solved consistently at this floor regime; the unbiased Chen-et-al-2021 Pass@1 estimator is deterministic on perfectly-solved or perfectly-failed problems, so differences between methods can only register on partial-success problems, of which there are essentially none at Pass@1

\approx 0.04

Method	Pass@1	Pass@8	Mode rate
independent	0.4116	0.6162	0.6655
PPFG-stag	0.4116	0.6128	0.6757
PPFG-random	0.4091	0.6094	0.6689
$\Delta$ (stag $-$ indep)	$+0.00$	$-0.34$	$+1.02$
$\Delta$ (random $-$ indep)	$-0.25$	$-0.68$	$+0.34$

Appendix

Table 4: NuminaMath-CoT test (

n{=}99

, 3 seeds, Qwen2.5-7B-Instruct). MDE

\pm 2.6

pp /

\pm 5.2

pp.

Math-Shepherd PRM
Method	Pass@1	Pass@8	Mode rate
independent	0.2363	0.6044	0.6530
PPFG-stag	0.2353	0.5892	0.6529
PPFG-random	0.2348	0.5976	0.6554
VersaPRM
independent	0.1109	0.3872	0.7650

Appendix

Table 5: GPQA-Diamond cross-PRM (

n{=}198

, 3 seeds, Qwen2.5-7B-Instruct). MDE

\pm 1.9

pp /

\pm 3.7

pp.

Math-Shepherd PRM
Method	Pass@1	Pass@8	Mode rate
independent	0.3940	0.6840	0.7213
PPFG-stag	0.3937	0.6853	0.7186
PPFG-random	0.3933	0.6847	0.7181
VersaPRM
independent	0.3247	0.6340	0.7735

Appendix

Table 6: MMLU-Pro Law/Biology/Philosophy stratified subset (

n{=}500

, 3 seeds).

Figure 1: Four-bucket classification of the 55 PPFG-stag injection events. Buckets are not mutually exclusive. Only

6/55

(

11\%

) events fall in plausibly-helped (the complement of the union of the other three); equivalently,

49/55

(

89\%

) fall into at least one state, {high-flat-PRM, near-completion, target-succeeded}, in which a rescue graft cannot change the outcome. We report the

6/55

well-targeted share as the primary quantity and the complement as the share of wasted fires.

Bucket	Count	%
high-flat-PRM (cosmetic plateau)	42	76.4
near-completion (graft no room)	24	43.6
target-succeeded (no rescue needed)	13	23.6
plausibly-helped (rule fired correctly)	6	10.9

Appendix

Table 7: Four-bucket counts on the first-100 slice (55 events); buckets are not mutually exclusive.

Tercile	mean indep	$n$ events	well-	high-flat-
	Pass@1		targeted	PRM
hard	$0.108$	$212$ ( $66\%$ )	$15.6\%$	$62.3\%$
medium	$0.721$	$102$ ( $32\%$ )	$11.8\%$	$74.5\%$
easy	$0.973$	$8$ ( $2\%$ )	$0.0\%$	$100.0\%$
all	—	$322$	$14.0\%$	$67.1\%$

Appendix

Table 8: Targeting quality by problem-difficulty tercile (322 PPFG-stag events, Qwen2.5-7B-Instruct + MATH500

n{=}500

, 3 seeds). Terciles are defined by pooled independent-baseline Pass@1. The stagnation rule concentrates firing on hard problems, but the well-targeted share barely rises there, while high-flat-PRM remains dominant.

Method	Pass@1	Pass@4	Pass@8	mode rate $\downarrow$	distinct-4 $\uparrow$	tpp	inj. rate
independent ( $N{=}8$ )	$0.644\pm.006$	$0.809\pm.009$	$0.852\pm.009$	$0.789\pm.006$	$0.535\pm.003$	4055	—
PPFG-stag ( $N{=}8$ )	$0.643\pm.006$	$0.807\pm.009$	$0.849\pm.011$	$0.790\pm.008$	$0.535\pm.003$	4048	0.215 (322/1500)
PPFG-stag-compound ( $t{=}0.60$ , $N{=}8$ )	$0.649\pm.006$	$0.810\pm.008$	$0.855\pm.007$	$0.790\pm.006$	$0.536\pm.003$	3974	0.021 (32/1500)
PPFG-stag-compound ( $t{=}0.70$ , $N{=}8$ )	$0.649\pm.006$	$0.810\pm.008$	$0.855\pm.007$	$0.791\pm.005$	$0.536\pm.003$	3971	0.037 (56/1500)

Appendix

Table 9: End-to-end measurement of the compound-gate variant at two thresholds (3 seeds

\times

500 problems each). Pre-comparator-correction values (§ 5.2 ); the within-table parity finding is not expected to change.

Method	inj. rate	P@1	P@8	mode rate
indep	—	$.871_{\pm.007}$	$.958_{\pm.003}$	$.959_{\pm.004}$
PPFG-stag	$2.8_{\pm 1.0}\%$	$.871_{\pm.007}$	$.958_{\pm.003}$	$.958_{\pm.005}$
PPFG-random	$9.5_{\pm 1.4}\%$	$.871_{\pm.007}$	$.958_{\pm.003}$	$.961_{\pm.004}$

Appendix

Table 10: GSM8K replication (

n{=}500

, 3 seeds; tpp

\approx 1891

for all rows).

Arch	Method	Pass@1	Pass@2	Pass@4	Pass@8	Mode rate	$\Delta$ P@1 vs indep
AIME 2024+2025 ( $n{=}120$ ) MDE $\pm 2.4/4.7$ pp
LLaMA-3.1-8B-Instruct	independent	0.0108	0.0186	0.0297	0.0444	0.6020	—
	PPFG-stag	0.0101	0.0175	0.0282	0.0417	0.6219	$-0.07$
	PPFG-random	0.0094	0.0161	0.0256	0.0389	0.6534	$-0.14$
DeepSeek-R1-Distill-Qwen-7B	independent	0.0094	0.0173	0.0301	0.0500	0.6647	—
	PPFG-stag	0.0090	0.0166	0.0287	0.0472	0.6764	$-0.04$

Appendix

Table 11: Cross-architecture sweep on extended benchmarks. All contrasts within per-benchmark MDE.

Figure 2: The PPFG mechanism. When a chain is pruned by the PRM (red status box), its high-PRM prefix is extracted; the four PPFG sub-operations (extract / target-select / gate / inject) select a still-decoding sibling chain and splice the fragment into the sibling’s prompt context as a bracketed in-place demonstration.

Arch.	$n$ scores	mean	q25	q75	$\tau_{\mathrm{kill}}$ fire
Qwen	89,091	0.768	0.624	0.917	2.4%
LLaMA	112,022	0.723	0.593	0.881	4.3%
DeepSeek	90,323	0.715	0.562	0.893	6.4%

Appendix

Table 12: PRM-distribution audit: PPFG fires more often on LLaMA and DeepSeek than on Qwen, ruling out a “PRM never prunes” artifact.

Figure 3: Cross-architecture parity panel. Pass@

k

k

on full MATH500 (

n=500

, 3 seeds, mean

\pm

seed-

\sigma

) for three base architectures. PPFG-stag and PPFG-random sit within seed-

\sigma

of the independent baseline at every

k

on every architecture.

Architecture	Method	Pass@1	Pass@2	Pass@4	Pass@8	mode rate $\downarrow$	distinct-4 $\uparrow$	tpp
Qwen2.5-7B-Instruct	independent	$0.592\pm.007$	$0.683\pm.006$	$0.742\pm.005$	$0.783\pm.006$	$0.788\pm.009$	$0.535\pm.003$	4030
	PPFG-stag	$0.591\pm.008$	$0.682\pm.007$	$0.742\pm.007$	$0.783\pm.011$	$0.789\pm.010$	$0.535\pm.003$	4025
	PPFG-random	$0.591\pm.007$	$0.682\pm.006$	$0.743\pm.005$	$0.786\pm.007$	$0.788\pm.010$	$0.535\pm.002$	4019
LLaMA-3.1-8B-Instruct	independent	$0.316\pm.007$	$0.415\pm.007$	$0.504\pm.007$	$0.584\pm.011$	$0.606\pm.012$	$0.555\pm.006$	3629
	PPFG-stag	$0.313\pm.011$	$0.409\pm.011$	$0.497\pm.010$	$0.572\pm.013$	$0.612\pm.009$	$0.553\pm.005$	3595
	PPFG-random	$0.311\pm.009$	$0.406\pm.007$	$0.493\pm.005$	$0.566\pm.009$	$0.596\pm.007$	$0.551\pm.006$	3601

Appendix

Table 13: Cross-architecture results on full MATH500 (

n=500

), mean

\pm

std, canonical scoring path (§ 5.2 ). independent rows: 5 seeds (

\{42,\ldots,46\}

); LLaMA/DeepSeek PPFG rows: 3 seeds (a seed-45/46 relaunch ambiguity for those cells is unresolved). LLaMA/DeepSeek’s low absolute Pass@1 is substantially a censoring effect, not weaker per-answer accuracy—see § 5.2 .

Figure 4: Pass@4 vs tokens-per-problem on MATH500 (Qwen2.5-7B,

n{=}500

, log-

x

). BoN at matched compute dominates the matched-tpp column; BoN at

N{=}16

reaches Pass@16

=0.89

2\times

compute.

$\xi_{\min}$	inj_rate	Pass@1	Pass@4	Pass@8	mode rate	distinct-4	tpp
0.00	16.0 $\pm$ 1.1	64.3 $\pm$ 0.5	80.7 $\pm$ 0.8	84.9 $\pm$ 0.9	79.0 $\pm$ 0.7	53.5 $\pm$ 0.2	4048
0.10	12.5 $\pm$ 0.7	64.7 $\pm$ 0.5	80.9 $\pm$ 0.7	85.3 $\pm$ 0.7	78.7 $\pm$ 0.6	53.6 $\pm$ 0.2	3967
0.30	11.0 $\pm$ 0.3	64.8 $\pm$ 0.5	80.9 $\pm$ 0.7	85.3 $\pm$ 0.7	78.9 $\pm$ 0.6	53.6 $\pm$ 0.2	3970
0.50	7.0 $\pm$ 0.3	64.8 $\pm$ 0.5	80.9 $\pm$ 0.7	85.3 $\pm$ 0.7	78.7 $\pm$ 0.5	53.6 $\pm$ 0.2	3974
0.70	2.4 $\pm$ 0.7	64.8 $\pm$ 0.5	80.9 $\pm$ 0.7	85.3 $\pm$ 0.7	78.7 $\pm$ 0.6	53.6 $\pm$ 0.2	3973

Appendix

Table 14:

\xi_{\min}

sensitivity sweep on Qwen2.5-7B + MATH500 (

n=500

, 3 seeds per cell). Values in percentage points except tpp. Pre-comparator-correction values (§ 5.2 ); the within-column null finding is not expected to change.

		Verdict (PPFG vs. independent)
Arch.	Rule	Pass@1	Pass@8	Mode rate
Qwen2.5-7B	stag.	eq.	eq.	eq.
Qwen2.5-7B	random	eq.	eq.	eq.
LLaMA-3.1-8B	stag.	eq.	eq.	eq.
LLaMA-3.1-8B	random	eq.	eq.	eq.
DSR1-Qwen-7B	stag.	eq.	incon.	incon.

Appendix

Table 15: TOST equivalence verdicts. Bounds pre-registered at the 3-seed MDE. “eq.” = both one-sided nulls rejected (positive equivalence); “upper” = only upper-side null rejected; “lower” = only lower; “incon.” = neither rejected at this

n

/variance. All three architectures now at

n{=}5

seeds (

\{42,\ldots,46\}

Property	SNDT	RSE	T-STAR	LATS	Metacog. Reuse	PPFG
Trigger event	none (offline)	search dead-ends	none (training)	failure reflection	across problems	PRM prune
Granularity	step-level	distilled exper.	trajectory	verbal summary	reusable behav.	step span
Injection point	solver pre-cond.	subseq. rollouts	training loss	next iteration	across runs	mid-decoding sibling
Verbatim or summary	verbatim	distilled	verbatim	summary	distilled skill	verbatim
Inference-time	✓	✓	✗ (training)	✓	✗ (cross-run)	✓

Appendix

Table 16: Differentiator table. PPFG composes three operational properties (PRM-pruned trigger, in-place sibling injection mid-decoding, verbatim fragment) not jointly realized by prior fragment-grafting work.

Outcome	$\Delta$ mean	CI 95 LB	CI 95 UB
pop_correctness	$+0.0154$	$-0.0308$	$+0.0615$
per_chain_promotion	$-0.0111$	$-0.0365$	$+0.0125$
per_chain_tokens	$-20.20$	$-34.21$	$-7.35$

Appendix

Table 17: Surviving-sibling counterfactual: deltas between PPFG-stag treated populations (injection-and-prune-within-3) and matched-problem-seed independent baseline. Mean and percentile bootstrap CI 95 over

5000

resamples of

N=65

matched pairs.

Architecture	$n$ ev.	post-inj.	baseline	ratio
Qwen2.5-7B-Instr. (3 sd)	$322$	$25.8\%$	$9.4\%$	$2.75\times$
DeepSeek-R1-Dist. (5 sd)	$825$	$21.2\%$	$9.7\%$	$2.18\times$
LLaMA-3.1-8B-Instr. (5 sd)	$2418$	$26.3\%$	$14.4\%$	$1.83\times$

Appendix

Table 18: Cross-architecture replication of the post-injection pruning hazard on MATH500. The matched-step baseline is the prune-within-3 rate over non-injected siblings at the same step indices. CIs are

95\%

; in all three architectures the post-injection and baseline intervals are non-overlapping.

$k$	PPFG-stag	indep	Oracle	Oracle $-$ indep
$1$	$0.5053\pm.0072$	$0.5063\pm.0074$	$0.5070\pm.0075$	$+0.07$ pp
$2$	$0.5828\pm.0075$	$0.5838\pm.0076$	$0.5847\pm.0076$	$+0.09$ pp
$4$	$0.6334\pm.0060$	$0.6340\pm.0059$	$0.6351\pm.0057$	$+0.11$ pp
$8$	$0.6693\pm.0042$	$0.6693\pm.0042$	$0.6707\pm.0031$	$+0.13$ pp

Appendix

Table 19: Oracle targeting upper bound vs. PPFG-stag and independent on Qwen2.5-7B-Instruct + Math-Shepherd + MATH500 (

n{=}500

, 3 seeds). The oracle is the per-chain OR across methods; the Oracle

-

indep delta column is the maximum Pass@

k

a hindsight method-chooser could extract beyond the independent baseline. All deltas fall inside the seed-

\sigma

band (

0.003

–

0.008

Method	Pass@1	Pass@2	Pass@4	Pass@8
Indep (canon)	$0.6442\pm.0050$	—	—	$0.8520\pm.0075$
PPFG-stag (canon)	$0.6428\pm.0049$	—	—	$0.8493\pm.0093$
E1: seamless	$0.6461\pm.0054$	—	—	$0.8520\pm.0075$
E2: re-attention	$0.6470\pm.0045$	—	—	$0.8533\pm.0062$

Appendix

Table 20: Injection-format ablation on Qwen2.5-7B-Instruct + Math-Shepherd + MATH500 (

n{=}500

, 3 seeds). Seed-

\sigma

\approx 0.005

for Pass@1,

0.008

for Pass@8. Pre-comparator-correction values (§ 5.2 ); the parity finding is not expected to change.

Reference: canonical PPFG-stag (no immunity), Pass@1 $0.6233\pm.0069$
$k$	(a) indep	(b) immun. $+$ null	(c) immun. $+$ graft	(c) $-$ (b)
$1$	$0.6238\pm.0050$	$0.6312\pm.0082$	$0.6308\pm.0100$	$-0.04$ pp
$2$	$0.7204\pm.0009$	$0.7254\pm.0023$	$0.7245\pm.0058$	$-0.08$ pp
$4$	$0.7888\pm.0047$	$0.7946\pm.0078$	$0.7929\pm.0063$	$-0.17$ pp
$8$	$0.8367\pm.0115$	$0.8467\pm.0153$	$0.8433\pm.0115$	$-0.33$ pp

Appendix

Table 21: Prune-immunity content control on the first-100 MATH500 slice (Qwen2.5-7B-Instruct + Math-Shepherd,

N{=}8

, 3 seeds). All four arms are scored by one identical extractor/comparator on the identical 100 problems. The immunity lift (c

-

a) and the graft effect (c

-

b) both fall inside seed-

\sigma

; canonical PPFG-stag is shown for reference.

$\Delta$ (pp)	$N{=}16$	$N{=}32$	$N{=}64$	$N{=}128$	$N{=}256$
Pass@1	$-0.06$	$-0.04$	$-0.08$	$-0.15$	$-0.11$
Pass@2	$-0.07$	$-0.07$	$-0.09$	$-0.19$	$-0.11$
Pass@4	$-0.12$	$-0.08$	$-0.07$	$-0.20$	$-0.06$
Pass@8	$-0.27$	$-0.07$	$-0.01$	$-0.15$	$-0.00$
mode rate	$+0.33$	$+0.67$	$+0.33$	$-0.33$	$+0.00$

Appendix

Table 22: Population-size sweep: PPFG-stag minus independent, in percentage points, on the first-100 MATH500 slice (3 seeds per cell, matched populations). Every Pass@

k

delta is

\leq 0.27

pp in magnitude and none is positive at any

N

. The mode rate row changes sign across

N

, consistent with a diversity rather than an accuracy metric.

Inference-Time Intervention

Inference-time intervention changes model inputs, internal signals, or decisions during use to improve safety, reliability, or task performance.

See all 33 papers

Feb 27, 2026

When does Chain-of-Thought Help: A Markovian Perspective

Chain-of-Thought (CoT) prompting is a widely used inference-time technique for improving reasoning, yet its gains are uneven across tasks. We analyze when and why CoT helps by modeling the step-wise reasoning trajectory as a Markov chain. Each intermediate step is a state and the dependence between steps is captured by a transition kernel. Our theory identifies transition alignment, whether instances share a common step-wise transition kernel, as the key determinant of CoT's effectiveness. When transitions are identical across steps, CoT reduces inference-time sample complexity: fewer context sample trajectories suffice to recover the final decision. In contrast, when transitions differ across steps, these gains can vanish. We further quantify how noise in intermediate steps modulates CoT's benefit. Beyond theory, we design synthetic benchmarks that isolate these factors to complement prior results on real-world tasks and to empirically validate our predictions.

Zihan Wang, Yijun Dong, Qi Lei

New York University

Markov Chains CoT Reasoning LLM Reasoning CoT Evaluation

Ask PDF

Jul 13, 2026

OS-Pruner: Pruning Chains-of-Thought of Reasoning Models via Optimal Stopping

Large Language Models (LLMs) have achieved remarkable success in complex reasoning tasks through Chain-of-Thought (CoT) prompting. However, these models often exhibit "computational overthinking," generating redundant reasoning steps that increase latency and cost without improving accuracy. Recent studies suggest that CoT trajectories can be significantly pruned, yet existing methods often rely on forcing a static thinking budget, heuristic filtering, sub-optimal early exit via classification, or expensive re-training. In this paper, we introduce OS-Pruner, a lightweight plug-in framework that formulates chain-of-thought pruning as an optimal stopping problem. Given a reasoning prefix, OS-Pruner learns whether further reasoning is worth its token cost by optimizing an explicit utility that trades off final-answer accuracy against generated length. Our novel formulation enables the model to dynamically assess the sufficient point of termination for a reasoning chain. OS-Pruner is designed to be lightweight during both training and inference, and to provide users with fine-grained control over the reasoning-effort vs. accuracy trade-off. On diverse reasoning benchmarks and base models, OS-Pruner achieves 20-60% reduction in generation length with minimal accuracy sacrifice.

Mohammed Ehab, Aymane El Gadarri, Vivek F. Farias +2

MIT · Columbia Business School

Efficient Language Model Reasoning Sequential Decision Making Optimal Stopping Overthinking in Language Models Large Reasoning Models Efficient Language Model Inference+1

Ask PDF

May 27, 2026

Reasoning that Travels: Dissecting How Chain-of-Thought Transfers Across Models

Large reasoning models (LRMs) often generate extensive chain-of-thought (CoT) traces before producing a final answer. As explicit textual artifacts, these traces can be passed to other models to solve the same task, enabling cross-model reasoning transfer. Yet successful transfer alone does not reveal how the provided CoT contributes to another model's answer. We study this question with a controlled provider--receiver framework, where a provider generates a reasoning trace and a receiver solves the same problem from increasingly longer trace prefixes. We compare force-answer, where the receiver answers directly from the prefix, with free-generation, where it may continue reasoning before answering. Across models and benchmarks, full traces often transfer successfully, but prefix trajectories reveal distinct mechanisms. In force-answer mode, AIME transfer is largely driven by explicit answer availability. MMLU-Pro instead reflects a larger role for receiver competence, while ZebraLogic depends on partial structured-answer information rather than complete-answer leakage alone. In free-generation mode, partial CoTs improve performance across benchmarks, indicating that prefixes can guide continued reasoning. Finally, answer agreement among receivers provides a gold-free signal for stopping provider reasoning early. Overall, cross-model CoT transfer is not a single phenomenon: it can reflect answer extraction, reasoning scaffolding, or receiver-dependent competence.

Xinyuan Cheng, Beiduo Chen, Philipp Mondorf +1

UMaiNLP, Center for Information and Language Processing, LMU Munich, Germany · ÆMunich Center for Machine Learning, Germany

LLM Prompting LLM Reasoning Large Reasoning Models Reasoning Trace Analysis

Ask PDF

Abstract

Inference-Time Intervention

Explore similar work

When does Chain-of-Thought Help: A Markovian Perspective

OS-Pruner: Pruning Chains-of-Thought of Reasoning Models via Optimal Stopping

Reasoning that Travels: Dissecting How Chain-of-Thought Transfers Across Models