Selective Regenerative Decoding: Trajectory-Level Intervention for Inference-Time Reasoning

Sophia Xiao Pu; Yumo Xu; Sailik Sengupta; Millennium Bismay; Ruixue Lian; James Gung; Yi-an Lai; Arshit Gupta

cs.AIAug 25, 2026

Selective Regenerative Decoding: Trajectory-Level Intervention for Inference-Time Reasoning

Authors: Sophia Xiao Pu, Yumo Xu, Sailik Sengupta, Millennium Bismay, Ruixue Lian, James Gung, Yi-an Lai, Arshit Gupta

Organizations: University of California, Santa Barbara · Netflix · Amazon Science · Meta

Abstract

Inference-time decoding methods improve LLM reasoning by exploring multiple candidate trajectories, yet treat each trajectory as atomic: either retaining it whole or discarding it irreversibly. This wastes computation on partially promising candidates whose high-quality prefixes are abandoned alongside degraded suffixes. We introduce Selective Regenerative Decoding (SRD), which routes each candidate to discard, keep, or refine only the degraded portion of the suffix while preserving the useful prefix of borderline candidates, without requiring a larger target model. Under mild assumptions, SRD achieves a provable 1.28-to-1.36-fold gain in sample efficiency over rejection sampling with strictly higher expected trajectory quality, with the gain growing as the candidate pool grows. Across MATH500, GPQA Diamond, HotpotQA, and AlpacaEval with multiple generation-reward model pairs, SRD matches Best-of-N accuracy with substantially fewer generated tokens and outperforms speculative rejection in low-compute regimes. By enabling segment-level intervention rather than whole-trajectory selection, SRD opens a previously underexplored region of the accuracy-compute tradeoff for inference-time reasoning.

Figures & tables

Panel of Figure 1 — Figure 1: Comparison of decoding strategies on three trajectories ( $\tau_{1}$ : high quality throughout; $\tau_{2}$ : high-quality prefix with degraded suffix; $\tau_{3}$ : low quality). Best-of- $N$ fully generates all candidates and selects the best, discarding the rest. Speculative Rejection terminates trajectories when quality drops, saving compute but permanently losing useful prefixes. SRD preserves $\tau_{2}$ ’s high-quality prefix and selectively regenerates only its degraded suffix (teal), recovering a trajectory that both baselines waste.

Generation Model	Reward Model
MATH500
Llama-3.1-8B-Instruct	AceMath-7B-RM
Qwen2.5-Math-1.5B-Instruct	AceMath-7B-RM
GPQA DIAMOND
Llama-3.1-8B-Instruct	Skywork-o1-Open-PRM-7B
Qwen3-4B-Instruct	Skywork-o1-Open-PRM-7B

Table 1: Generation and reward model combinations used for each dataset.

Figure 2: Accuracy-compute trade-offs of inference-time decoding methods across four benchmarks and multiple generation-reward model combinations. Compute is measured by the average number of generated tokens from the language model. SRD consistently achieves competitive performance with respect to Best-of-

N

and Speculative Rejection under lower or comparable generation budgets by selectively reusing and refining partial trajectories.

$N$	Acc.	Time (s)	In Tok.	Out Tok.	Drafter (time/calls)	Scorer (time/calls)	Editor (time/calls)	Router (time/calls)
10	0.544	7.21	6629	2166	4.73 / 4.1	1.29 / 5.6	1.18 / 1.0	<0.01 / 3.8
50	0.611	18.07	32895	10927	7.43 / 4.2	6.98 / 6.6	3.59 / 2.2	<0.01 / 5.3
100	0.640	29.76	65737	21840	10.96 / 4.2	13.53 / 6.7	5.13 / 2.3	<0.01 / 5.5

Table 2: Component-level breakdown of SRD on MATH500 under different values of

N

. We report overall accuracy, runtime, token statistics, and per-component time and call frequency in the format of time / calls .

$\theta_{\text{high}}$	$\theta_{\text{low}}$	Accuracy	Out Tokens
0.5	0.3	0.5444	2166
0.5	0.5	0.5400	1973
0.5	0.2	0.5267	2325
0.8	0.5	0.4667	1871
0.7	0.3	0.4467	2091

Table 3: Effect of routing thresholds on MATH500 (Llama-3.1-8B-Instruct,

N=10

Refinement Policy	Accuracy	Out Tokens
Dataset: MATH500
Reroute (global)	0.5444	2166
Keep on refine	0.5333	2149
Force keep	0.5267	2162
Refine-Bo3	0.5222	2521
Refine-Bo10	0.5244	3819

Table 4: Effect of different refinement policies on representative reasoning tasks (

N=10

Scoring Interval	Accuracy	Out Tokens
Dataset: MATH500
1	0.5533	2389
5	0.5422	2383
10	0.5667	2387
20	0.5556	2388
50	0.5689	2387

Table 5: Ablation on scoring interval (localization granularity), which determines where SRD begins suffix regeneration within a trajectory routed to Refine .

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

$N$	$p_{H}$	$p_{M}$	$p_{L}$	$\hat{\rho}$	Efficiency Gain
10	0.522	0.148	0.329	1.00 (100%)	1.28 $\times$
20	0.513	0.180	0.305	1.00 (100%)	1.35 $\times$
30	0.509	0.182	0.308	1.00 (100%)	1.36 $\times$

Appendix

Table 6: Empirical routing distributions and refinement efficacy (

\hat{\rho}

) on MATH500 (Qwen2.5-Math-1.5B, AceMath-7B-RM, 50 samples per

N

). The efficiency gain is computed as

1+\hat{\rho}\cdot p_{M}/p_{H}

per Theorem .

$N$	SRD $\bar{R}_{\max}$	RS $\bar{R}_{\max}$	$\Delta_{n}$	SRD wins	RS wins	Ties	Refine improved	Pre $\to$ Post
10	17.70	17.34	$+0.36\pm 1.70$	46%	18%	36%	78%	2.85 $\to$ 5.96
20	18.75	18.47	$+0.28\pm 4.11$	54%	24%	22%	98%	3.41 $\to$ 6.79
30	19.96	18.66	$+1.29\pm 3.41$	52%	24%	24%	100%	3.69 $\to$ 6.87

Appendix

Table 7: Theorem validation on MATH500 (Qwen2.5-Math-1.5B, AceMath-7B-RM, 50 samples per

N

). Both methods start from identical initial drafts via seeded generation.

\Delta_{n}=\bar{R}_{\max}-\bar{R}_{\max}

(mean

\pm

std). “Pre

\to

Post” reports mean reward before and after refinement.

Efficient Language Model Reasoning

Efficient language model reasoning studies how to improve problem-solving accuracy while reducing inference, memory, latency, and training costs.

See all 371 papers

Entropy-Aware Token Rejection for Improving Speculative Decoding
Dec 29, 2025Tiancheng Su, Meicong Zhang, Guoxiu HeSpeculative Decoding LLM Inference Acceleration
School of Economics and Management East China Normal University
$\nabla$ -Reasoner: LLM Reasoning via Test-Time Gradient Descent in Latent Space
Mar 5, 2026Peihao Wang, Ruisi Cai, Zhen Wang +4LLM Reasoning Test-Time Optimization
The University of Texas at Austin · UC San Diego · TTIC +1
Calibrated Speculative Decoding: Frequency-Guided Candidate Selection for Efficient Inference
Apr 15, 2026Xuwen Zhou, Fangxin Liu, Chao Wang +5Speculative Decoding LLM Inference Acceleration
Shanghai Jiao Tong University · Alibaba Cloud Computing

Dec 29, 2025J/K move · Enter open · S save

Entropy-Aware Token Rejection for Improving Speculative Decoding

Tiancheng Su, Meicong Zhang, Guoxiu He

School of Economics and Management East China Normal University

Speculative Decoding LLM Inference Acceleration LLM Reasoning

Open paper AskPDF arXiv

Speculative decoding (SD) accelerates large language model (LLM) inference by using a lightweight draft model to propose tokens and a stronger target model to verify them. However, standard SD is mainly designed for acceleration, and its output quality is typically constrained by the target model. In this work, we propose Entropy-Aware Speculative Decoding (EASD), a lightweight and training-free extension of SD that improves reasoning quality through token-level entropy-guided rejection. EASD detects cases where both draft and target models exhibit high uncertainty while strongly overlapping in their top predictions. In such uncertain-agreement cases, EASD rejects the aligned token and resamples from the target distribution, preventing low-confidence errors from propagating. Experiments on challenging reasoning benchmarks show that EASD consistently improves accuracy over standard SD and reward-guided variants while maintaining comparable inference efficiency. Notably, EASD can surpass the standalone performance of the target model, suggesting that speculative decoding can serve not only as an acceleration method but also as an effective mechanism for improving reasoning quality. The code is available at https://github.com/ECNU-Text-Computing/EASD.

Abstract

Efficient Language Model Reasoning

Explore similar work

Entropy-Aware Token Rejection for Improving Speculative Decoding

∇\nabla∇-Reasoner: LLM Reasoning via Test-Time Gradient Descent in Latent Space

Calibrated Speculative Decoding: Frequency-Guided Candidate Selection for Efficient Inference

$\nabla$ -Reasoner: LLM Reasoning via Test-Time Gradient Descent in Latent Space