Adaptive Replay Refinement: Allocating Test-Time Compute Between Revision and Resampling

Ahsan Bilal; Muhammad Ahmed Mohsin; Muhammad Umer; Ayesha Mohsin; Sarmad Bilal; Dean F. Hougen

cs.CLOct 20, 2026

Adaptive Replay Refinement: Allocating Test-Time Compute Between Revision and Resampling

Authors: Ahsan Bilal, Muhammad Ahmed Mohsin, Muhammad Umer, Ayesha Mohsin, Sarmad Bilal, Dean F. Hougen

Organizations: University of Oklahoma · Stanford University · National University of Sciences and Technology

Abstract

Test-time scaling allocates additional inference compute to refining existing solutions or generating new ones. This raises a fundamental question: Should I keep improving this solution, or stop working on it and try a completely new solution? Existing methods make this decision using fixed refinement schedules, self-verification judgments, or learned controllers, which can waste compute or rely on signals that may be incorrect or require additional training. We introduce Adaptive Replay Refinement (ARR), a training-free method that detects when a stationary revision reaches a fixed point and reallocates the released compute to a new independent sample; replay decoding treats the current solution as a draft and verifies unchanged tokens in parallel, enabling efficient fixed-point detection. Across seven models from five families and seven benchmarks, ARR matches majority voting on three of five mathematical reasoning benchmarks at equal compute, while reducing critical-path serial passes by up to 2.4 times. On MATH500, ARR achieves 84.6% accuracy with only 41% of the serial passes required by majority voting. These results demonstrate that detecting refinement exhaustion enables adaptive reallocation from revision to independent sampling, improving inference efficiency while highlighting the limits of self-revision. Code is available at https://github.com/AhsanBilal7/adaptive_refine.

Figures & tables

Figure 1: Revision on MATH500 with Qwen2.5-Math-7B-Instruct. (a) Fraction of the four initial roots of ARR Depth (

B{=}800

) at a fixed point: 35% after one revision and 75% by budget exhaustion. (b) Fraction of Self-Refine compute wasted on already converged solutions.

Figure 2: Overview of ARR. (a) Replay decoding accelerates revision. (b) Exact reproduction identifies a fixed point and retires the root. (c) Released compute is reallocated to new roots until the budget is exhausted.

Figure 3: Accuracy against median critical-path serial passes (Qwen2.5-Math-7B-Instruct); each point is one budget.

Figure 4: Revision in ARR on MATH500 (

B{=}800

). (a) Accuracy of roots by revision depth; bars: roots that reach depth

k

. (b) Answer changes per revision; bars: fixed-point rate. (c) Accuracy of the running vote.

Figure 5: (a) Net repair of

F_{x}

vs. ARR

-

Majority Voting; open = LiveBench, circled = repaired answer format. (b) Accuracy of the other stopping rules and Majority Voting relative to ARR, with 95% CIs. (c) Their roots and serial passes relative to ARR.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Accuracy against the FLOP budget

B

(Qwen2.5-Math-7B-Instruct). Error bars show the standard deviation over seeds.

Figure 7: Compute and accuracy over all settings: (a) TFLOPs used, (b) serial passes relative to ARR, and (c) accuracy difference from ARR.

Figure 8: Cost of revision for Qwen2.5-Math-7B-Instruct. (a) Ratio of the median serial passes of ARR to those of ARR AR . Replay gives a larger decrease at larger budgets. (b) Output tokens for each question at

B{=}800

Figure 9: Accuracy (%) of every method on the other models at their largest budget (

B{=}800

for Ministral-8B and Qwen3-8B, 520 for Qwen3.5-2B, 1,400 for Gemma-4-E4B, and 2,600 for Qwen3.5-9B).

Method	Update of a root	Decoding of the update	Explicit verifier	A root stops when	Slot of a stopped root
ARR	$F_{x}(y)$	replay ( $W{=}32$ )	no	$F_{x}(y)=y$ or $F_{x}(y)\notin\mathcal{Y}$	new root
ARR AR	$F_{x}(y)$	autoregressive greedy	no	same as ARR	new root
ARR PLD	$F_{x}(y)$	prompt lookup	no	same as ARR	new root
ARR Depth	$F_{x}(y)$	replay	no	same as ARR	not reused
Self-Refine	$F_{x}(y)$	autoregressive greedy	no	after $D{=}4$ revisions	not reused
SETS	$F_{x}^{\mathrm{vc}}(y)$	autoregressive greedy	yes	$F_{x}^{\mathrm{vc}}(y)=y$ or truncated	new root

Appendix

Table 1: Compared methods. The rows from ARR AR to SETS change one component of ARR each.

Figure 10: Accuracy gains from revision depth and replacement breadth across benchmarks, budgets, and models.

Figure 11: Depth-to-breadth reallocation under fixed-point replacement across budgets and models.

Benchmark	$B$	ARR	Answer	Once	Majority Voting	$\Delta$	Roots	Passes
MATH500	200	81.6	81.6	81.8	83.2	$+0.0$ $[-1.1,+1.2]$	5.8 / 9.3	1.20
	400	83.8	83.0	83.4	83.2	$-0.8$ $[-2.4,+0.8]$	9.5 / 19.1	1.19
	800	84.0	84.6	84.4	84.0	$+0.6$ $[-0.8,+1.8]$	16.1 / 38.5	1.16
OlympiadBench	200	44.0	45.0	44.0	46.4	$+1.0$ $[-0.7,+2.7]$	5.2 / 7.0	1.14
	400	45.7	46.9	47.6	48.1	$+1.2$ $[-1.2,+3.6]$	8.2 / 14.0	1.19
	800	47.6	49.9	48.2	49.9	$+2.2$ $[+0.3,+4.2]$	13.3 / 27.8	1.18

Appendix

Table 2: Stopping rules with the replacement of ARR for Qwen2.5-Math-7B-Instruct. Answer: retire a root when its extracted answer does not change; Once: retire a root after one revision.

\Delta

: Answer

-

ARR with a 95% paired CI. Roots: roots per question of ARR / Answer. Passes: serial passes of Answer / ARR. All columns use the seeds of the stopping-rule runs, so

\Delta

is paired. On MATH500, OlympiadBench, and GSM8K, these are a subset of the main-run seeds, so ARR and Majority Voting differ from the all-seed means of Figure 6 (e.g., 84.0% vs. 84.6% on MATH500 at

B{=}800

). In all tables, blue marks the best value and red the second-best value.

Figure 12: Accuracy up to

B{=}1{,}600

Figure 13: Change of accuracy from the default

\tau{=}0.7

at (a)

\tau{=}0.5

and (b)

\tau{=}1.0

, for each benchmark and budget

B

Method	Answer	Votes	Serial passes
Majority Voting	circle ✗	34 vs. 27 (61 samples)	13,049
SETS	circle ✗	-	-
ARR	ellipse ✓	13 vs. 5	7,877

Appendix

Table 3: Outcome on MATH500 problem 358 at

B{=}800

TFLOPs. Votes give the winning answer against the runner-up. Serial passes are counted on the critical path.

	Sample	Rev. 1	Rev. 2	Rev. 3	Rev. 4
Draft	-	428	76	667	620
Accepted	-	0	0	602	620
Discarded	-	0	0	6	0
Decoded	428	75	666	17	0
Passes	428	76	667	37	21
Answer	circle	circle	circle	ellipse	ellipse (FP)

Appendix

Table 4: Trajectory of root 0. Accepted and discarded denote replay-verified draft tokens; decoded denotes greedy single-token steps, and FP denotes a fixed point.

Figure 14: (a) Accuracy and (b) serial passes relative to the default

W{=}32

, for each benchmark and budget.

Figure 15: Change of accuracy from the default

N{=}4

at (a)

N{=}2

and (b)

N{=}8

, for each benchmark and budget. (c) Accuracy relative to ARR (

N{=}8

) on AMC23 and AIME24; dots denote individual settings, large markers denote means.

Method	TF	Roots	FP	W $\to$ C	C $\to$ W
ARR	789	16.2	.23	663	357
ARR AR	788	15.9	.22	666	374
ARR PLD	797	13.3	.30	463	230
ARR Depth	381	4.0	.08	300	234
Self-Refine	268	4.0	.53	173	96
SETS	793	14.8	.69	244	258

Appendix

Table 5: Mechanism on MATH500 at

B{=}800

TFLOPs with Qwen2.5-Math-7B-Instruct. TF: TFLOPs per question; Roots: roots per question; FP: fixed-point rate; W

\to

C, C

\to

W: answer changes (Appendix G ).

Figure 16: Accuracy of ARR with the revision prompt

\rho

and two paraphrases; (g) uses Qwen2.5-7B-Instruct.

Figure 17: Accuracy against a budget of serial passes per question. Sequential Revision is omitted for scale.

Figure 18: First seed against the mean over all seeds at the largest budget. (a) Accuracy of ARR and Majority Voting. (b) Accuracy differences between methods.

Benchmark	$B$	ARR	Answer	Once
AMC23	200	62.5	62.5	62.5
	400	65.0	72.5	65.0
	800	75.0	72.5	77.5
AIME24	200	26.7	23.3	20.0
	400	26.7	23.3	23.3
	800	33.3	23.3	26.7

Appendix

Table 6: Accuracy (%) of the stopping rules of Table 2 for Qwen3-8B.

Method	Latency (s)	Passes	TFLOPs	Acc.
ARR	78.5	2,748	398	90.0
ARR AR	83.3	3,926	398	90.0
ARR PLD	42.5	1,593	399	90.0
Majority Voting	176.5	6,878	381	90.0
ConMA	18.4	644	93	87.5

Appendix

Table 7: Wall-clock latency on one A100 GPU (Qwen2.5-Math-7B-Instruct, a subset of MATH500 problems,

B{=}400

, one question at a time). Latency and serial passes are medians for each question; accuracy is on this subset.

Method	Training-	No step	Released compute	Signal that stops or redirects refinement
Method	free	structure	$\rightarrow$ new attempts	Signal that stops or redirects refinement
Self-Refine ( Madaan et al., 2023 )	✓	✓	✗	stop word in self-feedback, or iteration limit
SETS ( Chen et al., 2025a )	✓	✓	✗	self-verification verdict “correct”, or round limit
Thought-ICS ( Samanta et al., 2026 )	✓	✗	✗	self-localization of the first incorrect thought
Adaptive-Consistency ( Aggarwal et al., 2023 )	✓	✓	✗	agreement of the vote distribution (stops sampling)
ConMA ( Wang et al., 2026 )	✓	✓	✓	token-probability confidence and answer frequency

Appendix

Table 8: Signals that end or redirect refinement in the closest methods. The signal of ARR is an exact property of a deterministic operator, not a verdict or a score. ARR gets the signal with parallel draft verification, which comes from prior work ( Sun et al., 2021 ; Leviathan et al., 2023 ; Saxena, 2023 ) .

Inference-Time Compute Allocation

Inference-time compute allocation decides where, when, and how much computation models spend during prediction to improve quality under resource limits.

See all 82 papers

What If We Allocate Test-Time Compute Adaptively?
Feb 1, 2026Ahsan Bilal, Ahmed Mohsin, Muhammad Umer +4Inference-Time Search Test-Time Scaling
University of Oklahoma, OK, USA · Stanford University, CA, USA · Universitat Pompeu Fabra, Barcelona, Spain +2
SVR: Self-Verifying Refinement via Joint Verdict-Confidence Reinforcement Learning for Adaptive Test-Time Compute
Jul 30, 2026Hongyu Chen, Liang Lin, Guangrun WangLanguage Model Self-Verification RL for Language Model Reasoning
1Sun Yat-sen University · 2Guangdong Key Laboratory of Big Data Analysis and Processing · 3X-Era AI Lab
Refining Over Resampling: Test-Time Self-Correction for LLM Reasoning
Aug 6, 2026Ahsan Bilal, Muhammad Ahmed Mohsin, Muhammad Umer +4Numerical Reasoning in Language Models LLM Self-Correction
University of Oklahoma · Stanford University · Universitat Pompeu Fabra +1

Feb 1, 2026J/K move · Enter open · S save

What If We Allocate Test-Time Compute Adaptively?

Ahsan Bilal, Ahmed Mohsin, Muhammad Umer, Ali Subhan+3

University of Oklahoma, OK, USA · Stanford University, CA, USA · Universitat Pompeu Fabra, Barcelona, Spain+2

Inference-Time Search Test-Time Scaling Inference-Time Compute Allocation Inference-Time Guidance

Open paper AskPDF arXiv

Test-time compute scaling allocates inference computation uniformly, uses fixed sampling strategies, and applies verification only for reranking. In contrast, we propose a verifier-guided adaptive framework treating reasoning as iterative trajectory generation and selection. For each problem, the agent runs multiple inference iterations. In each iteration, it optionally produces a high-level plan, selects a set of reasoning tools and a compute strategy together with an exploration parameter, and then generates a candidate reasoning trajectory. A process reward model (PRM) serves as a unified control signal: within each iteration, step-level PRM scores are aggregated to guide pruning and expansion during generation, and across iterations, aggregated trajectory rewards are used to select the final response. Across datasets, our dynamic, PRM-guided approach consistently outperforms direct test-time scaling, yielding large gains on MATH-500 and several-fold improvements on harder benchmarks such as AIME24 and AMO-Bench. We characterize efficiency using theoretical FLOPs and a compute intensity metric penalizing wasted generation and tool overhead, demonstrating that verification-guided allocation concentrates computation on high-utility reasoning paths.

Abstract

Inference-Time Compute Allocation

Explore similar work

What If We Allocate Test-Time Compute Adaptively?

SVR: Self-Verifying Refinement via Joint Verdict-Confidence Reinforcement Learning for Adaptive Test-Time Compute

Refining Over Resampling: Test-Time Self-Correction for LLM Reasoning