Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneck

Davide Romano; Kanak Raj; Jerrod Parker; Daniele Giofrè

cs.CLAug 19, 2026

Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneck

Authors: Davide Romano, Kanak Raj, Jerrod Parker, Daniele Giofrè

Organizations: Thomson Reuters Foundational Research, 5 Canada Square, London

Abstract

Test-time scaling (TTS) improves language model outputs by spending additional inference compute - generating multiple candidates, searching over partial sequences, or iteratively refining drafts. These techniques yield large gains on mathematics and code, but have been developed and stress-tested almost exclusively on tasks where verification is straightforward. We conduct the first compute-normalised comparison of five TTS families across five open-ended generation benchmarks spanning medicine, law, finance, general chat, and creative writing - grounded in a unified framework that decomposes the effectiveness of each method's token budget into exploration and exploitation. The answer depends on which side of that decomposition you examine. Scaling exploration works: the best candidate in the pool improves steadily with compute across all settings. What breaks is exploitation - the step that converts a rich candidate pool into a final output. With state-of-the-art generators, reward models correlate at only $ρ_v \approx 0.12$ with true quality, rendering selection near-random regardless of budget. Tree search amplifies this failure through diversity collapse. Refinement helps on one of five benchmarks; its apparent gains elsewhere are confounded. Only synthesis across candidates (Fusion) consistently improves over single-sample baselines, yet still recovers only ~40% of available quality. The candidate pool is not the bottleneck - choosing from it is.

Figures & tables

Method	$T_{x}$	$T_{e}$	$N$	Exploitation type
BoN + ORM	$\approx 0$	$\approx T$	$\lfloor T/\bar{\ell}\rfloor$	External scoring
Tree search	$\approx 0$	$\approx T$	$B\times W^{d}$ (collapses)	External scoring
SR	$K\bar{\ell}_{c}$	$T-K\bar{\ell}_{c}$	$K=T/(\bar{\ell}+\bar{\ell}_{c})$	Generative (sequential)
Fusion	$\bar{\ell}_{f}$	$T-\bar{\ell}_{f}$	$N-1$	Generative (synthesis)
Budget Forcing	$0$	$T$	$1$	None (single trajectory)

Table 1: Token cost accounting per TTS family under budget

T

\bar{\ell}

: mean generation length;

\bar{\ell}_{c}

: mean critique length;

\bar{\ell}_{f}

: mean fusion output length;

K

: refinement steps;

N

: candidates;

B

: beam width;

W

: branching factor;

d

: depth.

Benchmark	Domain	$n$	Evaluation	Source	Scoring	Avg source items
LEXam [ 7 ]	Law	516	Reference-guided	Professor solutions	Holistic 0–1	1
HealthBench [ 2 ]	Medicine	5000	Rubric-only	Physician rubrics	Binary per criterion	11.4
PRBench [ 1 ]	Finance, Law	1650	Rubric-only	Expert rubrics	Binary per criterion	17.7
WildBench [ 13 ]	Chat/General	1024	Rubric-only	LLM-generated checklist	Holistic 1–10	$\sim$ 11
WritingBench [ 30 ]	Writing	555	Rubric-only	LLM-generated rubric	1–10 per criterion	5

Table 2: Benchmark characteristics.

Figure 1: Schema for the six TTS methods in this paper. PF is the only method reproduced as it is, the other methods are variations from previous papers

	HealthBench					PRBench					LEXam
	N=1	Low	Mid	High	XHigh	N=1	Low	Mid	High	XHigh	N=1	Low	Mid	High	XHigh
BoN+Skywork	0.536	0.538	0.539	0.539	0.539	0.292	0.293	0.294	0.293	0.292	0.525	0.527	0.529	0.531	0.530
BoN+Llama70B	0.536	0.539	0.541	0.541	0.541	0.292	0.293	0.295	0.296	0.298	0.525	0.528	0.532	0.533	0.534
Fusion	—	—	0.574	0.587	0.588	—	—	0.315	0.324	0.331	—	—	0.546	0.546	0.547
Beam Search	—	—	0.534	0.526	0.532	—	—	0.289	0.285	0.285	—	—	0.516	0.526	0.525
Particle Filter	—	—	0.532	0.532	0.530	—	—	0.286	0.282	0.282	—	—	0.531	0.527	0.526

Table 3: Realised quality for Qwen3.5-35B-A3B across five benchmarks and four compute levels. Cell colour encodes delta from the single-sample baseline (BoN@1): green = improvement, red = regression. Bold = gain

\geq 0.03

over baseline.

Figure 2: Oracle (full bar) vs. realised (hatched) quality per method at matched compute ( XHigh ), Qwen3.5-35B-A3B. All qualitative patterns above replicate under the naive oracle (Appendix G ). In that appendix figure, the naive estimator systematically overestimates oracle quality, so the exploitation gaps shown there are conservative lower bounds on the true exploitation gap.

Method	Oracle quality	Headroom capture	Failure mode
BoN + ORM	High	${\sim}15\%$ ( $\hat{\rho}_{v}\approx 0.12$ )	RM miscalibrated (§ 6.1 )
Tree search	Collapsed (diversity loss)	Negative on most tasks	PRM destroys diversity (§ 6.2 )
Fusion	$\geq$ BoN	${\sim}40\%$ (best, still low)	Residual gap (§ 6.3 )
SR	Lower than BoN on 3/5 benchmarks	Genuine gain on 1/5 (PRBench)	Refinement mostly hurts; where it helps, gains trace to subtask composition or verbosity (§ 6.3 )

Table 4: Diagnosis per method family. Oracle quality reflects exploration; headroom capture reflects exploitation.

Figure 3: Left:

\hat{h}

vs.

\hat{\rho}_{v}

across

N=152

task-level points. Identity line

y=x

is the theoretical prediction (Equation 4 ); empirical fit

y=1.198\,\hat{\rho}_{v}-0.011

confirms it (

R^{2}=0.66

\rho=0.81

p<10^{-36}

). Right: Headroom capture per method at XHigh compute. The headrooms for all models are in Appendix J

Figure 4: Left: Mean pairwise cosine distance across final outputs at High compute. Right: Particle trajectory similarity over normalised generation depth.

Figure 5: Left: Fusion output quality vs. a random candidate drawn from the candidates not included in the shared sub-pool common to both BoN and Fusion, per benchmark. Right: Oracle-position frequency (bars, left axis) and mean score trajectory (line, right axis) across SR iterations.

Appendix figures & tables65 assets

Supplementary material from the paper’s appendix.

Appendix

Band	BoN	Beam Search	PF	Sequential Refinement	Budget Forcing	Fusion
Low	$n=2$	—	—	iter $=3$	$2\bar{\ell}_{t}+\bar{\ell}$	—
Mid	$n=4$	$W=2,k=2$	$N=4$	iter $=6$	$4\bar{\ell}_{t}+\bar{\ell}$	subset $=3$
High	$n=8$	$W=2,k=4$	$N=8$	iter $=12$	$8\bar{\ell}_{t}+\bar{\ell}$	subset $=6/7$
XHigh	$n=16$	$W=4,k=4^{\dagger}$	$N=16$	iter $=16$	$16\bar{\ell}_{t}+\bar{\ell}$	subset $=15$

Appendix

Table 5: Compute band configurations. Fusion High uses subset

=7

for Qwen3.5 and subset

=6

for OLMo3. Entries marked — were not run.

\bar{\ell}_{t}

: mean thinking tokens per BoN sample;

\bar{\ell}

: mean answer tokens per BoN sample. † Two tasks ran at

W=3,k=4

instead of

W=4,k=4

(see Table 48 ).

Weight class	Integer weight
critically important	$\{8,9,10\}$
important	$\{4,5,6,7\}$
slightly important	$\{1,2,3\}$
slightly detrimental	$\{-3,-2,-1\}$
detrimental	$\{-7,-6,-5,-4\}$
critically detrimental	$\{-10,-9,-8\}$

Appendix

Table 6: PRBench weight class to integer point ranges.

Benchmark	Native judge	Protocol	Calls/ex
LEXam	GPT-4o + Qwen3-32B + DeepSeek-V3	holistic, min-agg ensemble	$3$
HealthBench	GPT-4.1	per-criterion binary	$\sim 11$
PRBench	o4-mini	per-criterion binary	$\sim 17$
WildBench	GPT-4.1	holistic + checklist	$1$
WritingBench	Claude 3.7 Sonnet	per-criterion $1$ – $10$	$5$

Appendix

Table 7: Native judges adopted verbatim from each benchmark’s upstream release. Calls/example is the average number of judge invocations per candidate completion under the native protocol; per-criterion designs (HealthBench, PRBench, WritingBench) fan out into many calls per example.

Generator	Temperature	Top- $p$	Top- $k$	Presence penalty	Thinking mode
Qwen3.5 (9B, 35B-A3B)	1.0	0.95	20	1.5	enabled
OLMo3 (7B, 32B)	0.8	0.9	-1	0	enabled

Appendix

Table 8: Canonical sampling profile per generator family.

Method	Settings
BoN	XHigh pool ( $N{=}16$ ) generated once; lower bands subsampled without replacement; selection $=\operatorname{argmax}$ ORM score.
Beam Search	$W$ beams, $k$ candidates/step (Table 5 ); delimiter \n\n ; scored by min per-step PRM on full path; top- $W$ kept globally; EOS $=$ beam complete; PRM: VersaPRM-8B; depth 300 (500 for deterministic).
Particle Filter	$N$ particles (Table 5 ); delimiter \n\n ; loop order: score $\to$ resample $\to$ generate; multinomial resampling with replacement, weights $=\mathrm{softmax}(\mathbf{w},\,T{=}1.0)$ ; PRM: VersaPRM-8B; max iterations 300.
Sequential Refinement	Forced iterations (stop-check ignored); history window $K{=}3$ (draft, feedback) pairs; feedback & stop-check: thinking disabled, stop-check at $T{=}0$ ; </think> stripped from drafts before each prompt call.
Budget Forcing	2-phase decode: Phase 1 suppresses end-of-thinking tag and injects a wait-string to extend thinking; Phase 2 generates answer freely from accumulated context.
Fusion	Random subset without replacement from precomputed BoN- $16$ pool; candidates shuffled and </think> -stripped before synthesis; single-shot direct synthesis with thinking enabled.

Appendix

Table 9: Method-specific algorithmic settings beyond the compute-band axis.

Algorithm	HealthBench	LEXam	PRBench	WildBench	WritingBench	Total
BoN (oracle, $n=16$ )	4,556	202	4,995	173	506	10,432
PF-High (oracle, $n=16$ )	1,519	74	1,704	54	176	3,527
Fusion	1,524	61	2,219	48	164	4,015
Beam Search	1,146	44	1,657	37	120	3,004
PF (normal)	1,146	44	1,657	37	120	3,004
Budget Forcing	764	29	1,105	25	80	2,003

Appendix

Table 10: Total LLM-Judge input tokens (millions) by algorithm family and benchmark.

	Tasks	Wall-clock (h)	GPU-hours
Open-QA	164	2,642.6	21,104
Verifiable	93	342.2	2,737
Total	257	2,984.8	$\approx$ 23,800

Appendix

Table 11: Combined paper-level totals across Open-QA and verifiable evaluation tracks. Wall-clock hours measure end-to-end run duration; GPU-hours scale by node size.

Judge	Macro F1
GPT-4.1	0.709
o4-mini	0.692
o3	0.681
Qwen3.5-397B-A17B (ours)	0.679
GPT-4.1 mini	0.661
Inter-physician mean	0.655 ( $\pm\,0.102$ )

Appendix

Table 12: HealthBench judge agreement with physician annotations (Macro F1). The inter-physician baseline is the mean pairwise Macro F1 across 184 physicians with sufficient annotation data.

Benchmark	$n$ pairs	Metric	Agreement
PRBench	1,186	$\kappa$ / Macro F1	0.679 / 0.838
LEXam	77	Pearson $r$ / MAE	0.813 / 22.2
WildBench	147	QWK / MAE	0.564 / 1.35
WritingBench	404	QWK / MAE	0.408 / 1.51

Appendix

Table 13: Criterion-level agreement between the unified (Qwen3.5-397B-A17B) and each benchmark’s native judge. PRBench is evaluated on a

5\%

sample; the remaining benchmarks on a

15\%

sample. Metrics are reported in the form most natural to each native protocol and are not directly comparable across rows.

Benchmark	Square root of mean var (normalized)
HealthBench	0.076
PRBench	0.034
LEXam	0.055
WildBench	0.055
WritingBench	0.030

Appendix

Table 14: Square root of the per-benchmark mean per-entry judge variance (micro-averaged across entries).

Figure 6: Naive (unadjusted) vs ideal oracle scores across compute for qwen3.5-9b BoN.

Figure 7: Oracle (full bar) vs. realised (hatched) quality per method at matched compute ( XHigh ), Qwen3.5-35B-A3B, using the naive (uncorrected) oracle.

Figure 8: Headroom capture per method at XHigh compute, using the naive (uncorrected) oracle. Compare with Figure 3 (right) which uses the bias-corrected oracle.

Figure 9:

h

vs.

\rho_{v}

across

N=152

task-level points using the naive (uncorrected) oracle and measured (non-denoised)

\rho_{v}

. The empirical fit closely tracks the identity line

y=x

, mirroring the linear relationship

\hat{h}\approx\hat{\rho}_{v}

that Figure 3 (left) demonstrates under the bias-corrected oracle and denoised

\hat{\rho}_{v}

. Same conclusion holds under both conventions: headroom capture is bounded by verifier correlation regardless of whether we de-noise the oracle and the correlation estimate.

	HealthBench				PRBench				LEXam
	Low	Mid	High	XHigh	Low	Mid	High	XHigh	Low	Mid	High	XHigh
BoN+Skywork	+0.047	+0.038	+0.030	+0.025	+0.038	+0.023	+0.014	-0.005	+0.047	+0.062	+0.065	+0.051
Fusion	—	+0.670	+0.580	+0.495	—	+0.390	+0.357	+0.339	—	+0.355	+0.246	+0.209
Particle Filter	—	—	—	-0.168	—	—	—	-0.293	—	—	—	+0.006
Self - Refine	-1.096	-0.907	-0.553	-0.399	+0.455	+0.467	+0.495	+0.496	-1.148	-0.833	-0.959	-0.858

Appendix

Table 15: Headroom capture for Qwen3.5-35B :

h=(\text{realized}-\mu)/(\text{oracle}_{\text{ideal}}-\mu)

, where

\mu

is the BoN

N{=}1

baseline (single-inference mean).

h=1

means full exploitation of the oracle gap;

h=0

matches

\mu

;

h<0

underperforms

\mu

. green = positive (gain captured), red = negative (below baseline). Bold when

|h|\geq 0.20

	HealthBench				PRBench				LEXam
	Low	Mid	High	XHigh	Low	Mid	High	XHigh	Low	Mid	High	XHigh
BoN+Skywork	+0.084	+0.080	+0.084	+0.089	+0.077	+0.071	+0.061	+0.050	+0.073	+0.058	+0.059	+0.056
Fusion	—	+0.615	+0.512	+0.473	—	+0.301	+0.346	+0.313	—	+0.419	+0.445	+0.339
Self - Refine	-3.615	-1.681	-1.175	-0.918	+0.251	+0.323	+0.404	+0.329	-1.104	-0.970	-0.952	-1.147

Appendix

Table 16: Headroom capture for Qwen3.5-9B :

h=(\text{realized}-\mu)/(\text{oracle}_{\text{ideal}}-\mu)

, where

\mu

is the BoN

N{=}1

baseline (single-inference mean).

h=1

means full exploitation of the oracle gap;

h=0

matches

\mu

;

h<0

underperforms

\mu

. green = positive (gain captured), red = negative (below baseline). Bold when

|h|\geq 0.20

	HealthBench				PRBench				LEXam
	Low	Mid	High	XHigh	Low	Mid	High	XHigh	Low	Mid	High	XHigh
BoN+Skywork	+0.177	+0.165	+0.160	+0.165	+0.170	+0.137	+0.119	+0.110	+0.216	+0.171	+0.152	+0.157
Fusion	—	+0.207	+0.292	—	—	+0.059	+0.224	—	—	+0.353	+0.286	—

Appendix

Table 17: Headroom capture for OLMo3-7B :

h=(\text{realized}-\mu)/(\text{oracle}_{\text{ideal}}-\mu)

, where

\mu

is the BoN

N{=}1

baseline (single-inference mean).

h=1

means full exploitation of the oracle gap;

h=0

matches

\mu

;

h<0

underperforms

\mu

. green = positive (gain captured), red = negative (below baseline). Bold when

|h|\geq 0.20

	HealthBench				PRBench				LEXam
	Low	Mid	High	XHigh	Low	Mid	High	XHigh	Low	Mid	High	XHigh
BoN+Skywork	+0.099	+0.096	+0.090	+0.086	+0.102	+0.082	+0.070	+0.076	+0.085	+0.065	+0.049	+0.025
Fusion	—	-0.426	-0.326	—	—	-0.979	-1.016	—	—	+0.231	+0.228	—

Appendix

Table 18: Headroom capture for OLMo3-32B :

h=(\text{realized}-\mu)/(\text{oracle}_{\text{ideal}}-\mu)

, where

\mu

is the BoN

N{=}1

baseline (single-inference mean).

h=1

means full exploitation of the oracle gap;

h=0

matches

\mu

;

h<0

underperforms

\mu

. green = positive (gain captured), red = negative (below baseline). Bold when

|h|\geq 0.20

	HealthBench					PRBench					LEXam
	N=1	Low	Mid	High	XHigh	N=1	Low	Mid	High	XHigh	N=1	Low	Mid	High	XHigh
BoN oracle	0.536	0.571	0.599	0.620	0.637	0.292	0.325	0.354	0.380	0.403	0.525	0.560	0.587	0.609	0.626
Fusion	—	—	0.593	0.624	0.641	—	—	0.349	0.382	0.406	—	—	0.584	0.612	0.629
Particle Filter	—	—	—	—	0.575	—	—	—	—	0.327	—	—	—	—	0.580
Self - Refine	—	0.551	0.558	0.572	0.594	—	0.313	0.333	0.363	0.393	—	0.543	0.551	0.564	0.578

Appendix

Table 19: Oracle quality for Qwen3.5-35B across five benchmarks and four compute levels. Cell colour encodes delta from BoN oracle N=1 baseline: steel blue = large gain, pale blue = small gain over BoN N=1 baseline. Bold = gain

\geq 0.03

over baseline.

	HealthBench					PRBench					LEXam
	N=1	Low	Mid	High	XHigh	N=1	Low	Mid	High	XHigh	N=1	Low	Mid	High	XHigh
BoN+Skywork	0.508	0.511	0.514	0.516	0.518	0.259	0.262	0.264	0.265	0.265	0.463	0.466	0.467	0.469	0.470
BoN+Llama70B	0.508	0.511	0.514	0.517	0.519	0.259	0.261	0.262	0.263	0.262	0.463	0.468	0.471	0.475	0.475
Fusion	—	—	0.548	0.560	0.566	—	—	0.275	0.290	0.293	—	—	0.487	0.502	0.501
Beam Search	—	—	0.508	0.505	0.506	—	—	0.254	0.253	0.254	—	—	0.475	0.480	0.463
Particle Filter	—	—	0.503	0.506	0.502	—	—	0.251	0.251	0.248	—	—	0.471	0.465	0.467

Appendix

Table 20: Realised quality for Qwen3.5-9B across benchmarks and four compute levels. Cell colour encodes delta from the single-sample baseline (BoN@1): green = improvement, red = regression. Bold = gain

\geq 0.03

over baseline.

	HealthBench					PRBench					LEXam
	N=1	Low	Mid	High	XHigh	N=1	Low	Mid	High	XHigh	N=1	Low	Mid	High	XHigh
BoN oracle	0.508	0.549	0.581	0.606	0.627	0.259	0.291	0.319	0.344	0.365	0.463	0.499	0.528	0.552	0.572
Fusion	—	—	0.574	0.609	0.631	—	—	0.312	0.347	0.368	—	—	0.521	0.549	0.573
Self - Refine	—	0.516	0.529	0.545	0.566	—	0.276	0.289	0.317	0.343	—	0.481	0.490	0.499	0.508

Appendix

Table 21: Oracle quality for Qwen3.5-9B across five benchmarks and four compute levels. Cell colour encodes delta from BoN oracle N=1 baseline: steel blue = large gain, pale blue = small gain over BoN N=1 baseline. Bold = gain

\geq 0.03

over baseline.

	HealthBench					PRBench					LEXam
	N=1	Low	Mid	High	XHigh	N=1	Low	Mid	High	XHigh	N=1	Low	Mid	High	XHigh
BoN+Skywork	0.480	0.484	0.487	0.489	0.490	0.211	0.213	0.215	0.215	0.217	0.395	0.398	0.399	0.399	0.398
BoN+Llama70B	0.480	0.483	0.486	0.487	0.488	0.211	0.212	0.214	0.215	0.218	0.395	0.399	0.401	0.405	0.408
Fusion	—	—	0.454	0.450	—	—	—	0.174	0.164	—	—	—	0.409	0.414	—
Beam Search	—	—	0.479	0.480	—	—	—	0.211	0.212	—	—	—	0.393	0.386	—
Particle Filter	—	—	0.479	0.480	0.477	—	—	0.210	0.207	0.210	—	—	0.390	0.406	0.395

Appendix

Table 22: Realised quality for OLMo3-32B across five benchmarks and four compute levels. Cell colour encodes delta from the single-sample baseline (BoN@1): green = improvement, red = regression. Bold = gain

\geq 0.03

over baseline.

	HealthBench					PRBench					LEXam
	N=1	Low	Mid	High	XHigh	N=1	Low	Mid	High	XHigh	N=1	Low	Mid	High	XHigh
BoN+Skywork	0.335	0.342	0.348	0.353	0.358	0.142	0.145	0.147	0.148	0.150	0.281	0.288	0.291	0.294	0.298
BoN+Llama70B	0.335	0.342	0.346	0.350	0.354	0.142	0.144	0.146	0.147	0.150	0.281	0.289	0.294	0.297	0.298
Fusion	—	—	0.349	0.365	—	—	—	0.144	0.154	—	—	—	0.300	0.304	—
Beam Search	—	—	0.332	0.328	0.329	—	—	0.142	0.139	0.141	—	—	0.278	0.278	0.272
Particle Filter	—	—	0.334	0.340	0.336	—	—	0.139	0.141	0.140	—	—	0.273	0.281	0.279

Appendix

Table 23: Realised quality for OLMo3-7B across five benchmarks and four compute levels. Cell colour encodes delta from the single-sample baseline (BoN@1): green = improvement, red = regression. Bold = gain

\geq 0.03

over baseline.

	HealthBench					PRBench					LEXam
	N=1	Low	Mid	High	XHigh	N=1	Low	Mid	High	XHigh	N=1	Low	Mid	High	XHigh
BoN oracle	0.335	0.376	0.413	0.445	0.473	0.142	0.161	0.180	0.197	0.213	0.281	0.313	0.342	0.368	0.391
Fusion	—	—	0.403	0.438	—	—	—	0.175	0.199	—	—	—	0.335	0.361	—

Appendix

Table 24: Oracle quality for OLMo3-7B across five benchmarks and four compute levels. Cell colour encodes delta from BoN oracle N=1 baseline: steel blue = large gain, pale blue = small gain over BoN N=1 baseline. Bold = gain

\geq 0.03

over baseline.

	HealthBench					PRBench					LEXam
	N=1	Low	Mid	High	XHigh	N=1	Low	Mid	High	XHigh	N=1	Low	Mid	High	XHigh
BoN oracle	0.480	0.521	0.554	0.580	0.601	0.211	0.236	0.259	0.281	0.300	0.395	0.429	0.459	0.483	0.501
Fusion	—	—	0.540	0.571	—	—	—	0.248	0.256	—	—	—	0.456	0.478	—

Appendix

Table 25: Oracle quality for OLMo3-32B across five benchmarks and four compute levels. Cell colour encodes delta from BoN oracle N=1 baseline: steel blue = large gain, pale blue = small gain over BoN N=1 baseline. Bold = gain

\geq 0.03

over baseline.

	HealthBench				PRBench				LEXam
	XLow	Low	Mid	XHigh	XLow	Low	Mid	XHigh	XLow	Low	Mid	XHigh
BoN+Skywork	+0.047	+0.038	+0.030	+0.025	+0.038	+0.023	+0.014	-0.005	+0.047	+0.062	+0.065	+0.051
Fusion	—	+0.670	+0.580	+0.495	—	+0.390	+0.357	+0.339	—	+0.355	+0.246	+0.209
Particle Filter	—	—	—	-0.168	—	—	—	-0.293	—	—	—	+0.006
Self - Refine	-1.096	-0.907	-0.553	-0.399	+0.455	+0.467	+0.495	+0.496	-1.148	-0.833	-0.959	-0.858

Appendix

Table 26: Headroom capture for Qwen3.5-35B :

h=(\text{realized}-\mu)/(\text{oracle}_{\text{ideal}}-\mu)

, where

\mu

is the BoN

N{=}1

baseline (single-inference mean).

h=1

means full exploitation of the oracle gap;

h=0

matches

\mu

;

h<0

underperforms

\mu

. green = positive (gain captured), red = negative (below baseline). Bold when

|h|\geq 0.20

	HealthBench				PRBench				LEXam
	XLow	Low	Mid	XHigh	XLow	Low	Mid	XHigh	XLow	Low	Mid	XHigh
BoN+Skywork	+0.084	+0.080	+0.084	+0.089	+0.077	+0.071	+0.061	+0.050	+0.073	+0.058	+0.059	+0.056
Fusion	—	+0.615	+0.512	+0.473	—	+0.301	+0.346	+0.313	—	+0.419	+0.445	+0.339
Self - Refine	-3.615	-1.681	-1.175	-0.918	+0.251	+0.323	+0.404	+0.329	-1.104	-0.970	-0.952	-1.147

Appendix

Table 27: Headroom capture for Qwen3.5-9B :

h=(\text{realized}-\mu)/(\text{oracle}_{\text{ideal}}-\mu)

, where

\mu

is the BoN

N{=}1

baseline (single-inference mean).

h=1

means full exploitation of the oracle gap;

h=0

matches

\mu

;

h<0

underperforms

\mu

. green = positive (gain captured), red = negative (below baseline). Bold when

|h|\geq 0.20

	HealthBench				PRBench				LEXam
	XLow	Low	Mid	XHigh	XLow	Low	Mid	XHigh	XLow	Low	Mid	XHigh
BoN+Skywork	+0.099	+0.096	+0.090	+0.086	+0.102	+0.082	+0.070	+0.076	+0.085	+0.065	+0.049	+0.025
Fusion	—	-0.426	-0.326	—	—	-0.979	-1.016	—	—	+0.231	+0.228	—

Appendix

Table 28: Headroom capture for OLMo3-32B :

h=(\text{realized}-\mu)/(\text{oracle}_{\text{ideal}}-\mu)

, where

\mu

is the BoN

N{=}1

baseline (single-inference mean).

h=1

means full exploitation of the oracle gap;

h=0

matches

\mu

;

h<0

underperforms

\mu

. green = positive (gain captured), red = negative (below baseline). Bold when

|h|\geq 0.20

	HealthBench				PRBench				LEXam
	XLow	Low	Mid	XHigh	XLow	Low	Mid	XHigh	XLow	Low	Mid	XHigh
BoN+Skywork	+0.177	+0.165	+0.160	+0.165	+0.170	+0.137	+0.119	+0.110	+0.216	+0.171	+0.152	+0.157
Fusion	—	+0.207	+0.292	—	—	+0.059	+0.224	—	—	+0.353	+0.286	—

Appendix

Table 29: Headroom capture for OLMo3-7B :

h=(\text{realized}-\mu)/(\text{oracle}_{\text{ideal}}-\mu)

, where

\mu

is the BoN

N{=}1

baseline (single-inference mean).

h=1

means full exploitation of the oracle gap;

h=0

matches

\mu

;

h<0

underperforms

\mu

. green = positive (gain captured), red = negative (below baseline). Bold when

|h|\geq 0.20

Generator	RM	Health.	LEXam	PR	Wild	Writing
Qwen3.5-9B	Skywork V2	0.063	0.045	0.064	0.255	0.197
	Llama-3.1-70B	0.068	0.058	0.050	0.178	0.189
Qwen3.5-35B	Skywork V2	0.036	0.028	0.023	0.260	0.220
	Llama-3.1-70B	0.067	0.046	0.031	0.146	0.173
Olmo-3-7B	Skywork V2	0.127	0.136	0.113	0.180	0.123
	Llama-3.1-70B	0.119	0.154	0.101	0.166	0.095

Appendix

Table 30: Verifier correlation

\hat{\rho}_{v}

(Spearman between RM scores and judge scores), averaged over tasks within each benchmark (

N=152

, 4 generators

\times

2 RMs

\times

19 tasks). Both RMs produce near-identical profiles (Skywork mean

\hat{\rho}_{v}=0.120

, Llama mean

\hat{\rho}_{v}=0.107

, computed at task level before benchmark aggregation), confirming the failure is structural rather than model-specific.

Generator	RM	Health.	LEXam	PR	Wild	Writing
Qwen3.5-9B	Skywork V2	0.080	0.056	0.041	0.345	0.208
	Llama-3.1-70B	0.092	0.102	0.015	0.182	0.182
Qwen3.5-35B	Skywork V2	0.010	0.051	0.002	0.413	0.275
	Llama-3.1-70B	0.056	0.090	0.046	0.243	0.253
Olmo-3-7B	Skywork V2	0.139	0.157	0.119	0.182	0.123
	Llama-3.1-70B	0.127	0.157	0.110	0.130	0.126

Appendix

Table 31: Headroom capture

\hat{h}=(\text{BoN@16}-\mu)\,/\,(\text{Oracle@16}-\mu)

for the same generator–RM–benchmark combinations. Bold negative values indicate the RM selects worse than random. Comparing cell-by-cell with Table 30 confirms

\hat{h}\approx\hat{\rho}_{v}

(regression:

\hat{h}=1.20\,\hat{\rho}_{v}-0.011

R^{2}=0.66

\rho=0.81

p<10^{-36}

), validating Equation 4 .

	MATH-500					GPQA Diamond
	N=1	Low	Mid	High	XHigh	N=1	Low	Mid	High	XHigh
BoN+Skywork	0.922	0.930	0.933	0.935	0.936	0.823	0.841	0.851	0.852	0.843
BoN+Llama70B	0.922	0.929	0.932	0.934	0.938	0.823	0.836	0.842	0.841	0.838
Fusion	—	—	0.936	0.934	0.932	—	—	0.859	0.833	0.823
Beam Search	—	—	0.942	0.940	0.934	—	—	0.818	0.803	0.818
Particle Filter	—	—	0.912	0.892	0.890	—	—	—	—	—

Appendix

Table 32: Realised quality on deterministic benchmarks for Qwen3.5-35B-A3B . Cell colour encodes delta from the single-sample baseline (BoN@1): green = improvement, red = regression. Bold =

|\Delta|\geq 0.03

. Italic Pass@K row reports the BoN-pool oracle. Particle Filter and Budget Forcing on GPQA were not run.

Figure 10: Reproduction of BoN scaling from the Skywork-Reward-V2 paper [ 14 ] on MATH-500 (left) and GPQA Diamond (right). Dashed grey: oracle accuracy. Solid coloured: realised accuracy under Skywork-Reward-V2-Llama-3.1-8B selection. Results match the authors’ reported curves, validating our BoN pipeline before deployment on open-ended benchmarks.

Figure 11: Left: Mean score trajectory across Sequential Refinement iterations per WildBench task category. Coding & Debugging rises steeply; most other categories are flat or decline. Right: Oracle and mean score trajectories on WildBench with Coding & Debugging excluded. Without this subtask, mean quality regresses across iterations while the oracle continues to rise—confirming the exploitation failure pattern observed on other benchmarks.

Figure 12: Token length distributions per method and benchmark for Qwen3.5-35B-A3B. BoN (blue) and Fusion (pink) show stable length distributions across compute levels. Sequential Refinement (orange gradient, SR $k{=}1$ through $k{=}16$ ) shows monotonically increasing median length on every benchmark, with the steepest growth on WritingBench and PRBench.

Benchmark	$n$	$\hat{\rho}$ [95% CI]		% $\rho{>}0$	$t$ -test $p$	Wilcoxon $p$
HealthBench	4,619	$+$ 0.133	[ $+$ 0.124, $+$ 0.142]	67.5%	$7.76\times 10^{-180}$	$8.20\times 10^{-167}$
PRBench	1,634	$+$ 0.112	[ $+$ 0.099, $+$ 0.126]	64.7%	$6.07\times 10^{-57}$	$1.87\times 10^{-51}$
LEXam	502	$+$ 0.014	[ $-$ 0.011, $+$ 0.039]	52.0%	$2.73\times 10^{-1}$	$3.81\times 10^{-1}$
WildBench	917	$+$ 0.044	[ $+$ 0.024, $+$ 0.063]	57.6%	$1.93\times 10^{-5}$	$4.72\times 10^{-6}$
WritingBench	554	$+$ 0.198	[ $+$ 0.174, $+$ 0.222]	74.5%	$4.97\times 10^{-46}$	$4.10\times 10^{-39}$

Appendix

Table 33: Within-prompt Spearman rank correlation between response length and judge score for BoN candidates (16 per prompt), Qwen3.5-35B-A3B, unified judge.

\hat{\rho}

= mean per-prompt Spearman correlation; 95% CI is bootstrapped; %

\rho{>}0

= fraction of prompts with a positive correlation;

p

-values from a one-sample

t

-test and Wilcoxon signed-rank test against

H_{0}\colon\rho=0

Benchmark	$n$	mean CV	med CV	%CV ${<}0.1$	%CV ${<}0.2$	mean std	med std
HealthBench	5,000	0.150	0.127	29.7%	83.2%	101.3	88.0
PRBench	1,650	0.083	0.076	81.2%	98.6%	141.6	128.8
LEXam	516	0.119	0.096	54.3%	90.9%	113.5	110.7
WildBench	1,023	0.186	0.131	30.2%	73.4%	295.2	160.7
WritingBench	555	0.145	0.107	45.2%	76.2%	392.7	244.0

Appendix

Table 34: Within-prompt length variance for BoN candidates (16 per prompt), Qwen3.5-35B-A3B. CV = coefficient of variation (std/mean) of token lengths within a prompt; %CV

{<}0.1

and %CV

{<}0.2

report the fraction of prompts with low length dispersion; mean and median std are in tokens. PRBench shows the lowest length variance (mean CV

=0.083

), consistent with its structured rubric format constraining response length. WritingBench and WildBench show the highest variance, reflecting open-ended prompts that admit responses of widely varying length.

Benchmark	$n$	CV score	CV len	mean ratio	%ratio ${>}2$	%both ${<}0.05$
HealthBench	5,000	0.329	0.150	1.57	37.8%	0.1%
PRBench	1,650	0.356	0.083	3.17	76.5%	0.2%
LEXam	516	0.217	0.119	1.65	39.1%	0.0%
WildBench	1,023	0.137	0.186	0.69	8.0%	0.4%
WritingBench	555	0.084	0.145	0.55	5.2%	1.4%

Appendix

Table 35: Score variability versus length variability for BoN candidates (16 per prompt), Qwen3.5-35B-A3B, unified judge. CV score and CV len are the per-prompt coefficient of variation of judge scores and token lengths respectively; mean ratio

=

mean(CV score /CV len ). A ratio

{>}1

indicates score variation is driven by content; a ratio

{<}1

indicates length is the dominant axis of variation. PRBench (ratio

=3.17

) and HealthBench (ratio

=1.57

) are content-driven. WritingBench (ratio

=0.55

) and WildBench (ratio

=0.69

) are length-driven even under BoN, before any iterative elaboration.

Benchmark	$n$	$\hat{\rho}$ [95% CI]		% $\rho{>}0$	$t$ -test $p$	Wilcoxon $p$
HealthBench	4,703	$+$ 0.059	[ $+$ 0.047, $+$ 0.069]	56.2%	$1.26\times 10^{-23}$	$2.94\times 10^{-23}$
PRBench	1,634	$+$ 0.252	[ $+$ 0.232, $+$ 0.272]	72.2%	$6.68\times 10^{-116}$	$1.75\times 10^{-99}$
LEXam	505	$-$ 0.060	[ $-$ 0.092, $-$ 0.026]	41.2%	$2.46\times 10^{-4}$	$2.29\times 10^{-4}$
WildBench	925	$+$ 0.023	[ $-$ 0.001, $+$ 0.046]	52.0%	$5.23\times 10^{-2}$	$3.82\times 10^{-2}$
WritingBench	555	$+$ 0.352	[ $+$ 0.319, $+$ 0.385]	80.7%	$1.12\times 10^{-73}$	$1.21\times 10^{-55}$

Appendix

Table 36: Within-prompt Spearman rank correlation between response length and judge score for Sequential Refinement drafts (

k=1,\ldots,16

per prompt), Qwen3.5-35B-A3B, unified judge. Columns as in Table 33 . Compared with BoN, Sequential Refinement shows a substantially stronger length–score correlation on WritingBench (

+0.352

vs.

+0.198

) and PRBench (

+0.252

vs.

+0.112

), confirming that iterative elaboration amplifies the length–score relationship. LEXam is the only benchmark with a significantly negative correlation (

\hat{\rho}=-0.060

), consistent with refinement producing longer but worse responses on knowledge-intensive tasks.

Benchmark	$n$	CV score	CV len	mean ratio	%ratio ${>}2$	%both ${<}0.05$
HealthBench	5,000	0.358	0.257	0.91	19.3%	0.0%
PRBench	1,650	0.297	0.256	0.84	15.6%	0.0%
LEXam	516	0.237	0.218	0.86	13.2%	0.0%
WildBench	1,024	0.126	0.256	0.46	5.8%	0.2%
WritingBench	555	0.083	0.198	0.36	2.3%	0.5%

Appendix

Table 37: Score variability versus length variability for Sequential Refinement drafts (16 iterations per prompt), Qwen3.5-35B-A3B, unified judge. Columns as in Table 35 . Compared with BoN, Sequential Refinement shows higher CV len on every benchmark, reflecting monotonic length growth across iterations. On WritingBench, CV len substantially exceeds CV score (ratio

=0.36

), confirming that length growth is the primary driver of score gains. On HealthBench, the ratio drops from

1.57

(BoN) to

0.91

(Sequential Refinement): iterative conditioning increases length variance more than score variance, a sign of elaboration without quality improvement.

Benchmark	$n$	$\hat{\rho}$ [95% CI]		% $\rho{>}0$	$t$ -test $p$	Wilcoxon $p$
HealthBench	4,619	$+$ 0.133	[ $+$ 0.124, $+$ 0.142]	67.5%	$7.76\times 10^{-180}$	$8.20\times 10^{-167}$
PRBench	1,634	$+$ 0.112	[ $+$ 0.099, $+$ 0.125]	64.7%	$6.07\times 10^{-57}$	$1.87\times 10^{-51}$
LEXam	502	$+$ 0.014	[ $-$ 0.010, $+$ 0.038]	52.0%	$2.73\times 10^{-1}$	$3.81\times 10^{-1}$
WildBench	917	$+$ 0.044	[ $+$ 0.023, $+$ 0.064]	57.6%	$1.93\times 10^{-5}$	$4.72\times 10^{-6}$
WritingBench	555	$+$ 0.370	[ $+$ 0.347, $+$ 0.394]	87.9%	$6.24\times 10^{-121}$	$7.28\times 10^{-77}$

Appendix

Table 38: Within-prompt Spearman rank correlation between response length and judge score for BoN candidates (16 per prompt), Qwen3.5-35B-A3B, native WritingBench judge . WritingBench

\hat{\rho}

rises from

+0.198

(unified judge, Table 33 ) to

+0.370

; all other benchmarks are stable, confirming the amplification is WritingBench-specific.

Benchmark	$n$	CV score	CV len	mean ratio	%ratio ${>}2$	%both ${<}0.05$
HealthBench	5,000	0.329	0.150	1.57	37.8%	0.1%
PRBench	1,650	0.356	0.083	3.17	76.5%	0.2%
LEXam	516	0.217	0.119	1.65	39.1%	0.0%
WildBench	1,023	0.137	0.186	0.69	8.0%	0.4%
WritingBench	555	0.034	0.145	0.25	0.0%	2.0%

Appendix

Table 39: Score variability versus length variability for BoN candidates (16 per prompt), Qwen3.5-35B-A3B, native WritingBench judge . Under the native judge, WritingBench CV score collapses to

0.034

(vs.

0.084

under the unified judge, Table 35 ), yielding a ratio of

0.25

: score variation is almost entirely explained by length. Length statistics are identical to Table 34 since they are judge-independent.

Benchmark	$n$	$\hat{\rho}$ [95% CI]		% $\rho{>}0$	$t$ -test $p$	Wilcoxon $p$
HealthBench	4,703	$+$ 0.059	[ $+$ 0.047, $+$ 0.070]	56.2%	$1.26\times 10^{-23}$	$2.94\times 10^{-23}$
PRBench	1,634	$+$ 0.252	[ $+$ 0.232, $+$ 0.272]	72.2%	$6.68\times 10^{-116}$	$1.75\times 10^{-99}$
LEXam	505	$-$ 0.060	[ $-$ 0.091, $-$ 0.029]	41.2%	$2.46\times 10^{-4}$	$2.29\times 10^{-4}$
WildBench	925	$+$ 0.023	[ $+$ 0.000, $+$ 0.045]	52.0%	$5.23\times 10^{-2}$	$3.82\times 10^{-2}$
WritingBench	551	$+$ 0.539	[ $+$ 0.513, $+$ 0.566]	92.2%	$1.07\times 10^{-159}$	$1.82\times 10^{-82}$

Appendix

Table 40: Within-prompt Spearman rank correlation between response length and judge score for Sequential Refinement drafts (

k=1,\ldots,16

per prompt), Qwen3.5-35B-A3B, native WritingBench judge . WritingBench

\hat{\rho}

rises to

+0.539

(vs.

+0.352

under the unified judge, Table 36 ), with 92.2% of prompts showing a positive correlation. All other benchmarks are stable.

Benchmark	$n$	CV score	CV len	mean ratio	%ratio ${>}2$	%both ${<}0.05$
HealthBench	5,000	0.358	0.257	0.91	19.3%	0.0%
PRBench	1,650	0.297	0.256	0.84	15.6%	0.0%
LEXam	516	0.237	0.218	0.86	13.2%	0.0%
WildBench	1,024	0.126	0.256	0.46	5.8%	0.2%
WritingBench	555	0.038	0.198	0.17	0.5%	0.4%

Appendix

Table 41: Score variability versus length variability for Sequential Refinement drafts (16 iterations per prompt), Qwen3.5-35B-A3B, native WritingBench judge . WritingBench CV score

=0.038

(vs.

0.083

under the unified judge, Table 37 ), with a ratio of

0.17

: under the native judge, virtually all score variation in Sequential Refinement on WritingBench is attributable to response length.

Figure 13: WritingBench old vs. new judge at

k{=}16

(Qwen3.5-35B, micro-averaged across 555 samples). Left: Oracle (solid) and realized (hatched) scores for BoN+Skywork, BoN+Llama70B, and Self-Refine under our judge (Qwen3.5-397B, dark) and the authors’ fine-tuned WritingBench-Critic-Model-Qwen-7B (light); horizontal lines show the candidate-mean

\mu

for each judge. Center: Headroom capture

(\text{realized}-\mu)/(\text{oracle}-\mu)

. Right: Decomposition into denominator (oracle

-\mu

, solid) and numerator (realized

-\mu

, hatched).

Figure 14: Absolute score gain over the BoN@1 baseline across compute levels, by model family and benchmark.

Figure 15: Fraction of remaining gap to a perfect score, $(p_{k}-p_{1})/(1-p_{1})$ , closed at compute level $k$ , faceted by benchmark. $p_{1}$ is the BoN@1 baseline.

Figure 16: Sequential Refinement oracle vs BoN oracle across compute levels, by model family and benchmark.

Figure 17: Sequential Refinement oracle vs BoN oracle across compute levels, by model family and benchmark. Note that x-axis refers to number of candidates in the pool so compare k self-revisions to k i.i.d. samples here.

Figure 18: Oracle gaps over a BoN@1 baseline across models for fusion at medium compute (k=8). Note that Olmo models used 6 candidates instead of 7 due to context length limits. Additionally, context-length issues persisted for Olmo on PrBench and WildBench so they are not included here.

Figure 19: Oracle gap and headroom capture comparison between different families

Figure 20: Demand profiles for HealthBench (left) and LEXam (right)

Figure 21: Demand profiles for PRBench (left) and WildBench (right)

Figure 22: WritingBench demand profile

Benchmark	Qwen3.5-9B		Qwen3.5-35B-A3B		OLMo-3-7B		OLMo-3.1-32B
	Tokens	Band [min–max]	Tokens	Band [min–max]	Tokens	Band [min–max]	Tokens	Band [min–max]
healthbench_communication	39.6M	[28.9M–50.3M]	35.7M	[26.1M–45.4M]	32.9M	[24.0M–41.8M]	34.0M	[24.8M–43.2M]
healthbench_complex_responses	17.3M	[12.6M–22.0M]	16.2M	[11.8M–20.6M]	14.1M	[10.3M–17.9M]	16.0M	[11.7M–20.3M]
healthbench_context_seeking	19.3M	[14.1M–24.5M]	17.8M	[13.0M–22.6M]	15.6M	[11.4M–19.8M]	15.6M	[11.4M–19.8M]
healthbench_emergency_referrals	15.9M	[11.6M–20.1M]	14.5M	[10.6M–18.4M]	12.6M	[9.2M–16.0M]	12.1M	[8.8M–15.3M]
healthbench_global_health	41.3M	[30.1M–52.4M]	37.5M	[27.4M–47.7M]	34.5M	[25.2M–43.9M]	34.7M	[25.3M–44.0M]

Appendix

Table 42: BoN

n{=}16

total completion tokens and

\pm 27\%

compute band (min–max) per benchmark.

Benchmark	Qwen3.5-9B	Qwen3.5-35B-A3B	OLMo-3-7B	OLMo-3.1-32B
healthbench_communication	10.2M	9.0M	8.3M	8.6M
healthbench_complex_responses	4.5M	4.3M	3.4M	3.9M
healthbench_context_seeking	5.0M	4.6M	3.9M	3.9M
healthbench_emergency_referrals	4.1M	3.7M	3.1M	3.0M
healthbench_global_health	10.5M	9.4M	8.7M	8.9M
healthbench_health_data_tasks	5.5M	5.0M	4.3M	5.1M

Appendix

Table 43: Particle Filter (Mid (

n{=}4

)) total completion tokens per benchmark.

Benchmark	Qwen3.5-9B	Qwen3.5-35B-A3B	OLMo-3-7B	OLMo-3.1-32B
healthbench_communication	20.3M	18.2M	16.5M	17.1M
healthbench_complex_responses	9.1M	8.6M	6.8M	8.0M
healthbench_context_seeking	10.0M	9.1M	7.8M	8.0M
healthbench_emergency_referrals	8.2M	7.3M	6.4M	6.1M
healthbench_global_health	21.2M	18.9M	17.5M	17.7M
healthbench_health_data_tasks	10.9M	10.2M	8.3M	9.9M

Appendix

Table 44: Particle Filter (High (

n{=}8

)) total completion tokens per benchmark.

Benchmark	Qwen3.5-9B	Qwen3.5-35B-A3B	OLMo-3-7B	OLMo-3.1-32B
healthbench_communication	40.5M	36.1M	33.0M	35.0M
healthbench_complex_responses	18.1M	17.0M	13.7M	15.9M
healthbench_context_seeking	20.0M	18.2M	16.0M	16.2M
healthbench_emergency_referrals	16.5M	14.7M	12.9M	12.5M
healthbench_global_health	42.3M	37.8M	35.2M	36.3M
healthbench_health_data_tasks	21.9M	20.5M	16.8M	20.4M

Appendix

Table 45: Particle Filter (XHigh (

n{=}16

)) total completion tokens per benchmark.

Benchmark	Qwen3.5-9B	Qwen3.5-35B-A3B	OLMo-3-7B	OLMo-3.1-32B
healthbench_communication	10.2M	9.0M	8.0M	8.4M
healthbench_complex_responses	4.7M	4.2M	3.4M	4.1M
healthbench_context_seeking	5.0M	4.6M	3.8M	3.8M
healthbench_emergency_referrals	4.1M	3.7M	2.9M	2.9M
healthbench_global_health	10.5M	9.3M	8.4M	8.5M
healthbench_health_data_tasks	5.5M	5.1M	3.9M	5.0M

Appendix

Table 46: Beam Search (Mid (

\mathrm{{bw}}{=}2,\,n{=}2

)) total completion tokens per benchmark.

Benchmark	Qwen3.5-9B	Qwen3.5-35B-A3B	OLMo-3-7B	OLMo-3.1-32B
healthbench_communication	19.8M	17.7M	15.7M	16.4M
healthbench_complex_responses	9.4M	8.8M	6.6M	7.9M
healthbench_context_seeking	9.7M	8.8M	7.4M	7.5M
healthbench_emergency_referrals	8.1M	7.2M	5.7M	5.5M
healthbench_global_health	20.6M	18.4M	16.4M	16.6M
healthbench_health_data_tasks	10.7M	10.2M	8.0M	9.6M

Appendix

Table 47: Beam Search (High (

\mathrm{{bw}}{=}2,\,n{=}4

)) total completion tokens per benchmark.

Benchmark	Qwen3.5-9B	Qwen3.5-35B-A3B	OLMo-3-7B	OLMo-3.1-32B
healthbench_communication	40.4M	34.9M	32.0M	34.0M
healthbench_complex_responses	21.6M	19.8M	14.0M	16.7M
healthbench_context_seeking	21.0M	18.1M	15.1M	15.5M
healthbench_emergency_referrals	16.6M	14.4M	11.3M	11.8M
healthbench_global_health	42.6M	36.6M	33.3M	34.9M
healthbench_health_data_tasks	23.3M	21.4M	16.6M	20.4M

Appendix

Table 48: Beam Search (XHigh (

\mathrm{{bw}}{=}4,\,n{=}4

)) total completion tokens per benchmark. † Run at

\mathrm{{bw}}{=}3,\,n{=}4

Benchmark	Qwen3.5-9B	Qwen3.5-35B-A3B	OLMo-3-7B	OLMo-3.1-32B
healthbench_communication	83.0M	82.6M	–	–
healthbench_complex_responses	33.0M	32.4M	–	–
healthbench_context_seeking	45.6M	44.7M	–	–
healthbench_emergency_referrals	36.4M	36.0M	–	–
healthbench_global_health	91.6M	88.3M	–	–
healthbench_health_data_tasks	40.1M	40.2M	–	–

Appendix

Table 49: Sequential Refinement (

\text{iter}{=}16

) total completion tokens per benchmark.

Benchmark	Qwen3.5-9B	Qwen3.5-35B-A3B	OLMo-3-7B	OLMo-3.1-32B
healthbench_communication	7.4M	6.7M	6.2M	6.4M
healthbench_complex_responses	3.2M	3.0M	2.6M	3.0M
healthbench_context_seeking	3.6M	3.3M	2.9M	2.9M
healthbench_emergency_referrals	3.0M	2.7M	2.4M	2.3M
healthbench_global_health	7.7M	7.0M	6.5M	6.5M
healthbench_health_data_tasks	3.9M	3.7M	3.2M	3.8M

Appendix

Table 50: Fusion (Mid (

s{=}3

)) total completion tokens per benchmark.

Benchmark	Qwen3.5-9B	Qwen3.5-35B-A3B	OLMo-3-7B	OLMo-3.1-32B
healthbench_communication	14.9M	13.4M	12.3M	12.8M
healthbench_complex_responses	6.5M	6.1M	5.3M	6.0M
healthbench_context_seeking	7.2M	6.7M	5.8M	5.8M
healthbench_emergency_referrals	5.9M	5.4M	4.7M	4.5M
healthbench_global_health	15.5M	14.1M	13.0M	13.0M
healthbench_health_data_tasks	7.8M	7.4M	6.5M	7.5M

Appendix

Table 51: Fusion (High,

s{=}6or7

) total completion tokens per benchmark.

Benchmark	Qwen3.5-9B	Qwen3.5-35B-A3B	OLMo-3-7B	OLMo-3.1-32B
healthbench_communication	37.1M	33.5M	–	–
healthbench_complex_responses	16.2M	15.2M	–	–
healthbench_context_seeking	18.1M	16.7M	–	–
healthbench_emergency_referrals	14.9M	13.6M	–	–
healthbench_global_health	38.7M	35.2M	–	–
healthbench_health_data_tasks	19.5M	18.5M	–	–

Appendix

Table 52: Fusion (XHigh (

s{=}15

)) total completion tokens per benchmark.

Explore similar work

LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling
May 8, 2026Tong Zheng, Haolin Liu, Chengsong Huang +10LLM Inference Efficiency Automated Algorithm Discovery
UMD · UVA · WUSTL +3
Test-Time Scaling for Scientific Equation Discovery
Aug 21, 2026Haowei Lin, Hubert Lim, Xiangyu Wang +2Inference-Time Search Test-Time Scaling
Peking University
Test-Time Personalization: A Diagnostic Framework and Probabilistic Fix for Scaling Failures
May 9, 2026Linhai Zhang, Yulan HeReward Modeling Reward Hacking
King’s College London · The Alan Turing Institute

May 8, 2026J/K move · Enter open · S save

LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling

Tong Zheng, Haolin Liu, Chengsong Huang, Huiwen Bao+9

UMD · UVA · WUSTL+3

LLM Inference Efficiency Automated Algorithm Discovery Test-Time Optimization Large Language Model-Guided Optimization Test-Time Scaling

Open paper AskPDF arXiv

Test-time scaling (TTS) has become an effective approach for improving large language model performance by allocating additional computation during inference. However, existing TTS strategies are largely hand-crafted: researchers manually design reasoning patterns and tune heuristics by intuition, leaving much of the computation-allocation space unexplored. We propose an environment-driven framework, AutoTTS, that changes what researchers design: from individual TTS heuristics to environments where TTS strategies can be discovered automatically. The key to AutoTTS lies in environment construction: the discovery environment must make the control space tractable and provide cheap, frequent feedback for TTS search. As a concrete instantiation, we formulate width--depth TTS as controller synthesis over pre-collected reasoning trajectories and probe signals, where controllers decide when to branch, continue, probe, prune, or stop and can be evaluated cheaply without repeated LLM calls. We further introduce beta parameterization to make the search tractable and fine-grained execution trace feedback to improve discovery efficiency by helping the agent diagnose why a TTS program fails. Experiments on mathematical reasoning benchmarks show that the discovered strategies improve the overall accuracy--cost tradeoff over strong manually designed baselines. The discovered strategies generalize to held-out benchmarks and model scales, while the entire discovery costs only $39.9 and 160 minutes. Our data, and code will be open-source at https://github.com/zhengkid/AutoTTS.

Abstract

Explore similar work

LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling

Test-Time Scaling for Scientific Equation Discovery

Test-Time Personalization: A Diagnostic Framework and Probabilistic Fix for Scaling Failures