cs.CLAug 19, 2026

Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneck

Authors: Davide Romano, Kanak Raj, Jerrod Parker, Daniele Giofrè

Organizations: Thomson Reuters Foundational Research, 5 Canada Square, London

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Test-time scaling (TTS) improves language model outputs by spending additional inference compute - generating multiple candidates, searching over partial sequences, or iteratively refining drafts. These techniques yield large gains on mathematics and code, but have been developed and stress-tested almost exclusively on tasks where verification is straightforward. We conduct the first compute-normalised comparison of five TTS families across five open-ended generation benchmarks spanning medicine, law, finance, general chat, and creative writing - grounded in a unified framework that decomposes the effectiveness of each method's token budget into exploration and exploitation. The answer depends on which side of that decomposition you examine. Scaling exploration works: the best candidate in the pool improves steadily with compute across all settings. What breaks is exploitation - the step that converts a rich candidate pool into a final output. With state-of-the-art generators, reward models correlate at only ρv≈0.12ρ_v \approx 0.12 with true quality, rendering selection near-random regardless of budget. Tree search amplifies this failure through diversity collapse. Refinement helps on one of five benchmarks; its apparent gains elsewhere are confounded. Only synthesis across candidates (Fusion) consistently improves over single-sample baselines, yet still recovers only ~40% of available quality. The candidate pool is not the bottleneck - choosing from it is.

Figures & tables

MethodTxT_{x}TeT_{e}NNExploitation type
BoN + ORM≈0\approx 0≈T\approx T⌊T/ℓˉ⌋\lfloor T/\bar{\ell}\rfloorExternal scoring
Tree search≈0\approx 0≈T\approx TB×WdB\times W^{d} (collapses)External scoring
SRKℓˉcK\bar{\ell}_{c}T−KℓˉcT-K\bar{\ell}_{c}K=T/(ℓˉ+ℓˉc)K=T/(\bar{\ell}+\bar{\ell}_{c})Generative (sequential)
Fusionℓˉf\bar{\ell}_{f}T−ℓˉfT-\bar{\ell}_{f}N−1N-1Generative (synthesis)
Budget Forcing00TT11None (single trajectory)
Table 1: Token cost accounting per TTS family under budget TT . ℓˉ\bar{\ell} : mean generation length; ℓˉc\bar{\ell}_{c} : mean critique length; ℓˉf\bar{\ell}_{f} : mean fusion output length; KK : refinement steps; NN : candidates; BB : beam width; WW : branching factor; dd : depth.
BenchmarkDomainnnEvaluationSourceScoringAvg source items
LEXam [ 7 ]Law516Reference-guidedProfessor solutionsHolistic 0–11
HealthBench [ 2 ]Medicine5000Rubric-onlyPhysician rubricsBinary per criterion11.4
PRBench [ 1 ]Finance, Law1650Rubric-onlyExpert rubricsBinary per criterion17.7
WildBench [ 13 ]Chat/General1024Rubric-onlyLLM-generated checklistHolistic 1–10∼\sim 11
WritingBench [ 30 ]Writing555Rubric-onlyLLM-generated rubric1–10 per criterion5
Table 2: Benchmark characteristics.
Figure 1: Schema for the six TTS methods in this paper. PF is the only method reproduced as it is, the other methods are variations from previous papers
 HealthBenchPRBenchLEXam
 N=1LowMidHighXHighN=1LowMidHighXHighN=1LowMidHighXHigh
BoN+Skywork0.5360.5380.5390.5390.5390.2920.2930.2940.2930.2920.5250.5270.5290.5310.530
BoN+Llama70B0.5360.5390.5410.5410.5410.2920.2930.2950.2960.2980.5250.5280.5320.5330.534
Fusion——0.5740.5870.588——0.3150.3240.331——0.5460.5460.547
Beam Search——0.5340.5260.532——0.2890.2850.285——0.5160.5260.525
Particle Filter——0.5320.5320.530——0.2860.2820.282——0.5310.5270.526
Table 3: Realised quality for Qwen3.5-35B-A3B across five benchmarks and four compute levels. Cell colour encodes delta from the single-sample baseline (BoN@1): green = improvement, red = regression. Bold = gain ≥0.03\geq 0.03 over baseline.
Figure 2: Oracle (full bar) vs. realised (hatched) quality per method at matched compute ( XHigh ), Qwen3.5-35B-A3B. All qualitative patterns above replicate under the naive oracle (Appendix G ). In that appendix figure, the naive estimator systematically overestimates oracle quality, so the exploitation gaps shown there are conservative lower bounds on the true exploitation gap.
MethodOracle qualityHeadroom captureFailure mode
BoN + ORMHigh∼15%{\sim}15\% ( ρ^v≈0.12\hat{\rho}_{v}\approx 0.12 )RM miscalibrated (§ 6.1 )
Tree searchCollapsed (diversity loss)Negative on most tasksPRM destroys diversity (§ 6.2 )
Fusion≥\geq BoN∼40%{\sim}40\% (best, still low)Residual gap (§ 6.3 )
SRLower than BoN on 3/5 benchmarksGenuine gain on 1/5 (PRBench)Refinement mostly hurts; where it helps, gains trace to subtask composition or verbosity (§ 6.3 )
Table 4: Diagnosis per method family. Oracle quality reflects exploration; headroom capture reflects exploitation.
Figure 3: Left: h^\hat{h} vs. ρ^v\hat{\rho}_{v} across N=152N=152 task-level points. Identity line y=xy=x is the theoretical prediction (Equation 4 ); empirical fit y=1.198 ρ^v−0.011y=1.198\,\hat{\rho}_{v}-0.011 confirms it ( R2=0.66R^{2}=0.66 , ρ=0.81\rho=0.81 , p<10−36p<10^{-36} ). Right: Headroom capture per method at XHigh compute. The headrooms for all models are in Appendix J
Figure 4: Left: Mean pairwise cosine distance across final outputs at High compute. Right: Particle trajectory similarity over normalised generation depth.
Figure 5: Left: Fusion output quality vs. a random candidate drawn from the candidates not included in the shared sub-pool common to both BoN and Fusion, per benchmark. Right: Oracle-position frequency (bars, left axis) and mean score trajectory (line, right axis) across SR iterations.
Appendix figures & tables65 assets

Supplementary material from the paper’s appendix.

Appendix

BandBoNBeam SearchPFSequential RefinementBudget ForcingFusion
Lown=2n=2——iter =3=32ℓˉt+ℓˉ2\bar{\ell}_{t}+\bar{\ell}—
Midn=4n=4W=2,k=2W=2,k=2N=4N=4iter =6=64ℓˉt+ℓˉ4\bar{\ell}_{t}+\bar{\ell}subset =3=3
Highn=8n=8W=2,k=4W=2,k=4N=8N=8iter =12=128ℓˉt+ℓˉ8\bar{\ell}_{t}+\bar{\ell}subset =6/7=6/7
XHighn=16n=16W=4,k=4†W=4,k=4^{\dagger}N=16N=16iter =16=1616ℓˉt+ℓˉ16\bar{\ell}_{t}+\bar{\ell}subset =15=15
Appendix
Table 5: Compute band configurations. Fusion High uses subset =7=7 for Qwen3.5 and subset =6=6 for OLMo3. Entries marked — were not run. ℓˉt\bar{\ell}_{t} : mean thinking tokens per BoN sample; ℓˉ\bar{\ell} : mean answer tokens per BoN sample. † Two tasks ran at W=3,k=4W=3,k=4 instead of W=4,k=4W=4,k=4 (see Table 48 ).
Weight classInteger weight
critically important{8,9,10}\{8,9,10\}
important{4,5,6,7}\{4,5,6,7\}
slightly important{1,2,3}\{1,2,3\}
slightly detrimental{−3,−2,−1}\{-3,-2,-1\}
detrimental{−7,−6,−5,−4}\{-7,-6,-5,-4\}
critically detrimental{−10,−9,−8}\{-10,-9,-8\}
Appendix
Table 6: PRBench weight class to integer point ranges.
BenchmarkNative judgeProtocolCalls/ex
LEXamGPT-4o + Qwen3-32B + DeepSeek-V3holistic, min-agg ensemble33
HealthBenchGPT-4.1per-criterion binary∼11\sim 11
PRBencho4-miniper-criterion binary∼17\sim 17
WildBenchGPT-4.1holistic + checklist11
WritingBenchClaude 3.7 Sonnetper-criterion 11 – 101055
Appendix
Table 7: Native judges adopted verbatim from each benchmark’s upstream release. Calls/example is the average number of judge invocations per candidate completion under the native protocol; per-criterion designs (HealthBench, PRBench, WritingBench) fan out into many calls per example.
GeneratorTemperatureTop- ppTop- kkPresence penaltyThinking mode
Qwen3.5 (9B, 35B-A3B)1.00.95201.5enabled
OLMo3 (7B, 32B)0.80.9-10enabled
Appendix
Table 8: Canonical sampling profile per generator family.
MethodSettings
BoNXHigh pool ( N=16N{=}16 ) generated once; lower bands subsampled without replacement; selection =argmax⁡=\operatorname{argmax} ORM score.
Beam SearchWW beams, kk candidates/step (Table 5 ); delimiter \n\n ; scored by min per-step PRM on full path; top- WW kept globally; EOS == beam complete; PRM: VersaPRM-8B; depth 300 (500 for deterministic).
Particle FilterNN particles (Table 5 ); delimiter \n\n ; loop order: score →\to resample →\to generate; multinomial resampling with replacement, weights =softmax(w, T=1.0)=\mathrm{softmax}(\mathbf{w},\,T{=}1.0) ; PRM: VersaPRM-8B; max iterations 300.
Sequential RefinementForced iterations (stop-check ignored); history window K=3K{=}3 (draft, feedback) pairs; feedback & stop-check: thinking disabled, stop-check at T=0T{=}0 ; </think> stripped from drafts before each prompt call.
Budget Forcing2-phase decode: Phase 1 suppresses end-of-thinking tag and injects a wait-string to extend thinking; Phase 2 generates answer freely from accumulated context.
FusionRandom subset without replacement from precomputed BoN- 1616 pool; candidates shuffled and </think> -stripped before synthesis; single-shot direct synthesis with thinking enabled.
Appendix
Table 9: Method-specific algorithmic settings beyond the compute-band axis.
AlgorithmHealthBenchLEXamPRBenchWildBenchWritingBenchTotal
BoN (oracle, n=16n=16 )4,5562024,99517350610,432
PF-High (oracle, n=16n=16 )1,519741,704541763,527
Fusion1,524612,219481644,015
Beam Search1,146441,657371203,004
PF (normal)1,146441,657371203,004
Budget Forcing764291,10525802,003
Appendix
Table 10: Total LLM-Judge input tokens (millions) by algorithm family and benchmark.
 TasksWall-clock (h)GPU-hours
Open-QA1642,642.621,104
Verifiable93342.22,737
Total2572,984.8≈\approx 23,800
Appendix
Table 11: Combined paper-level totals across Open-QA and verifiable evaluation tracks. Wall-clock hours measure end-to-end run duration; GPU-hours scale by node size.
JudgeMacro F1
GPT-4.10.709
o4-mini0.692
o30.681
Qwen3.5-397B-A17B (ours)0.679
GPT-4.1 mini0.661
Inter-physician mean0.655 ( ± 0.102\pm\,0.102 )
Appendix
Table 12: HealthBench judge agreement with physician annotations (Macro F1). The inter-physician baseline is the mean pairwise Macro F1 across 184 physicians with sufficient annotation data.
Benchmarknn pairsMetricAgreement
PRBench1,186κ\kappa / Macro F10.679 / 0.838
LEXam77Pearson rr / MAE0.813 / 22.2
WildBench147QWK / MAE0.564 / 1.35
WritingBench404QWK / MAE0.408 / 1.51
Appendix
Table 13: Criterion-level agreement between the unified (Qwen3.5-397B-A17B) and each benchmark’s native judge. PRBench is evaluated on a 5%5\% sample; the remaining benchmarks on a 15%15\% sample. Metrics are reported in the form most natural to each native protocol and are not directly comparable across rows.
BenchmarkSquare root of mean var (normalized)
HealthBench0.076
PRBench0.034
LEXam0.055
WildBench0.055
WritingBench0.030
Appendix
Table 14: Square root of the per-benchmark mean per-entry judge variance (micro-averaged across entries).
Figure 6: Naive (unadjusted) vs ideal oracle scores across compute for qwen3.5-9b BoN.
Figure 7: Oracle (full bar) vs. realised (hatched) quality per method at matched compute ( XHigh ), Qwen3.5-35B-A3B, using the naive (uncorrected) oracle.
Figure 8: Headroom capture per method at XHigh compute, using the naive (uncorrected) oracle. Compare with Figure 3 (right) which uses the bias-corrected oracle.
Figure 9: hh vs. ρv\rho_{v} across N=152N=152 task-level points using the naive (uncorrected) oracle and measured (non-denoised) ρv\rho_{v} . The empirical fit closely tracks the identity line y=xy=x , mirroring the linear relationship h^≈ρ^v\hat{h}\approx\hat{\rho}_{v} that Figure 3 (left) demonstrates under the bias-corrected oracle and denoised ρ^v\hat{\rho}_{v} . Same conclusion holds under both conventions: headroom capture is bounded by verifier correlation regardless of whether we de-noise the oracle and the correlation estimate.
 HealthBenchPRBenchLEXam
 LowMidHighXHighLowMidHighXHighLowMidHighXHigh
BoN+Skywork+0.047+0.038+0.030+0.025+0.038+0.023+0.014-0.005+0.047+0.062+0.065+0.051
Fusion—+0.670+0.580+0.495—+0.390+0.357+0.339—+0.355+0.246+0.209
Particle Filter———-0.168———-0.293———+0.006
Self - Refine-1.096-0.907-0.553-0.399+0.455+0.467+0.495+0.496-1.148-0.833-0.959-0.858
Appendix
Table 15: Headroom capture for Qwen3.5-35B : h=(realized−μ)/(oracleideal−μ)h=(\text{realized}-\mu)/(\text{oracle}_{\text{ideal}}-\mu) , where μ\mu is the BoN N=1N{=}1 baseline (single-inference mean). h=1h=1 means full exploitation of the oracle gap; h=0h=0 matches μ\mu ; h<0h<0 underperforms μ\mu . green = positive (gain captured), red = negative (below baseline). Bold when ∣h∣≥0.20|h|\geq 0.20 .
 HealthBenchPRBenchLEXam
 LowMidHighXHighLowMidHighXHighLowMidHighXHigh
BoN+Skywork+0.084+0.080+0.084+0.089+0.077+0.071+0.061+0.050+0.073+0.058+0.059+0.056
Fusion—+0.615+0.512+0.473—+0.301+0.346+0.313—+0.419+0.445+0.339
Self - Refine-3.615-1.681-1.175-0.918+0.251+0.323+0.404+0.329-1.104-0.970-0.952-1.147
Appendix
Table 16: Headroom capture for Qwen3.5-9B : h=(realized−μ)/(oracleideal−μ)h=(\text{realized}-\mu)/(\text{oracle}_{\text{ideal}}-\mu) , where μ\mu is the BoN N=1N{=}1 baseline (single-inference mean). h=1h=1 means full exploitation of the oracle gap; h=0h=0 matches μ\mu ; h<0h<0 underperforms μ\mu . green = positive (gain captured), red = negative (below baseline). Bold when ∣h∣≥0.20|h|\geq 0.20 .
 HealthBenchPRBenchLEXam
 LowMidHighXHighLowMidHighXHighLowMidHighXHigh
BoN+Skywork+0.177+0.165+0.160+0.165+0.170+0.137+0.119+0.110+0.216+0.171+0.152+0.157
Fusion—+0.207+0.292——+0.059+0.224——+0.353+0.286—
Appendix
Table 17: Headroom capture for OLMo3-7B : h=(realized−μ)/(oracleideal−μ)h=(\text{realized}-\mu)/(\text{oracle}_{\text{ideal}}-\mu) , where μ\mu is the BoN N=1N{=}1 baseline (single-inference mean). h=1h=1 means full exploitation of the oracle gap; h=0h=0 matches μ\mu ; h<0h<0 underperforms μ\mu . green = positive (gain captured), red = negative (below baseline). Bold when ∣h∣≥0.20|h|\geq 0.20 .
 HealthBenchPRBenchLEXam
 LowMidHighXHighLowMidHighXHighLowMidHighXHigh
BoN+Skywork+0.099+0.096+0.090+0.086+0.102+0.082+0.070+0.076+0.085+0.065+0.049+0.025
Fusion—-0.426-0.326——-0.979-1.016——+0.231+0.228—
Appendix
Table 18: Headroom capture for OLMo3-32B : h=(realized−μ)/(oracleideal−μ)h=(\text{realized}-\mu)/(\text{oracle}_{\text{ideal}}-\mu) , where μ\mu is the BoN N=1N{=}1 baseline (single-inference mean). h=1h=1 means full exploitation of the oracle gap; h=0h=0 matches μ\mu ; h<0h<0 underperforms μ\mu . green = positive (gain captured), red = negative (below baseline). Bold when ∣h∣≥0.20|h|\geq 0.20 .
 HealthBenchPRBenchLEXam
 N=1LowMidHighXHighN=1LowMidHighXHighN=1LowMidHighXHigh
BoN oracle0.5360.5710.5990.6200.6370.2920.3250.3540.3800.4030.5250.5600.5870.6090.626
Fusion——0.5930.6240.641——0.3490.3820.406——0.5840.6120.629
Particle Filter————0.575————0.327————0.580
Self - Refine—0.5510.5580.5720.594—0.3130.3330.3630.393—0.5430.5510.5640.578
Appendix
Table 19: Oracle quality for Qwen3.5-35B across five benchmarks and four compute levels. Cell colour encodes delta from BoN oracle N=1 baseline: steel blue = large gain, pale blue = small gain over BoN N=1 baseline. Bold = gain ≥0.03\geq 0.03 over baseline.
 HealthBenchPRBenchLEXam
 N=1LowMidHighXHighN=1LowMidHighXHighN=1LowMidHighXHigh
BoN+Skywork0.5080.5110.5140.5160.5180.2590.2620.2640.2650.2650.4630.4660.4670.4690.470
BoN+Llama70B0.5080.5110.5140.5170.5190.2590.2610.2620.2630.2620.4630.4680.4710.4750.475
Fusion——0.5480.5600.566——0.2750.2900.293——0.4870.5020.501
Beam Search——0.5080.5050.506——0.2540.2530.254——0.4750.4800.463
Particle Filter——0.5030.5060.502——0.2510.2510.248——0.4710.4650.467
Appendix
Table 20: Realised quality for Qwen3.5-9B across benchmarks and four compute levels. Cell colour encodes delta from the single-sample baseline (BoN@1): green = improvement, red = regression. Bold = gain ≥0.03\geq 0.03 over baseline.
 HealthBenchPRBenchLEXam
 N=1LowMidHighXHighN=1LowMidHighXHighN=1LowMidHighXHigh
BoN oracle0.5080.5490.5810.6060.6270.2590.2910.3190.3440.3650.4630.4990.5280.5520.572
Fusion——0.5740.6090.631——0.3120.3470.368——0.5210.5490.573
Self - Refine—0.5160.5290.5450.566—0.2760.2890.3170.343—0.4810.4900.4990.508
Appendix
Table 21: Oracle quality for Qwen3.5-9B across five benchmarks and four compute levels. Cell colour encodes delta from BoN oracle N=1 baseline: steel blue = large gain, pale blue = small gain over BoN N=1 baseline. Bold = gain ≥0.03\geq 0.03 over baseline.
 HealthBenchPRBenchLEXam
 N=1LowMidHighXHighN=1LowMidHighXHighN=1LowMidHighXHigh
BoN+Skywork0.4800.4840.4870.4890.4900.2110.2130.2150.2150.2170.3950.3980.3990.3990.398
BoN+Llama70B0.4800.4830.4860.4870.4880.2110.2120.2140.2150.2180.3950.3990.4010.4050.408
Fusion——0.4540.450———0.1740.164———0.4090.414—
Beam Search——0.4790.480———0.2110.212———0.3930.386—
Particle Filter——0.4790.4800.477——0.2100.2070.210——0.3900.4060.395
Appendix
Table 22: Realised quality for OLMo3-32B across five benchmarks and four compute levels. Cell colour encodes delta from the single-sample baseline (BoN@1): green = improvement, red = regression. Bold = gain ≥0.03\geq 0.03 over baseline.
 HealthBenchPRBenchLEXam
 N=1LowMidHighXHighN=1LowMidHighXHighN=1LowMidHighXHigh
BoN+Skywork0.3350.3420.3480.3530.3580.1420.1450.1470.1480.1500.2810.2880.2910.2940.298
BoN+Llama70B0.3350.3420.3460.3500.3540.1420.1440.1460.1470.1500.2810.2890.2940.2970.298
Fusion——0.3490.365———0.1440.154———0.3000.304—
Beam Search——0.3320.3280.329——0.1420.1390.141——0.2780.2780.272
Particle Filter——0.3340.3400.336——0.1390.1410.140——0.2730.2810.279
Appendix
Table 23: Realised quality for OLMo3-7B across five benchmarks and four compute levels. Cell colour encodes delta from the single-sample baseline (BoN@1): green = improvement, red = regression. Bold = gain ≥0.03\geq 0.03 over baseline.
 HealthBenchPRBenchLEXam
 N=1LowMidHighXHighN=1LowMidHighXHighN=1LowMidHighXHigh
BoN oracle0.3350.3760.4130.4450.4730.1420.1610.1800.1970.2130.2810.3130.3420.3680.391
Fusion——0.4030.438———0.1750.199———0.3350.361—
Appendix
Table 24: Oracle quality for OLMo3-7B across five benchmarks and four compute levels. Cell colour encodes delta from BoN oracle N=1 baseline: steel blue = large gain, pale blue = small gain over BoN N=1 baseline. Bold = gain ≥0.03\geq 0.03 over baseline.
 HealthBenchPRBenchLEXam
 N=1LowMidHighXHighN=1LowMidHighXHighN=1LowMidHighXHigh
BoN oracle0.4800.5210.5540.5800.6010.2110.2360.2590.2810.3000.3950.4290.4590.4830.501
Fusion——0.5400.571———0.2480.256———0.4560.478—
Appendix
Table 25: Oracle quality for OLMo3-32B across five benchmarks and four compute levels. Cell colour encodes delta from BoN oracle N=1 baseline: steel blue = large gain, pale blue = small gain over BoN N=1 baseline. Bold = gain ≥0.03\geq 0.03 over baseline.
 HealthBenchPRBenchLEXam
 XLowLowMidXHighXLowLowMidXHighXLowLowMidXHigh
BoN+Skywork+0.047+0.038+0.030+0.025+0.038+0.023+0.014-0.005+0.047+0.062+0.065+0.051
Fusion—+0.670+0.580+0.495—+0.390+0.357+0.339—+0.355+0.246+0.209
Particle Filter———-0.168———-0.293———+0.006
Self - Refine-1.096-0.907-0.553-0.399+0.455+0.467+0.495+0.496-1.148-0.833-0.959-0.858
Appendix
Table 26: Headroom capture for Qwen3.5-35B : h=(realized−μ)/(oracleideal−μ)h=(\text{realized}-\mu)/(\text{oracle}_{\text{ideal}}-\mu) , where μ\mu is the BoN N=1N{=}1 baseline (single-inference mean). h=1h=1 means full exploitation of the oracle gap; h=0h=0 matches μ\mu ; h<0h<0 underperforms μ\mu . green = positive (gain captured), red = negative (below baseline). Bold when ∣h∣≥0.20|h|\geq 0.20 .
 HealthBenchPRBenchLEXam
 XLowLowMidXHighXLowLowMidXHighXLowLowMidXHigh
BoN+Skywork+0.084+0.080+0.084+0.089+0.077+0.071+0.061+0.050+0.073+0.058+0.059+0.056
Fusion—+0.615+0.512+0.473—+0.301+0.346+0.313—+0.419+0.445+0.339
Self - Refine-3.615-1.681-1.175-0.918+0.251+0.323+0.404+0.329-1.104-0.970-0.952-1.147
Appendix
Table 27: Headroom capture for Qwen3.5-9B : h=(realized−μ)/(oracleideal−μ)h=(\text{realized}-\mu)/(\text{oracle}_{\text{ideal}}-\mu) , where μ\mu is the BoN N=1N{=}1 baseline (single-inference mean). h=1h=1 means full exploitation of the oracle gap; h=0h=0 matches μ\mu ; h<0h<0 underperforms μ\mu . green = positive (gain captured), red = negative (below baseline). Bold when ∣h∣≥0.20|h|\geq 0.20 .
 HealthBenchPRBenchLEXam
 XLowLowMidXHighXLowLowMidXHighXLowLowMidXHigh
BoN+Skywork+0.099+0.096+0.090+0.086+0.102+0.082+0.070+0.076+0.085+0.065+0.049+0.025
Fusion—-0.426-0.326——-0.979-1.016——+0.231+0.228—
Appendix
Table 28: Headroom capture for OLMo3-32B : h=(realized−μ)/(oracleideal−μ)h=(\text{realized}-\mu)/(\text{oracle}_{\text{ideal}}-\mu) , where μ\mu is the BoN N=1N{=}1 baseline (single-inference mean). h=1h=1 means full exploitation of the oracle gap; h=0h=0 matches μ\mu ; h<0h<0 underperforms μ\mu . green = positive (gain captured), red = negative (below baseline). Bold when ∣h∣≥0.20|h|\geq 0.20 .
 HealthBenchPRBenchLEXam
 XLowLowMidXHighXLowLowMidXHighXLowLowMidXHigh
BoN+Skywork+0.177+0.165+0.160+0.165+0.170+0.137+0.119+0.110+0.216+0.171+0.152+0.157
Fusion—+0.207+0.292——+0.059+0.224——+0.353+0.286—
Appendix
Table 29: Headroom capture for OLMo3-7B : h=(realized−μ)/(oracleideal−μ)h=(\text{realized}-\mu)/(\text{oracle}_{\text{ideal}}-\mu) , where μ\mu is the BoN N=1N{=}1 baseline (single-inference mean). h=1h=1 means full exploitation of the oracle gap; h=0h=0 matches μ\mu ; h<0h<0 underperforms μ\mu . green = positive (gain captured), red = negative (below baseline). Bold when ∣h∣≥0.20|h|\geq 0.20 .
GeneratorRMHealth.LEXamPRWildWriting
Qwen3.5-9BSkywork V20.0630.0450.0640.2550.197
 Llama-3.1-70B0.0680.0580.0500.1780.189
Qwen3.5-35BSkywork V20.0360.0280.0230.2600.220
 Llama-3.1-70B0.0670.0460.0310.1460.173
Olmo-3-7BSkywork V20.1270.1360.1130.1800.123
 Llama-3.1-70B0.1190.1540.1010.1660.095
Appendix
Table 30: Verifier correlation ρ^v\hat{\rho}_{v} (Spearman between RM scores and judge scores), averaged over tasks within each benchmark ( N=152N=152 , 4 generators ×\times 2 RMs ×\times 19 tasks). Both RMs produce near-identical profiles (Skywork mean ρ^v=0.120\hat{\rho}_{v}=0.120 , Llama mean ρ^v=0.107\hat{\rho}_{v}=0.107 , computed at task level before benchmark aggregation), confirming the failure is structural rather than model-specific.
GeneratorRMHealth.LEXamPRWildWriting
Qwen3.5-9BSkywork V20.0800.0560.0410.3450.208
 Llama-3.1-70B0.0920.1020.0150.1820.182
Qwen3.5-35BSkywork V20.0100.0510.0020.4130.275
 Llama-3.1-70B0.0560.0900.0460.2430.253
Olmo-3-7BSkywork V20.1390.1570.1190.1820.123
 Llama-3.1-70B0.1270.1570.1100.1300.126
Appendix
Table 31: Headroom capture h^=(BoN@16−μ) / (Oracle@16−μ)\hat{h}=(\text{BoN@16}-\mu)\,/\,(\text{Oracle@16}-\mu) for the same generator–RM–benchmark combinations. Bold negative values indicate the RM selects worse than random. Comparing cell-by-cell with Table 30 confirms h^≈ρ^v\hat{h}\approx\hat{\rho}_{v} (regression: h^=1.20 ρ^v−0.011\hat{h}=1.20\,\hat{\rho}_{v}-0.011 , R2=0.66R^{2}=0.66 , ρ=0.81\rho=0.81 , p<10−36p<10^{-36} ), validating Equation 4 .
 MATH-500GPQA Diamond
 N=1LowMidHighXHighN=1LowMidHighXHigh
BoN+Skywork0.9220.9300.9330.9350.9360.8230.8410.8510.8520.843
BoN+Llama70B0.9220.9290.9320.9340.9380.8230.8360.8420.8410.838
Fusion——0.9360.9340.932——0.8590.8330.823
Beam Search——0.9420.9400.934——0.8180.8030.818
Particle Filter——0.9120.8920.890—————
Appendix
Table 32: Realised quality on deterministic benchmarks for Qwen3.5-35B-A3B . Cell colour encodes delta from the single-sample baseline (BoN@1): green = improvement, red = regression. Bold = ∣Δ∣≥0.03|\Delta|\geq 0.03 . Italic Pass@K row reports the BoN-pool oracle. Particle Filter and Budget Forcing on GPQA were not run.
Figure 10: Reproduction of BoN scaling from the Skywork-Reward-V2 paper [ 14 ] on MATH-500 (left) and GPQA Diamond (right). Dashed grey: oracle accuracy. Solid coloured: realised accuracy under Skywork-Reward-V2-Llama-3.1-8B selection. Results match the authors’ reported curves, validating our BoN pipeline before deployment on open-ended benchmarks.
Figure 11: Left: Mean score trajectory across Sequential Refinement iterations per WildBench task category. Coding & Debugging rises steeply; most other categories are flat or decline. Right: Oracle and mean score trajectories on WildBench with Coding & Debugging excluded. Without this subtask, mean quality regresses across iterations while the oracle continues to rise—confirming the exploitation failure pattern observed on other benchmarks.
Figure 12: Token length distributions per method and benchmark for Qwen3.5-35B-A3B. BoN (blue) and Fusion (pink) show stable length distributions across compute levels. Sequential Refinement (orange gradient, SR k=1k{=}1 through k=16k{=}16 ) shows monotonically increasing median length on every benchmark, with the steepest growth on WritingBench and PRBench.
Benchmarknnρ^\hat{\rho} [95% CI]% ρ>0\rho{>}0tt -test ppWilcoxon pp
HealthBench4,619++ 0.133[ ++ 0.124, ++ 0.142]67.5%7.76×10−1807.76\times 10^{-180}8.20×10−1678.20\times 10^{-167}
PRBench1,634++ 0.112[ ++ 0.099, ++ 0.126]64.7%6.07×10−576.07\times 10^{-57}1.87×10−511.87\times 10^{-51}
LEXam502++ 0.014[ −- 0.011, ++ 0.039]52.0%2.73×10−12.73\times 10^{-1}3.81×10−13.81\times 10^{-1}
WildBench917++ 0.044[ ++ 0.024, ++ 0.063]57.6%1.93×10−51.93\times 10^{-5}4.72×10−64.72\times 10^{-6}
WritingBench554++ 0.198[ ++ 0.174, ++ 0.222]74.5%4.97×10−464.97\times 10^{-46}4.10×10−394.10\times 10^{-39}
Appendix
Table 33: Within-prompt Spearman rank correlation between response length and judge score for BoN candidates (16 per prompt), Qwen3.5-35B-A3B, unified judge. ρ^\hat{\rho} = mean per-prompt Spearman correlation; 95% CI is bootstrapped; % ρ>0\rho{>}0 = fraction of prompts with a positive correlation; pp -values from a one-sample tt -test and Wilcoxon signed-rank test against H0 ⁣:ρ=0H_{0}\colon\rho=0 .
Benchmarknnmean CVmed CV%CV <0.1{<}0.1%CV <0.2{<}0.2mean stdmed std
HealthBench5,0000.1500.12729.7%83.2%101.388.0
PRBench1,6500.0830.07681.2%98.6%141.6128.8
LEXam5160.1190.09654.3%90.9%113.5110.7
WildBench1,0230.1860.13130.2%73.4%295.2160.7
WritingBench5550.1450.10745.2%76.2%392.7244.0
Appendix
Table 34: Within-prompt length variance for BoN candidates (16 per prompt), Qwen3.5-35B-A3B. CV = coefficient of variation (std/mean) of token lengths within a prompt; %CV <0.1{<}0.1 and %CV <0.2{<}0.2 report the fraction of prompts with low length dispersion; mean and median std are in tokens. PRBench shows the lowest length variance (mean CV =0.083=0.083 ), consistent with its structured rubric format constraining response length. WritingBench and WildBench show the highest variance, reflecting open-ended prompts that admit responses of widely varying length.
BenchmarknnCV scoreCV lenmean ratio%ratio >2{>}2%both <0.05{<}0.05
HealthBench5,0000.3290.1501.5737.8%0.1%
PRBench1,6500.3560.0833.1776.5%0.2%
LEXam5160.2170.1191.6539.1%0.0%
WildBench1,0230.1370.1860.698.0%0.4%
WritingBench5550.0840.1450.555.2%1.4%
Appendix
Table 35: Score variability versus length variability for BoN candidates (16 per prompt), Qwen3.5-35B-A3B, unified judge. CV score and CV len are the per-prompt coefficient of variation of judge scores and token lengths respectively; mean ratio == mean(CV score /CV len ). A ratio >1{>}1 indicates score variation is driven by content; a ratio <1{<}1 indicates length is the dominant axis of variation. PRBench (ratio =3.17=3.17 ) and HealthBench (ratio =1.57=1.57 ) are content-driven. WritingBench (ratio =0.55=0.55 ) and WildBench (ratio =0.69=0.69 ) are length-driven even under BoN, before any iterative elaboration.
Benchmarknnρ^\hat{\rho} [95% CI]% ρ>0\rho{>}0tt -test ppWilcoxon pp
HealthBench4,703++ 0.059[ ++ 0.047, ++ 0.069]56.2%1.26×10−231.26\times 10^{-23}2.94×10−232.94\times 10^{-23}
PRBench1,634++ 0.252[ ++ 0.232, ++ 0.272]72.2%6.68×10−1166.68\times 10^{-116}1.75×10−991.75\times 10^{-99}
LEXam505−- 0.060[ −- 0.092, −- 0.026]41.2%2.46×10−42.46\times 10^{-4}2.29×10−42.29\times 10^{-4}
WildBench925++ 0.023[ −- 0.001, ++ 0.046]52.0%5.23×10−25.23\times 10^{-2}3.82×10−23.82\times 10^{-2}
WritingBench555++ 0.352[ ++ 0.319, ++ 0.385]80.7%1.12×10−731.12\times 10^{-73}1.21×10−551.21\times 10^{-55}
Appendix
Table 36: Within-prompt Spearman rank correlation between response length and judge score for Sequential Refinement drafts ( k=1,…,16k=1,\ldots,16 per prompt), Qwen3.5-35B-A3B, unified judge. Columns as in Table 33 . Compared with BoN, Sequential Refinement shows a substantially stronger length–score correlation on WritingBench ( +0.352+0.352 vs. +0.198+0.198 ) and PRBench ( +0.252+0.252 vs. +0.112+0.112 ), confirming that iterative elaboration amplifies the length–score relationship. LEXam is the only benchmark with a significantly negative correlation ( ρ^=−0.060\hat{\rho}=-0.060 ), consistent with refinement producing longer but worse responses on knowledge-intensive tasks.
BenchmarknnCV scoreCV lenmean ratio%ratio >2{>}2%both <0.05{<}0.05
HealthBench5,0000.3580.2570.9119.3%0.0%
PRBench1,6500.2970.2560.8415.6%0.0%
LEXam5160.2370.2180.8613.2%0.0%
WildBench1,0240.1260.2560.465.8%0.2%
WritingBench5550.0830.1980.362.3%0.5%
Appendix
Table 37: Score variability versus length variability for Sequential Refinement drafts (16 iterations per prompt), Qwen3.5-35B-A3B, unified judge. Columns as in Table 35 . Compared with BoN, Sequential Refinement shows higher CV len on every benchmark, reflecting monotonic length growth across iterations. On WritingBench, CV len substantially exceeds CV score (ratio =0.36=0.36 ), confirming that length growth is the primary driver of score gains. On HealthBench, the ratio drops from 1.571.57 (BoN) to 0.910.91 (Sequential Refinement): iterative conditioning increases length variance more than score variance, a sign of elaboration without quality improvement.
Benchmarknnρ^\hat{\rho} [95% CI]% ρ>0\rho{>}0tt -test ppWilcoxon pp
HealthBench4,619++ 0.133[ ++ 0.124, ++ 0.142]67.5%7.76×10−1807.76\times 10^{-180}8.20×10−1678.20\times 10^{-167}
PRBench1,634++ 0.112[ ++ 0.099, ++ 0.125]64.7%6.07×10−576.07\times 10^{-57}1.87×10−511.87\times 10^{-51}
LEXam502++ 0.014[ −- 0.010, ++ 0.038]52.0%2.73×10−12.73\times 10^{-1}3.81×10−13.81\times 10^{-1}
WildBench917++ 0.044[ ++ 0.023, ++ 0.064]57.6%1.93×10−51.93\times 10^{-5}4.72×10−64.72\times 10^{-6}
WritingBench555++ 0.370[ ++ 0.347, ++ 0.394]87.9%6.24×10−1216.24\times 10^{-121}7.28×10−777.28\times 10^{-77}
Appendix
Table 38: Within-prompt Spearman rank correlation between response length and judge score for BoN candidates (16 per prompt), Qwen3.5-35B-A3B, native WritingBench judge . WritingBench ρ^\hat{\rho} rises from +0.198+0.198 (unified judge, Table 33 ) to +0.370+0.370 ; all other benchmarks are stable, confirming the amplification is WritingBench-specific.
BenchmarknnCV scoreCV lenmean ratio%ratio >2{>}2%both <0.05{<}0.05
HealthBench5,0000.3290.1501.5737.8%0.1%
PRBench1,6500.3560.0833.1776.5%0.2%
LEXam5160.2170.1191.6539.1%0.0%
WildBench1,0230.1370.1860.698.0%0.4%
WritingBench5550.0340.1450.250.0%2.0%
Appendix
Table 39: Score variability versus length variability for BoN candidates (16 per prompt), Qwen3.5-35B-A3B, native WritingBench judge . Under the native judge, WritingBench CV score collapses to 0.0340.034 (vs. 0.0840.084 under the unified judge, Table 35 ), yielding a ratio of 0.250.25 : score variation is almost entirely explained by length. Length statistics are identical to Table 34 since they are judge-independent.
Benchmarknnρ^\hat{\rho} [95% CI]% ρ>0\rho{>}0tt -test ppWilcoxon pp
HealthBench4,703++ 0.059[ ++ 0.047, ++ 0.070]56.2%1.26×10−231.26\times 10^{-23}2.94×10−232.94\times 10^{-23}
PRBench1,634++ 0.252[ ++ 0.232, ++ 0.272]72.2%6.68×10−1166.68\times 10^{-116}1.75×10−991.75\times 10^{-99}
LEXam505−- 0.060[ −- 0.091, −- 0.029]41.2%2.46×10−42.46\times 10^{-4}2.29×10−42.29\times 10^{-4}
WildBench925++ 0.023[ ++ 0.000, ++ 0.045]52.0%5.23×10−25.23\times 10^{-2}3.82×10−23.82\times 10^{-2}
WritingBench551++ 0.539[ ++ 0.513, ++ 0.566]92.2%1.07×10−1591.07\times 10^{-159}1.82×10−821.82\times 10^{-82}
Appendix
Table 40: Within-prompt Spearman rank correlation between response length and judge score for Sequential Refinement drafts ( k=1,…,16k=1,\ldots,16 per prompt), Qwen3.5-35B-A3B, native WritingBench judge . WritingBench ρ^\hat{\rho} rises to +0.539+0.539 (vs. +0.352+0.352 under the unified judge, Table 36 ), with 92.2% of prompts showing a positive correlation. All other benchmarks are stable.
BenchmarknnCV scoreCV lenmean ratio%ratio >2{>}2%both <0.05{<}0.05
HealthBench5,0000.3580.2570.9119.3%0.0%
PRBench1,6500.2970.2560.8415.6%0.0%
LEXam5160.2370.2180.8613.2%0.0%
WildBench1,0240.1260.2560.465.8%0.2%
WritingBench5550.0380.1980.170.5%0.4%
Appendix
Table 41: Score variability versus length variability for Sequential Refinement drafts (16 iterations per prompt), Qwen3.5-35B-A3B, native WritingBench judge . WritingBench CV score =0.038=0.038 (vs. 0.0830.083 under the unified judge, Table 37 ), with a ratio of 0.170.17 : under the native judge, virtually all score variation in Sequential Refinement on WritingBench is attributable to response length.
Figure 13: WritingBench old vs. new judge at k=16k{=}16 (Qwen3.5-35B, micro-averaged across 555 samples). Left: Oracle (solid) and realized (hatched) scores for BoN+Skywork, BoN+Llama70B, and Self-Refine under our judge (Qwen3.5-397B, dark) and the authors’ fine-tuned WritingBench-Critic-Model-Qwen-7B (light); horizontal lines show the candidate-mean μ\mu for each judge. Center: Headroom capture (realized−μ)/(oracle−μ)(\text{realized}-\mu)/(\text{oracle}-\mu) . Right: Decomposition into denominator (oracle −μ-\mu , solid) and numerator (realized −μ-\mu , hatched).
Figure 14: Absolute score gain over the BoN@1 baseline across compute levels, by model family and benchmark.
Figure 15: Fraction of remaining gap to a perfect score, (pk−p1)/(1−p1)(p_{k}-p_{1})/(1-p_{1}) , closed at compute level kk , faceted by benchmark. p1p_{1} is the BoN@1 baseline.
Figure 16: Sequential Refinement oracle vs BoN oracle across compute levels, by model family and benchmark.
Figure 17: Sequential Refinement oracle vs BoN oracle across compute levels, by model family and benchmark. Note that x-axis refers to number of candidates in the pool so compare k self-revisions to k i.i.d. samples here.
Figure 18: Oracle gaps over a BoN@1 baseline across models for fusion at medium compute (k=8). Note that Olmo models used 6 candidates instead of 7 due to context length limits. Additionally, context-length issues persisted for Olmo on PrBench and WildBench so they are not included here.
Figure 19: Oracle gap and headroom capture comparison between different families
Figure 20: Demand profiles for HealthBench (left) and LEXam (right)
Figure 21: Demand profiles for PRBench (left) and WildBench (right)
Figure 22: WritingBench demand profile
BenchmarkQwen3.5-9BQwen3.5-35B-A3BOLMo-3-7BOLMo-3.1-32B
 TokensBand [min–max]TokensBand [min–max]TokensBand [min–max]TokensBand [min–max]
healthbench_communication39.6M[28.9M–50.3M]35.7M[26.1M–45.4M]32.9M[24.0M–41.8M]34.0M[24.8M–43.2M]
healthbench_complex_responses17.3M[12.6M–22.0M]16.2M[11.8M–20.6M]14.1M[10.3M–17.9M]16.0M[11.7M–20.3M]
healthbench_context_seeking19.3M[14.1M–24.5M]17.8M[13.0M–22.6M]15.6M[11.4M–19.8M]15.6M[11.4M–19.8M]
healthbench_emergency_referrals15.9M[11.6M–20.1M]14.5M[10.6M–18.4M]12.6M[9.2M–16.0M]12.1M[8.8M–15.3M]
healthbench_global_health41.3M[30.1M–52.4M]37.5M[27.4M–47.7M]34.5M[25.2M–43.9M]34.7M[25.3M–44.0M]
Appendix
Table 42: BoN n=16n{=}16 total completion tokens and ±27%\pm 27\% compute band (min–max) per benchmark.
BenchmarkQwen3.5-9BQwen3.5-35B-A3BOLMo-3-7BOLMo-3.1-32B
healthbench_communication10.2M9.0M8.3M8.6M
healthbench_complex_responses4.5M4.3M3.4M3.9M
healthbench_context_seeking5.0M4.6M3.9M3.9M
healthbench_emergency_referrals4.1M3.7M3.1M3.0M
healthbench_global_health10.5M9.4M8.7M8.9M
healthbench_health_data_tasks5.5M5.0M4.3M5.1M
Appendix
Table 43: Particle Filter (Mid ( n=4n{=}4 )) total completion tokens per benchmark.
BenchmarkQwen3.5-9BQwen3.5-35B-A3BOLMo-3-7BOLMo-3.1-32B
healthbench_communication20.3M18.2M16.5M17.1M
healthbench_complex_responses9.1M8.6M6.8M8.0M
healthbench_context_seeking10.0M9.1M7.8M8.0M
healthbench_emergency_referrals8.2M7.3M6.4M6.1M
healthbench_global_health21.2M18.9M17.5M17.7M
healthbench_health_data_tasks10.9M10.2M8.3M9.9M
Appendix
Table 44: Particle Filter (High ( n=8n{=}8 )) total completion tokens per benchmark.
BenchmarkQwen3.5-9BQwen3.5-35B-A3BOLMo-3-7BOLMo-3.1-32B
healthbench_communication40.5M36.1M33.0M35.0M
healthbench_complex_responses18.1M17.0M13.7M15.9M
healthbench_context_seeking20.0M18.2M16.0M16.2M
healthbench_emergency_referrals16.5M14.7M12.9M12.5M
healthbench_global_health42.3M37.8M35.2M36.3M
healthbench_health_data_tasks21.9M20.5M16.8M20.4M
Appendix
Table 45: Particle Filter (XHigh ( n=16n{=}16 )) total completion tokens per benchmark.
BenchmarkQwen3.5-9BQwen3.5-35B-A3BOLMo-3-7BOLMo-3.1-32B
healthbench_communication10.2M9.0M8.0M8.4M
healthbench_complex_responses4.7M4.2M3.4M4.1M
healthbench_context_seeking5.0M4.6M3.8M3.8M
healthbench_emergency_referrals4.1M3.7M2.9M2.9M
healthbench_global_health10.5M9.3M8.4M8.5M
healthbench_health_data_tasks5.5M5.1M3.9M5.0M
Appendix
Table 46: Beam Search (Mid ( bw=2, n=2\mathrm{{bw}}{=}2,\,n{=}2 )) total completion tokens per benchmark.
BenchmarkQwen3.5-9BQwen3.5-35B-A3BOLMo-3-7BOLMo-3.1-32B
healthbench_communication19.8M17.7M15.7M16.4M
healthbench_complex_responses9.4M8.8M6.6M7.9M
healthbench_context_seeking9.7M8.8M7.4M7.5M
healthbench_emergency_referrals8.1M7.2M5.7M5.5M
healthbench_global_health20.6M18.4M16.4M16.6M
healthbench_health_data_tasks10.7M10.2M8.0M9.6M
Appendix
Table 47: Beam Search (High ( bw=2, n=4\mathrm{{bw}}{=}2,\,n{=}4 )) total completion tokens per benchmark.
BenchmarkQwen3.5-9BQwen3.5-35B-A3BOLMo-3-7BOLMo-3.1-32B
healthbench_communication40.4M34.9M32.0M34.0M
healthbench_complex_responses21.6M19.8M14.0M16.7M
healthbench_context_seeking21.0M18.1M15.1M15.5M
healthbench_emergency_referrals16.6M14.4M11.3M11.8M
healthbench_global_health42.6M36.6M33.3M34.9M
healthbench_health_data_tasks23.3M21.4M16.6M20.4M
Appendix
Table 48: Beam Search (XHigh ( bw=4, n=4\mathrm{{bw}}{=}4,\,n{=}4 )) total completion tokens per benchmark. † Run at bw=3, n=4\mathrm{{bw}}{=}3,\,n{=}4 .
BenchmarkQwen3.5-9BQwen3.5-35B-A3BOLMo-3-7BOLMo-3.1-32B
healthbench_communication83.0M82.6M––
healthbench_complex_responses33.0M32.4M––
healthbench_context_seeking45.6M44.7M––
healthbench_emergency_referrals36.4M36.0M––
healthbench_global_health91.6M88.3M––
healthbench_health_data_tasks40.1M40.2M––
Appendix
Table 49: Sequential Refinement ( iter=16\text{iter}{=}16 ) total completion tokens per benchmark.
BenchmarkQwen3.5-9BQwen3.5-35B-A3BOLMo-3-7BOLMo-3.1-32B
healthbench_communication7.4M6.7M6.2M6.4M
healthbench_complex_responses3.2M3.0M2.6M3.0M
healthbench_context_seeking3.6M3.3M2.9M2.9M
healthbench_emergency_referrals3.0M2.7M2.4M2.3M
healthbench_global_health7.7M7.0M6.5M6.5M
healthbench_health_data_tasks3.9M3.7M3.2M3.8M
Appendix
Table 50: Fusion (Mid ( s=3s{=}3 )) total completion tokens per benchmark.
BenchmarkQwen3.5-9BQwen3.5-35B-A3BOLMo-3-7BOLMo-3.1-32B
healthbench_communication14.9M13.4M12.3M12.8M
healthbench_complex_responses6.5M6.1M5.3M6.0M
healthbench_context_seeking7.2M6.7M5.8M5.8M
healthbench_emergency_referrals5.9M5.4M4.7M4.5M
healthbench_global_health15.5M14.1M13.0M13.0M
healthbench_health_data_tasks7.8M7.4M6.5M7.5M
Appendix
Table 51: Fusion (High, s=6or7s{=}6or7 ) total completion tokens per benchmark.
BenchmarkQwen3.5-9BQwen3.5-35B-A3BOLMo-3-7BOLMo-3.1-32B
healthbench_communication37.1M33.5M––
healthbench_complex_responses16.2M15.2M––
healthbench_context_seeking18.1M16.7M––
healthbench_emergency_referrals14.9M13.6M––
healthbench_global_health38.7M35.2M––
healthbench_health_data_tasks19.5M18.5M––
Appendix
Table 52: Fusion (XHigh ( s=15s{=}15 )) total completion tokens per benchmark.

Explore similar work

CardsList
May 8, 2026

LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling

Test-time scaling (TTS) has become an effective approach for improving large language model performance by allocating additional computation during inference. However, existing TTS strategies are largely hand-crafted: researchers manually design reasoning patterns and tune heuristics by intuition, leaving much of the computation-allocation space unexplored. We propose an environment-driven framework, AutoTTS, that changes what researchers design: from individual TTS heuristics to environments where TTS strategies can be discovered automatically. The key to AutoTTS lies in environment construction: the discovery environment must make the control space tractable and provide cheap, frequent feedback for TTS search. As a concrete instantiation, we formulate width--depth TTS as controller synthesis over pre-collected reasoning trajectories and probe signals, where controllers decide when to branch, continue, probe, prune, or stop and can be evaluated cheaply without repeated LLM calls. We further introduce beta parameterization to make the search tractable and fine-grained execution trace feedback to improve discovery efficiency by helping the agent diagnose why a TTS program fails. Experiments on mathematical reasoning benchmarks show that the discovered strategies improve the overall accuracy--cost tradeoff over strong manually designed baselines. The discovered strategies generalize to held-out benchmarks and model scales, while the entire discovery costs only $39.9 and 160 minutes. Our data, and code will be open-source at https://github.com/zhengkid/AutoTTS.

Tong Zheng, Haolin Liu, Chengsong Huang +10

UMD · UVA · WUSTL +3

LLM Inference EfficiencyAutomated Algorithm DiscoveryTest-Time OptimizationLarge Language Model-Guided OptimizationTest-Time Scaling
AskPDF
Aug 21, 2026

Test-Time Scaling for Scientific Equation Discovery

Test-time scaling (TTS) improves language model reasoning by allocating additional test-time compute, but prior work mainly studies closed-ended tasks such as math and coding. We study TTS for automated equation discovery, an open-ended setting where models search over candidate equations and rely on observed datapoints for feedback. We formulate LLM-driven equation discovery as an iterative search process that unifies Best-of-N, sequential refinement, tree search, and evolution-style methods under a common compute-allocation view. To isolate allocation effects from prompt engineering and other heuristics, we compare minimal parallel controllers under fixed budgets. On LLM-SRBench equation-discovery tasks, we find that search width is the dominant allocation parameter: the best width in our sweep generally increases with the compute budget, while the population--branching split and controller choice matter less. Appropriate width selection also improves wall-clock efficiency by increasing parallelism. These results suggest that, given an informative verifier, controlling exploration and exploitation is central to scaling LLM-based equation discovery.

Haowei Lin, Hubert Lim, Xiangyu Wang +2

Peking University

Inference-Time SearchTest-Time ScalingLarge Language Model-Guided Scientific DiscoverySymbolic Regression
AskPDF
May 9, 2026

Test-Time Personalization: A Diagnostic Framework and Probabilistic Fix for Scaling Failures

Existing approaches to LLM personalization focus on constructing better personalized models or inputs, while treating inference as a single-shot process. In this work, we study Test-Time Personalization (TTP) along an unexplored axis: scaling inference-time computation by sampling N candidates from a personalized policy model and selecting the best with a personalized reward model. We prove that oracle selection yields expected utility growing logarithmically with the number of sampled candidates, establishing a theoretical ceiling for test-time scaling. However, standard reward models fail to realize this potential. To diagnose why, we derive a unified scaling law that decomposes any reward model's Best-of-N curve into four measurable quantities and reveals two failure modes, user-level collapse (near-constant prediction for some users) and query-level reward hacking (negative correlation with true quality for some queries). Guided by this law, we propose a probabilistic personalized reward model whose learned variance effectively mitigates both failure modes. Experiments confirm both elements of our framework: TTP delivers consistent scaling across multiple policy models and personalized text generation tasks, and our scaling law closely matches observed scaling curves across reward-model variants.

Linhai Zhang, Yulan He

King’s College London · The Alan Turing Institute

Reward ModelingReward HackingLLM PersonalizationTest-Time Scaling
AskPDF