Test-Time Scaling for Open-Ended Reasoning at Small Model Scale

Adam Y. Ahmed; Abdullah G. Elafifi

cs.LGNov 1, 2026

Test-Time Scaling for Open-Ended Reasoning at Small Model Scale

Authors: Adam Y. Ahmed, Abdullah G. Elafifi

Abstract

A vote over a model's samples can only help when the samples disagree. For open-ended generation the binding question is therefore not how to combine samples but whether a model produces samples worth combining, a question test-time scaling has answered for math but not for free-form text. We study it at small scale with semantic cluster voting, a judge-free aggregator that groups a small instruct model's samples by meaning and returns the largest cluster's representative, with no reward model or self-judging. On a 250-prompt benchmark with three tiers of answer checkability, the same pattern holds across three model families (Qwen2.5-3B, Llama-3.2-3B, Gemma-2-2B) and two generation seeds: voting raises accuracy on the math control, where samples vary but stay checkable (a gain of +10 to +29 points across the three models), yet adds at most +0.30 judge points on open-ended text. On a larger 200-problem MATH-500 run with the primary model, the math gain is +8.5 points (95% CI [+4.0, +13.0]). Almost every open-ended sample falls into one meaning cluster, so there is little to select over. The collapse is consistent with instruction-tuning rather than model size, and raising decoding temperature does not recover usable diversity, because it costs more quality than it adds (net -0.64 judge points, 95% CI [-0.93, -0.37]).

Figures & tables

Tier	Single	USC	Cluster
Open-ended (1–10)	7.69	8.02 ( $N{=}8$ )	7.93 ( $N{=}64$ )
Long-form QA (1–10)	6.26	6.37 ( $N{=}4$ )	6.44 ( $N{=}32$ )
Math (acc. %, 50)	64	64	74 ( $N{=}32$ )

Table 1: Main results across the three tiers, for the primary model (Qwen2.5-3B-Instruct, seed 42). Each method’s best score over the sample-budget sweep

N\in\{1,4,8,16,32,64\}

(the budget in parentheses); single is budget-independent. Text tiers are LLM-judge scores (1–10); the math tier is exact-match accuracy (%) on the 50-problem control. The best value in each row is in bold. Paired differences against single with 95% CIs at every

N

are in Appendix C , and the math gain is confirmed on a larger set in Appendix B . The same comparison across three model families and two seeds is in Table 2 .

Open-ended (judge 1–10)
Model	Single	USC	Cluster
Qwen2.5-3B	7.62 $\pm$ 0.11	7.85 $\pm$ 0.24	7.91 $\pm$ 0.03
Llama-3.2-3B	7.83 $\pm$ 0.05	8.01 $\pm$ 0.04	8.12 $\pm$ 0.04
Gemma-2-2B	7.92 $\pm$ 0.03	8.15 $\pm$ 0.05	8.10 $\pm$ 0.08
Long-form QA (judge 1–10)
Qwen2.5-3B	6.36 $\pm$ 0.14	6.46 $\pm$ 0.21	6.56 $\pm$ 0.28

Table 2: Generality across three small instruct families and two seeds: each method’s best score over the budget sweep

N\in\{1,4,8,16,32,64\}

, mean

\pm

s.d. over seeds 42 and 123 (single is budget-independent). Text tiers are judge scores (1–10); math is exact-match accuracy (%) on the 50-problem control. Best in each row in bold. The best open-ended gain over single is at most

+0.30

on every model while the math gain scales. Qwen2.5-3B is the primary model (Table 1 ).

Model	Prompts	Median share
Qwen2.5-3B (base)	120	0.38
Qwen2.5-3B-Instruct	120	0.97 $[0.95,0.97]$
Qwen2.5-7B-Instruct	20	1.00

Table 3: Median largest-cluster share on the open-ended tier (8 samples per prompt,

T=1.0

, median over prompts, adaptive cut). A share near 1 means the samples collapse into one meaning cluster. The aligned 3B model collapses while its pre-alignment base does not, and a larger aligned model collapses as hard.

Contrast	$\Delta$ judge /10	95% CI
Cluster $-$ single, $T=1.0$	$+0.22$	$[+0.03,+0.40]$
Cluster $-$ single, $T=1.5$	$+0.17$	$[-0.11,+0.42]$
Temperature cost (single $T{=}1.0$ $-$ single $T{=}1.5$ )	$+0.81$	$[+0.56,+1.07]$
Net (cluster $T{=}1.5$ $-$ single $T{=}1.0$ )	$-0.64$	$[-0.93,-0.37]$

Table 4: Temperature decomposition on the open-ended tier (instruct 3B, 120 prompts), judge points with 95% bootstrap CIs over prompts. Higher temperature recovers some diversity, but the quality it costs (

+0.81

) outweighs the voting gain it enables (

+0.17

), so voting at

T=1.5

lands below the default single-sample baseline.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

$N$	Single	Cluster	Cluster $-$ Single
1	59.0	59.0	$+0.0\ [+0.0,+0.0]$
4	59.0	63.0	$+4.0\ [+1.0,+7.5]$
8	59.0	66.0	$+7.0\ [+3.0,+11.5]$
16	59.0	67.0	$+8.0\ [+3.5,+12.5]$
32	59.0	67.5	$+8.5\ [+4.0,+13.0]$

Appendix

Table 5: Math power run: single versus cluster voting on 200 MATH-500 problems (accuracy %), with the paired gain and its 95% bootstrap CI. The gain excludes zero at every

N\geq 4

Condition	Quality /10	95% CI
Base, single	5.53	$[4.67,6.35]$
Base, cluster	6.20	$[5.38,6.95]$
Instruct, single	7.75	$[7.08,8.35]$

Appendix

Table 6: Base-model diagnostic on 40 open-ended prompts (8 samples each), judged 1 to 10 with 95% bootstrap CIs. The base model is more diverse (largest-cluster share 0.75 versus the instruct model’s 1.00) but far lower quality, and voting over it does not reach the instruct single-sample floor.

Open-ended
$N$	Cluster $-$ Single	USC $-$ Single
1	$+0.00\ [+0.00,+0.00]$	$+0.00\ [+0.00,+0.00]$
4	$+0.03\ [-0.14,+0.19]$	$+0.10\ [-0.06,+0.28]$
8	$+0.13\ [-0.03,+0.31]$	$+0.33\ [+0.15,+0.53]$
16	$+0.12\ [-0.07,+0.31]$	$+0.21\ [+0.04,+0.40]$
32	$+0.15\ [-0.03,+0.35]$	$+0.05\ [-0.13,+0.23]$

Appendix

Table 7: Paired differences against single (95% bootstrap CI). Text in judge points, math in accuracy points on the 50-problem tier.

Test-Time Scaling

Test-time scaling studies how extra inference-time computation can improve model answers, decisions, and generated outputs.

See all 232 papers

Interpretable Adaptive Sampling for LLM Test-Time Scaling
Aug 4, 2026Mobina Kashaniyan, Ali JannesariLLM Inference Efficiency Test-Time Scaling
Iowa State University Ames, IA, USA
Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneck
Aug 19, 2026Davide Romano, Kanak Raj, Jerrod Parker +1Test-Time Scaling Language Model Decoding
Thomson Reuters Foundational Research, 5 Canada Square, London
Ranking Reasoning LLMs under Test-Time Scaling
Mar 11, 2026Mohsen Hariri, Michael Hinczewski, Jing Ma +1LLM Evaluation Test-Time Scaling
Department of Computer and Data Sciences, Case Western Reserve University, Cleveland, OH, USA · Department of Physics, Case Western Reserve University, Cleveland, OH, USA

Aug 4, 2026J/K move · Enter open · S save

Interpretable Adaptive Sampling for LLM Test-Time Scaling

Mobina Kashaniyan, Ali Jannesari

Iowa State University Ames, IA, USA

LLM Inference Efficiency Test-Time Scaling Efficient Language Model Reasoning Adaptive Sampling

Open paper AskPDF arXiv

Test-time scaling improves LLM reasoning by generating and aggregating multiple candidate answers, yet many pipelines use fixed per-query budgets that spend the same compute on easy and difficult prompts. These fixed budgets are also difficult to inspect because they do not explain why a given prompt receives a particular number of samples. We propose adaptive} test-time scaling with a lightweight fuzzy controller that maps interpretable signals, including estimated prompt complexity and model confidence, to a per-query sampling budget. The controller assigns fewer samples to easier or more confident prompts and more samples to harder or less certain prompts, making inference-time compute inspectable rather than fixed or opaque. We evaluate under a fair-alignment protocol with matched decoding settings and controlled answer selection, and compare against best-of-

N

, compute-aware scaling, and self-certainty-based baselines on question-answering and mathematical reasoning tasks. Across models and datasets, adaptive fuzzy control improves over several standard baselines and remains close to a selector-matched full-budget control while reducing the average number of samples. These findings suggest that interpretable adaptive sampling is a practical direction for more efficient test-time reasoning in large language models.

Abstract

Test-Time Scaling

Explore similar work

Interpretable Adaptive Sampling for LLM Test-Time Scaling

Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneck

Ranking Reasoning LLMs under Test-Time Scaling