Test-time scaling investigates whether a fixed diffusion language model (DLM) can generate better outputs when given more inference compute, without additional training. However, naive best-of-K sampling is fundamentally limited because it repeatedly draws from the same base diffusion distribution, whose high-probability regions are often misaligned with high-quality outputs. We propose S3 (Stratified Scaling Search), a classical verifier-guided search method that improves generation by reallocating compute during the denoising process rather than only at the final output stage. At each denoising step, S3 expands multiple candidate trajectories, evaluates them with a lightweight reference-free verifier, and selectively resamples promising candidates while preserving diversity within the search frontier. This procedure effectively approximates a reward-tilted sampling distribution that favors higher-quality outputs while remaining anchored to the model prior. Experiments with LLaDA-8B-Instruct on MATH-500, GSM8K, ARC-Challenge, and TruthfulQA demonstrate that S3 consistently improves performance across benchmarks, achieving the largest gains on mathematical reasoning tasks while leaving the underlying model and decoding schedule unchanged. These results show that classical search over denoising trajectories provides a practical mechanism for test-time scaling in DLMs.
Figures & tables
Figure 1: Density–quality mismatch and trajectory search under S3 for MATH-500 . Panels (a)-(c) illustrate the distributional mismatch between p0 , the verifier landscape f(x) , and the reward-tilted target p~0 . Panels (d)-(f) show the corresponding inference behavior of direct sampling versus S3 trajectory search and the resulting shift in verifier score distribution.
Figure 2: S3 : N particles are initialized at t=T [P1], then at each step expanded into Nb candidates, scored via one-step clean predictions x^0(i,j,t) [P2], and resampled to N particles via SSP [P3]. Final output is selected by majority voting at t=0 .
Dataset
Baseline Diffusion
Best-of- K
S3
GSM-8K
68.16
69.56
70.21
MATH-500
25.60
28.20
30.20
TruthfulQA
46.49
49.36
49.57
ARC-Challenge
76.11
79.30
77.86
Table 1: Accuracy (%) of standard diffusion decoding, Best-of- K ( K=8 ), and S3 ( N=4 , b=2 , K=64 ) on LLaDA-8B-Instruct.
Figure 3: Inference-time scaling with S3 across datasets. Top row: accuracy vs. compute ( NFE=steps×N×b ) across multiple (N,b) settings. Bottom row: mean top-1 token confidence over denoising progress. Curves shown for Baseline, S3 ( N=4 , b=2 , λ=1.0 ), and BoK ( K=8 ).
Figure 4: Accuracy (%) across block lengths K∈{2,4,8,16,32,64} on four benchmarks. Results use LLaDA-8B-Instruct with N=4 , b=2 , and K=8 . Orange bars denote S3 , blue bars denote Baseline, and green bars denote BoK. An upward arrow ( ↑ ) indicates configurations where the leading method improves over the Baseline.
Method
Look-ahead
Tilting
Accuracy (%)
GSM8K
MATH-500
ARC-C
TruthfulQA
Baseline diffusion
×
×
68.16
25.60
76.11
46.49
Best-of- K
×
×
69.56
28.20
79.30
49.36
Tilting only
×
✓
69.54
26.40
76.30
48.10
Look-ahead only
✓
×
69.69
26.20
76.71
46.49
S3 (full, ours)
✓
✓
70.21
30.20
77.86
49.57
Table 2: Ablation study decomposing the contributions of look-ahead search and Gibbs tilting in S3 (LLaDA-8B-Instruct, N=4 , b=2 , K=N⋅b=8 , block length 64, generation length 128). ✓/× indicate whether each mechanism is active. Scores represent task accuracy (%).
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Distributional shift of verifier scores under S3 search on MATH-500. Sequential resampling during denoising concentrates particles in higher-reward regions compared to direct sampling from p0 .
Dataset
α1
α2
α3
α4
α5
αc
GSM8K
0.20
0.25
0.25
0.10
0.20
0.00
MATH-500
0.25
0.20
0.25
0.10
0.20
0.00
ARC-Challenge
0.20
0.00
0.00
0.15
0.20
0.45
TruthfulQA
0.20
0.00
0.00
0.15
0.20
0.45
Countdown
0.15
0.10
0.10
0.10
0.15
0.40
Sudoku
0.05
0.00
0.00
0.05
0.10
0.80
Appendix
Table 3: Weight profiles (α1,…,α5,αc) per dataset, corresponding to structural completeness, arithmetic consistency, and answer reachability, model confidence, non-degeneracy, and constraint satisfaction respectively. All rows sum to 1.0 .
Dataset
Method
Avg Eff. Tokens (↓)
Total Latency (s) (↓)
Wall Time (s) (↓)
ARC-C
Baseline
5.67
3648.21
1829.87
Best-of- K
5.43
28803.25
14431.86
Look-ahead only
6.08
29450.49
14756.30
Tilting only
5.43
28804.53
14431.20
S3
4.34
29100.42
14580.21
GSM8K
Baseline
3.03
3919.06
1964.63
Appendix
Table 4: Average effective output tokens ( ↓ ), total latency ( ↓ , seconds), and wall time ( ↓ , seconds) for each ablation variant and dataset ( N=4 , b=2 , K=8 , LLaDA-8B-Instruct, block_length =64).
Dataset
Block
Mode
Avg Tok. ↓
Latency (s) ↓
Wall (s) ↓
ARC-C
2
Base
16.92
7.42
6.10
S3
17.10
71262.09
35680.83
4
Base
8.92
7.43
6.12
S3
8.46
71038.65
35550.89
8
Base
8.04
7.51
6.18
S3
7.45
37840.20
18952.40
Appendix
Table 5: Average effective tokens ( ↓ ), total latency ( ↓ , s), and wall time ( ↓ , s) across all block sizes for LLaDA-8B-Instruct. Base : single-trajectory decoding; S3 : multi-block decoding with BFS search and SSP resampling.
Test-time scaling (TTS) improves language model outputs by spending additional inference compute - generating multiple candidates, searching over partial sequences, or iteratively refining drafts. These techniques yield large gains on mathematics and code, but have been developed and stress-tested almost exclusively on tasks where verification is straightforward. We conduct the first compute-normalised comparison of five TTS families across five open-ended generation benchmarks spanning medicine, law, finance, general chat, and creative writing - grounded in a unified framework that decomposes the effectiveness of each method's token budget into exploration and exploitation. The answer depends on which side of that decomposition you examine. Scaling exploration works: the best candidate in the pool improves steadily with compute across all settings. What breaks is exploitation - the step that converts a rich candidate pool into a final output. With state-of-the-art generators, reward models correlate at only ρv≈0.12 with true quality, rendering selection near-random regardless of budget. Tree search amplifies this failure through diversity collapse. Refinement helps on one of five benchmarks; its apparent gains elsewhere are confounded. Only synthesis across candidates (Fusion) consistently improves over single-sample baselines, yet still recovers only ~40% of available quality. The candidate pool is not the bottleneck - choosing from it is.
Davide Romano, Kanak Raj, Jerrod Parker +1
Thomson Reuters Foundational Research, 5 Canada Square, London
Test-Time Scaling (TTS) paradigm offers a promising perspective for enhancing the generation performance of diffusion models. However, current solutions largely restrict their search to predefined noise candidates or suffer from inflexible exploration across the denoising trajectory. To bridge this gap, we propose RTS, a novel Reward-guided Trajectory Scaling method to fully unlock the generative potential of diffusion models. Unlike existing methods, RTS facilitates the synthesis of refined, high-fidelity images via two core innovations: 1) a coarse-to-fine noise optimization mechanism that exploits historical search experience to actively steer the exploration toward high-reward regions and 2) a unified sparse test-time scaling framework featuring PCA-driven curvature analysis, which eliminates temporal redundancy by flexiblely allocating compute to a sparse set of key timesteps that represent critical shifts in the denoising direction. Extensive experiments across SD v3, FLUX, and Qwen-Image architectures demonstrate that RTS outperforms baselines, improving the GenEval score by 20.7%, 15.6%, and 12.2%, respectively. Notably, empirical findings indicate that these key points primarily cluster in the mid-stage of the trajectory, distinct from the structure-sensitive early phases and the late attribute refinement phases.
Gang Dai, Yining Huang, Yiming Xia +2
Guangdong University of Technology · South China University of Technology · Nanyang Technological University
Scaling inference-time computation has emerged as a reliable method to improve the performance of large language models on complex reasoning and programming tasks. However, standard approaches such as independent sampling and sequential multi-turn refinement operate without token-level credit assignment, resulting in computational inefficiency, since valid reasoning prefixes are frequently discarded. In this work, we introduce Test-Time Scaling via Error Localization (TTEL), an inference-time algorithm that utilizes fixed or environment feedback to perform token-level error localization. By comparing conditional probabilities under informed feedback against a null-context baseline, TTEL isolates the step at which an error occurred. The algorithm then truncates the trajectory and branches a new generation, maximally reusing the valid prefix. Extensive evaluations demonstrate that TTEL establishes strictly dominating Pareto frontiers across sequential reasoning domains, measured by pass-at-k vs. generated-token cost. With Qwen3-8B on LiveCodeBench, TTEL attains a pass@64 of 71.0% while generating approximately half as many tokens as independent sampling (360.4k vs. 735.0k). Generalizing to math benchmarks AIME-2025 and HMMT-2025, TTEL cleanly outperforms competing test-time baselines across both Qwen3-8B and Qwen3-4B-Thinking-2507.