A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Models
Authors: Theo X. Olausson, Metod Jazbec, Xi Wang, Armando Solar-Lezama, Christian A. Naesseth, Stephan Mandt, Eric Nalisnick
Organizations: Massachusetts Institute of Technology · UvA-Bosch Delta Lab, University of Amsterdam · Johns Hopkins University · University of California, Irvine
Much work has been done on designing fast and accurate sampling for diffusion language models (dLLMs). However, these efforts have largely focused on the tradeoff between speed and quality of individual samples; how to additionally ensure diversity across samples remains less well understood. In this work, we show that diversity can be increased by using softened, tempered versions of familiar confidence-based remasking heuristics, retaining their computational benefits and offering simple implementations. We motivate this approach by introducing an idealized formal model of fork tokens and studying the impact of remasking on the expected entropy at the forks. Empirically, the proposed tempered heuristics close the exploration gap (pass@k) between existing confidence-based and autoregressive sampling, hence outperforming both when controlling for cost (pass@NFE). We further study how the increase in diversity translates to downstream post-training and test-time compute scaling. Overall, our findings demonstrate that simple, efficient, and diverse sampling from dLLMs is possible.
Figures & tables
Figure 1: An example illustrating our model of entropy degradation. The fork token ( Definition 1 ) has high marginal uncertainty under the dLLM, indicating that both the numpy and statistics solutions are plausible. However, the dLLM is simultaneously overly confident in its prediction at the anchor tokens in the function body, biasing low-confidence remasking towards one solution as it reveals those tokens first. Tempered remasking allows the fork to still be revealed first, increasing semantic diversity without sacrificing parallelism. Note that while both paths lead to generations that match the prompt ( ), they have subtly different semantics ( ); sampling both is important to achieve good coverage.
Figure 2: Pass@ k for TLC ( Tpos=1 ) on LLaDA-8B-Instruct and Dream-7B-Instruct across four benchmarks. TLC closely tracks the AR baseline, recovering almost all of the diversity lost by low-confidence remasking.
Figure 3: Pass@ k (top) and pass@NFE (bottom) for TCT on LLaDA-8B-Instruct. Both rows plot the same data; only the metric changes (pass@ k vs pass@NFE). TCT slightly underperforms AR in pass@ k , but substantially outperforms it when the cost of each sample is taken into account. TCT also outperforms deterministic confidence-thresholding (Fast-dLLM; Wu et al. 2026a ) thanks to its additional source of randomness via Tpos .
Figure 4: Test-time scaling for LLaDA-8B-Instruct in terms of Best@NFE when using either self-consistency or an ORM for final answer selection. TCT performs worse compared to AR at higher NFEs when using self-consistency; however, when using an ORM, TCT improves the Pareto frontier compared to autoregressive rollouts. See Figure 11 for Best@ k results.
Figure 5: GRPO post-training results for LLaDA-8B-Instruct on GSM8K (top) and MATH (bottom). Left : Group mean training reward. Middle : Group standard deviation of the reward. Right : Evaluation accuracy under greedy decoding with confidence thresholding at varying λ . All models are trained for a fixed wall-clock budget of 72 hours; vertical dashed lines indicate the number of completed training steps for each sampler. TCT completes around twice as many training steps compared to AR due to its adaptive parallelism, maintains the highest reward diversity throughout training due to Tpos>0 , and yields evaluation performance comparable to or better than both AR and Fast-dLLM baselines.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6Figure 7
Figure 6: Empirical validation of Assumption 1 . Left : Entropy degradation at potential fork tokens as a function of how many potential anchors have been revealed, aggregated over all forks and samples. Middle : Slopes of per-fork linear fits. Right : R2 correlation between each fork’s entropy degradation and a linear fit on that fork’s data. In all cases, we see that forks tend to decrease in entropy as their anchors are revealed. While different forks vary in how much their entropies change (causing large error bars and lower R2 on the left), most exhibit an internally nearly-linear relationship (the median per-fork R2 being ∼0.7 across datasets in the rightmost plot), and almost all decrease in entropy (middle).
Figure 7: Number of semantically distinct and well-formed outcomes when setting a fork position from Figure 6 to each of its top 5 most likely values. Overall, a majority of the fork positions produce ≥2 valid and semantically distinct outcomes, indicating that they play an important role in determining the semantics of the outcome (rather than just syntax).
Figure 8: TLC Tpos ablation on HumanEval with LLaDA-8B-Instruct at Ttoken=0.8 . Too little tempering ( Tpos=0.1 ) remains close to low-confidence remasking, while too much ( Tpos=2 ) approaches random remasking and degrades quality. Intermediate values ( Tpos∈{0.5,1} ) strike the best balance, matching or exceeding the autoregressive baseline at high k .
Figure 9: Varying Ttoken for LLaDA-8B-Instruct on HumanEval. Autoregressive generation favors low temperatures, obtaining its best pass@64 at Ttoken=0.6 . Meanwhile, low-confidence remasking scales very poorly at low token temperatures due to its greedy position selection. TLC consistently shows strong scaling and most robustness to the exact choice of Ttoken , matching or exceeding the best of autoregressive and low-confidence remasking in pass@64 at all temperatures. Moreover, it exhibits stronger scaling compared to low-confidence across all considered Ttoken , suggesting it achieves diversity through an alternative mechanism.
Figure 10: Pass@ k (top) and pass@NFE (bottom) for TCT on Dream-7B-Instruct. Note that both rows plot the same data, we just vary the x-axis measure ( k vs NFEs). While TCT slightly underperforms AR in pass@ k , it outperforms AR when the cost of each rollout is taken into account.
Figure 11: Test-time scaling for LLaDA-8B-Instruct in terms of Best@ k (top) and Best@NFE (bottom) when using either self-consistency or an ORM for final answer selection. TCT performs worse compared to AR at higher NFEs when using self-consistency; however, when using an ORM, TCT improves the Pareto frontier compared to autoregressive rollouts.
Figure 12: Mean empirical entropy of the per-question answer distributions (see Figure 13 ) as a function of the group size k . TCT consistently yields higher-entropy answer distributions than autoregressive and confidence thresholding, particularly on GSM8K.
Figure 13: Per-question answer distributions over k=64 samples for three GSM8K questions, comparing autoregressive, Fast-dLLM, and TCT sampling. Correct answers are marked with an asterisk (*). We observe that TCT sometimes spreads mass across a wider set of candidates (left). This illustrates the higher-entropy behavior observed in Figure 12 : TCT produces more diverse answers, which benefits pass@ k and ORM-based selection but can hurt majority voting when the correct answer no longer dominates the distribution.
Figure 14: Pass@ k for TLC ( Tpos=1 ) on MBPP across dLLMs with different amounts of diffusion training. LLaDA-8B was trained as a diffusion model from scratch on approximately 2.3T tokens, whereas Dream-7B, LLaDA-2-mini-16B, and SDAR-1.7B were initialized from autoregressive models and subsequently diffusion-trained on approximately 580B, fewer than 200B, and 50B tokens, respectively. The LLaDA-8B and Dream-7B panels reproduce the MBPP results from Figure 2 . TLC closely tracks autoregressive sampling on the more extensively diffusion-trained LLaDA-8B and Dream-7B models, but leaves a larger gap on LLaDA-2-mini-16B and especially SDAR-1.7B.
Figure 15: Pass@ k and pass@NFE for TCT on LLaDA-2-mini ( Bie et al., 2025 ) . In addition to pass@ k ( Left ) and pass@NFE ( Middle ), we here additionally report pass@wall-clock-time ( Right ) to show that our proposed TCT has a negligible overhead in terms of latency compared to Fast-dLLM.
k
Dataset
Method
1
2
4
8
16
32
64
LLaDA-8B-Instruct
GSM8K
Autoregressive
75.9 ± 3.1
86.9 ± 2.7
92.7 ± 2.1
96.0 ± 1.7
97.8 ± 1.4
98.6 ± 1.2
99.0 ± 1.2
Low Confidence ( Tpos=0 )
78.8 ± 3.2
87.3 ± 2.9
91.9 ± 2.5
94.6 ± 2.1
96.3 ± 1.9
97.2 ± 1.7
98.0 ± 1.5
Random ( Tpos=∞ )
55.5 ± 2.9
73.8 ± 3.1
85.9 ± 2.8
92.4 ± 2.2
95.9 ± 1.7
97.9 ± 1.3
99.3 ± 0.8
TLC ( Tpos=1 )
72.7 ± 3.0
85.7 ± 2.7
92.3 ± 2.2
95.8 ± 1.8
97.5 ± 1.5
98.4 ± 1.2
99.0 ± 1.2
Appendix
Table 1: Pass@ k (%) with bootstrapped 95% confidence intervals (10,000 resamples over questions/tasks, 64 samples per question, Ttoken=0.8 ). Values are mean ± half-width of the 95% CI. Best value per column within each dataset block in bold. Evaluation sets: GSM8K 300 questions, MATH 500, HumanEval 164 tasks, MBPP 500.
k
Dataset
Method
NFE
1
2
4
8
16
32
64
96
GSM8K
Autoregressive
228
75.8 ± 3.1
86.7 ± 2.7
92.6 ± 2.1
96.0 ± 1.6
97.8 ± 1.3
98.8 ± 1.0
–
–
Fast-dLLM
46
73.9 ± 3.1
85.5 ± 2.9
91.2 ± 2.5
94.1 ± 2.2
96.0 ± 1.9
97.1 ± 1.7
97.5 ± 1.7
97.7 ± 1.7
TCT ( Tpos=0.1 )
44
65.7 ± 2.9
81.7 ± 2.9
90.2 ± 2.4
94.5 ± 2.0
96.7 ± 1.7
97.8 ± 1.5
98.2 ± 1.5
98.3 ± 1.5
MATH
Autoregressive
243
29.6 ± 3.0
39.2 ± 3.4
48.1 ± 3.6
56.3 ± 3.7
64.0 ± 3.6
70.8 ± 3.5
–
–
Fast-dLLM
61
30.3 ± 3.1
39.0 ± 3.6
46.4 ± 3.8
53.4 ± 3.8
60.1 ± 3.8
66.0 ± 3.8
70.6 ± 3.9
72.6 ± 3.9
Appendix
Table 2: Pass@ k (%) for LLaDA-8B-Instruct with bootstrapped 95% confidence intervals (10,000 resamples over questions/tasks, Ttoken=0.8 , λ=0.6 ). Values are mean ± half-width of the 95% CI. Evaluation sets: GSM8K 300 questions, MATH 500, HumanEval 164 tasks, MBPP 500. NFE is the average number of function evaluations per sample; for Autoregressive it is the effective generation length (tokens up to EOS), matching the pass@NFE axis of the figures.
k
Dataset
Method
NFE
1
2
4
8
16
32
64
96
GSM8K
Autoregressive
200
64.6 ± 3.3
79.1 ± 2.8
89.4 ± 2.2
95.2 ± 1.6
97.8 ± 1.1
99.0 ± 0.8
–
–
Fast-dLLM
55
69.0 ± 3.0
83.3 ± 2.8
90.9 ± 2.4
94.5 ± 2.1
96.3 ± 1.8
97.3 ± 1.7
97.8 ± 1.6
98.0 ± 1.5
TCT ( Tpos=0.1 )
51
58.2 ± 2.7
77.1 ± 2.7
89.2 ± 2.2
95.0 ± 1.7
97.6 ± 1.3
98.6 ± 1.1
99.1 ± 1.0
99.3 ± 0.8
MATH
Autoregressive
223
34.3 ± 2.9
45.9 ± 3.4
56.0 ± 3.6
64.7 ± 3.6
71.8 ± 3.5
77.3 ± 3.4
–
–
Fast-dLLM
66
36.4 ± 3.1
47.6 ± 3.5
57.2 ± 3.6
65.3 ± 3.6
71.9 ± 3.5
77.1 ± 3.4
80.7 ± 3.3
82.2 ± 3.3
Appendix
Table 3: Pass@ k (%) for Dream-v0-Instruct-7B with bootstrapped 95% confidence intervals (10,000 resamples over questions/tasks, Ttoken=0.8 , λ=0.6 ). Values are mean ± half-width of the 95% CI. Fast-dLLM and TCT use 96 samples per question; Autoregressive uses 64–128 (dataset-dependent) and is reported to k≤32 . Evaluation sets: GSM8K 300 questions, MATH 500, HumanEval 164 tasks, MBPP 500. NFE is the average number of function evaluations per sample; for Autoregressive it is the effective generation length (tokens up to EOS), matching the pass@NFE axis of the figures.
Masked diffusion language models (dLLMs) have shown strong potential for faster inference through parallel token generation when combined with confidence-based samplers. However, recent work has shown that such methods can defer unmasking high-entropy fork positions at which multiple plausible continuations exist. This results in reduced generation diversity, as shown by worse pass@k scaling, and limits gains obtainable from RL post-training. To avoid this flexibility trap, prior work advocated for autoregressive (AR) sampling. Here, we show that discarding confidence-based sampling is unnecessary and, once inference cost is taken into account, wasteful. We first propose Fork-dLLM, a simple hybrid sampler that uses AR-style ordering only at uncertain fallback steps while retaining parallel generation otherwise. We then extend the same principle to post-training with ForkGRPO, which uses Fork-dLLM rollouts and applies the GRPO objective only at fallback steps, preserving exact policy-likelihood ratios while substantially reducing rollout and optimization cost. In our experiments, Fork-dLLM matches the strong pass@k scaling of AR sampling while being 2-3x more efficient, and ForkGRPO achieves downstream performance comparable to or better than AR-based GRPO baselines at a substantially lower training cost.
Stipe Frković, Metod Jazbec, Christian A. Naesseth
University of Amsterdam · UvA-Bosch Delta Lab, University of Amsterdam
Diffusion large language models (dLLMs) theoretically permit token decoding in arbitrary order, a flexibility that could enable richer exploration of reasoning paths than autoregressive (AR) LLMs. In practice, however, random-order decoding often hurts generation quality. To mitigate this, low-confidence remasking improves single-sample quality (e.g., Pass@1) by prioritizing confident tokens, but it also suppresses exploration and limits multi-sample gains (e.g., Pass@k), creating a fundamental quality--exploration dilemma. In this paper, we provide a unified explanation of this dilemma. We show that low-confidence remasking improves a myopic proxy for quality while provably constraining the entropy of the induced sequence distribution. To overcome this limitation, we characterize the optimal distribution that explicitly balances quality and exploration, and develop a simple Independent Metropolis--Hastings sampler that approximately targets this distribution during decoding. Experiments across a range of reasoning benchmarks including MATH500, AIME24/25, HumanEval, and MBPP show that our approach yields better exploration-quality tradeoff than both random and low-confidence remasking.
Liancheng Fang, Aiwei Liu, Henry Peng Zou +7
University of Illinois Chicago · Tsinghua University · MBZUAI +1
Diverse outputs in text generation are necessary for effective exploration in complex reasoning tasks, such as code generation and mathematical problem solving. Such Pass@k problems benefit from distinct candidates covering the solution space. However, traditional sampling approaches often waste computational resources on repetitive failure modes. While Diffusion Language Models have emerged as a competitive alternative to the prevailing Autoregressive paradigm, they remain susceptible to this redundancy, with independent samples frequently collapsing into similar modes. To address this, we propose a training free, low cost intervention to enhance generative diversity in Diffusion Language Models. Our approach modifies intermediate samples in a batch sequentially, where each sample is repelled from the feature space of previous samples, actively penalising redundancy. Unlike prior methods that require retraining or beam search, our strategy incurs negligible computational overhead, while ensuring that each sample contributes a unique perspective to the batch. We evaluate our method on the HumanEval and GSM8K benchmarks using the LLaDA-8B-Instruct model. Our results demonstrate significantly improved diversity and Pass@k performance across various temperature settings. As a simple modification to the sampling process, our method offers an immediate, low-cost improvement for current and future Diffusion Language Models in tasks that benefit from diverse solution search. We make our code available at https://github.com/sean-lamont/odd.
Sean Lamont, Christian Walder, Paul Montague +2
Australian National University · Defence Science and Technology Group · Google DeepMind +1