Locally Confident, Globally Stuck: The Quality-Exploration Dilemma in Diffusion Language Models

Liancheng Fang; Aiwei Liu; Henry Peng Zou; Yankai Chen; Enze Ma; Leyi Pan; Chunyu Miao; Wei-Chieh Huang

cs.CLApr 1, 2026

Locally Confident, Globally Stuck: The Quality-Exploration Dilemma in Diffusion Language Models

Authors: Liancheng Fang, Aiwei Liu, Henry Peng Zou, Yankai Chen, Enze Ma, Leyi Pan, Chunyu Miao, Wei-Chieh Huang,

Organizations: University of Illinois Chicago · Tsinghua University · MBZUAI · McGill University

Abstract

Diffusion large language models (dLLMs) theoretically permit token decoding in arbitrary order, a flexibility that could enable richer exploration of reasoning paths than autoregressive (AR) LLMs. In practice, however, random-order decoding often hurts generation quality. To mitigate this, low-confidence remasking improves single-sample quality (e.g., Pass@ $1$ ) by prioritizing confident tokens, but it also suppresses exploration and limits multi-sample gains (e.g., Pass@ $k$ ), creating a fundamental quality--exploration dilemma. In this paper, we provide a unified explanation of this dilemma. We show that low-confidence remasking improves a myopic proxy for quality while provably constraining the entropy of the induced sequence distribution. To overcome this limitation, we characterize the optimal distribution that explicitly balances quality and exploration, and develop a simple Independent Metropolis--Hastings sampler that approximately targets this distribution during decoding. Experiments across a range of reasoning benchmarks including MATH500, AIME24/25, HumanEval, and MBPP show that our approach yields better exploration-quality tradeoff than both random and low-confidence remasking.

Figures & tables

Figure 1: The quality–exploration dilemma in dLLM decoding. Confidence remasking achieves high sample quality (Pass@

1

) but plateaus Pass@

k

quickly due to limited exploration. Conversely, random remasking promotes exploration but degrades individual sample quality. Our global tempering reconciles this trade-off, establishing a new Pareto frontier with superior Pass@

1

and Pass@

16

performance.

Heuristic	Commit criterion	Mode	$\delta$ -gating
Confidence ( Nie et al., 2025 ; Ye et al., 2025 )	$\max_{v}\,p_{i}(v)\geq 1-\delta$	Sample-then-Filter	$\delta$ (directly)
Entropy ( Ben-Hamu et al., 2025 ; Fu et al., 2025 )	$H(p_{i})\leq\varepsilon$	Rank-then-Sample	$\delta=\delta_{V}(\varepsilon)$
Margin ( Kim et al., 2025a ; Hong et al., 2025b )	$p_{i}(v_{1})-p_{i}(v_{2})\geq\gamma$	Rank-then-Sample	$\delta=\frac{(\|V\|-1)(1-\gamma)}{\|V\|}$

Table 1: Taxonomy of uncertainty heuristics for uncertainty-based remasking. We categorize heuristics by their distributional scoring functions (

p_{i}(\cdot)\coloneqq p(\cdot\mid s_{t},i)

) and operational paradigms ( Sample-then-Filter vs. Rank-then-Sample ). The

\delta

-gating column establishes the equivalent confidence lower bound

\max_{v}p_{i}(v)\geq 1-\delta

for each criterion. See Appendix C for detailed definitions and formal derivations of these bounds.

		Math reasoning			Code generation
Method	Metric	MATH500	AIME’24	AIME’25	HumanEval	HumanEval+	MBPP	MBPP+
WeDLM-8B
Entropy †	Pass@1	0.528	0.093	0.071	0.732	0.684	0.784	0.677
	Pass@k	0.875	0.310	0.291	0.950	0.911	0.916	0.739
Confidence	Pass@1	0.528	0.079	0.074	0.725	0.678	0.707	0.598
	Pass@k	0.851	0.323	0.292	0.905	0.857	0.834	0.717

Table 2: Pass@

\bm{1}

and Pass@

\bm{k}

across benchmarks and models. Bold indicates the best result for a given metric. LLaDA-8B results on AIME24/25 are omitted as they are close to 0.

Figure 2: Pass@

\bm{k}

scaling curves. (Left) Performance on MATH500, HumanEval, and MBPP for WeDLM-8B (top) and LLaDA-8B (bottom). (Right) WeDLM-8B on AIME’24/25.

Panel of Figure 3 — Figure 3: Left: Quality-diversity trade-off on MATH500 (LLaDA) by sweeping temperature parameters. IMH strictly dominates local baselines. Middle: Pass@ $32$ on AIME 2024 stratified by difficulty ( Sun et al., 2025 ) (WeDLM-8B). IMH yields the largest gains on Hard problems. Right: Trajectory similarity matrix on AIME 2024. Local baselines collapse into similar paths, whereas IMH explores distinctly different reasoning strategies.

$T$	Pass@1	Pass@8	Acc.(%)
1	9.5	23.1	99.3
3	10.0	23.3	99.1
7	11.4	24.0	98.3
15	11.0	23.4	97.5
31	11.0	24.5	97.2
Baseline	9.4	21.4	—

Table 3: MCMC dynamics on AIME 2024 (WeDLM-8B). Baseline denotes entropy remasking without IMH.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Maximum Generation Length
MATH500	128
AIME 2024	512
AIME 2025	512
HumanEval	256
MBPP	256

Appendix

Table 4: Maximum generation lengths for LLaDA across different benchmarks.

Benchmark	Size ( $M$ )	WeDLM ( $n=128,k=32$ )	LLaDA ( $n=32,k=16$ )
MBPP	974	$\pm 1.4\%$	$\pm 2.0\%$
MATH500	500	$\pm 1.9\%$	$\pm 2.9\%$
HumanEval	164	$\pm 3.4\%$	$\pm 5.0\%$
AIME 2024 / 2025	30	$\pm 7.9\%$	-

Appendix

Table 5: Worst-case 95% margin of error for Pass@

k

estimation. The theoretical maximum noise introduced by the finite-sample estimator for each benchmark and model setting.

Diffusion Language Model Decoding

Diffusion language model decoding generates text through iterative parallel refinement, balancing fewer denoising steps with accuracy, coherence, and control.

See all 235 papers

A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Models
Apr 10, 2026Theo X. Olausson, Metod Jazbec, Xi Wang +4Diffusion Language Models Diversity-Aware Sampling
Massachusetts Institute of Technology · UvA-Bosch Delta Lab, University of Amsterdam · Johns Hopkins University +1
Answer First, Reason Later: Commitment Order in Diffusion LLMs
Aug 6, 2026Jewon Yeom, Jaewon Sok, Seonghyeon Park +3CoT Reasoning Diffusion Language Model Decoding
Graduate School of Data Science, Seoul National University · Department of Rural Systems Engineering, Seoul National University · Department of Aerospace Engineering, Seoul National University
Fork-dLLM: Avoiding the Flexibility Trap in Diffusion Language Models
Sep 30, 2026Stipe Frković, Metod Jazbec, Christian A. NaessethMasked Diffusion Models Speculative Decoding for Diffusion Language Models
University of Amsterdam · UvA-Bosch Delta Lab, University of Amsterdam

Apr 10, 2026J/K move · Enter open · S save

A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Models

Theo X. Olausson, Metod Jazbec, Xi Wang, Armando Solar-Lezama+3

Massachusetts Institute of Technology · UvA-Bosch Delta Lab, University of Amsterdam · Johns Hopkins University+1

Diffusion Language Models Diversity-Aware Sampling

Open paper AskPDF arXiv

Much work has been done on designing fast and accurate sampling for diffusion language models (dLLMs). However, these efforts have largely focused on the tradeoff between speed and quality of individual samples; how to additionally ensure diversity across samples remains less well understood. In this work, we show that diversity can be increased by using softened, tempered versions of familiar confidence-based remasking heuristics, retaining their computational benefits and offering simple implementations. We motivate this approach by introducing an idealized formal model of fork tokens and studying the impact of remasking on the expected entropy at the forks. Empirically, the proposed tempered heuristics close the exploration gap (pass@k) between existing confidence-based and autoregressive sampling, hence outperforming both when controlling for cost (pass@NFE). We further study how the increase in diversity translates to downstream post-training and test-time compute scaling. Overall, our findings demonstrate that simple, efficient, and diverse sampling from dLLMs is possible.

Abstract

Diffusion Language Model Decoding

Explore similar work

A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Models

Answer First, Reason Later: Commitment Order in Diffusion LLMs

Fork-dLLM: Avoiding the Flexibility Trap in Diffusion Language Models