Parallel decoding speeds up masked diffusion language models by writing several tokens per forward pass. Published commit rules choose these tokens by testing the predicted distributions. Under sampling the written tokens are drawn, and these tests do not see them. We find that answer quality follows the reliability of the tokens actually written. An answer's log probability splits into this reliability and the dependence within each step. We propose PACT, a training-free commit rule that draws the tokens first, certifies the set it will write and spaces the positions of each step apart. This certificate holds at every temperature, and we show that no rule with the same guarantee commits more positions. Experiments use four models of the LLaDA and Dream families and HumanEval, MBPP, GSM8K, MATH500 and IFEval benchmarks. PACT reaches the quality of published commit rules with fewer forward passes. Spacing removes positions from each step, yet it usually lowers the total cost because later steps then write more tokens. At the same speed, published decoders also write a word twice in a row far more often than PACT. These results suggest that commit rules should certify the tokens they write. Code is available at https://github.com/LiZaiyuan0619/PACT.
Figures & tables
Figure 1: A test that cannot see the draw writes the same word twice. Top, two answers of LLaDA-8B to HumanEval/48 at τ=0.1 . Bottom, the share of answers with one word written twice in a row at matched forward passes, with bars at the median over models (Appendix C.6 ).
Figure 2: Quality follows the reliability of what is written, and tests on the distributions misjudge it. (a) Pass@1 of LLaDA-8B against three axes. (b) Error of predicting pass@1 from each axis, holding out one rule family at a time. (c) Stand-in rules, built to hold another quantity fixed. (d) Steps where the reported reliability reaches τ and the actual reliability does not, under four orders with one shared test. (e) The other three orders against ours. Appendix C.6 defines each quantity.
Figure 3: Speed and quality of every decoder. (a) Speed-up of PACT over each published rule at matched pass@1 on code and math. (b) The sweeps behind a. (c) PACT against fixed k at the same forward passes, with dark bars significantly above one. Appendix C.6 gives the matched level, the tasks, the band of c and the test.
Figure 4: Decoding time per problem. Multiples are speed-ups over one token per step (Appendix C.4 ).
Figure 5: Ablations of PACT. (a) Forward passes on code to reach the pass@1 printed in each panel, as the parts of PACT are removed in turn. (b) Committed sets on code that report more than they write. (c) Forward passes of each problem with and without spacing, on the models of a. (d) The saving of spacing on code by minimum gap.
Figure 6: Effects of spacing and of the sampling temperature. (a) Dependence term and step size against the minimum gap. (b) Tokens written per forward pass along the answer. (c) Accuracy and forward passes against the sampling temperature, for three rules that share the set-level cut. Appendix C.6 gives the settings.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Checkpoint
Parameters
License
LLaDA-8B-Instruct
8B
MIT
LLaDA-MoE-7B-A1B-Instruct
7B
Apache-2.0
LLaDA2.1-mini
16B
Apache-2.0
Dream-v0-Instruct-7B
7B
Apache-2.0
Qwen3-8B
8B
Apache-2.0
Appendix
Table 1: Models and benchmarks. Checkpoints are the releases on Hugging Face. Every method decodes the same problems of each benchmark.
Rule
Parameter
Table
Figure 3
Fast-dLLM
threshold
0.9
0.1 to 0.95
Fast-dLLM++
δ
0
0 to 1
EB-Sampler
γ
0.5
0.5 to 24
KLASS
confidence
0.9
0.3 to 0.9
TACG
threshold
0.9
0.5 to 0.95
Fixed k
k
4
4 to 16
Appendix
Table 2: Settings of each rule. KLASS keeps its KL threshold at 0.001. PACT uses g=8 , without spacing on LLaDA2.1-mini, and Figure 3 also sweeps it without spacing.
Figure 7: Forward passes per problem under two orders. Each point is a HumanEval problem at one τ , under the top-choice order and under our order with the same set-level cut. Each panel gives the share of problems in the shaded triangle, where the top-choice order costs more.
Figure 8: Decoding time against forward passes. Each point is one setting on the code benchmarks. The dashed line is fitted through the origin on the settings of the other decoders, and its slope is printed as the time of one forward pass.
Figure 9: Speed-up in time at the accuracy of one token per step. Top, the best pass@1 of PACT minus that of one token per step. Bottom, the speed-up at the fastest setting of PACT that reaches the pass@1 of one token per step. A light bar is a lower bound, since the fastest setting already reaches it. A dashed outline marks a benchmark where no setting reaches it.
Figure 10: Sampling temperature on GSM8K. Pass@1 as a share of greedy pass@1 (top) and forward passes as a multiple of greedy (bottom), for the three rules of Figure 6 c on three models.
Diffusion language model decoding generates text through iterative parallel refinement, balancing fewer denoising steps with accuracy, coherence, and control.
Diffusion language models generate multiple token positions in parallel, but few-step decoding makes parallel commitment difficult: tokens that appear reliable individually may be jointly inconsistent, while premature commitments can prevent later context-dependent refinement. In this paper, we introduce SetCommit, a training-free method for coordinated parallel token commitment. Instead of selecting tokens independently by confidence, SetCommit selects a commitment set that balances token reliability against local joint commitment risk, preserving conditioning opportunities for subsequent denoising steps. It further performs budget-neutral local exchanges, replacing vulnerable earlier commitments when newly revealed context supports more reliable alternatives. SetCommit requires no retraining or architectural modification and is compatible with existing diffusion decoding strategies. Experiments with LLaDA-8B and Dream-7B on GSM8K, MATH-500, MBPP, and HumanEval show accuracy/pass@1 gains of up to 7.4 percentage points over native decoding at 32 model evaluations. At common quality targets, geometric-mean incremental speedups range from 1.44 to 1.74×. Code is available at https://github.com/Kissmetothemoon/SetCommit.
Yuannuo Feng, Yubing Ye, Wenshuai Yao +4
Beihang University · The University of Hong Kong · Peking University
Masked diffusion language models (MDLMs) can generate text efficiently by predicting multiple masked tokens in parallel, but predictions from the same forward pass are not necessarily reliable when committed together. We study when parallel commitment is reliable. Our diagnostics show that confidence alone does not determine a reliable commitment order: confident predictions near the end of the sequence can fix an answer before its supporting computations are established, and downstream predictions become less reliable as the uncertainty of their upstream context grows. At the same time, a single forward pass can already resolve several masked tokens, and predictions that remain stable across the final layers are more likely to be correct. Based on these findings, we propose Reliable Parallel Decoding (RPD), a training-free method that selects candidates by layerwise prediction stability and final confidence, and commits them under a cumulative entropy budget over their preceding masked positions. RPD defers predictions with uncertain upstream context while committing the remaining candidates in parallel, without relying on a fixed block schedule. Across mathematical reasoning and code generation benchmarks on LLaDA and Dream, RPD achieves the highest decoding throughput among the evaluated methods while maintaining or improving accuracy.
Zhenghao He, Bohan Liu, Guangzhi Xiong +1
Department of Computer Science, University of Virginia
Discrete diffusion language models enable parallel token generation, offering a pathway to low-latency decoding. However, selecting tokens independently by marginal confidence limits effective parallelism: tokens that appear reliable in isolation can form incompatible configurations when several positions are updated at once. We introduce a training-free decoding framework that coordinates these parallel updates. At each forward pass, the method assigns a commit score to each masked position and refines these scores using pairwise interactions derived from the model's predictive distributions. A variational relaxation yields a simple fixed-point update that suppresses conflicting simultaneous commitments within a single forward pass. This mechanism allows the decoder to commit more tokens in parallel while maintaining competitive generation quality. The method is lightweight, requires no auxiliary model or retraining, and drops into existing diffusion decoding pipelines without modification. Experiments on reasoning and code-generation benchmarks show consistent improvements in the quality-latency trade-off.
Tamim Zoabi, Ameen Ali, Liran Ringel +1
School of Electrical & Computer Engineering Tel Aviv University, Israel · School of Computer Science and AI Tel Aviv University, Israel · Department of Computer Science Technion, Israel Institute of Technology, Israel