cs.CLNov 3, 2026

PACT: Certifying What Is Written in Parallel Decoding of Diffusion Language Models

Authors: Yanchao Li, Yusu Zhang, Jiaqing Xie, Yi Ban, Ben Gao, Zhongyu Huang, Tianfan Fu, Yuqiang Li

Organizations: Nanjing University · Shanghai Artificial Intelligence Laboratory

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Parallel decoding speeds up masked diffusion language models by writing several tokens per forward pass. Published commit rules choose these tokens by testing the predicted distributions. Under sampling the written tokens are drawn, and these tests do not see them. We find that answer quality follows the reliability of the tokens actually written. An answer's log probability splits into this reliability and the dependence within each step. We propose PACT, a training-free commit rule that draws the tokens first, certifies the set it will write and spaces the positions of each step apart. This certificate holds at every temperature, and we show that no rule with the same guarantee commits more positions. Experiments use four models of the LLaDA and Dream families and HumanEval, MBPP, GSM8K, MATH500 and IFEval benchmarks. PACT reaches the quality of published commit rules with fewer forward passes. Spacing removes positions from each step, yet it usually lowers the total cost because later steps then write more tokens. At the same speed, published decoders also write a word twice in a row far more often than PACT. These results suggest that commit rules should certify the tokens they write. Code is available at https://github.com/LiZaiyuan0619/PACT.

Figures & tables

Figure 1: A test that cannot see the draw writes the same word twice. Top, two answers of LLaDA-8B to HumanEval/48 at τ=0.1\tau=0.1 . Bottom, the share of answers with one word written twice in a row at matched forward passes, with bars at the median over models (Appendix C.6 ).
Figure 2: Quality follows the reliability of what is written, and tests on the distributions misjudge it. (a) Pass@1 of LLaDA-8B against three axes. (b) Error of predicting pass@1 from each axis, holding out one rule family at a time. (c) Stand-in rules, built to hold another quantity fixed. (d) Steps where the reported reliability reaches τ\tau and the actual reliability does not, under four orders with one shared test. (e) The other three orders against ours. Appendix C.6 defines each quantity.
Figure 3: Speed and quality of every decoder. (a) Speed-up of PACT over each published rule at matched pass@1 on code and math. (b) The sweeps behind a. (c) PACT against fixed k at the same forward passes, with dark bars significantly above one. Appendix C.6 gives the matched level, the tasks, the band of c and the test.
Figure 4: Decoding time per problem. Multiples are speed-ups over one token per step (Appendix C.4 ).
Figure 5: Ablations of PACT. (a) Forward passes on code to reach the pass@1 printed in each panel, as the parts of PACT are removed in turn. (b) Committed sets on code that report more than they write. (c) Forward passes of each problem with and without spacing, on the models of a. (d) The saving of spacing on code by minimum gap.
Figure 6: Effects of spacing and of the sampling temperature. (a) Dependence term and step size against the minimum gap. (b) Tokens written per forward pass along the answer. (c) Accuracy and forward passes against the sampling temperature, for three rules that share the set-level cut. Appendix C.6 gives the settings.
Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

CheckpointParametersLicense
LLaDA-8B-Instruct8BMIT
LLaDA-MoE-7B-A1B-Instruct7BApache-2.0
LLaDA2.1-mini16BApache-2.0
Dream-v0-Instruct-7B7BApache-2.0
Qwen3-8B8BApache-2.0
Appendix
Table 1: Models and benchmarks. Checkpoints are the releases on Hugging Face. Every method decodes the same problems of each benchmark.
RuleParameterTableFigure 3
Fast-dLLMthreshold0.90.1 to 0.95
Fast-dLLM++δ\delta00 to 1
EB-Samplerγ\gamma0.50.5 to 24
KLASSconfidence0.90.3 to 0.9
TACGthreshold0.90.5 to 0.95
Fixed kkkk44 to 16
Appendix
Table 2: Settings of each rule. KLASS keeps its KL threshold at 0.001. PACT uses g=8g=8 , without spacing on LLaDA2.1-mini, and Figure 3 also sweeps it without spacing.
Figure 7: Forward passes per problem under two orders. Each point is a HumanEval problem at one τ\tau , under the top-choice order and under our order with the same set-level cut. Each panel gives the share of problems in the shaded triangle, where the top-choice order costs more.
Figure 8: Decoding time against forward passes. Each point is one setting on the code benchmarks. The dashed line is fitted through the origin on the settings of the other decoders, and its slope is printed as the time of one forward pass.
Figure 9: Speed-up in time at the accuracy of one token per step. Top, the best pass@1 of PACT minus that of one token per step. Bottom, the speed-up at the fastest setting of PACT that reaches the pass@1 of one token per step. A light bar is a lower bound, since the fastest setting already reaches it. A dashed outline marks a benchmark where no setting reaches it.
Figure 10: Sampling temperature on GSM8K. Pass@1 as a share of greedy pass@1 (top) and forward passes as a multiple of greedy (bottom), for the three rules of Figure 6 c on three models.
More in this topic

Diffusion Language Model Decoding

Diffusion language model decoding generates text through iterative parallel refinement, balancing fewer denoising steps with accuracy, coherence, and control.

See all 255 papers

Explore similar work

CardsList

  1. SetCommit: Coordinated Token Commitments for Few-Step Diffusion Language Model Decoding

    Oct 22, 2026Yuannuo Feng, Yubing Ye, Wenshuai Yao +4Diffusion Language Model InferenceDiffusion Language Models

    Beihang University · The University of Hong Kong · Peking University

  2. Reliable Parallel Decoding in Masked Diffusion Language Models

    Sep 29, 2026Zhenghao He, Bohan Liu, Guangzhi Xiong +1Masked Diffusion ModelsDiffusion Language Model Decoding

    Department of Computer Science, University of Virginia

  3. Mean-Field Parallel Decoding for Discrete Diffusion Language Models

    Jun 14, 2026Tamim Zoabi, Ameen Ali, Liran Ringel +1Parallel DecodingDiffusion Language Model Decoding

    School of Electrical & Computer Engineering Tel Aviv University, Israel · School of Computer Science and AI Tel Aviv University, Israel · Department of Computer Science Technion, Israel Institute of Technology, Israel