Fast at the Frontier, Wrong at the Fallback: A Commit-Level Analysis of Threshold Decoding in Diffusion Language Models
Organizations: MBZUAI Abu Dhabi, UAE · UIUC Urbana, IL, USA
Abstract
Diffusion language models (DLMs) can write several tokens per forward pass. Open-weight DLMs commit, in any order, every masked position whose confidence passes a threshold. If none does, a fallback commits the most confident one, and the commits are never revised. Wide windows let the decoder commit far ahead of the text, which should speed up decoding but often costs accuracy. It is unclear where the model is confident and which commits cause errors. We trace every commit of four off-the-shelf DLMs on math, code, translation and summarization, and rerun failing answers with one step changed. Most tokens become confident only near the frontier, the leftmost masked position, so committing only there, still several tokens per step, is about as fast as committing anywhere. The accuracy loss comes from committing tokens before the text to their left, and committing one token per step does not reduce it. The worst case is an early end-of-sequence token that cuts the answer short. Fallback commits are 11-35% of commits but start 61-72% of the failures that left-to-right decoding avoids. Changing only the step where such a failure starts repairs it more often than changing a similar earlier step. Committing only at the frontier recovers most of the lost accuracy on all four models. Our code is available at https://github.com/rmakarovv/fast-at-the-frontier.
Figures & tables
| Policy | Tokens committed per step |
|---|---|
| vanilla | One or more. Commit all tokens with confidence , otherwise the most confident one. |
| one-per-step | One. Commit the most confident masked position. |
| left-to-right | One. Commit the frontier position (left-to-right decoding). |
| frontier-only | One or more. Commit the confident positions in a row, starting at the frontier. If none exists, commit the frontier position (PARD; Hou and Kwok, 2026 ). |
| Variants of vanilla : one or more tokens per step | |
| frontier-fallback | Use the frontier position as the fallback (Fork-dLLM; Frković et al., 2026 ). |
| confidence order | frontier order | effect (points) | EOS only at the frontier | |||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Model | several vanilla | one one-per-step | one left-to-right | several frontier-only | order | paral- lelism | several eos-at-frontier | one one-per-step | order | |
| NLD-3B | 128 | 79.1 | 79.7 | 88.0 | 87.6 | 86.2 | 87.3 | |||
| NLD-3B | 256 | 54.5 | 55.0 | 88.0 | 88.1 | 87.2 | 88.0 | |||
| NLD-8B | 128 | 87.3 | 86.7 | 90.2 | 90.1 | 91.0 | 91.0 | |||
| NLD-8B | 256 | 73.8 | 73.3 | 88.9 | 88.8 | 90.4 | 90.2 | |||
| LLaDA, A | 256 | 50.6 | 49.9 | 71.9 | 71.6 | 54.0 | 53.9 | |||
| Policy | NLD 3B | NLD 8B | LLaDA A | LLaDA B | Fast- dLLM v2 |
|---|---|---|---|---|---|
| vanilla | 79.1 | 87.3 | 50.6 | 35.5 | 79.4 |
| frontier-fallback | 85.7 | 89.6 | 67.9 | 75.4 | 82.6 |
| frontier-gating | 87.3 | 90.9 | 72.6 | 80.7 | 80.8 |
| entropy-budget | 87.1 | 91.4 | 73.2 | 80.8 | 80.3 |
| eos-at-frontier | 86.2 | 91.0 | 54.0 | 57.4 | 79.5 |
| frontier-only | 87.6 | 90.1 | 71.6 | 82.3 | 85.4 |
| Fallback share (%) | Rate ratio | |||
|---|---|---|---|---|
| Model | commits | harmful roots | fallback / confident | ahead / frontier |
| NLD-3B | 14 | 61 | 20.2 | 9.6–14.6 |
| NLD-8B | 35 | 72 | 31.5 | 9.0–14.3 |
| Fast-dLLM v2 | 11 | 70 | 19.1 | 7.8–12.3 |
| LLaDA | 17 | 68 | 10.3 | 1.3–1.9 |
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
| Task | Source and split | Size | Metric | Models | Answer length |
|---|---|---|---|---|---|
| GSM8K ( Cobbe et al., 2021 ) | openai/gsm8k, test | 1319 | accuracy | all four | 179 tokens |
| MATH-500 ( Hendrycks et al., 2021 ; Lightman et al., 2024 ) | HuggingFaceH4/MATH-500 | 500 | accuracy | NLD, Fast-dLLM v2 | 591 tokens |
| HumanEval ( Chen et al., 2021 ) | EvalPlus ( Liu et al., 2023 ) , base tests | 164 | pass@1 | all four | 552 tokens |
| MBPP ( Austin et al., 2021b ) | EvalPlus ( Liu et al., 2023 ) , base tests | 378 | pass@1 | all four | 461 tokens |
| WMT16 de en ( Bojar et al., 2016 ) | wmt/wmt16, test, 500 random sentences | 500 | chrF ( Popović, 2015 ) | NLD-3B, LLaDA | 18 words |
| CNN/DailyMail ( Hermann et al., 2015 ; See et al., 2017 ) | abisee/cnn_dailymail 3.0.0, test, 500 random articles | 500 | ROUGE-L ( Lin, 2004 ) | NLD-3B, LLaDA | 72 words |
| Model | Task | Window | Ahead | vanilla | one-per-step | frontier-only | TPF vanilla | TPF frontier-only |
|---|---|---|---|---|---|---|---|---|
| NLD-3B | GSM8K | 4 | 6 | 88.6 | 88.9 | – | 1.51 | – |
| NLD-3B | GSM8K | 8 | 13 | 89.1 | 89.2 | – | 2.06 | – |
| NLD-3B | GSM8K | 16 | 23 | 87.9 | 88.8 | – | 2.46 | – |
| NLD-3B | GSM8K | 32 | 32 | 87.6 | 88.3 | 88.1 | 2.75 | 2.61 |
| NLD-3B | GSM8K | 64 | 41 | 87.0 | 88.9 | 88.4 | 2.72 | 2.74 |
| NLD-3B | GSM8K | 128 | 49 | 79.1 | 79.7 | 87.6 | 2.27 | 2.77 |
| Decoder | batch 1 | batch 8 | batch 32 |
|---|---|---|---|
| self-speculation | 4.72 | 3.42 | 1.93 |
| + LoRA drafter | 4.83 | 3.60 | 2.00 |
| threshold decoding | |||
| vanilla , | 2.95 | 1.57 | 0.63 |
| frontier-only , | 2.85 | 1.54 | 0.62 |
| vanilla , | 3.12 | 0.88 | 0.25 |
| Task | vanilla | frontier-only | frontier-fallback | frontier-gating | entropy-budget | eos-at-frontier | vanilla , |
|---|---|---|---|---|---|---|---|
| GSM8K | 79.1 | 87.6 | 85.7 | 87.3 | 87.1 | 86.2 | 87.6 |
| MATH-500 | 69.6 | 73.2 | 68.8 | 71.8 | 72.2 | 70.8 | 74.4 |
| HumanEval | 64.6 | 81.1 | 73.2 | 70.7 | 71.3 | 70.1 | 73.2 |
| MBPP | 63.5 | 72.5 | 65.3 | 72.0 | 71.7 | 69.8 | 71.2 |
| Model, window | task | vanilla | one-per-step | left-to-right | frontier-only | order | parallelism | interaction |
|---|---|---|---|---|---|---|---|---|
| NLD-3B, | GSM8K | 79.1 | 79.7 | 88.0 | 87.6 | 8.4 1.7 | 0.5 0.8 | 0.2 1.5 |
| NLD-3B, | MATH-500 | 69.6 | 73.8 | 76.6 | 73.2 | 3.2 2.9 | 3.8 2.0 | 0.8 3.3 |
| NLD-3B, | HumanEval | 64.6 | 70.1 | 80.5 | 81.1 | 13.4 6.3 | 2.4 2.7 | 6.1 5.2 |
| NLD-3B, | MBPP | 63.5 | 64.3 | 73.8 | 72.5 | 9.3 4.4 | 1.1 1.9 | 0.5 3.6 |
| NLD-3B, | GSM8K | 54.5 | 55.0 | 88.0 | 88.1 | 33.3 2.6 | 0.2 1.0 | 0.6 2.0 |
| NLD-8B, | GSM8K | 87.3 | 86.7 | 90.2 | 90.1 | 3.2 1.5 | 0.2 0.9 | 0.6 1.8 |
| accuracy, EOS only at the frontier | order effect | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Model | window | task | vanilla | one-per-step | left-to-right | frontier-only | without | with | par. | int. | kept |
| NLD-3B | 128 | GSM8K | 86.2 | 87.3 | 88.0 | 87.6 | +8.4 1.7 | +1.1 1.3 | 0.8 0.6 | +0.6 1.1 | 13% |
| NLD-3B | 128 | MATH-500 | 70.8 | 74.2 | 76.6 | 73.2 | +3.2 2.9 | +2.4 2.8 | 3.4 1.8 | +0.0 2.7 | 75% |
| NLD-3B | 128 | HumanEval | 70.1 | 73.2 | 80.5 | 81.1 | +13.4 6.3 | +9.1 5.3 | 1.2 2.4 | +3.7 5.2 | 68% |
| NLD-3B | 128 | MBPP | 69.8 | 72.2 | 73.8 | 72.5 | +9.3 4.4 | +2.1 4.0 | 1.9 1.7 | +1.1 3.0 | 23% |
| NLD-3B | 256 | GSM8K | 87.2 | 88.0 | 88.0 | 88.1 | +33.3 2.6 | +0.5 1.3 | 0.4 0.6 | +0.9 1.1 | 1% |
| Setting | 0.7 | 0.8 | 0.9 | 0.95 |
|---|---|---|---|---|
| NLD-3B, | 73.9 / 85.5 (+11.6) | 77.3 / 86.9 (+9.6) | 79.1 / 87.6 (+8.5) | 79.5 / 87.9 (+8.4) |
| TPF | 4.0 / 3.8 | 3.0 / 3.2 | 2.3 / 2.8 | 1.8 / 2.5 |
| NLD-3B, , MATH-500 | 61.0 / 67.2 (+6.2) | 67.0 / 70.8 (+3.8) | 69.6 / 73.2 (+3.6) | 70.2 / 75.0 (+4.8) |
| TPF | 5.2 / 4.4 | 4.4 / 4.1 | 3.5 / 3.4 | 3.1 / 3.0 |
| NLD-3B, | 81.0 / 85.4 (+4.4) | 84.7 / 87.1 (+2.4) | 87.6 / 88.1 (+0.5) | 88.6 / 88.2 ( ) |
| TPF | 4.0 / 3.4 | 3.3 / 3.1 | 2.7 / 2.6 | 2.4 / 2.3 |
| Model, task | 1 | 2 | 4 | 8 | 16 | 32 | |||
|---|---|---|---|---|---|---|---|---|---|
| NLD-3B, GSM8K | acc. | 87.6 | 87.9 | 87.9 | 87.8 | 87.6 | 87.1 | 86.4 | 79.1 |
| 0 | +0.4 | +0.4 | +0.2 | +0.1 | 0.5 | 1.1 | 8.5 | ||
| 1.3 | 1.4 | 1.5 | 1.4 | 1.4 | 1.5 | 2.0 | |||
| TPF | 2.77 | 2.81 | 2.82 | 2.85 | 2.87 | 2.86 | 2.80 | 2.27 | |
| NLD-3B, MATH-500 | acc. | 73.2 | 73.2 | 73.6 | 72.4 | 72.0 | 69.8 | 69.2 | 69.6 |
| 0 | +0.0 | +0.4 | 0.8 | 1.2 | 3.4 | 4.0 | 3.6 |
| Term | NLD-3B | NLD-8B | Fast-dLLM v2 | LLaDA |
|---|---|---|---|---|
| all departures | ||||
| distance 1 | 10.9 | 9.0 | 8.7 | 1.6 |
| distance 2–3 | 12.7 | 10.7 | 11.9 | 1.9 |
| distance 4–7 | 14.6 | 11.9 | 12.3 | 1.3 0.2 |
| distance 8–15 | 10.2 | 14.3 | 10.8 2.2 | 1.4 |
| distance | 9.6 | 12.0 | 7.8 | 1.5 |
| roots | fallback share | fallback share of roots | fallback / confident per commit | fallback share | ||||
|---|---|---|---|---|---|---|---|---|
| Model | harmful | helpful | of commits | harmful | helpful | harmful | helpful | of the net gap |
| NLD-3B | 299 | 81 | 14% | 61% 6 | 74% 9 | 20.2 | 19.1 | 56% 8 |
| NLD-8B | 163 | 105 | 35% | 72% 7 | 90% 5 | 31.5 | 19.9 | 38% |
| Fast-dLLM v2 | 239 | 75 | 11% | 70% 6 | 73% 10 | 19.1 | 22.3 | 69% |
| LLaDA | 894 | 125 | 17% | 68% 3 | 69% 8 | 10.3 | 10.7 | 68% 4 |
| Model | Task | Root | Ctrl. | Root ctrl. | ||
|---|---|---|---|---|---|---|
| NLD-3B | GSM8K | 128 | 77 | 44.2 | 29.2 | |
| NLD-3B | GSM8K | 256 | 325 | 29.5 | 22.3 | |
| NLD-3B | MATH | 128 | 33 | 24.2 | 6.3 | |
| Fast-dLLM v2 | GSM8K | 128 | 114 | 47.4 | 8.2 | |
| Fast-dLLM v2 | MATH | 128 | 73 | 28.8 | 10.6 |
| sampled steps: repaired | helpful roots: broken | |||||
|---|---|---|---|---|---|---|
| Setting | fallback | confident | root | sampled | ||
| NLD-3B GSM8K, | 440 | 29.6 | 17.3 | 19 | 36.8 | 7.9 |
| NLD-3B MATH-500, | 172 | 4.2 | 3.0 | 14 | 35.7 | 17.9 |
| NLD-3B GSM8K, | 1608 | 19.7 | 18.0 | 21 | 23.8 | 2.4 |
| Fast-dLLM v2 GSM8K, | 592 | 16.6 | 9.6 | 50 | 34.0 | 5.0 |
| Fast-dLLM v2 MATH-500, | 364 | 8.1 | 6.8 | 22 | 31.8 | 15.9 |
| Frozen tokens | acc. | vs default | share of loss |
|---|---|---|---|
| none (default) | 94.3 | – | – |
| frontier tokens | 93.8 | 15% | |
| step-0 tokens | 93.6 | 24% | |
| as many random (control) | 92.6 | 60% | |
| ahead tokens | 92.0 | 81% | |
| all accepted tokens | 91.5 | 100% |
Diffusion Language Model Decoding
Diffusion language model decoding generates text through iterative parallel refinement, balancing fewer denoising steps with accuracy, coherence, and control.
See all 264 papers