cs.CLNov 16, 2026

Fast at the Frontier, Wrong at the Fallback: A Commit-Level Analysis of Threshold Decoding in Diffusion Language Models

Authors: Roman Makarov, Zhe Xu, Jian Kang

Organizations: MBZUAI Abu Dhabi, UAE · UIUC Urbana, IL, USA

Abstract

Diffusion language models (DLMs) can write several tokens per forward pass. Open-weight DLMs commit, in any order, every masked position whose confidence passes a threshold. If none does, a fallback commits the most confident one, and the commits are never revised. Wide windows let the decoder commit far ahead of the text, which should speed up decoding but often costs accuracy. It is unclear where the model is confident and which commits cause errors. We trace every commit of four off-the-shelf DLMs on math, code, translation and summarization, and rerun failing answers with one step changed. Most tokens become confident only near the frontier, the leftmost masked position, so committing only there, still several tokens per step, is about as fast as committing anywhere. The accuracy loss comes from committing tokens before the text to their left, and committing one token per step does not reduce it. The worst case is an early end-of-sequence token that cuts the answer short. Fallback commits are 11-35% of commits but start 61-72% of the failures that left-to-right decoding avoids. Changing only the step where such a failure starts repairs it more often than changing a similar earlier step. Committing only at the frontier recovers most of the lost accuracy on all four models. Our code is available at https://github.com/rmakarovv/fast-at-the-frontier.

Figures & tables

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. SetCommit: Coordinated Token Commitments for Few-Step Diffusion Language Model Decoding

    Oct 22, 2026Yuannuo Feng, Yubing Ye, Wenshuai Yao +4Diffusion Language Model InferenceDiffusion Language Models

  2. PACT: Certifying What Is Written in Parallel Decoding of Diffusion Language Models

    Nov 3, 2026Yanchao Li, Yusu Zhang, Jiaqing Xie +5Diffusion Language Model DecodingParallel Decoding for Diffusion Language Models

  3. Commit Locally, Exit Globally: Coordinating Adaptive Sampling and Early Exit in Diffusion Language Models

    Jul 30, 2026Chia-Ming Lee, Shao-Kai Liu, Ming-Ching Chang +3Diffusion Language Model InferenceDiffusion Language Model Decoding