Parallel Decoding for Diffusion Language Models

Latest papers

37 papers11 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Introspective Diffusion Language Models
Apr 13, 2026Yifan Yu, Yuqing Jian, Junxiong Wang +12Diffusion Language Models Diffusion Language Model Decoding
Together AI · University of Illinois Urbana-Champaign · Princeton University +2
DMax: Aggressive Parallel Decoding for dLLMs
Apr 9, 2026Zigeng Chen, Gongfan Fang, Xinyin Ma +2Diffusion Language Model Training Parallel Decoding for Diffusion Language Models
National University of Singapore
DAPD: Dependency-Aware Parallel Decoding via Attention for Diffusion LLMs
Mar 13, 2026Bumjun Kim, Dongjae Jeon, Moongyu Jeon +1Parallel Decoding for Diffusion Language Models Parallel Decoding
Department of Artificial Intelligence, Yonsei University, Seoul, Korea · Department of Computer Science, Yonsei University, Seoul, Korea
LightningRL: Breaking the Accuracy-Parallelism Trade-off of Block-wise dLLMs via Reinforcement Learning
Mar 4, 2026Yanzhe Hu, Yijie Jin, Pengfei Liu +2Parallel Decoding for Diffusion Language Models Blockwise Diffusion Language Models
Shanghai Jiao Tong University · Huazhong University of Science and Technology
Locally Coherent Parallel Decoding in Diffusion Language Models
Mar 3, 2026Michael Hersche, Nicolas Menet, Ronan Tanios +1Parallel Decoding for Diffusion Language Models Diffusion Language Model Decoding
IBM Research - Zurich · Department of Computer Science, ETH Zürich
Learning Generation Orders for Masked Discrete Diffusion Models via Variational Inference
Feb 27, 2026David Fox, Sam Bowyer, Song Liu +3Parallel Decoding for Diffusion Language Models Masked Diffusion Models
Mistral AI
Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding?
Feb 26, 2026Pengxiang Li, Dilxat Muhtar, Tianlong Chen +2Parallel Decoding for Diffusion Language Models Diffusion Language Model Decoding
The Hong Kong Polytechnic University, Hong Kong, China · ELLIS Institute Tübingen, Tübingen, Germany · Max Planck Institute for Intelligent Systems, Tübingen, Germany +3

Apr 13, 2026J/K move · Enter open · S save

Introspective Diffusion Language Models

Yifan Yu, Yuqing Jian, Junxiong Wang, Zhongzhu Zhou+11

Together AI · University of Illinois Urbana-Champaign · Princeton University+2

Diffusion Language Models Diffusion Language Model Decoding Diffusion Language Model Inference Parallel Decoding for Diffusion Language Models

Open paper AskPDF arXiv

Diffusion language models promise parallel generation, yet still lag behind autoregressive (AR) models in quality. We stem this gap to a failure of introspective consistency: AR models agree with their own generations, while DLMs often do not. We define the introspective acceptance rate, which measures whether a model accepts its previously generated tokens. This reveals why AR training has a structural advantage: causal masking and logit shifting implicitly enforce introspective consistency. Motivated by this observation, we introduce Introspective Diffusion Language Model (I-DLM), a paradigm that retains diffusion-style parallel decoding while inheriting the introspective consistency of AR training. I-DLM uses a novel introspective strided decoding (ISD) algorithm, which enables the model to verify previously generated tokens while advancing new ones in the same forward pass. From a systems standpoint, we build I-DLM inference engine on AR-inherited optimizations and further customize it with a stationary-batch scheduler. To the best of our knowledge, I-DLM is the first DLM to match the quality of its same-scale AR counterpart while outperforming prior DLMs in both model quality and practical serving efficiency across 15 benchmarks. It reaches 69.6 on AIME-24 and 45.7 on LiveCodeBench-v6, exceeding LLaDA-2.1-mini (16B) by more than 26 and 15 points, respectively. Beyond quality, I-DLM is designed for the growing demand of large-concurrency serving, delivering about 3x higher throughput than prior state-of-the-art DLMs.