Diffusion Language Model Decoding

Follow this topic Ask about this topic

Diffusion language model decoding generates text by repeatedly refining masked or noisy token positions, often updating many positions in parallel instead of predicting one token at a time. Common approaches use confidence, attention, token dependencies, or prediction stability to choose what to reveal and when; blockwise schedules, remasking, and learned ordering policies aim to make those choices more reliable. Speculative drafting and verification, search, adaptive length control, and cache or hardware optimizations further target faster inference across reasoning, code, long-form, and multimodal tasks. The central tension is that parallel updates can miss dependencies or commit errors too early, while cautious decoding costs more, so improving speed without sacrificing accuracy, coherence, or diversity remains an open challenge.

Latest papers

251 papers47 papers in the last four weeks, up 161% on the four weeks before. 0.3% of all new papers.

Orthrus: Memory-Efficient Parallel Token Generation via Dual-View Diffusion
May 12, 2026Chien Van Nguyen, Chaitra Hegde, Van Cuong Pham +3Language Model Decoding Diffusion Language Model Inference
University of Oregon · Google DeepMind · Adobe Research
Multi-Token Residual Prediction
May 12, 2026Yufeng Xu, Zishuo Bao, Qian Wang +6Masked Diffusion Models Multi-Token Prediction
New York University · New York University Shanghai · Nous Research +1
Self-Distilled Trajectory-Aware Boltzmann Modeling: Bridging the Training-Inference Discrepancy in Diffusion Language Models
May 12, 2026Kecheng Chen, Ziru Liu, Xijia Tao +9Diffusion Model Distillation Diffusion Model Fine-Tuning
1City University of Hong Kong · 2Huawei Research · 3The University of Hong Kong
D-PACE: Dynamic Position-Aware Cross-Entropy for Parallel Speculative Drafting
May 12, 2026Tianyu Wu, Yu Yao, Zhenting Qi +7Speculative Decoding LLM Inference Acceleration
1Harvard · 2MIT
BitLM: Unlocking Multi-Token Language Generation with Bitwise Continuous Diffusion
May 12, 2026Shaobin Zhuang, Yuang Ai, Jiaming Han +7Multi-Token Prediction Autoregressive Language Modeling
Shanghai Jiao Tong University · MMLab, The Chinese University of Hong Kong · Institute of Automation, Chinese Academy of Sciences +1
Dystruct: Dynamically Structured Diffusion Language Model Decoding via Bayesian Inference
May 10, 2026Bian Sun, Kevin Zhai, Mubarak Shah +1Diffusion Language Model Decoding Diffusion Language Models
University of Central Florida
Edit-Based Refinement for Parallel Masked Diffusion Language Models
May 10, 2026Houxing Ren, Mingjie Zhan, Zimu Lu +5Masked Diffusion Models Parallel Decoding
CUHK MMLab · SenseTime Research · Shenzhen Loop Area Institute +1
TAD: Temporal-Aware Trajectory Self-Distillation for Fast and Accurate Diffusion LLM
May 10, 2026Haoyang Zhou, Li Kong, Shijie Ren +4Diffusion Model Distillation Diffusion Language Model Inference
Gaoling School of Artificial Intelligence, Renmin University of China · Ant Group
Path-Dependent Denoising: A Non-Conservative Field Perspective on Order Collapse in Diffusion Language Models
May 10, 2026Jeonseong KimDiffusion Language Model Inference Diffusion Language Model Decoding
LEAP: Unlocking dLLM Parallelism via Lookahead Early-Convergence Token Detection
May 9, 2026Haohui Zhang, Zhiye Wang, Xiaoying Gan +2Parallel Decoding Diffusion Language Model Decoding
Shanghai Jiao Tong University
Steering Without Breaking: Mechanistically Informed Interventions for Discrete Diffusion Language Models
May 8, 2026Hanhan Zhou, Shamik Roy, Rashmi GangadharaiahLanguage Model Steering Diffusion Language Model Decoding
AWS AI Labs Santa Clara, CA 95054
Fast Byte Latent Transformer
May 8, 2026Julie Kallini, Artidoro Pagnoni, Tomasz Limisiewicz +5Speculative Decoding LLM Inference Acceleration
FAIR at Meta · Stanford University · University of Washington
CoBit: Language Modeling with Bitstream Diffusion
May 7, 2026Georgios Batzolis, Mark Girolami, Luca AmbrogioniDiffusion Language Model Decoding Diffusion Language Models
Department of Engineering, University of Cambridge. · Devotion AI Labs Ltd. · Donders Institute for Brain, Cognition, and Behaviour, Radboud University.
Predict-then-Diffuse: Adaptive Response Length for Compute-Budgeted Inference in Diffusion LLMs
May 5, 2026Michael Rottoli, Subhankar Roy, Stefano ParaboschiDiffusion Language Model Inference Diffusion Language Model Decoding
Universit`a degli Studi di Bergamo, Italy
Break the Block: Dynamic-size Reasoning Blocks for Diffusion Large Language Models via Monotonic Entropy Descent with Reinforcement Learning
May 4, 2026Yan Jiang, Ruihong Qiu, Zi HuangBlockwise Diffusion Language Models Diffusion Language Model Decoding
School of Electrical Engineering and Computer Science, The University of Queensland, Brisbane, Queensland, Australia
Focus on the Core: Empowering Diffusion Large Language Models by Self-Contrast
May 2, 2026Jinyuan Feng, Xin Yu, Yiqun Chen +5Diffusion Language Model Inference Diffusion Language Model Decoding
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Renmin University of China +1
DPRM: A Plug-in Doob h transform-induced Token-Ordering Module for Diffusion Language Models
Apr 27, 2026Dake Bu, Wei Huang, Andi Han +4Doob H-Transform Process Reward Models
City University of Hong Kong · CFAR and IHPC, A*STAR · RIKEN AIP +4
When to Commit? Towards Variable-Size Self-Contained Blocks for Discrete Diffusion Language Models
Apr 27, 2026Danny Wang, Ruihong Qiu, Zi HuangBlockwise Diffusion Language Models Diffusion Language Model Inference
The University of Queensland
Where to Place the Query? Unveiling and Mitigating Positional Bias in In-Context Learning for Diffusion LLMs via Decoding Dynamics
Apr 26, 2026Zhengheng Li, Panrui Li, Xuyang Liu +1In-Context Learning Diffusion Language Model Decoding
Southeast University
$R^2$ -dLLM: Accelerating Diffusion Large Language Models via Spatio-Temporal Redundancy Reduction
Apr 21, 2026Zhenbang Du, Kejing Xia, Xinrui Zhong +6Diffusion Language Model Inference Diffusion Model Fine-Tuning
Georgia Institute of Technology · NVIDIA
Remask, Don't Replace: Token-to-Mask Refinement in Diffusion Large Language Models
Apr 20, 2026Lin YaoLLM Self-Correction Multi-Token Prediction
School of Computer Science, Shanghai Jiao Tong University, Shanghai, 200240, China · Zhongguancun Academy, Beijing, 100097, China
NI Sampling: Accelerating Discrete Diffusion Sampling by Token Order Optimization
Apr 20, 2026Enshu Liu, Xuefei Ning, Yu Wang +1Few-Step Diffusion Sampling Diffusion Language Model Decoding
Department of EE, Tsinghua University · Microsoft Research
Stability-Weighted Decoding for Diffusion Language Models
Apr 18, 2026Yue Wu, Jian HuangDiffusion Language Model Decoding Diffusion Language Models
Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University, Hong Kong, China.
DepCap: Adaptive Block-Wise Parallel Decoding for Efficient Diffusion LM Inference
Apr 17, 2026Xiang Xia, Wuyang Zhang, Jiazheng Liu +2Diffusion Language Model Inference Parallel Decoding
University of Science and Technology of China · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center
Diffusion Language Models for Speech Recognition
Apr 15, 2026Davyd Naveriani, Albert Zeyer, Ralf Schlüter +1Connectionist Temporal Classification Automatic Speech Recognition
Machine Learning and Human Language Technology Group, RWTH Aachen University, Germany · AppTek, Germany
BARD: Bridging AutoRegressive and Diffusion Vision-Language Models Via Highly Efficient Progressive Block Merging and Stage-Wise Distillation
Apr 15, 2026Baoyou Chen, Hanchen Xia, Peng Tu +5Vision-Language Models Diffusion VLMs
Shanghai Academy of AI for Science · Fudan University · Shanghai Innovation Institute +1
Introspective Diffusion Language Models
Apr 13, 2026Yifan Yu, Yuqing Jian, Junxiong Wang +12Diffusion Language Models Diffusion Language Model Decoding
Together AI · University of Illinois Urbana-Champaign · Princeton University +2
Early Decisions Matter: Proximity Bias and Initial Trajectory Shaping in Non-Autoregressive Diffusion Language Models
Apr 12, 2026Jiyeon Kim, Sungik Choi, Yongrae Jo +2Non-Autoregressive Text Generation Diffusion Language Model Decoding
KAIST AI · LG AI Research · University of Illinois Chicago
Lost in Diffusion: Uncovering Hallucination Patterns and Failure Modes in Diffusion Large Language Models
Apr 12, 2026Zhengnan Guo, Fei TanHallucination in Language Models Diffusion Language Models
East China Normal University · Zhejiang University of Technology
Breaking Block Boundaries: Anchor-based History-stable Decoding for Diffusion Large Language Models
Apr 10, 2026Shun Zou, Yong Wang, Zehui Chen +4Diffusion Language Model Decoding Diffusion Model Inference Acceleration
MoE Key Lab of BIPC, University of Science and Technology of China · AMAP, Alibaba Group

May 12, 2026J/K move · Enter open · S save

Orthrus: Memory-Efficient Parallel Token Generation via Dual-View Diffusion

Chien Van Nguyen, Chaitra Hegde, Van Cuong Pham, Ryan A. Rossi+2

University of Oregon · Google DeepMind · Adobe Research

Language Model Decoding Diffusion Language Model Inference LLM Inference Acceleration Diffusion Language Model Decoding Parallel Decoding for Diffusion Language Models

Open paper AskPDF arXiv

We introduce Orthrus, a simple and efficient dual-architecture framework that unifies the exact generation fidelity of autoregressive Large Language Models (LLMs) with the high-speed parallel token generation of diffusion models. The sequential nature of standard autoregressive decoding represents a fundamental bottleneck for high-throughput inference. While diffusion language models attempt to break this barrier via parallel generation, they suffer from significant performance degradation, high training costs, and a lack of rigorous convergence guarantees. Orthrus resolves this dichotomy natively. Designed to seamlessly integrate into existing Transformers, the framework augments a frozen LLM with a lightweight, trainable module to create a parallel diffusion view alongside the standard autoregressive view. In this unified system, both views attend to the exact same high-fidelity Key-Value (KV) cache; the autoregressive head executes context pre-filling to construct accurate KV representations, while the diffusion head executes parallel generation. By employing an exact consensus mechanism between the two views, Orthrus guarantees lossless inference, delivering up to a 7.8x speedup with only an O(1) memory cache overhead and minimal parameter additions.