Diffusion Language Model Decoding

Follow this topic Ask about this topic

Momentum

26 papers in the last four weeks, up 271% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 164

All topics

The Confidence Shortcut: A Reasoning Failure Mode of Masked Diffusion Models
May 27, 2026Dueun Kim, Albert NoMasked Diffusion Models Language Model Decoding
Department of Artificial Intelligence, Yonsei University
Diffusion Large Language Models for Visual Speech Recognition
May 27, 2026Jeong Hun Yeo, Chae Won Kim, Hyeongseop Rha +1Visual Speaker Recognition Diffusion Language Model Decoding
Integrated Vision Language Lab, KAIST, South Korea
When Confidence Misleads: Suffix Anchoring and Anchor-Proximity Confidence Modulation for Diffusion Language Models
May 27, 2026Jungwon Park, Jimyeong Kim, Jungmin Ko +2Non-Autoregressive Text Generation Language Model Decoding
1RICS · 2AIIS · 3IPAI +2
Dynamic-dLLM: Dynamic Cache-Budget and Adaptive Parallel Decoding for Training-Free Acceleration of Diffusion LLM
May 27, 2026Tianyi Wu, Xiaoxi Sun, Yanhua Jiao +5Diffusion Language Model Inference Diffusion Language Model Decoding
*Equal contribution. The work is done during Tianyi’s internship at CBG Celia DeviceAI Team, Huawei. · 1Harbin Institute of Technology, Shenzhen · 2Huawei +1
Visual-Redundancy-Controlled Parallel Decoding for Diffusion-Based Multimodal Large Language Models
May 25, 2026Yulin Yuan, Hongshuo Zhao, Xiangming MengLanguage Model Decoding Multimodal Large Language Models
Zhejiang University · ZJUI-UIUC Institute
The Path Matters: Learning a Token-Commitment Policy for Diffusion Language Models
May 23, 2026Bohang Sun, Max Zhu, Francesco Caso +5Diffusion Language Model Decoding Diffusion Language Models
Department of Computer Science and Technology, University of Cambridge · Department of Engineering Science, University of Oxford
Learned Relay Representations for Forward-Thinking Discrete Diffusion Models
May 21, 2026Benjamin Rozonoyer, Jacopo Minniti, Dhruvesh Patel +4Masked Diffusion Models Diffusion Language Model Inference
University of Massachusetts Amherst · University of Toronto · Stanford University +3
AnchorDiff: Topology-Aware Masked Diffusion with Confidence-based Rewriting for Radiology Report Generation
May 16, 2026Shiying Yu, Jielei Wang, Guoming LuRadiology Report Generation Diffusion Language Model Decoding
University of Electronic Science and Technology of China
Roll Out and Roll Back: Diffusion LLMs are Their Own Efficiency Teachers
May 16, 2026Fanqin Zeng, Feng Hong, Geng Yu +6Diffusion Model Distillation Diffusion Language Model Decoding
Shanghai Jiao Tong University, Shanghai, China · Apple MLR, Cupertino, CA, USA · Tongji University, Shanghai, China +2
Dynamic Chunking for Diffusion Language Models
May 15, 2026Yichen Zhu, Xiaoming Shi, Peng Zhao +3Document Chunking Autoregressive Diffusion
CSE, HKUST · Xiaohongshu Inc. · Alibaba group +1
Ligand-Conditioned Discrete Diffusion for Protein Sequence-Structure Co-Design
May 15, 2026Chen Wei, Fanding Xu, Minghao Sun +5Protein Language Models Diffusion Language Model Decoding
Xi’an University of Posts & Telecommunications · National University of Singapore · Xi’an Jiaotong University +1
PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding
May 15, 2026Shengyin Sun, Yiming Li, Renxi Liu +7Masked Diffusion Models Speculative Decoding for Diffusion Language Models
Department of Computer Science City University of Hong Kong · Huawei Technologies · City University of Hong Kong
Mitigating Mask Prior Drift and Positional Attention Collapse in Large Diffusion Vision-Language Models
May 14, 2026Sujung Hong, Chanyong Yoon, Seong Jae HwangVision-Language Models Diffusion Language Model Decoding
Department of Artificial Intelligence, Yonsei University, Seoul, Republic of Korea.
Factorization-Error-Free Discrete Diffusion Language Model via Speculative Decoding
May 14, 2026Xun Fang, Yunchen Li, Hang Yuan +1Speculative Decoding for Diffusion Language Models Diffusion Language Model Inference
East China Normal University · Beijing Zhongguancun Academy
Orthrus: Memory-Efficient Parallel Token Generation via Dual-View Diffusion
May 12, 2026Chien Van Nguyen, Chaitra Hegde, Van Cuong Pham +3Language Model Decoding Diffusion Language Model Inference
University of Oregon · Google DeepMind · Adobe Research
Multi-Token Residual Prediction
May 12, 2026Yufeng Xu, Zishuo Bao, Qian Wang +6Masked Diffusion Models Multi-Token Prediction
New York University · New York University Shanghai · Nous Research +1
Self-Distilled Trajectory-Aware Boltzmann Modeling: Bridging the Training-Inference Discrepancy in Diffusion Language Models
May 12, 2026Kecheng Chen, Ziru Liu, Xijia Tao +9Diffusion Model Distillation Diffusion Model Fine-Tuning
1City University of Hong Kong · 2Huawei Research · 3The University of Hong Kong
D-PACE: Dynamic Position-Aware Cross-Entropy for Parallel Speculative Drafting
May 12, 2026Tianyu Wu, Yu Yao, Zhenting Qi +7Speculative Decoding LLM Inference Acceleration
1Harvard · 2MIT
BitLM: Unlocking Multi-Token Language Generation with Bitwise Continuous Diffusion
May 12, 2026Shaobin Zhuang, Yuang Ai, Jiaming Han +7Multi-Token Prediction Autoregressive Language Modeling
Shanghai Jiao Tong University · MMLab, The Chinese University of Hong Kong · Institute of Automation, Chinese Academy of Sciences +1
Dystruct: Dynamically Structured Diffusion Language Model Decoding via Bayesian Inference
May 10, 2026Bian Sun, Kevin Zhai, Mubarak Shah +1Diffusion Language Model Decoding Diffusion Language Models
University of Central Florida
Edit-Based Refinement for Parallel Masked Diffusion Language Models
May 10, 2026Houxing Ren, Mingjie Zhan, Zimu Lu +5Masked Diffusion Models Parallel Decoding
CUHK MMLab · SenseTime Research · Shenzhen Loop Area Institute +1
TAD: Temporal-Aware Trajectory Self-Distillation for Fast and Accurate Diffusion LLM
May 10, 2026Haoyang Zhou, Li Kong, Shijie Ren +4Diffusion Model Distillation Diffusion Language Model Inference
Gaoling School of Artificial Intelligence, Renmin University of China · Ant Group
Path-Dependent Denoising: A Non-Conservative Field Perspective on Order Collapse in Diffusion Language Models
May 10, 2026Jeonseong KimDiffusion Language Model Inference Diffusion Language Model Decoding
LEAP: Unlocking dLLM Parallelism via Lookahead Early-Convergence Token Detection
May 9, 2026Haohui Zhang, Zhiye Wang, Xiaoying Gan +2Parallel Decoding Diffusion Language Model Decoding
Shanghai Jiao Tong University
Steering Without Breaking: Mechanistically Informed Interventions for Discrete Diffusion Language Models
May 8, 2026Hanhan Zhou, Shamik Roy, Rashmi GangadharaiahLanguage Model Steering Diffusion Language Model Decoding
AWS AI Labs Santa Clara, CA 95054
Fast Byte Latent Transformer
May 8, 2026Julie Kallini, Artidoro Pagnoni, Tomasz Limisiewicz +5Speculative Decoding LLM Inference Acceleration
FAIR at Meta · Stanford University · University of Washington
CoBit: Language Modeling with Bitstream Diffusion
May 7, 2026Georgios Batzolis, Mark Girolami, Luca AmbrogioniDiffusion Language Model Decoding Diffusion Language Models
Department of Engineering, University of Cambridge. · Devotion AI Labs Ltd. · Donders Institute for Brain, Cognition, and Behaviour, Radboud University.
Predict-then-Diffuse: Adaptive Response Length for Compute-Budgeted Inference in Diffusion LLMs
May 5, 2026Michael Rottoli, Subhankar Roy, Stefano ParaboschiDiffusion Language Model Inference Diffusion Language Model Decoding
Universit`a degli Studi di Bergamo, Italy
Break the Block: Dynamic-size Reasoning Blocks for Diffusion Large Language Models via Monotonic Entropy Descent with Reinforcement Learning
May 4, 2026Yan Jiang, Ruihong Qiu, Zi HuangBlockwise Diffusion Language Models Diffusion Language Model Decoding
School of Electrical Engineering and Computer Science, The University of Queensland, Brisbane, Queensland, Australia
Focus on the Core: Empowering Diffusion Large Language Models by Self-Contrast
May 2, 2026Jinyuan Feng, Xin Yu, Yiqun Chen +5Diffusion Language Model Inference Diffusion Language Model Decoding
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Renmin University of China +1

May 27, 2026, cs.AIJ/K move · Enter open · S save

The Confidence Shortcut: A Reasoning Failure Mode of Masked Diffusion Models

Dueun Kim, Albert No

Department of Artificial Intelligence, Yonsei University

Masked Diffusion Models Language Model Decoding Mathematical Reasoning Diffusion Language Model Decoding Diffusion Language Models

Open paper AskPDF arXiv

Chain-of-thought reasoning helps autoregressive models solve complex problems by generating intermediate steps that support later predictions. Masked diffusion models (MDMs) offer a similar opportunity through arbitrary-order generation: they can ideally reveal intermediate results along logical dependencies. In practice, however, standard decoding simply prioritizes high-confidence tokens, which need not align with this dependency order. We identify this discrepancy as the \emph{confidence shortcut}: models commit with high certainty to plausible tokens while neglecting long-range dependencies. In multi-digit addition, models predict higher-order digits without properly tracking carries through long chains. Controlled pretraining across diverse reasoning tasks confirms that confidence-guided ordering often selects suboptimal sequences, and confidence-aligned training schemes can exacerbate these failures---for example, increasing addition error rates by an order of magnitude. Our findings caution against relying solely on confidence to choose generation orders and against training objectives that reinforce this preference. The experimental code is available at https://github.com/jinha2536/mdm-arithmetic.

Momentum

Related topics

Latest papers 164