Masked Diffusion Models

Follow this topic Ask about this topic

Masked diffusion models learn to generate discrete sequences by corrupting tokens—often replacing them with masks—and iteratively predicting missing content using context on both sides. Work spans language, images, speech, molecules, DNA, and structured data, with methods including learned reveal-order policies, confidence- or attention-guided samplers, blockwise and speculative decoding, self-correction, and reinforcement-learning or distillation post-training. Evaluations on reasoning and coding benchmarks such as GSM8K, MATH-500, and HumanEval test whether parallel generation can match autoregressive quality, while multimodal and scientific-design tasks test the framework beyond text. The central tension is that committing more tokens per model pass improves speed but can ignore dependencies and reduce accuracy or diversity; sampler choices, likelihood evaluation, and real serving costs also complicate comparisons.

Latest papers

184 papers39 papers in the last four weeks, up 255% on the four weeks before. 0.2% of all new papers.

TUBE: Tangent Upper Bound on Evidence for Discrete Diffusion Language Models
May 22, 2026Arseny Ivanov, Sergei Kholkin, Vladislav Gromadskii +3Masked Diffusion Models Autoregressive Language Modeling
AXXX, Russia · Applied AI Institute, Moscow, Russia · MIRAI, Russia
Extracting Training Data from Diffusion Language Models via Infilling
May 22, 2026Yihan Wang, N. AsokanData Leakage Masked Diffusion Models
University of Waterloo · KTH Royal Institute of Technology
DiLaDiff: Distilled Latent-Augmented Diffusion for Language Modeling
May 22, 2026Jean-Marie Lemercier, Tomas Geffner, Karsten Kreis +3Diffusion Language Models Diffusion Language Model Distillation
NVIDIA
Learned Relay Representations for Forward-Thinking Discrete Diffusion Models
May 21, 2026Benjamin Rozonoyer, Jacopo Minniti, Dhruvesh Patel +4Masked Diffusion Models Diffusion Language Model Inference
University of Massachusetts Amherst · University of Toronto · Stanford University +3
Backdooring Masked Diffusion Language Models
May 19, 2026Daniel Yiming Cao, Chengzhong Wang, Sheng-Yen Chou +3LLM Backdoor Attacks Masked Diffusion Models
Cornell University · Virginia Tech · IBM Research
Machine Unlearning for Masked Diffusion Language Models
May 18, 2026Georu Lee, Seungwon Jeong, Hoki Kim +2Masked Diffusion Models Diffusion Model Unlearning
1Dongguk University-Seoul · 2Chung-Ang University · 3Korea Institute for Advanced Study
Elastic-dLLM: Position Preserving Context Compression and Augmentation of Diffusion LLMs
May 18, 2026Junyi Wu, Tianchen Zhao, Shaoqiu Zhang +3Masked Diffusion Models Long-Context Language Modeling
Tsinghua University · Shanghai Jiao Tong University · Infinigence AI
PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding
May 15, 2026Shengyin Sun, Yiming Li, Renxi Liu +7Parallel Speculative Decoding Masked Diffusion Models
Department of Computer Science City University of Hong Kong · Huawei Technologies · City University of Hong Kong
Membership Inference Attacks on Discrete Diffusion Language Models
May 15, 2026Shailesh KasivelrajanMasked Diffusion Models Membership Inference Attacks
Indian Institute of Technology Madras
Understanding and Accelerating the Training of Masked Diffusion Language Models
May 13, 2026Chunsan Hong, Sanghyun Lee, Chieh-Hsin Lai +5Masked Diffusion Models Diffusion Language Model Training
1KAIST · 2Sony AI · University of Tokyo +1
Image AID via continuous-time reinforcement learning
May 13, 2026Yilie Huang, Xun Yu ZhouDiffusion Model Guidance Masked Diffusion Models
Department of Industrial Engineering and Operations Research, Columbia University, New York, NY 10027, USA. · Department of Industrial Engineering and Operations Research & Data Science Institute, Columbia University, New York, NY 10027, USA.
Discrete Stochastic Localization for Non-autoregressive Generation
May 13, 2026Yunshu Wu, Jiayi Cheng, Longxuan Yu +4Non-Autoregressive Text Generation Masked Diffusion Models
University of California Riverside · New York University
Multi-Token Residual Prediction
May 12, 2026Yufeng Xu, Zishuo Bao, Qian Wang +6Masked Diffusion Models Multi-Token Prediction
New York University · New York University Shanghai · Nous Research +1
Infinite Mask Diffusion for Few-Step Distillation
May 11, 2026Jaehoon Yoo, Wonjung Kim, Chanhyuk Lee +1Masked Diffusion Models Diffusion Model Distillation
Korea Advanced Institute of Science and Technology (KAIST)
Edit-Based Refinement for Parallel Masked Diffusion Language Models
May 10, 2026Houxing Ren, Mingjie Zhan, Zimu Lu +5Masked Diffusion Models Parallel Decoding
CUHK MMLab · SenseTime Research · Shenzhen Loop Area Institute +1
Don't Retrain, Align: Adapting Autoregressive LMs to Diffusion LMs via Representation Alignment
May 7, 2026Fred Zhangzhi Peng, Alexis Fox, Anru R. Zhang +1Masked Diffusion Models Diffusion Language Models
Duke University · AITHYRA
Simple Self-Conditioning Adaptation for Masked Diffusion Models
Apr 28, 2026Michael Cardei, Huu Binh Ta, Ferdinando FiorettoMasked Diffusion Models Diffusion Model Fine-Tuning
University of Virginia
On the Trainability of Masked Diffusion Language Models via Blockwise Locality
Apr 27, 2026Yuxiang Wang, Yu Xiang, Baojian Zhou +4Masked Diffusion Models Masked Language Modeling
1Fudan University, Shanghai, China · 2Alibaba Group, China.
Discrete Tilt Matching
Apr 20, 2026Yuyuan Chen, Shiyi Wang, Peter Potaptchik +2Masked Diffusion Models RL for Language Model Reasoning
1Harvard University · University of Oxford · 3IAIFI +1
LaDA-Band: Language Diffusion Models for Vocal-to-Accompaniment Generation
Apr 13, 2026Qi Wang, Zhexu Shen, Meng Chen +4Music Accompaniment Generation Music Generation
State Key Lab of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences (CAS) University of Chinese Academy of Sciences Beijing, China · Lyra Lab, Tencent Music Entertainment Shenzhen, China · Pengcheng Laboratory Shenzhen, China
DMax: Aggressive Parallel Decoding for dLLMs
Apr 9, 2026Zigeng Chen, Gongfan Fang, Xinyin Ma +2Diffusion Language Model Training Parallel Decoding for Diffusion Language Models
National University of Singapore
DualDiffusion: A Speculative Decoding Strategy for Masked Diffusion Models
Apr 6, 2026Satyam Goyal, Kushal Patel, Tanush Mittal +1Diffusion Language Model Inference Speculative Decoding for Diffusion Language Models
Department of Computer Science and Engineering, University of Michigan, Ann Arbor, MI, USA.
MolDA: Molecular Understanding and Generation via Large Language Diffusion Model
Apr 6, 2026Seohyeon Shin, HanJun Choi, Jun-Hyung Park +2Multimodal Large Language Models Molecular Graph Generation
Gwangju Institute of Science and Technology, Gwangju, South Korea · Hankuk University of Foreign Studies, Seoul, South Korea
Unlocking Prompt Infilling Capability for Diffusion Language Models
Apr 4, 2026Yoshinari Fujinuma, Keisuke SakaguchiDiffusion Language Models Masked Diffusion Models
Patronus AI · Tohoku University, RIKEN
NeuralLVC: Neural Lossless Video Compression via Masked Diffusion with Temporal Conditioning
Apr 3, 2026Tiberio Uricchio, Marco BertiniMasked Diffusion Models Neural Video Compression
University of Pisa · Pisa, Italy · Università degli Studi di Firenze +1
Inverse Design of Optical Multilayer Thin Films using Robust Masked Diffusion Models
Apr 1, 2026Jonas Schaible, Asena Karolin Özdemir, Charlotte Debus +5Inverse Design Diffusion-Based Inverse Problems
Department Optics for Solar Energy, Helmholtz-Zentrum für Materialien und Energie GmbH, Berlin, Germany · Zuse Institute Berlin, Berlin, Germany · Scientific Computing Center, Karlsruhe Institute of Technology, Eggenstein-Leopoldshafen, Germany +3
TRIMS: Trajectory-Ranked Instruction Masked Supervision for Diffusion Language Models
Apr 1, 2026Lingjie Chen, Ruizhong Qiu, Yuyu Fan +2Diffusion Language Model Decoding Diffusion Language Models
UIUC · Independent Researcher
LogicDiff: Logic-Guided Denoising Improves Zero-Shot Reasoning in Masked Diffusion Language Models
Mar 24, 2026Shaik AmanMasked Diffusion Models Diffusion Language Model Decoding
Independent Researcher, Nellore, Andhra Pradesh, India
Diffutron: A Masked Diffusion Language Model for Turkish Language
Mar 20, 2026Şuayp Talha Kocabay, Talha Rüzgar AkkuşText Generation Diffusion Language Models
Hugging Face: huggingface.co/suayptalha · Hugging Face: huggingface.co/Q-bert
Cubic Discrete Diffusion: Discrete Visual Generation on High-Dimensional Representation Tokens
Mar 19, 2026Yuqing Wang, Chuofan Ma, Zhijie Lin +7Image Generation Diffusion-Based Image Generation
University of Hong Kong · ByteDance Seed · Carnegie Mellon University +2

May 22, 2026J/K move · Enter open · S save

TUBE: Tangent Upper Bound on Evidence for Discrete Diffusion Language Models

Arseny Ivanov, Sergei Kholkin, Vladislav Gromadskii, Grigoriy Ksenofontov+2

AXXX, Russia · Applied AI Institute, Moscow, Russia · MIRAI, Russia

Masked Diffusion Models Autoregressive Language Modeling Generative Modeling Diffusion Language Models Variational Inference Diffusion Models

Open paper AskPDF arXiv

Log-likelihood is a standard metric for evaluating generative models. Unfortunately, in contrast to autoregressive models (ARMs), discrete diffusion models generally do not admit exact computation of this quantity. Existing evaluations, therefore, rely on the evidence lower bound (ELBO), leaving unclear how much higher the true value may be. We address this by introducing the Tangent Upper Bound on Evidence (TUBE), a variational upper bound on log-likelihood that admits an unbiased Monte Carlo estimator. Our TUBE extends across latent-variable models, including masked diffusion models (MDMs), any-order ARMs (AO-ARMs), and block variants of both. Applied to block MDMs and block AO-ARMs, TUBE reveals our key empirical finding that these models lie strictly below the exact ARM baseline, showing that ARMs still dominate in likelihood.