Masked Diffusion Models

Follow this topic Ask about this topic

Masked diffusion models learn to generate discrete sequences by corrupting tokens—often replacing them with masks—and iteratively predicting missing content using context on both sides. Work spans language, images, speech, molecules, DNA, and structured data, with methods including learned reveal-order policies, confidence- or attention-guided samplers, blockwise and speculative decoding, self-correction, and reinforcement-learning or distillation post-training. Evaluations on reasoning and coding benchmarks such as GSM8K, MATH-500, and HumanEval test whether parallel generation can match autoregressive quality, while multimodal and scientific-design tasks test the framework beyond text. The central tension is that committing more tokens per model pass improves speed but can ignore dependencies and reduce accuracy or diversity; sampler choices, likelihood evaluation, and real serving costs also complicate comparisons.

Latest papers

184 papers39 papers in the last four weeks, up 255% on the four weeks before. 0.2% of all new papers.

HiLo-RD: Cross-Budget Self-Distillation for Masked Diffusion Language Models
Oct 27, 2026Xiang Li, Chengting Yu, Jungang Li +5Diffusion Language Model Decoding Diffusion Language Models
The Hong Kong University of Science and Technology (Guangzhou) · Zhejiang University · Nanyang Technological University +2
Unveil the Mask: Jointly-Optimized Diffusion Masked Pretraining for Appearance-Agnostic Operator Verification
Oct 26, 2026Katsuya Ogata, Ryoya Yoshiyama, Kawachi Hodaka +1Biometric Identification Video Representation Learning
The University of Osaka
What Makes a Good Unmasking Policy? A Controlled Study of Learned Unmasking in Diffusion Language Models
Oct 22, 2026Jinya Sakurai, Xun Xu, Jaehong YoonDiffusion Language Model Decoding Diffusion Language Models
NTU Singapore · Institute for Advanced Intelligence and Computing (IAIC), A*STAR · Centre for Frontier AI Research (CFAR), A*STAR
Random Remasking Scales Masked Diffusion Inference
Oct 22, 2026Zhaoyi Zhou, Frank Wu, Andrea ZanetteMasked Diffusion Models Diffusion Language Model Inference
Carnegie Mellon University
Block Diffusion for Longitudinal EHR: Accuracy, Efficiency, and Representation Trade-offs
Oct 21, 2026Ashvin Gupta, Joshua Placidi, Brendan Delaney +1Longitudinal EHR Modeling Blockwise Diffusion Language Models
Imperial College London
Hierarchical Masked Diffusion Language Models
Oct 21, 2026Parul Gupta, Satwik, Avni Aggarwal +5Masked Diffusion Models Diffusion Language Model Decoding
Indian Institute of Technology Delhi · IBM Research
Binary Discrete Diffusion: Candidate-Set Diffusion for Logic and Planning Tasks
Oct 20, 2026Rodrigo Casado Noguerales, Dimitri von Rütte, Bernhard Schölkopf +2Diffusion Models Diffusion-Based Planning
ETH Zurich · Max Planck Institute for Intelligent Systems · ELLIS Institute Tübingen
Enhanced Masked Diffusion Unlearning
Oct 18, 2026Tomer Garber, Tom TirerMachine Unlearning Diffusion Model Unlearning
Faculty of Engineering, Bar-Ilan University, Ramat Gan, Israel · Red Hat
LED: Improving Masked Diffusion Language Models by Lookahead Editing
Oct 16, 2026Yuyan Zhou, Zeyuan Yang, Yuxuan Song +5Diffusion Language Model Decoding Masked Diffusion Models
Department of Computer Science and Engineering, The Hong Kong University of Science and Technology · ByteDance Seed · Institute for AI Industry Research (AIR), Tsinghua University +1
Query-Efficient Rejection Sampling for Parallel Masked Diffusion
Oct 15, 2026Shiyuan Zhang, Xuheng Li, Qiwei Di +1Parallel Decoding for Diffusion Language Models Diffusion Sampling
Department of Computer Science, University of California, Los Angeles, CA 90095, USA
TIPS: A Tiny Inference-time Policy for Scaling in Diffusion Language Models
Oct 15, 2026Ya-Shi Zhang, Jian Tang, Pierre-André Noël +1Diffusion Language Model Decoding Inference-Time Adaptation of Language Models
ServiceNow AI Research · Mila - Quebec AI Institute · HEC Montréal
Diffusion Language Models as Open Quantum Systems
Oct 15, 2026Akinori Tanaka, Yukinao Akamatsu, Takuki Tomita +1Diffusion Language Models Quantum Generative Modeling
RIKEN Center for Advanced Intelligence Project (AIP), Japan · RIKEN Center for Interdisciplinary Theoretical and Mathematical Sciences (iTHEMS), Japan · Department of Mathematics, Keio University, Japan +2
Revise-TTS: A Non-Masking Refinement Framework for Masked Diffusion Models in Text-to-Speech
Oct 14, 2026Wanli Sun, Dan Su, Anton RagniTTS Synthesis Zero-Shot TTS
School of Computer Science, University of Sheffield, Sheffield, UK · HojoAI, Beijing, China
Multi-Stream Residual Computation in Diffusion Language Models
Oct 13, 2026Ekaterina Alimaskina, Suren Kazarian, Gleb Molodtsov +1Diffusion Language Models Masked Diffusion Models
BRAIn Lab, Yandex Research
Masked Diffusion for Travel Diaries Synthesis
Oct 12, 2026Pascal Jutras-Dubé, Raziq Zaman, Fabian Bastin +2Conditional Generative Modeling Masked Diffusion Models
Department of Computer Science, Purdue University, West Lafayette, United States · Department of Civil and Environmental Engineering, University of Maryland, College Park, United States · Department of Computer Science and Operations Research, Université de Montréal, Montréal, Canada
Transformer-Based Masked Diffusion Models Provably Learn Distributions with Latent Structure
Oct 8, 2026Jingda Wu, Changxiao CaiMasked Diffusion Models Diffusion Sampling
Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor, USA.
Ambient Discrete Diffusion: Using the Wrong Data at the Right Time for Data Efficient Learning
Oct 8, 2026Julian Kleutgens, Mauricio Tec, Claudio Battiloro +2Diffusion Models Masked Diffusion Models
ETH Zürich · Harvard University · MIT
DiffuPlex: Accelerating Full-Duplex Spoken Dialog Models via Rolling Masked Diffusion
Oct 8, 2026Heeseung KimFull-Duplex Spoken Dialogue Systems Diffusion Model Inference Acceleration
Department of Artificial Intelligence University of Seoul Seoul, Republic of Korea
Just Weather Scoring: Efficient End-to-end Nowcasting with Distributional Diffusion
Oct 8, 2026Jannik Wiese, Johannes Schusterbauer, Tommaso Martorella +1Precipitation Nowcasting Probabilistic Forecasting
DiMOS: Doob-Guided Inference-Time Multi-Objective Search for Scientific Design
Oct 5, 2026Ziqing Wang, Qijie Zhu, Weimin Wu +4Masked Diffusion Models Inference-Time Search
Northwestern University
CleanMDM: Clean Motion Diffusion Model for Multimodal Motion Cleanup
Oct 4, 2026Zhe Li, Shicheng Wang, Bowen Cai +1Masked Diffusion Models Motion Diffusion Models
Peking University · Light Origins · Alibaba
D-DOIT: Training-free Adaptation of Discrete Diffusion via Doob's h-Transform
Oct 4, 2026Jieke Wu, Qijie Zhu, Weimin Wu +3Diffusion Model Adaptation Diffusion Model Guidance
Department of Computer Science, King Abdullah University of Science and Technology · Department of Statistics and Data Science, Northwestern University · Center for Foundation Models and Generative AI, Northwestern University Department of Computer Science, Northwestern University +1
SUAVE: Unified Video-Action Models via Masked Diffusion
Oct 2, 2026Rhythm Syed, Jean Mercat, Sedrick Keh +5Vision-Language-Action Models Masked Diffusion Models
Columbia University, New York, NY, USA. · Toyota Research Institute, Los Altos, CA, USA.
Pivot-SD: Efficient Self-Distillation for Masked Diffusion Language Models
Oct 2, 2026Seo Hyun Kim, Sunwoo Hong, Younwoo Choi +3Diffusion Language Model Distillation Masked Diffusion Models
KAIST AI · University of Toronto & Vector Institute
IDRF: Inverse-Distilled Reward Fine-tuning of Masked Discrete Diffusion Models
Oct 2, 2026Vladislav Gromadskii, David Li, Samson Gourevitch +4Diffusion Model Fine-Tuning Masked Diffusion Models
Applied AI Institute, Moscow, Russia · MBZUAI · CMAP, École Polytechnique +3
Refinement Buys Intelligibility, Search Buys Identity: What Test-Time Compute Buys in Masked-Diffusion TTS
Oct 2, 2026Nityanand Mathur, Hamees Sayed, Ayush Pratap SinghTTS Synthesis Text-to-Speech
Work done outside of Blackstar Inc. · Work done at Smallest AI.
Fork-dLLM: Avoiding the Flexibility Trap in Diffusion Language Models
Sep 30, 2026Stipe Frković, Metod Jazbec, Christian A. NaessethMasked Diffusion Models Speculative Decoding for Diffusion Language Models
University of Amsterdam · UvA-Bosch Delta Lab, University of Amsterdam
CDMD: A Cross-Dataset Mixed-Type Diffusion Model for Tabular Data
Sep 30, 2026Mohamed Amine Ketata, Maximilian Schambach, Stephan GünnemannMasked Diffusion Models Tabular Diffusion Models
Munich Data Science Institute, Technical University of Munich, Germany · SAP SE, Germany
Conditional Generation of Creative Chess Puzzles with Diffusion Models
Sep 29, 2026Aatu Selkee, Severi Rissanen, Xidong Feng +2Masked Diffusion Models Computational Creativity
Aalto University · Google DeepMind
Fine-Tuning Diffusion Language Models with Context Selection and Target Weighting
Sep 29, 2026Loay Mualem, Lluís Pastor-Pérez, Vinh Tong +4Masked Diffusion Models Diffusion Model Fine-Tuning
IMPRS-IS · Institute for AI, University of Stuttgart · Bitdefender

Oct 27, 2026J/K move · Enter open · S save

HiLo-RD: Cross-Budget Self-Distillation for Masked Diffusion Language Models

Xiang Li, Chengting Yu, Jungang Li, Yilun Zheng+4

The Hong Kong University of Science and Technology (Guangzhou) · Zhejiang University · Nanyang Technological University+2

Diffusion Language Model Decoding Diffusion Language Models Diffusion Language Model Distillation Masked Diffusion Models

Open paper AskPDF arXiv

We propose HiLo-RD (High-to-Low Budget Representation Distillation) to improve masked diffusion language models (MDLMs) under a limited generation budget. Unlike autoregressive models, MDLMs choose both what to generate and where to commit it. This creates two coupled challenges: predicting useful content before reasoning is complete, and deciding when to fix answer tokens that shape later predictions. We identify budget-induced premature commitment: even when a correct solution fits, the model can fix answer fragments near the window boundary before resolving intermediate steps. These fragments become context for later predictions and redirect subsequent reasoning. Existing post-training methods do not directly transfer the stronger trajectories that the same model follows with a larger window. HiLo-RD addresses this gap by (1) pairing large-window teacher states with cropped small-window views and (2) matching their hidden representations at committed positions. This supervision guides both token prediction and commitment order without an external teacher. Across mathematics benchmarks, HiLo-RD improves low-budget accuracy and shortens responses; these gains extend to code generation without code-domain training. On GSM8K, accuracy rises by 9.4 percentage points as mean readable length falls by about 25%. Attention diagnostics and fixed-budget order interventions support the proposed mechanism. Together, these results show that large-window reasoning can improve small-window decoding.