Masked Diffusion Models

Follow this topic Ask about this topic

Momentum

21 papers in the last four weeks, up 250% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 112

All topics

VoidPadding: Let [VOID] Handle Padding in Masked Diffusion Language Models so that [EOS] Can Focus on Semantic Termination
Jun 16, 2026Chunyu Liu, Zhengyang Fan, Kaisen Yang +1Masked Diffusion Models Diffusion Language Model Inference
Tsinghua University
LESS Is More: Mutual-Stability Sampling for Diffusion Language Models
Jun 15, 2026Amr Mohamed, Guokan Shang, Michalis VazirgiannisMasked Diffusion Models Diffusion Language Model Decoding
1MBZUAI · 2Ecole Polytechnique
Multi-Turn Reflective Masking Elicits Reasoning in Mask Diffusion Models
Jun 15, 2026Yanming Zhang, Yihan Bian, Jingyuan Qi +3Masked Diffusion Models Diffusion Model Fine-Tuning
University of Maryland · Virginia Tech · Intuit +2
Who Should Lead Decoding Now? Tracking Reliable Trajectories for Ensembling Masked Diffusion Language Models
Jun 15, 2026Heecheol Yun, Joonhyung Park, Joowon Kim +1Masked Diffusion Models Diffusion Language Model Decoding
1KAIST · 2AITRICS
Neither Parallel Nor Sequential: How DiffusionGemma Actually Commits Tokens
Jun 12, 2026Ali Asaria, Tony Salomone, Deep GandhiMasked Diffusion Models Diffusion Sampling
Transformer Lab
TimeROME-DLM: Temporal Causal Tracing and Low-Rank Inference-Time Knowledge Editing for Masked Diffusion Language Models
Jun 11, 2026Zhengtao Yao, Liuyang Song, Hongbo Zhang +4Knowledge Editing Masked Diffusion Models
University of Southern California, USA · Peking University, China · Stevens Institute of Technology, USA +1
Beyond Fully Random Masking: Attention-Guided Denoising and Optimization for Diffusion Language Models
Jun 10, 2026Jia Deng, Junyi Li, Wayne Xin Zhao +3Masked Diffusion Models Diffusion Model Fine-Tuning
Gaoling School of Artificial Intelligence, Renmin University of China · Department of Data Science, City University of Hong Kong · Beijing Key Laboratory of Research on Large Models and Intelligent Governance +2
Re-evaluating Confidence Remasking in Masked Diffusion Language Models
Jun 10, 2026Stipe Frkovic, Metod Jazbec, Dan Zhang +3Masked Diffusion Models Diffusion Language Model Decoding
UvA-Bosch Delta Lab, University of Amsterdam · Bosch Center for AI · University of Basel +1
Attention-Discounted Adaptive Sampler for Masked Diffusion Language Models
Jun 9, 2026Yusuf Sahin, Ahmed Rockey Saikia, Volkan Cevher +1Masked Diffusion Models Parallel Decoding
University of Bern, Bern, Switzerland · EPFL, Lausanne, Switzerland
Self-Augmenting Retrieval for Diffusion Language Models
Jun 4, 2026Paul Jünger, Justin Lovelace, Linxi Zhao +2Multi-Hop QA Retrieval-Augmented Generation
Department of Computer Science, Cornell University.
NAVIRA: Decoupled Stochastic Remasking for Masked Diffusion Language Models
Jun 4, 2026Andrey Fomenko, Maksim Kryzhanovskiy, Svetlana Glazyrina +1Non-Autoregressive Text Generation Masked Diffusion Models
Lomonosov Moscow State University, Moscow, Russia · Institute for Artificial Intelligence, Lomonosov Moscow State University, Moscow, Russia
Supportive Token Revealing for Fast Diffusion Language Model Decoding
Jun 2, 2026Giries Abu Ayoub, Mario Barbara, Lluís Pastor-Pérez +4Masked Diffusion Models Diffusion Language Model Decoding
Department of Computer Science, University of Haifa · Institute for AI, University of Stuttgart · IMPRS-IS
Knowledge Editing for Masked Diffusion Language Models
Jun 2, 2026Haewon Park, Yohan JoKnowledge Editing Masked Diffusion Models
Graduate School of Data Science, Seoul National University
MaskForge: Structure-Aware Adaptive Attacks for Jailbreaking Diffusion Large Language Models
Jun 1, 2026Yingzi Ma, Zhengyue Zhao, Xiaogeng Liu +3Masked Diffusion Models Adversarial Attacks on LLMs
University of Wisconsin-Madison · Johns Hopkins University · Responsible AI Research (RAIR) Centre, The University of Adelaide +1
SimSD: Simple Speculative Decoding in Diffusion Language Models
Jun 1, 2026Junxia Cui, Haotian Ye, Runchu Tian +9Masked Diffusion Models Speculative Decoding for Diffusion Language Models
University of California San Diego · University of Illinois Urbana-Champaign · 3Google +1
BlockGen: Flexible Blockwise Sequence Modeling with Hybrid Samplers
Jun 1, 2026Justin Deschenaux, Caglar GulcehreMasked Diffusion Models Diffusion Sampling
EPFL Lausanne, Switzerland · EPFL, Lausanne, Switzerland · Microsoft AI
Revise, Don't Freeze: Sampler-Matched Training for Self-Correcting Masked Diffusion Language Models
May 31, 2026Longxuan Yu, Shaorong Zhang, Yu Fu +3Masked Diffusion Models Diffusion Model Fine-Tuning
University of California, Riverside · Microsoft
DSL-LLaDA: Scaling Continuous Denoising to 8B Masked Diffusion LMs
May 31, 2026Longxuan Yu, Yunshu Wu, Yu Fu +5Masked Diffusion Models Diffusion Language Model Decoding
University of California, Riverside · Georgia Institute of Technology · Microsoft
Adaptive Order Policies for Masked Diffusion
May 29, 2026Jama Hussein Mohamud, Mohsin Hasan, Mirco Ravanelli +1Masked Diffusion Models Diffusion Model Sampling
Université de Montréal · Mila · Concordia University +1
What Gets Unmasked First? Trajectory Analysis of Diffusion Models for Graph-to-Text Generation
May 29, 2026Qing Wang, Jacob Devasier, Chengkai LiMasked Diffusion Models LLM Fine-Tuning
The University of Texas at Arlington
Masked Diffusion Modeling for Anomaly Detection
May 28, 2026Lixing Zhang, Yuchen Liang, Liyan XieMasked Diffusion Models Tabular Anomaly Detection
University of Minnesota · Ohio State University
Cluster-Level Attention-Guided Parallel Decoding for Masked Diffusion Language Models
May 28, 2026Heqiang Qi, Wei Huang, Mingyuan Bai +1Masked Diffusion Models Diffusion Language Model Inference
Zhejiang University · RIKEN Center for Advanced Intelligence Project · The Institute of Statistical Mathematics +1
The Confidence Shortcut: A Reasoning Failure Mode of Masked Diffusion Models
May 27, 2026Dueun Kim, Albert NoMasked Diffusion Models Language Model Decoding
Department of Artificial Intelligence, Yonsei University
Looped Diffusion Language Models
May 25, 2026Sanghyun Lee, Chunsan Hong, Seungryong Kim +3Masked Diffusion Models Diffusion Language Model Inference
1KAIST · 2KRAFTON · University of California, Berkeley
DLLM-JEPA: Joint Embedding Predictive Architectures for Masked Diffusion Language Models
May 24, 2026Sangdae NamMasked Diffusion Models Joint-Embedding Predictive Architecture
VESSL AI
TUBE: Tangent Upper Bound on Evidence for Discrete Diffusion Language Models
May 22, 2026Arseny Ivanov, Sergei Kholkin, Vladislav Gromadskii +3Masked Diffusion Models Autoregressive Language Modeling
AXXX, Russia · Applied AI Institute, Moscow, Russia · MIRAI, Russia
Extracting Training Data from Diffusion Language Models via Infilling
May 22, 2026Yihan Wang, N. AsokanData Leakage Masked Diffusion Models
University of Waterloo · KTH Royal Institute of Technology
DiLaDiff: Distilled Latent-Augmented Diffusion for Language Modeling
May 22, 2026Jean-Marie Lemercier, Tomas Geffner, Karsten Kreis +3Masked Diffusion Models Diffusion Language Model Inference
NVIDIA
Learned Relay Representations for Forward-Thinking Discrete Diffusion Models
May 21, 2026Benjamin Rozonoyer, Jacopo Minniti, Dhruvesh Patel +4Masked Diffusion Models Diffusion Language Model Inference
University of Massachusetts Amherst · University of Toronto · Stanford University +3
Backdooring Masked Diffusion Language Models
May 19, 2026Daniel Yiming Cao, Chengzhong Wang, Sheng-Yen Chou +3LLM Backdoor Attacks Masked Diffusion Models
Cornell University · Virginia Tech · IBM Research

Jun 16, 2026, cs.CLJ/K move · Enter open · S save

VoidPadding: Let [VOID] Handle Padding in Masked Diffusion Language Models so that [EOS] Can Focus on Semantic Termination

Chunyu Liu, Zhengyang Fan, Kaisen Yang, Alex Lamb

Tsinghua University

Masked Diffusion Models Diffusion Language Model Inference Diffusion Language Model Decoding Diffusion Language Models

Open paper AskPDF arXiv

MDLMs generate text by denoising a preallocated masked response canvas, making response-length modeling central to instruction tuning. Existing MDLMs often inherit the autoregressive convention of using repeated \texttt{[EOS]} tokens for padding during instruction tuning, giving \texttt{[EOS]} a dual role as both a semantic terminator and a padding token. We show that this dual role is a root cause of \texttt{[EOS]} overflow under large-block decoding. To decouple these roles, we propose VoidPadding, which introduces \texttt{[VOID]} for padding and reserves \texttt{[EOS]} for termination. During inference, the learned \texttt{[EOS]} signal enables early stopping, while the learned \texttt{[VOID]} signal guides adaptive response canvas expansion. On Dream-7B-Instruct, VoidPadding improves the block-size-averaged four-task mean across mathematical reasoning and code generation benchmarks by

+17.84

points over the original model and

+6.95

points over RainbowPadding, while reducing decoding NFE by 55.7% on average. Code is available at https://github.com/Haru-LCY/VoidPadding.

Momentum

Related topics

Latest papers 112