Masked Diffusion Models

Follow this topic Ask about this topic

Momentum

21 papers in the last four weeks, up 250% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 112

All topics

Machine Unlearning for Masked Diffusion Language Models
May 18, 2026Georu Lee, Seungwon Jeong, Hoki Kim +2Masked Diffusion Models Diffusion Model Unlearning
1Dongguk University-Seoul · 2Chung-Ang University · 3Korea Institute for Advanced Study
Elastic-dLLM: Position Preserving Context Compression and Augmentation of Diffusion LLMs
May 18, 2026Junyi Wu, Tianchen Zhao, Shaoqiu Zhang +3Masked Diffusion Models Long-Context Language Modeling
Tsinghua University · Shanghai Jiao Tong University · Infinigence AI
PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding
May 15, 2026Shengyin Sun, Yiming Li, Renxi Liu +7Masked Diffusion Models Speculative Decoding for Diffusion Language Models
Department of Computer Science City University of Hong Kong · Huawei Technologies · City University of Hong Kong
Membership Inference Attacks on Discrete Diffusion Language Models
May 15, 2026Shailesh KasivelrajanMasked Diffusion Models Membership Inference Attacks
Indian Institute of Technology Madras
Understanding and Accelerating the Training of Masked Diffusion Language Models
May 13, 2026Chunsan Hong, Sanghyun Lee, Chieh-Hsin Lai +5Masked Diffusion Models Diffusion Model Acceleration
1KAIST · 2Sony AI · University of Tokyo +1
Image AID via continuous-time reinforcement learning
May 13, 2026Yilie Huang, Xun Yu ZhouDiffusion Model Guidance Masked Diffusion Models
Department of Industrial Engineering and Operations Research, Columbia University, New York, NY 10027, USA. · Department of Industrial Engineering and Operations Research & Data Science Institute, Columbia University, New York, NY 10027, USA.
Discrete Stochastic Localization for Non-autoregressive Generation
May 13, 2026Yunshu Wu, Jiayi Cheng, Longxuan Yu +4Non-Autoregressive Text Generation Masked Diffusion Models
University of California Riverside · New York University
Multi-Token Residual Prediction
May 12, 2026Yufeng Xu, Zishuo Bao, Qian Wang +6Masked Diffusion Models Multi-Token Prediction
New York University · New York University Shanghai · Nous Research +1
Infinite Mask Diffusion for Few-Step Distillation
May 11, 2026Jaehoon Yoo, Wonjung Kim, Chanhyuk Lee +1Masked Diffusion Models Diffusion Model Distillation
Korea Advanced Institute of Science and Technology (KAIST)
Edit-Based Refinement for Parallel Masked Diffusion Language Models
May 10, 2026Houxing Ren, Mingjie Zhan, Zimu Lu +5Masked Diffusion Models Parallel Decoding
CUHK MMLab · SenseTime Research · Shenzhen Loop Area Institute +1
Don't Retrain, Align: Adapting Autoregressive LMs to Diffusion LMs via Representation Alignment
May 7, 2026Fred Zhangzhi Peng, Alexis Fox, Anru R. Zhang +1Masked Diffusion Models Diffusion Language Models
Duke University · AITHYRA
Simple Self-Conditioning Adaptation for Masked Diffusion Models
Apr 28, 2026Michael Cardei, Huu Binh Ta, Ferdinando FiorettoMasked Diffusion Models Diffusion Model Fine-Tuning
University of Virginia
On the Trainability of Masked Diffusion Language Models via Blockwise Locality
Apr 27, 2026Yuxiang Wang, Yu Xiang, Baojian Zhou +4Masked Diffusion Models Masked Language Modeling
1Fudan University, Shanghai, China · 2Alibaba Group, China.
Discrete Tilt Matching
Apr 20, 2026Yuyuan Chen, Shiyi Wang, Peter Potaptchik +2Masked Diffusion Models RL for Language Model Reasoning
1Harvard University · University of Oxford · 3IAIFI +1
NeuralLVC: Neural Lossless Video Compression via Masked Diffusion with Temporal Conditioning
Apr 3, 2026Tiberio Uricchio, Marco BertiniMasked Diffusion Models Neural Video Compression
University of Pisa · Pisa, Italy · Università degli Studi di Firenze +1
Omni-Diffusion: Unified Multimodal Understanding and Generation with Masked Discrete Diffusion
Mar 6, 2026Lijiang Li, Zuwei Long, Yunhang Shen +6Masked Diffusion Models Unified Multimodal Models
Nanjing University · Tencent Youtu Lab · CASIA
ILRR: Inference-Time Steering Method for Masked Diffusion Language Models
Jan 29, 2026Eden Avrahami, Eliya NachmaniMasked Diffusion Models Language Model Steering
School of Computer Science, Tel Aviv University, Israel · Department of Electrical and Computer Engineering, Ben Gurion University, Beer Sheva, Israel.
Mechanism Shift During Post-training from Autoregressive to Masked Diffusion Language Models
Jan 21, 2026Injin Kong, Hyoungjoon Lee, Yohan JoMasked Diffusion Models Autoregressive Diffusion
Graduate School of Data Science, Seoul National University · Department of Biosystems & Biomaterials Science and Engineering, Seoul National University
Sparse-LaViDa: Sparse Multimodal Discrete Diffusion Language Models
Dec 16, 2025Shufan Li, Jiuxiang Gu, Kangning Liu +4Masked Diffusion Models Diffusion Language Model Inference
Adobe · UCLA
Enabling Approximate Joint Sampling in Diffusion LMs
Sep 25, 2025Parikshit Bansal, Sujay SanghaviMasked Diffusion Models Diffusion Language Model Decoding
UT Austin
Lavida-O: Elastic Large Masked Diffusion Models for Unified Multimodal Understanding and Generation
Sep 23, 2025Shufan Li, Jiuxiang Gu, Kangning Liu +4Masked Diffusion Models Unified Multimodal Models
Adobe · UCLA
Any-Order GPT as Masked Diffusion Model: Decoupling Formulation and Architecture
Jun 24, 2025Shuchen Xue, Tianyu Xie, Tianyang Hu +5Masked Diffusion Models Decoder-Only Language Models
University of Chinese Academy of Sciences · Academy of Mathematics and Systems Science, Chinese Academy of Sciences · School of Mathematical Sciences, Peking University +2

May 18, 2026, cs.CLJ/K move · Enter open · S save

Machine Unlearning for Masked Diffusion Language Models

Georu Lee, Seungwon Jeong, Hoki Kim, Jinseong Park+1

1Dongguk University-Seoul · 2Chung-Ang University · 3Korea Institute for Advanced Study

Masked Diffusion Models Diffusion Model Unlearning Diffusion Language Models Machine Unlearning LLM Unlearning

Open paper AskPDF arXiv

Recent masked diffusion language models (MDLMs), such as LLaDA and Dream, have achieved performance comparable to autoregressive large language models. Unlike autoregressive models, which generate text sequentially, MDLMs generate text by iteratively denoising masked positions in parallel. During fine-tuning, MDLMs learn to recover responses from masked response states conditioned on a prompt, thereby shifting their predictions from a prompt-masked unconditional distribution toward a prompt-conditional distribution. Despite this distinct generative and fine-tuning mechanism, machine unlearning for MDLMs remains largely unexplored. In this paper, we propose Masked Diffusion Unlearning (MDU), the first unlearning framework for MDLMs, by revisiting the process of learning specific knowledge in terms of diffusion. Specifically, MDU minimizes a forward KL divergence from the prompt-conditional prediction to a prompt-masked unconditional anchor at every masked response position, with a temperature scaling parameter to control the privacy-utility trade-off. Our empirical results on standard benchmarks and MDLM backbones show that MDU achieves high unlearning performance compared to existing LLM unlearning methods. Code is available at https://github.com/leegeoru/MDU.

Momentum

Related topics

Latest papers 112