Diffusion Language Models

Follow this topic Ask about this topic

Diffusion language models generate text by starting from corrupted or masked sequences and repeatedly refining them, using context from both earlier and later positions. Main approaches include discrete masked diffusion, continuous latent or flow-based models, and block or hybrid designs that combine parallel refinement with autoregressive structure. Work explores adaptive token ordering, remasking, distillation, and caching to improve reasoning, code generation, multimodal tasks, and inference speed. A central tension is balancing faster, more parallel decoding against token dependencies and output quality, while reliable evaluation and the cost of training from scratch remain open challenges.

Latest papers

473 papers99 papers in the last four weeks, up 130% on the four weeks before. 0.6% of all new papers.

DiPOD: Diffusion Policy Optimization without Drifting Apart
Jun 11, 2026Haozhe Jiang, Haiwen Feng, Pieter Abbeel +3Policy Gradient Methods RL for Diffusion Models
UC Berkeley · NVIDIA
TimeROME-DLM: Temporal Causal Tracing and Low-Rank Inference-Time Knowledge Editing for Masked Diffusion Language Models
Jun 11, 2026Zhengtao Yao, Liuyang Song, Hongbo Zhang +4Knowledge Editing Masked Diffusion Models
University of Southern California, USA · Peking University, China · Stevens Institute of Technology, USA +1
Detect, Remask, Repair: Diffusion Editing for Faithful Summarization of Evolving Contexts
Jun 11, 2026Hao Zou, Zachary Horvitz, Chandhru Karthick +2Text Summarization Diffusion Language Models
Columbia University New York, NY, USA
Beyond Fully Random Masking: Attention-Guided Denoising and Optimization for Diffusion Language Models
Jun 10, 2026Jia Deng, Junyi Li, Wayne Xin Zhao +3Masked Diffusion Models Diffusion Model Fine-Tuning
Gaoling School of Artificial Intelligence, Renmin University of China · Department of Data Science, City University of Hong Kong · Beijing Key Laboratory of Research on Large Models and Intelligent Governance +2
Re-evaluating Confidence Remasking in Masked Diffusion Language Models
Jun 10, 2026Stipe Frković, Metod Jazbec, Dan Zhang +3Masked Diffusion Models Diffusion Language Model Decoding
UvA-Bosch Delta Lab, University of Amsterdam · Bosch Center for AI · University of Basel +1
Attention-Discounted Adaptive Sampler for Masked Diffusion Language Models
Jun 9, 2026Yusuf Sahin, Ahmed Rockey Saikia, Volkan Cevher +1Masked Diffusion Models Parallel Decoding
University of Bern, Bern, Switzerland · EPFL, Lausanne, Switzerland
Speech Meets ELF: Audio Conditional Continuous-Target Diffusion for Speech Recognition and Translation
Jun 9, 2026Xuanchen Li, Tianrui Wang, Yuheng Lu +11Speech Translation Automatic Speech Recognition
1Tianjin University · 2Shanghai Jiao Tong University · 3Zhipu AI
A Continuous-Time Markov Chain Framework for Insertion Language Models
Jun 8, 2026Dhruvesh Patel, Benjamin Rozonoyer, Soumitra Das +3Markov Models Markov Chains
UMass Amherst · IBM Research · University of Toronto & Vijil
Unified Energy for Invariant and Independent Decoding in Diffusion Language Models
Jun 8, 2026Yuchen Yan, Minkai Xu, Zaiquan Yang +1Energy-Based Models Diffusion Language Model Decoding
Department of Computer Science National University of Singapore · Department of Computer Science Stanford University · Department of Computer Science City University of Hong Kong
Continuous Language Diffusion as a Decoder-Interface Problem
Jun 7, 2026Zhicheng Du, Lan MaLatent Diffusion Models Diffusion Language Model Decoding
Tsinghua Shenzhen International Graduate School, Tsinghua University
Back on Track: Aligning Rewards and States for Reasoning in Diffusion Large Language Models
Jun 7, 2026Yawen Shao, Jie Xiao, Kai Zhu +6RL for Language Model Reasoning Credit Assignment in RL
University of Science and Technology of China · Tongyi Lab · Northeastern University
Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs New Metrics
Jun 7, 2026Antonio Franca, Alexander TongNon-Autoregressive Text Generation LLM Evaluation
AITHYRA, Vienna, Austria
TimpaTeks: Automatic In-place Text Sequence Modification via Diffusion Language Model Steering
Jun 7, 2026Ryandito Diandaru, Ahmed Elshabrawy, Fadli Aulawi Al Ghiffari +2Language Model Steering Diffusion Language Models
MBZUAI
Forward-Free Diffusion Language Models
Jun 6, 2026Haotian Sun, Rushi Qiang, Yuqian Zheng +1Diffusion Language Models
Georgia Institute of Technology
Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation
Jun 4, 2026Xingyu Su, Jacob Helwig, Shubham Parashar +6Language Model Distillation Diffusion Language Models
Department of Computer Science and Engineering, Texas A&M University · Department of Bioinformatics and Systems Medicine,2026 University of Texas Health Science Center at Houston · Department of Electrical and Computer Engineering, Texas A&M UniversityJun 4
Self-Augmenting Retrieval for Diffusion Language Models
Jun 4, 2026Paul Jünger, Justin Lovelace, Linxi Zhao +2Adaptive RAG Retrieval-Augmented Generation
Department of Computer Science, Cornell University.
Adapting Diffusion Language Models for Lossless Pixel-Level Image Transmission
Jun 4, 2026Tianqi Ren, Rongpeng Li, Xianfu Chen +2Learned Image Compression Diffusion Language Models
College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou 310027, China · Shenzhen CyberAray Network Technology Co., Ltd, Shenzhen 518000, China · School of Mechanical Engineering and Electronic Information, China University of Geosciences, Wuhan 430074, China +1
NAVIRA: Decoupled Stochastic Remasking for Masked Diffusion Language Models
Jun 4, 2026Andrey Fomenko, Maksim Kryzhanovskiy, Svetlana Glazyrina +1Non-Autoregressive Text Generation Masked Diffusion Models
Lomonosov Moscow State University, Moscow, Russia · Institute for Artificial Intelligence, Lomonosov Moscow State University, Moscow, Russia
FAIR-Calib: Frontier-Aware Instability-Reweighted Calibration for Post-Training Quantization of Diffusion Large Language Models
Jun 4, 2026Haoyu Huang, Linlin Yang, Sheng Xu +5LLM Quantization Diffusion Model Quantization
National College for Excellent Engineers, Beihang University, Beijing, China · State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing, China · Independent researcher +4
SAID: Accelerating Diffusion-Based Language Models via Scaffold-Aware Iterative Decoding
Jun 3, 2026Na Li, Chengda Wang, Mingju Gao +1Diffusion Language Model Inference Adaptive Computation for Diffusion Models
School of Computer Science, Peking University
SemBlock: Semantic Boundary Dynamic Blocks for Diffusion LLMs
Jun 3, 2026Xinrui Song, Zhuoran Wang, Mingju Gao +1Blockwise Diffusion Language Models Diffusion Language Model Decoding
School of Computer Science, Peking University
STaR-Quant: State-Time Consistent Post-Training Quantization for Diffusion Large Language Models
Jun 3, 2026Xin Yan, Aqiang Wang, Zhenglin Wan +2LLM Quantization Diffusion Model Quantization
School of Artificial Intelligence, Beijing Normal University, Beijing, China · Department of Computer Science, National University of Singapore, Singapore · Centre for Frontier AI Research, Agency for Science, Technology and Research (A*STAR), Singapore
Dynamic Infilling Anchors for Format-Constrained Generation in Diffusion Large Language Models
Jun 3, 2026Boyan Han, Yiwei Wang, Yi Song +2Controllable Text Generation Diffusion Language Model Decoding
AGI Lab, Westlake University, China · University of California, Merced, USA · Teeni AI, China +1
Global Sketch-Based Watermarking for Diffusion Language Models
Jun 3, 2026Daniel ZhaoDiffusion Model Watermarking LLM Watermarking
Harvard University
Read the Trace, Steer the Path: Trajectory-Aware Reinforcement Learning for Diffusion Language Models
Jun 3, 2026Anant Khandelwal, Manish GuptaRL for Language Model Reasoning Credit Assignment in RL
Microsoft AI, India
Knowledge Editing for Masked Diffusion Language Models
Jun 2, 2026Haewon Park, Yohan JoKnowledge Editing Masked Diffusion Models
Graduate School of Data Science, Seoul National University
Fast-dLLM++: Fréchet Profile Decoding for Faster Diffusion LLM Inference
Jun 1, 2026Siva Rajesh Kasa, Yasong Dai, Sumit Negi +1Language Model Decoding Parallel Decoding
Amazon Inc. · School of Computing, Australian National University.
MaskForge: Structure-Aware Adaptive Attacks for Jailbreaking Diffusion Large Language Models
Jun 1, 2026Yingzi Ma, Zhengyue Zhao, Xiaogeng Liu +3Masked Diffusion Models Adversarial Attacks on LLMs
University of Wisconsin-Madison · Johns Hopkins University · Responsible AI Research (RAIR) Centre, The University of Adelaide +1
SimSD: Simple Speculative Decoding in Diffusion Language Models
Jun 1, 2026Junxia Cui, Haotian Ye, Runchu Tian +9Masked Diffusion Models Speculative Decoding for Diffusion Language Models
University of California San Diego · University of Illinois Urbana-Champaign · 3Google +1
BlockGen: Flexible Blockwise Sequence Modeling with Hybrid Samplers
Jun 1, 2026Justin Deschenaux, Caglar GulcehreMasked Diffusion Models Diffusion Sampling
EPFL Lausanne, Switzerland · EPFL, Lausanne, Switzerland · Microsoft AI

Jun 11, 2026J/K move · Enter open · S save

DiPOD: Diffusion Policy Optimization without Drifting Apart

Haozhe Jiang, Haiwen Feng, Pieter Abbeel, Jiantao Jiao+2

UC Berkeley · NVIDIA

Policy Gradient Methods RL for Diffusion Models Diffusion Language Models

Open paper AskPDF arXiv

RL post-training has become increasingly pivotal for improving diffusion policies, but existing diffusion policy-gradient methods are often unstable and cannot achieve reliable policy improvement. We identify the cause as the double-drift phenomenon: optimizing a variational surrogate can let the ELBO separate from the true log-likelihood, which then makes the resulting proxy policy gradient misaligned with the true policy gradient of expected return. We propose DiPOD, a diffusion policy optimization framework that maintains tight-bound behavior throughout training by interleaving self-distillation with policy-improving gradient updates. This leads to a simple and practical algorithm: augmenting each diffusion policy-gradient update with an on-policy ELBO regularizer. Across diffusion language model post-training and continuous-control diffusion policies, DiPOD substantially stabilizes training and reaches higher rewards than previous methods.