Diffusion Language Models

Follow this topic Ask about this topic

Diffusion language models generate text by starting from corrupted or masked sequences and repeatedly refining them, using context from both earlier and later positions. Main approaches include discrete masked diffusion, continuous latent or flow-based models, and block or hybrid designs that combine parallel refinement with autoregressive structure. Work explores adaptive token ordering, remasking, distillation, and caching to improve reasoning, code generation, multimodal tasks, and inference speed. A central tension is balancing faster, more parallel decoding against token dependencies and output quality, while reliable evaluation and the cost of training from scratch remain open challenges.

Latest papers

473 papers99 papers in the last four weeks, up 130% on the four weeks before. 0.6% of all new papers.

FLARE: Diffusion for Hybrid Language Model
Jun 1, 2026Yuchen Zhu, Jing Shi, Chongjian Ge +9Diffusion Model Serving Diffusion Language Model Inference
1Adobe Research · 2Georgia Institute of Technology
Revise, Don't Freeze: Sampler-Matched Training for Self-Correcting Masked Diffusion Language Models
May 31, 2026Longxuan Yu, Shaorong Zhang, Yu Fu +3Masked Diffusion Models Diffusion Model Fine-Tuning
University of California, Riverside · Microsoft
DSL-LLaDA: Scaling Continuous Denoising to 8B Masked Diffusion LMs
May 31, 2026Longxuan Yu, Yunshu Wu, Yu Fu +5Masked Diffusion Models Diffusion Language Model Decoding
University of California, Riverside · Georgia Institute of Technology · Microsoft
Decoding in Order-Agnostic Language Models: Chain-Rule Deviation and Uniform Spreading
May 31, 2026Lin YaoLanguage Model Decoding Confidence Estimation in Language Models
School of Computer Science, Shanghai Jiao Tong University, Shanghai, 200240, China · Zhongguancun Academy, Beijing, 100097, China
WaveFilter: Enhancing the Long-Context Capability of Diffusion LLMs via Wavelet-Guided KV Cache Filtering
May 30, 2026Jinnan Yang, Yan Wang, Zhen Bi +5Long-Context Language Model Inference Diffusion Language Model Inference
Nanjing University of Science and Technology · Alibaba Group · Huzhou Normal University +1
Adaptive Order Policies for Masked Diffusion
May 29, 2026Jama Hussein Mohamud, Mohsin Hasan, Mirco Ravanelli +1Masked Diffusion Models Diffusion Model Sampling
Université de Montréal · Mila · Concordia University +1
What Gets Unmasked First? Trajectory Analysis of Diffusion Models for Graph-to-Text Generation
May 29, 2026Qing Wang, Jacob Devasier, Chengkai LiMasked Diffusion Models LLM Fine-Tuning
The University of Texas at Arlington
AMix-2: Establishing Protein as a Native Modality in Large Language Models
May 29, 2026Keyue Qiu, Yixin Wu, Lihao Wang +19Diffusion Language Models Protein Language Models
1Shanghai Artificial Intelligence Laboratory · Institute for AI Industry Research (AIR), Tsinghua University
dMoE: dLLMs with Learnable Block Experts
May 29, 2026Sicheng Feng, Zigeng Chen, Gongfan Fang +2Mixture-of-Experts Language Models Memory-Efficient Inference
National University of Singapore
Efficient Diffusion LLMs via Temporal-Spatial Parallel Decoding and Confidence Extrapolation
May 29, 2026Zekai Li, Ji Liu, Yiqing Huang +3Adaptive Computation for Diffusion Models Diffusion Language Model Decoding
Advanced Micro Devices, Inc. (AMD)
Diffusion Models Preferentially Memorize Prototypical Examples or: Why Does My Diffusion Model Love Slop?
May 28, 2026Marta Aparicio Rodriguez, Anastasia Borovykh, Grigorios A. Pavliotis +1Privacy Leakage in Language Models Memorization in Generative Models
Department of Mathematics, Imperial College London, UK · ML Lab, Capital Fund Management, France · Department of Physics, École Polytechnique Fédérale de Lausanne (EPFL), Switzerland
NaRA: Noise-Aware LoRA for Parameter-Efficient Fine-Tuning of Diffusion LLMs
May 28, 2026Shuaidi Wang, Zhan Zhuang, Ruping Huang +1Low-Rank Adaptation Diffusion Language Models
Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China · Department of Computer Science, City University of Hong Kong, Hong Kong, China · Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Hong Kong, China
DLM-SWAI: Steering Diffusion Language Models Before They Unmask
May 28, 2026Hyeseon An, Yo-Sub HanControllable Text Generation Diffusion Language Model Inference
Department of Computer Science, Yonsei University, Seoul, South Korea
Decoding Strategies for Diffusion-Based ASR: A Systematic Evaluation of Confidence-Based Thresholding
May 28, 2026Jeong Hun Yeo, Minsu Kim, Hyeongseop Rha +1Language Model Decoding Automatic Speech Recognition
KAIST, Daejeon, Republic of Korea · Google DeepMind, Tokyo, Japan
Cluster-Level Attention-Guided Parallel Decoding for Masked Diffusion Language Models
May 28, 2026Heqiang Qi, Wei Huang, Mingyuan Bai +1Masked Diffusion Models Diffusion Language Model Inference
Zhejiang University · RIKEN Center for Advanced Intelligence Project · The Institute of Statistical Mathematics +1
GDSD: Reinforcement Learning as Guided Denoiser Self-Distillation for Diffusion Language Models
May 28, 2026Xiaohang Tang, Keyue Jiang, Che Liu +4Diffusion Model Distillation RL for Diffusion Models
UNIST · University of Basel
The Confidence Shortcut: A Reasoning Failure Mode of Masked Diffusion Models
May 27, 2026Dueun Kim, Albert NoMasked Diffusion Models Language Model Decoding
Department of Artificial Intelligence, Yonsei University
Diffusion Large Language Models for Visual Speech Recognition
May 27, 2026Jeong Hun Yeo, Chae Won Kim, Hyeongseop Rha +1Visual Speaker Recognition Diffusion Language Model Decoding
Integrated Vision Language Lab, KAIST, South Korea
When Confidence Misleads: Suffix Anchoring and Anchor-Proximity Confidence Modulation for Diffusion Language Models
May 27, 2026Jungwon Park, Jimyeong Kim, Jungmin Ko +2Non-Autoregressive Text Generation Language Model Decoding
1RICS · 2AIIS · 3IPAI +2
Enabling KV Caching of Shared Prefix for Diffusion Language Models
May 26, 2026Younghun Go, Jaehoon Han, Changyong Shin +2KV Caching KV-Cache Management
Korea University, South Korea
Looped Diffusion Language Models
May 25, 2026Sanghyun Lee, Chunsan Hong, Seungryong Kim +3Masked Diffusion Models Diffusion Language Model Inference
1KAIST · 2KRAFTON · University of California, Berkeley
Triplet-Block Diffusion RWKV
May 25, 2026Ke Lin, Yiyang Luo, Zhaolong Su +2Diffusion Language Model Inference Recurrent Transformers
William & Mary · HKUST · Cornell
$D^2$ -Monitor: Dynamic Safety Monitoring for Diffusion LLMs via Hesitation Signals
May 25, 2026Aoxi Liu, Yupeng Chen, James Oldfield +5Language Model Safety Evaluation LLM Safety Monitoring
Torr Vision Group, University of Oxford · The Chinese University of Hong Kong, Shenzhen
Reinforcement Learning from Denoising Feedback
May 25, 2026Qi He, Huan Chen, Ya Guo +3RL for Language Model Reasoning RL for Diffusion Models
Fudan University · Ant Group · Hong Kong University of Science and Technology
DLLM-JEPA: Joint Embedding Predictive Architectures for Masked Diffusion Language Models
May 24, 2026Sangdae NamMasked Diffusion Models Joint-Embedding Predictive Architecture
VESSL AI
Inference-Time Alignment of Diffusion Models via Trust-Region Iterative Twisted Sequential Monte Carlo
May 24, 2026Weixin Wang, Yu Yang, Wei Deng +1Diffusion Model Alignment Inference-Time Optimization
Duke University · Morgan Stanley
The Path Matters: Learning a Token-Commitment Policy for Diffusion Language Models
May 23, 2026Bohang Sun, Max Zhu, Francesco Caso +5Diffusion Language Model Decoding Diffusion Language Models
Department of Computer Science and Technology, University of Cambridge · Department of Engineering Science, University of Oxford
TUBE: Tangent Upper Bound on Evidence for Discrete Diffusion Language Models
May 22, 2026Arseny Ivanov, Sergei Kholkin, Vladislav Gromadskii +3Masked Diffusion Models Autoregressive Language Modeling
AXXX, Russia · Applied AI Institute, Moscow, Russia · MIRAI, Russia
Extracting Training Data from Diffusion Language Models via Infilling
May 22, 2026Yihan Wang, N. AsokanData Leakage Masked Diffusion Models
University of Waterloo · KTH Royal Institute of Technology
DiLaDiff: Distilled Latent-Augmented Diffusion for Language Modeling
May 22, 2026Jean-Marie Lemercier, Tomas Geffner, Karsten Kreis +3Diffusion Language Models Diffusion Language Model Distillation
NVIDIA

Jun 1, 2026J/K move · Enter open · S save

FLARE: Diffusion for Hybrid Language Model

Yuchen Zhu, Jing Shi, Chongjian Ge, Hao Tan+8

1Adobe Research · 2Georgia Institute of Technology

Diffusion Model Serving Diffusion Language Model Inference Diffusion Language Models Diffusion Model Inference Acceleration

Open paper AskPDF arXiv

Autoregressive (AR) large language models (LLMs) have achieved broad practical success, but sequential decoding remains a key bottleneck for low-latency deployment. Recent efficient-inference work has progressed along two axes: reducing the cost of each model invocation through efficient architectures, and reducing serial decoding steps through parallel generation. Hybrid attention backbones address the former, while diffusion language models (dLLMs) pursue the latter via iterative parallel denoising. Combining these advantages remains challenging: AR-to-dLLM conversion often fails to preserve seed-checkpoint capability, and hybrid-attention recurrent states and masking constraints make diffusion training and serving nontrivial. We present FLARE, a systematic conversion framework for hybrid-attention LLMs. Our analysis identifies transfer data quality as the primary determinant of capability preservation, outweighing loss formulation and attention-mask design. The resulting framework combines a token-equal AR-and-diffusion objective, hardware-aware kernels, and unified inference, enabling one checkpoint to support both AR-style verified decoding and diffusion-style parallel denoising. Starting from strong AR checkpoints with limited post-training data, FLARE is competitive with leading open-source dLLMs across model scales and delivers consistent throughput gains over open-source dLLM baselines in single-GPU concurrent serving. Our results further suggest that practical dLLMs are limited not only by decoding algorithms, but also by transfer data quality and the training inefficiency of current block-diffusion objectives, motivating joint design of data, objectives, architectures, and inference systems.