Diffusion Language Models

Follow this topic Ask about this topic

Diffusion language models generate text by starting from corrupted or masked sequences and repeatedly refining them, using context from both earlier and later positions. Main approaches include discrete masked diffusion, continuous latent or flow-based models, and block or hybrid designs that combine parallel refinement with autoregressive structure. Work explores adaptive token ordering, remasking, distillation, and caching to improve reasoning, code generation, multimodal tasks, and inference speed. A central tension is balancing faster, more parallel decoding against token dependencies and output quality, while reliable evaluation and the cost of training from scratch remain open challenges.

Latest papers

473 papers99 papers in the last four weeks, up 130% on the four weeks before. 0.6% of all new papers.

Language Diffusion Models are Associative Memories Capable of Retrieving Unseen Data
Apr 29, 2026Bao Pham, Mohammed J. Zaki, Luca Ambrogioni +2Memorization in Generative Models Neural Network Memorization
1Rensselaer Polytechnic Institute (RPI) · 2Donders Institute for Brain, Cognition, and Behaviour, Radboud University · 3Dynamical Mind +1
HIVE: Hidden-Evidence Verification for Hallucination Detection in Diffusion Large Language Models
Apr 28, 2026Guoshenghui Zhao, Tan Yu, Weijie ZhaoHallucination Detection Hallucination in Language Models
Rochester Institute of Technology · NVIDIA Corporation
Simple Self-Conditioning Adaptation for Masked Diffusion Models
Apr 28, 2026Michael Cardei, Huu Binh Ta, Ferdinando FiorettoMasked Diffusion Models Diffusion Model Fine-Tuning
University of Virginia
The Safety-Aware Denoiser for Text Diffusion Models
Apr 28, 2026Amman Yusuf, Zhejun Jiang, Mijung ParkDiffusion Model Guidance Diffusion Language Models
Department of Computer Science, The University of British Columbia, Vancouver, Canada.
On the Trainability of Masked Diffusion Language Models via Blockwise Locality
Apr 27, 2026Yuxiang Wang, Yu Xiang, Baojian Zhou +4Masked Diffusion Models Masked Language Modeling
1Fudan University, Shanghai, China · 2Alibaba Group, China.
Scaling Properties of Continuous Diffusion Spoken Language Models
Apr 27, 2026Jason Ramapuram, Eeshan Gunesh Dhekane, Amitis Shidani +6Language Model Scaling Laws Speech Language Models
Apple
DPRM: A Plug-in Doob h transform-induced Token-Ordering Module for Diffusion Language Models
Apr 27, 2026Dake Bu, Wei Huang, Andi Han +4Doob H-Transform Process Reward Models
City University of Hong Kong · CFAR and IHPC, A*STAR · RIKEN AIP +4
Factual and Edit-Sensitive Graph-to-Sequence Generation via Graph-Aware Adaptive Noising
Apr 27, 2026Aditya Hemant Shahane, Anuj Kumar Sirohi, Tanmoy Chakraborty +2Diffusion Language Models
Department of Electrical Engineering, Indian Institute of Technology Delhi, New Delhi, India · Indian Institute of Science, Bengaluru, India
BiMol-Diff: A Unified Diffusion Framework for Molecular Generation and Captioning
Apr 27, 2026Aditya Hemant Shahane, Anuj Kumar Sirohi, Devansh Arora +3Diffusion Sampling Molecular Generation
Indian Institute of Technology Delhi · Indian Institute of Technology Ropar · Indian Institute of Science Bengaluru
When to Commit? Towards Variable-Size Self-Contained Blocks for Discrete Diffusion Language Models
Apr 27, 2026Danny Wang, Ruihong Qiu, Zi HuangBlockwise Diffusion Language Models Diffusion Language Model Inference
The University of Queensland
Where to Place the Query? Unveiling and Mitigating Positional Bias in In-Context Learning for Diffusion LLMs via Decoding Dynamics
Apr 26, 2026Zhengheng Li, Panrui Li, Xuyang Liu +1In-Context Learning Diffusion Language Model Decoding
Southeast University
Measuring Temporal Linguistic Emergence in Diffusion Language Models
Apr 25, 2026Harry LuDiffusion Language Models
School of Mathematics, University of Minnesota, Minneapolis MN, United States.
LLaDA2.0-Uni: Unifying Multimodal Understanding and Generation with Diffusion Large Language Model
Apr 22, 2026Inclusion AI, Tiwei Bie, Haoxing Chen +15Unified Multimodal Models Multimodal Generation
AGI Research Center, Inclusion AI
On the Quantization Robustness of Diffusion Language Models in Coding Benchmarks
Apr 22, 2026Aarav Gupta, Gururaj Deshpande, Chandreyi ChakrabortyMixed-Precision Quantization Diffusion Model Quantization
College of Computing · Georgia Institute of Technology
Discrete Tilt Matching
Apr 20, 2026Yuyuan Chen, Shiyi Wang, Peter Potaptchik +2Masked Diffusion Models RL for Language Model Reasoning
1Harvard University · University of Oxford · 3IAIFI +1
Remask, Don't Replace: Token-to-Mask Refinement in Diffusion Large Language Models
Apr 20, 2026Lin YaoLLM Self-Correction Multi-Token Prediction
School of Computer Science, Shanghai Jiao Tong University, Shanghai, 200240, China · Zhongguancun Academy, Beijing, 100097, China
NI Sampling: Accelerating Discrete Diffusion Sampling by Token Order Optimization
Apr 20, 2026Enshu Liu, Xuefei Ning, Yu Wang +1Few-Step Diffusion Sampling Diffusion Language Model Decoding
Department of EE, Tsinghua University · Microsoft Research
Stability-Weighted Decoding for Diffusion Language Models
Apr 18, 2026Yue Wu, Jian HuangDiffusion Language Model Decoding Diffusion Language Models
Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University, Hong Kong, China.
FRIGID: Scaling Diffusion-Based Molecular Generation from Mass Spectra at Training and Inference Time
Apr 17, 2026Montgomery Bohde, Hongxuan Liu, Mrunali Manjrekar +4Inference-Time Scaling Molecular Generation
Massachusetts Institute of Technology, Cambridge, MA, United States · Texas A&M University, College Station, TX, United States
Reasoning on the Manifold: Bidirectional Consistency for Self-Verification in Diffusion Language Models
Apr 17, 2026Jiaoyang Ruan, Xin Gao, Yinda Chen +5Diffusion Model Alignment Diffusion Language Model Inference
Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China · University of Science and Technology of China, Hefei, China +1
Neural Continuous-Time Markov Chain: Discrete Diffusion via Decoupled Jump Timing and Direction
Apr 17, 2026Jingyuan Li, Xiaoyi Jiang, Fukang Wen +5Markov Models Diffusion Language Models
2Beijing Institute of Mathematical Sciences and Applications · 3Wuhan University · 4MathonAI +1
Diffusion Language Models for Speech Recognition
Apr 15, 2026Davyd Naveriani, Albert Zeyer, Ralf Schlüter +1Connectionist Temporal Classification Automatic Speech Recognition
Machine Learning and Human Language Technology Group, RWTH Aachen University, Germany · AppTek, Germany
Dataset-Level Metrics Attenuate Non-Determinism: A Fine-Grained Non-Determinism Evaluation in Diffusion Language Models
Apr 15, 2026Zhengyu Fang, Zhimeng Jiang, Huiyuan Chen +5Diffusion Language Models Diffusion Language Model Inference
Department of Computer and Data Sciences, Case Western Reserve University, Cleveland, USA · Department of Computer Science & Engineering, Texas A&M University, College Station, USA · Department of Biochemistry, Case Western Reserve University, Cleveland, USA +2
LoSA: Locality Aware Sparse Attention for Block-Wise Diffusion Language Models
Apr 13, 2026Haocheng Xi, Harman Singh, Yuezhou Hu +9Diffusion Language Model Inference Diffusion Model Inference Acceleration
University of California, Berkeley · FuriosaAI · ICSI +2
LangFlow: Continuous Diffusion Rivals Discrete in Language Modeling
Apr 13, 2026Yuxin Chen, Chumeng Liang, Hangke Sui +4Self-Conditioning Diffusion Language Models
UIUC
CAGenMol: Condition-Aware Diffusion Language Model for Goal-Directed Molecular Generation
Apr 13, 2026Yanting Li, Zhuoyang Jiang, Enyan Dai +3Molecular Generation Molecular Optimization
The Hong Kong University of Science and Technology (Guangzhou) · Jinan University, Guangzhou
Introspective Diffusion Language Models
Apr 13, 2026Yifan Yu, Yuqing Jian, Junxiong Wang +12Diffusion Language Models Diffusion Language Model Decoding
Together AI · University of Illinois Urbana-Champaign · Princeton University +2
Early Decisions Matter: Proximity Bias and Initial Trajectory Shaping in Non-Autoregressive Diffusion Language Models
Apr 12, 2026Jiyeon Kim, Sungik Choi, Yongrae Jo +2Non-Autoregressive Text Generation Diffusion Language Model Decoding
KAIST AI · LG AI Research · University of Illinois Chicago
Lost in Diffusion: Uncovering Hallucination Patterns and Failure Modes in Diffusion Large Language Models
Apr 12, 2026Zhengnan Guo, Fei TanHallucination in Language Models Diffusion Language Models
East China Normal University · Zhejiang University of Technology
From AR to Diffusion: Efficiently Adapting Large Language Models with Strictly Causal and Elastic Horizons
Apr 11, 2026Xiangyu Ma, Teng Xiao, Zuchao Li +1Diffusion Language Models Diffusion Language Model Inference
School of Artificial Intelligence, Wuhan University, Wuhan, China · School of Computer Science, Wuhan University, Wuhan, China

Apr 29, 2026J/K move · Enter open · S save

Language Diffusion Models are Associative Memories Capable of Retrieving Unseen Data

Bao Pham, Mohammed J. Zaki, Luca Ambrogioni, Dmitry Krotov+1

1Rensselaer Polytechnic Institute (RPI) · 2Donders Institute for Brain, Cognition, and Behaviour, Radboud University · 3Dynamical Mind+1

Memorization in Generative Models Neural Network Memorization Diffusion Models Memorization in Language Models Diffusion Language Models Associative Memory+1

Open paper AskPDF arXiv

When do language diffusion models memorize their training data, and how to quantitatively assess their true generative regime? We address these questions by showing that Uniform-based Discrete Diffusion Models (UDDMs) fundamentally behave as Associative Memories (AMs)

\textit{with emergent creative capabilities}

. The core idea of an AM is to reliably recover stored data points as

\textit{memories}

by establishing distinct basins of attraction around them. Historically, models like Hopfield networks use an explicit energy function to guarantee these stable attractors. We broaden this perspective by leveraging the observation that energy is not strictly necessary, as basins of attraction can also be formed via conditional likelihood maximization. By evaluating token recovery of

\textit{training}

and

\textit{test}

examples, we identify in UDDMs a sharp memorization-to-generalization transition governed by the size of the training dataset: as it increases, basins around training examples shrink and basins around unseen test examples expand, until both later converge to the same level. Crucially, we can detect this transition using only the conditional entropy of predicted token sequences: memorization is characterized by vanishing conditional entropy, while in the generalization regime the conditional entropy of most tokens remains finite. Thus, conditional entropy offers a practical probe for the memorization-to-generalization transition in deployed models.