Diffusion Language Models

Follow this topic Ask about this topic

Diffusion language models generate text by starting from corrupted or masked sequences and repeatedly refining them, using context from both earlier and later positions. Main approaches include discrete masked diffusion, continuous latent or flow-based models, and block or hybrid designs that combine parallel refinement with autoregressive structure. Work explores adaptive token ordering, remasking, distillation, and caching to improve reasoning, code generation, multimodal tasks, and inference speed. A central tension is balancing faster, more parallel decoding against token dependencies and output quality, while reliable evaluation and the cost of training from scratch remain open challenges.

Latest papers

465 papers87 papers in the last four weeks, up 129% on the four weeks before. 0.5% of all new papers.

LLaCO: Continuous Diffusion for Large Language Models with Competition-Aware Semantic Scheduling
Oct 23, 2026Zijian Jia, Peixuan Zhang, Genying Li +5Diffusion Language Models Continuous Diffusion Language Models
School of Artificial Intelligence, Beijing University of Posts and Telecommunications · Yuanpei College, Peking University · Beijing Key Laboratory of Multimodal Data Intelligent Perception and Governance +3
Watermarking Diffusion Language Models with Learned Foresight and Balanced Bands
Oct 23, 2026Hyojun Kim, Hyeseon An, Yo-Sub HanLLM Watermarking Diffusion Language Models
Yonsei University, Seoul, Republic of Korea
DELA: Learning Controllable Edits for Diffusion Language Models
Oct 23, 2026Xinsong Feng, Chenan Wang, Leshu Tang +4Diffusion Language Models Diffusion Language Model Decoding
Georgia Institute of Technology · William & Mary · UCLA +1
What Makes a Good Unmasking Policy? A Controlled Study of Learned Unmasking in Diffusion Language Models
Oct 22, 2026Jinya Sakurai, Xun Xu, Jaehong YoonDiffusion Language Model Decoding Diffusion Language Models
NTU Singapore · Institute for Advanced Intelligence and Computing (IAIC), A*STAR · Centre for Frontier AI Research (CFAR), A*STAR
Harmful Responses Are Jailbreak Templates: Transferring Autoregressive Jailbreaks to Diffusion Language Models
Oct 22, 2026Jinwei Gan, Zifeng Cheng, Zhiwei Jiang +3LLM Jailbreak Attacks Adversarial Attacks on Diffusion Models
State Key Laboratory for Novel Software Technology, Nanjing University · Nanjing University of Posts and Telecommunications
Trion-1: Unifying Language, Protein Sequence, and Structure with Diffusion Language Models
Oct 22, 2026Tianyu Lu, Kaicheng Yang, Jiangtao Feng +7Protein Language Models Protein Structure Generation
Different Tasks, Different Timesteps: Temporal Task-Aware Quantization for Diffusion Language Models
Oct 22, 2026Ofek Avivi, Eliron Rahimi, Avi Mendelson +1Neural Network Quantization Mixed-Precision Quantization
Technion – Israel Institute of Technology · INSIGHT Lab, Ben-Gurion University of the Negev
SetCommit: Coordinated Token Commitments for Few-Step Diffusion Language Model Decoding
Oct 22, 2026Yuannuo Feng, Yubing Ye, Wenshuai Yao +4Diffusion Language Model Inference Diffusion Language Models
Beihang University · The University of Hong Kong · Peking University
DConstrain: Compiling Context-Parameterized Transitions for Diffusion Language Model Constrained Decoding
Oct 22, 2026Shidong Cao, Jing Ma, Wenhao Chai +3Constrained Decoding Grammar-Constrained Decoding
Hong Kong Baptist University · Princeton University · Ant Group
Random Remasking Scales Masked Diffusion Inference
Oct 22, 2026Zhaoyi Zhou, Frank Wu, Andrea ZanetteMasked Diffusion Models Diffusion Language Model Inference
Carnegie Mellon University
Loopholing All the Way: Language Modeling with pure Self-Conditioning
Oct 21, 2026Adrien Deschenaux, Nadezhda Ilieva, Justin Deschenaux +6Diffusion Language Models Continuous Diffusion Language Models
EPFL, Lausanne, Switzerland · University of Basel, Switzerland · ELLIS Institute Tübingen, MPI for Intelligent Systems, Tübingen AI Center, Germany +1
Looped Uniform Diffusion Language Models
Oct 21, 2026Saiteja Utpala, Chieh-Hsin Lai, Yuki Mitsufuji +1Diffusion Language Models Diffusion Language Model Training
MBZUAI · Sony Group Corporation · MIT
A Diagnostic Study of Faithfulness in Diffusion Language Models for Table-to-Text Generation
Oct 21, 2026Zaynab Merimi, Noussayba El Marrakchi, Arthur SatoufText Generation Grounded Text Generation
CY Tech, CY Cergy Paris Université, France · MILA – Quebec AI Institute, Canada · Université Paris-Saclay & CentraleSupélec, France
Hierarchical Masked Diffusion Language Models
Oct 21, 2026Parul Gupta, Satwik, Avni Aggarwal +5Masked Diffusion Models Diffusion Language Model Decoding
Indian Institute of Technology Delhi · IBM Research
DiffResearch: Accelerating Deep Research with Diffusion Language Models
Oct 20, 2026Alan-Barsag Gazzaev, Alexey Gavrilov, Mikhail Mozikov +1Diffusion Language Models Diffusion Language Model Decoding
ITMO University, Kronverksky Pr. 49, bldg. A, St. Petersburg, 197101, Russia · AXXX, 5 Nizhny Susalny Pereulok, bldg. 19, Moscow, 105064, Russia
Hidden Messages Between Diffusion Language Model Agents: Causal Evidence for Latent Communication
Oct 20, 2026Alan-Barsag Gazzaev, Alexey Gavrilov, Sergey Muravyov +1Multi-Agent LLMs Diffusion Language Models
ITMO University · HSE University
Beyond the Sequentiality Bottleneck: A Hybrid Diffusion-Autoregressive Framework for Efficient Multimodal Reasoning
Oct 20, 2026Wei Cai, Jian Zhao, Yuchen Yuan +1Hybrid Language Models Parallel Reasoning in Language Models
Peking University · Institute of Artificial Intelligence (TeleAI), China Telecom
Self-Decomposing Diffusion for Code Generation
Oct 20, 2026Lize Shao, Zichen Xie, Zhizhen Chen +2Code Generation LLM Code Generation
University of Virginia
Beyond Model Weights: Sampling-Dependent Fairness in Diffusion Language Models
Oct 20, 2026Koshiro Saito, Daisuke Oba, Danushka Bollegala +1Social Bias in Language Models Diffusion Language Models
Institute of Science Tokyo · University of Liverpool · Amazon +2
Factual Access States in Diffusion Language Models: From Mechanism to Knowledge Editing and Retrieval
Oct 20, 2026Yuchen Guo, Zhuoyuan Mao, Yiwen GuoFactual Knowledge in Language Models Diffusion Language Models
The University of Tokyo · LIGHTSPEED · Independent Researcher
Guided Diffusion for Trustworthy Synthetic Text Generation
Oct 19, 2026Qi Pang, Hanshen Xiao, Andre Manoel +4Synthetic Data Generation for Language Models Differentially Private Synthetic Data Generation
NVIDIA · Carnegie Mellon University · Purdue University
Breaking Diffusion: Parallel Reasoning with Draft Refinement
Oct 19, 2026Jaesang Yu, Seoyeon Byun, Yun Wang +2Diffusion Language Models Parallel Reasoning in Language Models
KAIST · Microsoft Research Asia · SkillBench
Diffusion Language Models Change Speed Across Tasks and Languages
Oct 19, 2026Mohsen Fayyaz, Ali Modarressi, Daniel Israel +2Diffusion Language Models Diffusion Language Model Inference
University of California, Los Angeles · CIS, LMU Munich · Munich Center for Machine Learning
Learning to Couple: Context-Adaptive Joint Denoising for Diffusion Language Models
Oct 19, 2026Zilei Shao, Ian Li, Heng Zhao +4Diffusion Language Model Inference Diffusion Language Models
University of California, Los Angeles · University of California, San Diego · University of Virginia +1
Enhanced Masked Diffusion Unlearning
Oct 18, 2026Tomer Garber, Tom TirerMachine Unlearning Diffusion Model Unlearning
Faculty of Engineering, Bar-Ilan University, Ramat Gan, Israel · Red Hat
Who Gets Unmasked? Sampler-Induced Bias in Diffusion Language Models
Oct 17, 2026Xujun Che, Farsheed Haque, Depeng Xu +2Diffusion Language Models Language Model Bias Evaluation
University of North Carolina at Charlotte, Charlotte, NC, USA
DiffuPlan: Editable Planning State for Diffusion Language Model Agents
Oct 17, 2026Qisheng Zhou, Zhen Xiong, Qiaoyu TanLLM Agents LLM Agent Planning
New York University Shanghai · New York University
CoTA++: Understanding the Repeat Curse in dMLLMs from an Information Flow Perspective
Oct 16, 2026Qiyan Zhao, Xiaofeng Zhang, Xu-Yao Zhang +4Diffusion Language Model Decoding Diffusion Language Models
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China. · Q. Zhao and X.-Y. Zhang are the School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS), Beijing 100049, China · Shanghai Jiao Tong University, Shanghai 200240, China. +1
On the SPOT: Enhancing Speculative Decoding for Diffusion Language Models via Commit-Order Prediction
Oct 15, 2026Eliezer Shlomi, Eliron Rahimi, Avi Mendelson +1Speculative Decoding for Diffusion Language Models Diffusion Language Models
Technion – Israel Institute of Technology · INSIGHT Lab, School of Electrical and Computer Engineering, Ben-Gurion University of the Negev
Designing Transformers for Continuous Diffusion Language Models
Oct 15, 2026Weijian Luo, Yujian Chen, Hao Jiang +2Diffusion Language Models Continuous Diffusion Language Models
Dots Studio, Xiaohongshu · Polyadic Research Group · Peking University +2

Oct 23, 2026J/K move · Enter open · S save

LLaCO: Continuous Diffusion for Large Language Models with Competition-Aware Semantic Scheduling

Zijian Jia, Peixuan Zhang, Genying Li, Di Wu+4

School of Artificial Intelligence, Beijing University of Posts and Telecommunications · Yuanpei College, Peking University · Beijing Key Laboratory of Multimodal Data Intelligent Perception and Governance+3

Diffusion Language Models Continuous Diffusion Language Models Diffusion Models Diffusion Timestep Scheduling

Open paper AskPDF arXiv

Continuous diffusion models have emerged as a promising non-autoregressive paradigm for text generation over continuous embedding spaces. Along the continuous trajectory, the residual token identity directly shapes the denoising objective. However, the geometric trajectory misaligns with temporal progression, triggering severe temporal compression. In this paper, we show that embedding norm decay causes Euclidean distance to yield spurious overtaking, indicating that token competition must be evaluated directionally. In this space, a candidate overtakes the source token only when the effective noise magnitude surpasses their semantic separation. We formalize these pairwise dynamics as the Competition Threshold Theory (CTT). Crucially, high-dimensional overtaking requires extreme noise levels, causing standard trajectories to concentrate identity decay into a narrow window near the noisy endpoint. To resolve this, we propose Semantic Retention Scheduling (SRS) to align time allocation with the progression of identity decay, and demonstrate that existing token identity metrics are fundamentally statistical aggregations of these pairwise crossings. Building on these theoretical insights, we introduce LLaCO, a generative framework powered by competition-aware semantic scheduling. LLaCO adapts a pretrained backbone to continuous representations via an embedding directional adapter and timestep-conditioned modulation. To operate within the directional space, LLaCO employs a spherical noising process formulated as a geodesic interpolation toward a mask-biased endpoint, eliminating norm decay while preserving accurate token competition. Guided by SRS, LLaCO reallocates the denoising trajectory proportionally to the logarithmic rank decay. Instantiated as an 8B model, LLaCO outperforms previous continuous diffusion baselines and rivals discrete diffusion models of comparable scale.