Diffusion Language Models

Follow this topic Ask about this topic

Diffusion language models generate text by starting from corrupted or masked sequences and repeatedly refining them, using context from both earlier and later positions. Main approaches include discrete masked diffusion, continuous latent or flow-based models, and block or hybrid designs that combine parallel refinement with autoregressive structure. Work explores adaptive token ordering, remasking, distillation, and caching to improve reasoning, code generation, multimodal tasks, and inference speed. A central tension is balancing faster, more parallel decoding against token dependencies and output quality, while reliable evaluation and the cost of training from scratch remain open challenges.

Latest papers

473 papers99 papers in the last four weeks, up 130% on the four weeks before. 0.6% of all new papers.

Diffusion-GR2: Diffusion Generative Reasoning Re-ranker
Jul 1, 2026Zhuoxuan Zhang, Kangqi Ni, Yuhang Chen +12Blockwise Diffusion Language Models Efficient Language Model Reasoning
Yang · 2UNC Chapel Hill · 1Meta AI
Low Perplexity is Repetition: A One-Dimensional Self-Conditioning Attractor in Continuous Diffusion LMs
Jul 1, 2026Shuai Zhang, Zijie Chen, Hongliang He +2Repetition in Language Generation Language Model Generation Evaluation
Zhejiang University · Westlake University · Ant Group
SLIM-RL: Risk-Budgeted Random-Masking RL for Diffusion LLMs Without Trajectory Slicing
Jun 30, 2026Ruikang Zhao, Zhenting Wang, Han Gao +1Reinforcement Learning RL for Language Model Reasoning
Technical University of Denmark · MBZUAI Institute of Foundation Models · Iowa State University +1
TAG-DLM: Diffusion Language Models for Text-Attributed Graph Learning
Jun 30, 2026Lingjie Chen, Yuanchen Bei, Haobo Xu +3Masked Diffusion Models Link Prediction
UIUC · VISA
Why Do Few-Step Text Latents Fail When Image Latents Work? Non-Commitment at Sharp Categorical Readouts
Jun 29, 2026Zhongyao WangFew-Step Diffusion Sampling Generative Modeling
Fudan University · Shanghai Innovation Institute
Adaptive Block Diffusion: Resolving Training-Inference Mismatch in Diffusion Language Models
Jun 28, 2026Gagan JainDiffusion Language Model Training Blockwise Diffusion Language Models
Work done while at Microsoft AI.
Multi-Block Diffusion Language Models
Jun 28, 2026Yijie Jin, Jiajun Xu, Yuxuan Liu +8Blockwise Diffusion Language Models Diffusion Language Model Decoding
Shanghai Jiao Tong University · Xi’an Jiao Tong University · Huawei
DiLaServe: High SLO Attainment Serving for Diffusion Language Models
Jun 27, 2026Tzu-Tao Chang, Benjamin Yuanyang Hong, Kiet Pham +1LLM Serving Diffusion Model Serving
University of Wisconsin-Madison USA
Bifocal Diffusion Language Models: Asymmetric Bidirectional Context for Parallel Generation
Jun 26, 2026Yuhang Chen, Xianfeng Wu, Jinhao Duan +11Diffusion Language Model Inference Parallel Decoding
University of North Carolina at Chapel Hill · Meta AI
Nemotron-Labs-TwoTower: Diffusion Language Modeling with Pretrained Autoregressive Context
Jun 25, 2026Fitsum Reda, John Kamalu, Roger Waleffe +3Autoregressive Diffusion Blockwise Diffusion Language Models
NVIDIA
Improved Large Language Diffusion Models
Jun 24, 2026Shen Nie, Qiyang Min, Shaoxuan Xu +7Language Model Pretraining Masked Diffusion Models
Gaoling School of Artificial Intelligence, Renmin University of China · Beijing Key Laboratory of Research on Large Models and Intelligent Governance · Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE +1
Accelerating Disaggregated RL for Visual Generative LLMs with Diffusion-Based Parallelism and Trainer-Assisted Generation
Jun 23, 2026Sijie Wang, Zhengyu Qing, Zhiqiang Tan +6Reinforcement Learning Distributed RL
School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen · Data Science and Analytics Thrust, The Hong Kong University of Science and Technology (Guangzhou) · Huawei Technologies Ltd.
When Top-1 Fails: Calibrating LoRA Monitors for Masked Diffusion LMs
Jun 23, 2026Lucky Verma, Pratik YadavDiffusion Model Fine-Tuning Diffusion Language Models
Independent Researcher · University of Maryland, Baltimore County
Scheduling Thoughts: Learning the Order of Thought in Diffusion Language Models
Jun 22, 2026Jiawei Xu, Minghui Liu, Aakriti Agrawal +2Masked Diffusion Models Language Model Decoding
University of Maryland, College Park · University of California, Los Angeles.
When to Plan, When to Polish: Noise Level as a Granularity Axis for Diffusion Language Models
Jun 19, 2026Peihong Li, Yuanjie Shi, Yan YanLLM Planning Diffusion Language Model Decoding
Washington State University
How Transparent is DiffusionGemma?
Jun 18, 2026Joshua Engels, Callum McDougall, Bilal Chughtai +11LLM Interpretability Diffusion Language Models
Google DeepMind
Diffusion Language Models: An Experimental Analysis
Jun 17, 2026Thomas Bertolani, Davide Bucciarelli, Leonardo Zini +2Diffusion Language Model Inference Diffusion Language Model Decoding
University of Modena and Reggio Emilia, Italy · University of Pisa, Italy
Diffusion-Proof: Recipe for Formal Theorem Proving Beyond Auto-Regressive Generation
Jun 17, 2026Ruida Wang, Rui Pan, Pengcheng Wang +2Automated Theorem Proving Lean Theorem Proving
University of Illinois Urbana-Champaign · NVIDIA
DreamReasoner-8B: Block-Size Curriculum Learning for Diffusion Reasoning Models
Jun 17, 2026Zirui Wu, Lin Zheng, Jiacheng Ye +5Blockwise Diffusion Language Models Efficient Language Model Reasoning
The University of Hong Kong · Peking University
Sumi: Open Uniform Diffusion Language Model from Scratch
Jun 17, 2026Mengyu Ye, Keito Kudo, Wataru Ikeda +3Language Model Pretraining Diffusion Language Models
1Tohoku University
Learning from the Self-future: On-policy Self-distillation for dLLMs
Jun 16, 2026Yifu Luo, Zeyu Chen, Haoyu Wang +4On-Policy Self-Distillation Diffusion Language Model Training
Tsinghua University · Technical University of Munich · Nanyang Technological University +5
VoidPadding: Let [VOID] Handle Padding in Masked Diffusion Language Models so that [EOS] Can Focus on Semantic Termination
Jun 16, 2026Chunyu Liu, Zhengyang Fan, Kaisen Yang +1Masked Diffusion Models Diffusion Language Model Inference
Tsinghua University
Self-Generated Error Training for Token Editing in Diffusion Language Models
Jun 15, 2026Lin YaoLLM Self-Correction Diffusion Model Fine-Tuning
School of Computer Science, Shanghai Jiao Tong University, Shanghai, 200240, China · Zhongguancun Academy, Beijing, 100097, China
LESS Is More: Mutual-Stability Sampling for Diffusion Language Models
Jun 15, 2026Amr Mohamed, Guokan Shang, Michalis VazirgiannisMasked Diffusion Models Diffusion Language Model Decoding
1MBZUAI · 2Ecole Polytechnique
Follow the Latent Roadmap: Navigating Revocable Decoding for Diffusion LLMs with Anchor Tokens
Jun 15, 2026Yizhen Yao, Qinglin Zhu, Runcong Zhao +4Diffusion Language Model Inference Diffusion Language Model Decoding
King’s College London · The Chinese University of Hong Kong · The Alan Turing Institute, UK
Multi-Turn Reflective Masking Elicits Reasoning in Mask Diffusion Models
Jun 15, 2026Yanming Zhang, Yihan Bian, Jingyuan Qi +3Masked Diffusion Models Diffusion Model Fine-Tuning
University of Maryland · Virginia Tech · Intuit +2
Who Should Lead Decoding Now? Tracking Reliable Trajectories for Ensembling Masked Diffusion Language Models
Jun 15, 2026Heecheol Yun, Joonhyung Park, Joowon Kim +1Masked Diffusion Models Diffusion Language Model Decoding
1KAIST · 2AITRICS
Mean-Field Parallel Decoding for Discrete Diffusion Language Models
Jun 14, 2026Tamim Zoabi, Ameen Ali, Liran Ringel +1Parallel Decoding Diffusion Language Model Decoding
School of Electrical & Computer Engineering Tel Aviv University, Israel · School of Computer Science and AI Tel Aviv University, Israel · Department of Computer Science Technion, Israel Institute of Technology, Israel
Semantic DLM+: Improving Diffusion Language Models through Bias-variance Trade-off in Transition Kernel Design
Jun 13, 2026Keyue Jiang, Yuxiang Wang, Yanan Zhao +7Diffusion Sampling Diffusion Language Models
Alibaba Group · Fudan University · Nanyang Technological University +1
Neither Parallel Nor Sequential: How DiffusionGemma Actually Commits Tokens
Jun 12, 2026Ali Asaria, Tony Salomone, Deep GandhiMasked Diffusion Models Diffusion Sampling
Transformer Lab

Jul 1, 2026J/K move · Enter open · S save

Diffusion-GR2: Diffusion Generative Reasoning Re-ranker

Zhuoxuan Zhang, Kangqi Ni, Yuhang Chen, Mingfu Liang+11

Yang · 2UNC Chapel Hill · 1Meta AI

Blockwise Diffusion Language Models Efficient Language Model Reasoning LLM Reranking Diffusion Language Model Inference Large Language Model-Based Recommendation Diffusion Language Models

Open paper AskPDF arXiv

Generative reasoning re-rankers achieve strong recommendation accuracy by emitting a chain-of-thought before re-ordering a candidate list, but they are slow at inference: an autoregressive (AR) decoder spends one sequential forward pass per reasoning token, and the reasoning trace far exceeds the ranking it produces. To reduce this cost, block-diffusion language models decode many positions in parallel over a few denoising steps and are substantially faster, yet naively converting an AR re-ranker into one opens two accuracy gaps: (1) a structural gap: answer positions are denoised in parallel and scored independently, so the decoder emits invalid rankings (duplicated, dropped, or out-of-set identifiers) that AR avoids through left-to-right masking; and (2) a distributional gap: fine-tuning the converted model on fixed teacher trajectories is off-policy relative to its own decoding at inference, leaving a residual accuracy gap. To close both gaps while keeping the speedup, we propose Diffusion-GR2, a recipe that converts our AR reasoning re-ranker (GR2) into a block-diffusion re-ranker. First, conversion fine-tuning (CFT) adapts the AR-initialized diffusion model to denoise the answer into a valid permutation on its own, without an external constrained decoder. Next, on-policy distillation (OPD) then supervises the model on its own decoded trajectories with dense per-token targets from the AR teacher. Finally, we apply a reinforcement-learning (RL) stage against a re-ranking reward on top of OPD's on-policy policy. Experiments on Amazon Beauty demonstrate that Diffusion-GR2 recovers to near-parity with the AR re-ranker, while block-parallel decoding raises decode throughput by

2.4

3.5\times

at the model's reasoning output length. Ablations show that CFT recovers most of the conversion gap, and that on-policy distillation further closes it to the AR reference.