Diffusion Language Model Training

Follow this topic Ask about this topic

Latest papers

40 papers11 papers in the last four weeks, up 175% on the four weeks before. 0.1% of all new papers.

Noise Your Prompt: Noising Conditioning Tokens in Continuous Diffusion Language Models
Oct 6, 2026Justin JungDiffusion Language Model Training Continuous Diffusion Language Models
Lateral Intelligence
Enhancing Diffusion Language Models with Autoregressive Post-Training Weights
Oct 6, 2026Yiming Qin, Ke Wang, Amel Abdelraheem +2Diffusion Language Model Training Model Merging
EPFL, Lausanne, Switzerland
Representation-Space MMD for Diffusion Language Models
Oct 5, 2026Ilya Drobyshevskiy, Ilia Sudakov, Maksim Semenov +7Diffusion Model Alignment Maximum Mean Discrepancy
Yandex Research · HSE University · T-Tech +3
ALoDLM: Adaptively Looped Diffusion Language Models
Oct 3, 2026Liancheng Fang, Zhuowei Li, Youngeun Kim +10Adaptive Computation for Diffusion Models Diffusion Language Models
University of Illinois Chicago · Amazon AGI · Korea University
Scaling and Distilling Text Embeddings for Better Diffusibility
Oct 1, 2026Zekai Zhang, Yunjie Tian, Yanjin He +4Word Embeddings Language Model Distillation
University of Michigan
Clock Diffusion: Efficient Semi-Autoregressive Continuous Diffusion Language Models
Oct 1, 2026Yair Schiff, Omer Belhasin, Roy Uziel +7Diffusion Language Model Inference Language Modeling
NVIDIA · Cornell University
Distribution Matching Distillation for Continuous Diffusion Language Models
Sep 30, 2026Paul Le Van Kiem, Dario Shariatian, Umut Simsekli +1Continuous Diffusion Language Models Diffusion Language Model Distillation
Inria, PSL Research University · Cohere · CMAP, Ecole Polytechnique
Alpha Diffusion Language Models: Factorization Alone Is Not the Problem
Sep 29, 2026Nikita Gushchin, Dmitry Baranchuk, Alexander KorotinDiffusion Language Models Diffusion Language Model Training
Applied AI Institute, Moscow, Russia · AXXX, Russia · Yandex Research
Time-Anchored Diffusion Language Models: Latent-Space Caching for Fast Generation
Sep 29, 2026Joel Anto Paul, Litu Rout, Aditya Akella +1Diffusion Language Model Training Diffusion Language Model Inference
UT Austin
One Latent, Many Tokens: Jointly Learning Compressed Embeddings for Efficient Language Diffusion
Sep 27, 2026Yulin Yuan, Ying Zhang, Xiangming MengDiffusion Language Model Inference Diffusion Model Compression
Zhejiang University · University of Cambridge · ZJU-UIUC Institute, Zhejiang University
Towards Efficient Reasoning: Learning Causal Shortcuts for Diffusion Language Models
Sep 23, 2026Dian Jin, Kairong Han, Baohong Li +5Diffusion Language Model Training Efficient Language Model Reasoning
Zhejiang University · Shanghai AI Laboratory
Block Parallelism For Efficient Distributed Long-Context Diffusion Language Model Training
Sep 16, 2026Tarun Suresh, Pranshu Chaturvedi, Hangoo Kang +4Communication-Efficient Distributed Training Long-Context Language Modeling
Stanford University
DA-DLM: Explicitly Modeling Token Dependencies in Diffusion Language Models
Sep 14, 2026Pengyu Ji, Zichen Zhang, Xiang Hu +1Non-Autoregressive Text Generation Diffusion Language Model Decoding
‡School of Information Science and Technology, ShanghaiTech University · §Shanghai Engineering Research Center of Intelligent Vision and Imaging · ¶Tencent
A Ticket from Marginals to Joints: Coupled-Noise Distillation for One-Step Block Generation in Diffusion Language Models
Sep 6, 2026Lin YaoOne-Step Generative Modeling Diffusion Language Models
School of Computer Science, Shanghai Jiao Tong University, Shanghai, 200240, China · Zhongguancun Academy, Beijing, 100097, China
SelFusion: Self-distillation for Diffusion Language Models
Aug 24, 2026Hyeongsoo Lim, Jinyoung Kim, Eunseo Seo +2Diffusion Language Model Training Diffusion Language Models
Department of Artificial Intelligence, Chung-Ang University
DiffusionGemma Technical Report
Jul 31, 2026DiffusionGemma Team, Adrien Ali Taïga, James Assiene +41Diffusion Language Model Decoding Blockwise Diffusion Language Models
Adaptive Block Diffusion: Resolving Training-Inference Mismatch in Diffusion Language Models
Jun 28, 2026Gagan JainDiffusion Language Model Training Blockwise Diffusion Language Models
Work done while at Microsoft AI.
Sumi: Open Uniform Diffusion Language Model from Scratch
Jun 17, 2026Mengyu Ye, Keito Kudo, Wataru Ikeda +3Language Model Pretraining Diffusion Language Models
1Tohoku University
Learning from the Self-future: On-policy Self-distillation for dLLMs
Jun 16, 2026Yifu Luo, Zeyu Chen, Haoyu Wang +4On-Policy Self-Distillation Diffusion Language Model Training
Tsinghua University · Technical University of Munich · Nanyang Technological University +5
Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation
Jun 4, 2026Xingyu Su, Jacob Helwig, Shubham Parashar +6Language Model Distillation Diffusion Language Models
Department of Computer Science and Engineering, Texas A&M University · Department of Bioinformatics and Systems Medicine,2026 University of Texas Health Science Center at Houston · Department of Electrical and Computer Engineering, Texas A&M UniversityJun 4
Revise, Don't Freeze: Sampler-Matched Training for Self-Correcting Masked Diffusion Language Models
May 31, 2026Longxuan Yu, Shaorong Zhang, Yu Fu +3Masked Diffusion Models Diffusion Model Fine-Tuning
University of California, Riverside · Microsoft
DiLaDiff: Distilled Latent-Augmented Diffusion for Language Modeling
May 22, 2026Jean-Marie Lemercier, Tomas Geffner, Karsten Kreis +3Diffusion Language Models Diffusion Language Model Distillation
NVIDIA
Learnability-Informed Fine-Tuning of Diffusion Language Models
May 21, 2026Shubham Parashar, Atharv Chagi, Jacob Helwig +5Supervised Fine-Tuning Diffusion Language Models
Department of Computer Science and Engineering, Texas A&M University, College Station, TX, USA · Department of Electrical and Computer Engineering, Texas A&M University, College Station, TX, USA
Drifting Objectives for Refining Discrete Diffusion Language Models
May 19, 2026Daisuke Oba, Hiroki Furuta, Naoaki OkazakiDiffusion Model Fine-Tuning Diffusion Language Models
Institute of Science Tokyo · AIST · NII LLMC
Continuous Diffusion Scales Competitively with Discrete Diffusion for Language
May 18, 2026Zhihan Yang, Wei Guo, Shuibai Zhang +5Language Model Scaling Laws Diffusion Language Models
NVIDIA & Cornell · NVIDIA & Georgia Tech · UW-Madison +3
Beyond Mode-Seeking RL: Trajectory-Balance Post-Training for Diffusion Language Models
May 13, 2026Saba Ahmadi, Prasanna Parthasarathi, Yufei CuiRL for Language Model Reasoning RL for Diffusion Models
Noah’s Ark Lab
Understanding and Accelerating the Training of Masked Diffusion Language Models
May 13, 2026Chunsan Hong, Sanghyun Lee, Chieh-Hsin Lai +5Masked Diffusion Models Diffusion Language Model Training
1KAIST · 2Sony AI · University of Tokyo +1
How to Train Your Latent Diffusion Language Model Jointly With the Latent Space
May 8, 2026Viacheslav Meshchaninov, Alexander Shabalin, Egor Chimbulatov +4Non-Autoregressive Text Generation Latent Diffusion Models
Constructor University, Bremen, Germany · HSE University, Moscow, Russia · Applied AI Institute, Moscow, Russia +2
TextLDM: Language Modeling with Continuous Latent Diffusion
May 8, 2026Jiaxiu Jiang, Jingjing Ren, Wenbo Li +10Diffusion Transformer Latent Diffusion Models
Joy Future Academy · HIT · HKUST(GZ)
Don't Retrain, Align: Adapting Autoregressive LMs to Diffusion LMs via Representation Alignment
May 7, 2026Fred Zhangzhi Peng, Alexis Fox, Anru R. Zhang +1Masked Diffusion Models Diffusion Language Models
Duke University · AITHYRA

Oct 6, 2026J/K move · Enter open · S save

Noise Your Prompt: Noising Conditioning Tokens in Continuous Diffusion Language Models

Justin Jung

Lateral Intelligence

Diffusion Language Model Training Continuous Diffusion Language Models Conditional Diffusion Models

Open paper AskPDF arXiv

We revisit a standard accepted practice in the continuous diffusion language model literature of fixing conditioning prompt tokens clean during training. We make a very simple modification: also noise the conditioning prompt tokens during training. We demonstrate that under this modified training objective, we achieve better generalization in combinatorial reasoning tasks such as Sudoku and N-Queens, with the largest gains on harder variants (

3.73\% \to 24.65\%

solve rate on Sudoku Hard), and increased diversity of generated solutions (

50.60\% \to 73.79\%

coverage on 10x10 N-Queens). We also show measurable improvements to natural language generation quality in modest dataset regimes with Gigaword summarization, but notably demonstrate that gains do not transfer to all natural language tasks (e.g open ended dialogue generation). Our method is a single line change to the training objective, requires no additional inference costs by default, and provides the flexibility of classifier-free guidance inspired guided sampling. Our code is publicly available.