Diffusion Language Models

Follow this topic Ask about this topic

Diffusion language models generate text by starting from corrupted or masked sequences and repeatedly refining them, using context from both earlier and later positions. Main approaches include discrete masked diffusion, continuous latent or flow-based models, and block or hybrid designs that combine parallel refinement with autoregressive structure. Work explores adaptive token ordering, remasking, distillation, and caching to improve reasoning, code generation, multimodal tasks, and inference speed. A central tension is balancing faster, more parallel decoding against token dependencies and output quality, while reliable evaluation and the cost of training from scratch remain open challenges.

Latest papers

467 papers99 papers in the last four weeks, up 130% on the four weeks before. 0.6% of all new papers.

LaViDa-R1: Advancing Reasoning for Unified Multimodal Diffusion Language Models
Feb 15, 2026Shufan Li, Yuchen Zhu, Jiuxiang Gu +6Unified Multimodal Models Multimodal Reasoning
1Adobe · 2UCLA · 3Georgia Tech
Few-Step Diffusion Language Models via Trajectory Self-Distillation
Feb 12, 2026Tunyu Zhang, Xinxi Zhang, Ligong Han +10Diffusion Model Distillation Few-Step Diffusion Sampling
Rutgers University · Red Hat AI Innovation · MIT-IBM Watson AI Lab
Just on Time: Token-Level Early Stopping for Diffusion Language Models
Feb 11, 2026Zakhar Kohut, Severyn Shykula, Mykola Vysotskyi +7Diffusion Language Model Inference Diffusion Language Model Decoding
SoftServe Inc.
FlashBlock: Attention Caching for Efficient Long-Context Block Diffusion
Feb 5, 2026Zhuokun Chen, Jianfei Cai, Bohan ZhuangVideo Diffusion Models Self-Attention
1Monash University · 2Zhejiang University
ILRR: Inference-Time Steering Method for Masked Diffusion Language Models
Jan 29, 2026Eden Avrahami, Eliya NachmaniMasked Diffusion Models Language Model Steering
School of Computer Science, Tel Aviv University, Israel · Department of Electrical and Computer Engineering, Ben Gurion University, Beer Sheva, Israel.
Mechanism Shift During Post-training from Autoregressive to Masked Diffusion Language Models
Jan 21, 2026Injin Kong, Hyoungjoon Lee, Yohan JoMasked Diffusion Models Autoregressive Diffusion
Graduate School of Data Science, Seoul National University · Department of Biosystems & Biomaterials Science and Engineering, Seoul National University
A Comprehensive Study on Visual Token Redundancy for Discrete Diffusion-based Multimodal Large Language Models
Nov 19, 2025Duo Li, Zuhao Yang, Xiaoqin Zhang +2Multimodal Large Language Models Diffusion Language Model Inference
CCDS, NTU, Singapore · CCST, ZJUT, China · Terminus AI Lab, UCAS, China
ParallelBench: Understanding the Trade-offs of Parallel Decoding in Diffusion LLMs
Oct 6, 2025Wonjun Kang, Kevin Galim, Seunghyuk Oh +8Diffusion Language Model Inference Parallel Decoding
FuriosaAI · Seoul National University · UW-Madison +4
Demystifying MaskGIT Sampler and Beyond: Adaptive Order Selection in Masked Diffusion
Oct 6, 2025Satoshi Hayakawa, Yuhta Takida, Masaaki Imaizumi +2Diffusion Model Sampling Diffusion Sampling
Sony Group Corporation · Sony AI · The University of Tokyo
RFG: Self-Improving Diffusion Large Language Models with Reward-Free Guidance
Sep 29, 2025Tianlang Chen, Minkai Xu, Jure Leskovec +1Diffusion Guidance Diffusion Language Model Inference
Stanford University
Watermarking Diffusion Language Models
Sep 29, 2025Thibaud Gloaguen, Robin Staab, Nikola Jovanović +1Diffusion Model Watermarking LLM Watermarking
ETH Zurich
Enabling Approximate Joint Sampling in Diffusion LMs
Sep 25, 2025Parikshit Bansal, Sujay SanghaviMasked Diffusion Models Diffusion Language Model Decoding
UT Austin
Cross-Modality Controlled Molecule Generation with Diffusion Language Model
Aug 20, 2025Yunzhe Zhang, Yifei Wang, Khanh Vinh Nguyen +1Diffusion Model Guidance Multimodal Generation
Brandeis University, Waltham, MA, USA
Any-Order GPT as Masked Diffusion Model: Decoupling Formulation and Architecture
Jun 24, 2025Shuchen Xue, Tianyu Xie, Tianyang Hu +5Masked Diffusion Models Decoder-Only Language Models
University of Chinese Academy of Sciences · Academy of Mathematics and Systems Science, Chinese Academy of Sciences · School of Mathematical Sciences, Peking University +2
DLM-One: Diffusion Language Models for One-Step Sequence Generation
May 30, 2025Tianqi Chen, Shujian Zhang, Mingyuan ZhouDiffusion Language Models Diffusion Model Distillation
The University of Texas at Austin, Austin, TX 78712, USA
Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding
May 22, 2025Runpeng Yu, Xinyin Ma, Xinchao WangMultimodal Large Language Models LLM Inference Acceleration
National University of Singapore
LaViDa: A Large Diffusion Language Model for Multimodal Understanding
May 22, 2025Shufan Li, Konstantinos Kallidromitis, Hritik Bansal +7Multimodal Understanding Vision-Language Models
1UCLA · 2Panasonic AI Research · 4Salesforce Research +1

Feb 15, 2026J/K move · Enter open · S save

LaViDa-R1: Advancing Reasoning for Unified Multimodal Diffusion Language Models

Shufan Li, Yuchen Zhu, Jiuxiang Gu, Kangning Liu+5

1Adobe · 2UCLA · 3Georgia Tech

Unified Multimodal Models Multimodal Reasoning Multimodal Generation RL for Diffusion Models Diffusion Language Models Multimodal Diffusion Models+1

Open paper AskPDF arXiv

Diffusion language models (dLLMs) recently emerged as a promising alternative to auto-regressive LLMs. The latest works further extended it to multimodal understanding and generation tasks. In this work, we propose LaViDa-R1, a multimodal, general-purpose reasoning dLLM. Unlike existing works that build reasoning dLLMs through task-specific reinforcement learning, LaViDa-R1 incorporates diverse multimodal understanding and generation tasks in a unified manner. In particular, LaViDa-R1 is built with a novel unified post-training framework that seamlessly integrates supervised finetuning (SFT) and multi-task reinforcement learning (RL). It employs several novel training techniques, including answer-forcing, tree search, and complementary likelihood estimation, to enhance effectiveness and scalability. Extensive experiments demonstrate LaViDa-R1's strong performance on a wide range of multimodal tasks, including visual math reasoning, reason-intensive grounding, and image editing.