Diffusion Language Models

Follow this topic Ask about this topic

Diffusion language models generate text by starting from corrupted or masked sequences and repeatedly refining them, using context from both earlier and later positions. Main approaches include discrete masked diffusion, continuous latent or flow-based models, and block or hybrid designs that combine parallel refinement with autoregressive structure. Work explores adaptive token ordering, remasking, distillation, and caching to improve reasoning, code generation, multimodal tasks, and inference speed. A central tension is balancing faster, more parallel decoding against token dependencies and output quality, while reliable evaluation and the cost of training from scratch remain open challenges.

Latest papers

473 papers99 papers in the last four weeks, up 130% on the four weeks before. 0.6% of all new papers.

Fast-dDrive: Efficient Block-Diffusion VLM for Autonomous Driving
May 22, 2026Kewei Zhang, Jin Wang, Sensen Gao +9VLMs for Autonomous Driving End-to-End Autonomous Driving
1Peking University · 3The University of Hong Kong · 2NVIDIA +1
Learned Relay Representations for Forward-Thinking Discrete Diffusion Models
May 21, 2026Benjamin Rozonoyer, Jacopo Minniti, Dhruvesh Patel +4Masked Diffusion Models Diffusion Language Model Inference
University of Massachusetts Amherst · University of Toronto · Stanford University +3
Learnability-Informed Fine-Tuning of Diffusion Language Models
May 21, 2026Shubham Parashar, Atharv Chagi, Jacob Helwig +5Supervised Fine-Tuning Diffusion Language Models
Department of Computer Science and Engineering, Texas A&M University, College Station, TX, USA · Department of Electrical and Computer Engineering, Texas A&M University, College Station, TX, USA
Uniform Diffusion Models Revisited: Leave-One-Out Denoiser and Absorbing State Reformulation
May 21, 2026Samson Gourevitch, Yazid Janati, Dario Shariatian +4Diffusion Model Sampling Diffusion Language Models
CMAP, Ecole polytechnique · Institute of Foundation Models · Inria, PSL Research University +2
Efficient Long-Context Modeling in Diffusion Language Models via Block Approximate Sparse Attention
May 19, 2026Wenhu Zhang, Yiming Wu, Huanyu Wang +6Self-Attention Long-Context Language Modeling
The Hong Kong University of Science and Technology · The University of Hong Kong · Zhejiang University +1
Drifting Objectives for Refining Discrete Diffusion Language Models
May 19, 2026Daisuke Oba, Hiroki Furuta, Naoaki OkazakiDiffusion Model Fine-Tuning Diffusion Language Models
Institute of Science Tokyo · AIST · NII LLMC
Backdooring Masked Diffusion Language Models
May 19, 2026Daniel Yiming Cao, Chengzhong Wang, Sheng-Yen Chou +3LLM Backdoor Attacks Masked Diffusion Models
Cornell University · Virginia Tech · IBM Research
Continuous Diffusion Scales Competitively with Discrete Diffusion for Language
May 18, 2026Zhihan Yang, Wei Guo, Shuibai Zhang +5Language Model Scaling Laws Diffusion Language Models
NVIDIA & Cornell · NVIDIA & Georgia Tech · UW-Madison +3
Machine Unlearning for Masked Diffusion Language Models
May 18, 2026Georu Lee, Seungwon Jeong, Hoki Kim +2Masked Diffusion Models Diffusion Model Unlearning
1Dongguk University-Seoul · 2Chung-Ang University · 3Korea Institute for Advanced Study
Elastic-dLLM: Position Preserving Context Compression and Augmentation of Diffusion LLMs
May 18, 2026Junyi Wu, Tianchen Zhao, Shaoqiu Zhang +3Masked Diffusion Models Long-Context Language Modeling
Tsinghua University · Shanghai Jiao Tong University · Infinigence AI
Prompt Compression in Diffusion Large Language Models: Evaluating LLMLingua-2 on LLaDA
May 18, 2026Sterling Huang, Abigayle Brown, Jiyoo Noh +4LLM Evaluation LLM Compression
University of Toronto Toronto, Canada · King Mongkut’s University of Technology Thonburi Bangkok, Thailand
AnchorDiff: Topology-Aware Masked Diffusion with Confidence-based Rewriting for Radiology Report Generation
May 16, 2026Shiying Yu, Jielei Wang, Guoming LuRadiology Report Generation Diffusion Language Model Decoding
University of Electronic Science and Technology of China
Roll Out and Roll Back: Diffusion LLMs are Their Own Efficiency Teachers
May 16, 2026Fanqin Zeng, Feng Hong, Geng Yu +6Diffusion Model Distillation Diffusion Language Model Decoding
Shanghai Jiao Tong University, Shanghai, China · Apple MLR, Cupertino, CA, USA · Tongji University, Shanghai, China +2
Dynamic Chunking for Diffusion Language Models
May 15, 2026Yichen Zhu, Xiaoming Shi, Peng Zhao +3Document Chunking Autoregressive Diffusion
CSE, HKUST · Xiaohongshu Inc. · Alibaba group +1
Ligand-Conditioned Discrete Diffusion for Protein Sequence-Structure Co-Design
May 15, 2026Chen Wei, Fanding Xu, Minghao Sun +5Protein Language Models Diffusion Language Model Decoding
Xi’an University of Posts & Telecommunications · National University of Singapore · Xi’an Jiaotong University +1
PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding
May 15, 2026Shengyin Sun, Yiming Li, Renxi Liu +7Parallel Speculative Decoding Masked Diffusion Models
Department of Computer Science City University of Hong Kong · Huawei Technologies · City University of Hong Kong
When Latent Geometry Is Not Enough: Draft-Conditioned Latent Refinement for Non-Autoregressive Text Generation
May 15, 2026De Shuai ZhangNon-Autoregressive Text Generation Diffusion Language Models
Beijing Wuzi University
Membership Inference Attacks on Discrete Diffusion Language Models
May 15, 2026Shailesh KasivelrajanMasked Diffusion Models Membership Inference Attacks
Indian Institute of Technology Madras
Uncertainty Quantification for Large Language Diffusion Models
May 14, 2026Artem Vazhentsev, Vladislav Smirnov, David Li +3Uncertainty Quantification LLM Hallucination Detection
1MBZUAI · 2The University of Melbourne
Language Generation as Optimal Control: Closed-Loop Diffusion in Latent Control Space
May 14, 2026ZiYi Dong, Yuliang Huang, Weijian Deng +3Language Model-Based Control Stochastic Optimal Control
Sun Yat-sen University, Guangzhou, Guangdong, China · Australian National University, Canberra, ACT, Australia · Tsinghua University, Beijing, China
From Table to Cell: Attention for Better Reasoning with TABALIGN
May 14, 2026Tung Sum Thomas Kwok, Zeyong Zhang, Xinyu Wang +6Table QA Table Reasoning
University of California, Los Angeles · SimpleWay · New Jersey Institute of Technology +4
Where Should Diffusion Enter a Language Model? Geometry-Guided Hidden-State Replacement
May 14, 2026Injin Kong, Hyoungjoon Lee, Yohan JoRepresentation Geometry Diffusion Language Models
Graduate School of Data Science, Seoul National University · Department of Biosystems & Biomaterials Science and Engineering, Seoul National University
Factorization-Error-Free Discrete Diffusion Language Model via Speculative Decoding
May 14, 2026Xun Fang, Yunchen Li, Hang Yuan +1Speculative Decoding for Diffusion Language Models Diffusion Language Model Inference
East China Normal University · Beijing Zhongguancun Academy
Support Before Frequency in Discrete Diffusion
May 13, 2026Adrian Müller, Antoine Gonon, Zebang Shen +2Diffusion Language Models Diffusion Models
Department of Computer Science, ETH Zürich, Switzerland · Institute of Mathematics, EPFL, Switzerland · Institute of Statistical Science, Academia Sinica, Taiwan
Beyond Mode-Seeking RL: Trajectory-Balance Post-Training for Diffusion Language Models
May 13, 2026Saba Ahmadi, Prasanna Parthasarathi, Yufei CuiRL for Language Model Reasoning RL for Diffusion Models
Noah’s Ark Lab
BlockVLA: Accelerating Autoregressive VLA via Block Diffusion Finetuning
May 13, 2026Ruiheng Wang, Shuanghao Bai, Haoran Zhang +2Robotic Manipulation Vision-Language-Action Models
Xi'an Jiaotong University
Adaptive Steering and Remasking for Safe Generation in Diffusion Language Models
May 13, 2026Yejin Lee, Ungsik Kim, Yo-Sub HanAdversarial Attacks on LLMs Diffusion Language Model Inference
Department of Computer Science, Yonsei University, Seoul 03722, Republic of Korea · Department of Computer Science and Engineering, Gyeongsang National University, Jinju 52828, Republic of Korea
Understanding and Accelerating the Training of Masked Diffusion Language Models
May 13, 2026Chunsan Hong, Sanghyun Lee, Chieh-Hsin Lai +5Masked Diffusion Models Diffusion Language Model Training
1KAIST · 2Sony AI · University of Tokyo +1
The Efficiency Gap in Byte Modeling
May 13, 2026Celine Lee, Jing Nathan Yan, Chen Liang +9Language Model Scaling Laws Autoregressive Language Modeling
1Google DeepMind · 2Work done while at Google DeepMind · Department of Computer Science, Cornell University.
Discrete Stochastic Localization for Non-autoregressive Generation
May 13, 2026Yunshu Wu, Jiayi Cheng, Longxuan Yu +4Non-Autoregressive Text Generation Masked Diffusion Models
University of California Riverside · New York University

May 22, 2026J/K move · Enter open · S save

Fast-dDrive: Efficient Block-Diffusion VLM for Autonomous Driving

Kewei Zhang, Jin Wang, Sensen Gao, Chengyue Wu+8

1Peking University · 3The University of Hong Kong · 2NVIDIA+1

VLMs for Autonomous Driving End-to-End Autonomous Driving Efficient VLM Inference Blockwise Diffusion Language Models Speculative Decoding Vision-Language-Action Models+2

Open paper AskPDF arXiv

End-to-end autonomous driving via Vision-Language-Action (VLA) models demands a precarious balance between high-fidelity trajectory planning and efficient inference. Existing paradigms typically fall short: autoregressive (AR) VLAs are memory-bandwidth-bound on edge hardware and prone to exposure-bias drift, while full-sequence diffusion models preclude KV-cache reuse and suffer from "logical leakage" that violates the fundamental perceive-then-plan causality. We present Fast-dDrive, a block-diffusion VLA that performs bidirectional refinement within semantic units while enforcing strict causal ordering across them. Leveraging the observation that driving VLAs often emit structured JSON-like outputs, Fast-dDrive freezes structural tokens into a section scaffold and employs a section-aware training recipe that prioritizes safety-critical planning. We further introduce Scaffold Speculative Decoding to achieve AR-equivalent quality at significantly higher throughput. Finally, we propose a low-overhead test-time scaling scheme: by forking

N

stochastic trajectory rollouts from a single shared-prefix KV cache and averaging them, we effectively suppress prediction variance at a fractional computational cost. Empirical results demonstrate that Fast-dDrive redefines the speed-accuracy frontier for driving agents. On the WOD-E2E test set, Fast-dDrive achieves SOTA ADE@3s and ADE@5s, alongside the highest RFS among diffusion-based VLAs; on nuScenes, it reduces average L2 error to

0.32

m (a

22\%

improvement). When integrated with SGLang, our framework delivers

12\times

throughput speedup over the AR baseline, narrowing the gap between high-capacity VLAs and the efficiency demands of real-time on-vehicle deployment.