Diffusion Language Models

Follow this topic Ask about this topic

Recent momentum

-25%

24 papers in the last 28 days · 0.4% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

13 new papers

A weekly snapshot of new work published in Diffusion Language Models.

Period ending 2026-09-14

4 new papers

A weekly snapshot of new work published in Diffusion Language Models.

Period ending 2026-09-07

7 new papers

A weekly snapshot of new work published in Diffusion Language Models.

268 papers

Latest in Diffusion Language Models

Browse all topics →

Prefilling-dLLM: Predictive Prefilling for Long-Context Inference in Diffusion Language Models
Jun 9, 2026Jing Xiong, Qi Han, Shansan Gong +5Diffusion Language Models Prefill
A Continuous-Time Markov Chain Framework for Insertion Language Models
Jun 8, 2026Dhruvesh Patel, Benjamin Rozonoyer, Soumitra Das +3Diffusion Language Models Graph Language Models
Optimality of FSQ Tokens for Continuous Diffusion for Categorical Data with Application to Text-to-Speech
Jun 8, 2026Vadim Popov, Wenju Gu, Tasnima Sadekova +2Diffusion Language Models Audio Tokens
Unified Energy for Invariant and Independent Decoding in Diffusion Language Models
Jun 8, 2026Yuchen Yan, Minkai Xu, Zaiquan Yang +1Diffusion Language Models Invariance
Continuous Language Diffusion as a Decoder-Interface Problem
Jun 7, 2026Zhicheng Du, Lan MaDiffusion Language Models Large Language Model Decoder
Back on Track: Aligning Rewards and States for Reasoning in Diffusion Large Language Models
Jun 7, 2026Yawen Shao, Jie Xiao, Kai Zhu +6Diffusion Language Models
Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs Distributional Metrics
Jun 7, 2026Antonio Franca, Alexander TongLanguage Model Perplexity Diffusion Language Models
AsyncLane: Decoupling Refinement from Advancement in Diffusion Language Model Decoding
Jun 7, 2026Yingxuan Ren, Yuxuan Lou, Yong Liu +4Diffusion Language Models Autoregressive Decoding
TimpaTeks: Automatic In-place Text Sequence Modification via Diffusion Language Model Steering
Jun 7, 2026Ryandito Diandaru, Ikhlasul Akmal Hanif, Fadli Aulawi Al Ghiffari +2Diffusion Language Models Text Generation
Forward-Free Diffusion Language Models
Jun 6, 2026Haotian Sun, Rushi Qiang, Yuqian Zheng +1Diffusion Language Models Autoregressive Generation
Diffusion Language Model Parallel Decoding via Product-of-Experts Bridge
Jun 6, 2026Juntong Shi, Brian L. Trippe, Jure Leskovec +2Diffusion Language Models Parallel
Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation
Jun 4, 2026Xingyu Su, Jacob Helwig, Shubham Parashar +6Diffusion Language Models On-Policy Distillation
Self-Augmenting Retrieval for Diffusion Language Models
Jun 4, 2026Paul Jünger, Justin Lovelace, Linxi Zhao +2Diffusion Language Models Retrieval-Augmented Generation Systems
Adapting Diffusion Language Models for Lossless Pixel-Level Image Transmission
Jun 4, 2026Tianqi Ren, Rongpeng Li, Xianfu Chen +2Source-Channel Coding Entropy Coding
NAVIRA: Decoupled Stochastic Remasking for Masked Diffusion Language Models
Jun 4, 2026Andrey Fomenko, Maksim Kryzhanovskiy, Svetlana Glazyrina +1Masked Diffusion Language Models Diffusion Language Models
SAID: Accelerating Diffusion-Based Language Models via Scaffold-Aware Iterative Decoding
Jun 3, 2026Na Li, Chengda Wang, Mingju Gao +1Diffusion Language Models Autoregressive Generation
SemBlock: Semantic Boundary Dynamic Blocks for Diffusion LLMs
Jun 3, 2026Xinrui Song, Zhuoran Wang, Mingju Gao +1Diffusion Language Models
STaR-Quant: State-Time Consistent Post-Training Quantization for Diffusion Large Language Models
Jun 3, 2026Xin Yan, Aqiang Wang, Zhenglin Wan +2Large Language Model Quantization Weight-Only Quantization
Dynamic Infilling Anchors for Format-Constrained Generation in Diffusion Large Language Models
Jun 3, 2026Boyan Han, Yiwei Wang, Yi Song +2Diffusion Language Models Residual Anchoring
Global Sketch-Based Watermarking for Diffusion Language Models
Jun 3, 2026Daniel ZhaoWatermarking Watermarks
D^2SD: Accelerating Speculative Decoding with Dual Diffusion Draft Models
Jun 3, 2026Liyuan Zhang, Jiarui Zhang, Jinwei Yao +6Speculative Decoding Diffusion Language Models
Read the Trace, Steer the Path: Trajectory-Aware Reinforcement Learning for Diffusion Language Models
Jun 3, 2026Anant Khandelwal, Manish GuptaDiffusion Language Models Denoising Trajectory
Supportive Token Revealing for Fast Diffusion Language Model Decoding
Jun 2, 2026Giries Abu Ayoub, Mario Barbara, Lluís Pastor-Pérez +4Diffusion Language Models
Fast-dLLM++: Fréchet Profile Decoding for Faster Diffusion LLM Inference
Jun 1, 2026Siva Rajesh Kasa, Yasong Dai, Sumit Negi +1Diffusion Language Models LLM Inference Efficiency
SimSD: Simple Speculative Decoding in Diffusion Language Models
Jun 1, 2026Junxia Cui, Haotian Ye, Runchu Tian +9Diffusion Language Models Speculative Decoding
BlockGen: Flexible Blockwise Sequence Modeling with Hybrid Samplers
Jun 1, 2026Justin Deschenaux, Caglar GulcehreDiffusion Language Models Diffusion Processes
Cost-Aware Diffusion Draft Trees for Speculative Decoding
Jun 1, 2026Shuai Zhang, Huachuan Qiu, Hongliang He +1Speculative Decoding Diffusion Language Models
FLARE: Diffusion for Hybrid Language Model
Jun 1, 2026Yuchen Zhu, Jing Shi, Chongjian Ge +9Diffusion Language Models Image Denoising
Revise, Don't Freeze: Sampler-Matched Training for Self-Correcting Masked Diffusion Language Models
May 31, 2026Longxuan Yu, Shaorong Zhang, Yu Fu +3Masked Diffusion Language Models Diffusion Language Models
WaveFilter: Enhancing the Long-Context Capability of Diffusion LLMs via Wavelet-Guided KV Cache Filtering
May 30, 2026Jinnan Yang, Yan Wang, Zhen Bi +5Efficient Long-Context Inference Key-Value Cache
EPIC: Efficient and Parallel Inference under CFG Constraints for Diffusion Language Models
May 30, 2026Hyundong Jin, Yo-Sub HanDiffusion Language Models Context-Free Grammars
What Gets Unmasked First? Trajectory Analysis of Diffusion Models for Graph-to-Text Generation
May 29, 2026Qing Wang, Jacob Devasier, Chengkai LiMasked Diffusion Language Models Diffusion Language Models
AMix-2: Establishing Protein as a Native Modality in Large Language Models
May 29, 2026Keyue Qiu, Yixin Wu, Lihao Wang +19Protein Design Protein Data Bank
dMoE: dLLMs with Learnable Block Experts
May 29, 2026Sicheng Feng, Zigeng Chen, Gongfan Fang +2Mixture-of-Experts Models Diffusion Language Models
Efficient Diffusion LLMs via Temporal-Spatial Parallel Decoding and Confidence Extrapolation
May 29, 2026Zekai Li, Ji Liu, Yiqing Huang +3Diffusion Language Models LLM Inference Efficiency
NaRA: Noise-Aware LoRA for Parameter-Efficient Fine-Tuning of Diffusion LLMs
May 28, 2026Shuaidi Wang, Zhan Zhuang, Ruping Huang +1Diffusion Language Models Parameter-Efficient Fine-Tuning
DLM-SWAI: Steering Diffusion Language Models Before They Unmask
May 28, 2026Hyeseon An, Yo-Sub HanDiffusion Language Models Activation Steering
Decoding Strategies for Diffusion-Based ASR: A Systematic Evaluation of Confidence-Based Thresholding
May 28, 2026Jeong Hun Yeo, Minsu Kim, Hyeongseop Rha +1Llm-Based Automatic Speech Recognition Autoregressive Decoding
Cluster-Level Attention-Guided Parallel Decoding for Masked Diffusion Language Models
May 28, 2026Heqiang Qi, Wei Huang, Mingyuan Bai +1Masked Diffusion Language Models Diffusion Language Models
GDSD: Reinforcement Learning as Guided Denoiser Self-Distillation for Diffusion Language Models
May 28, 2026Xiaohang Tang, Keyue Jiang, Che Liu +4Diffusion Language Models Unsupervised On-Policy Self-Distillation
BlockBatch: Multi-Scale Consensus Decoding for Efficient Diffusion Language Model Inference
May 28, 2026Xiaoyou Wu, Cheng-Jhih Shih, Binfei Ji +2Diffusion Language Models LLM Inference Efficiency
When Confidence Misleads: Suffix Anchoring and Anchor-Proximity Confidence Modulation for Diffusion Language Models
May 27, 2026Jungwon Park, Jimyeong Kim, Jungmin Ko +2Diffusion Language Models Residual Anchoring
Dynamic-dLLM: Dynamic Cache-Budget and Adaptive Parallel Decoding for Training-Free Acceleration of Diffusion LLM
May 27, 2026Tianyi Wu, Xiaoxi Sun, Yanhua Jiao +5Diffusion Language Models LLM Inference Efficiency
Enabling KV Caching of Shared Prefix for Diffusion Language Models
May 26, 2026Younghun Go, Jaehoon Han, Changyong Shin +2Cache Diffusion Language Models
Looped Diffusion Language Models
May 25, 2026Sanghyun Lee, Chunsan Hong, Seungryong Kim +3Masked Diffusion Language Models Diffusion Language Models
Reinforcement Learning from Denoising Feedback
May 25, 2026Qi He, Huan Chen, Ya Guo +3Diffusion Language Models Reinforcement Learning From Human Feedback
The Path Matters: Learning a Token-Commitment Policy for Diffusion Language Models
May 23, 2026Bohang Sun, Max Zhu, Francesco Caso +5Diffusion Language Models Frozen Encoder
TUBE: Tangent Upper Bound on Evidence for Discrete Diffusion Language Models
May 22, 2026Arseny Ivanov, Sergei Kholkin, Vladislav Gromadskii +3Diffusion Language Models Likelihood
Extracting Training Data from Diffusion Language Models via Infilling
May 22, 2026Yihan Wang, N. AsokanDiffusion Language Models Memorization
DiLaDiff: Distilled Latent-Augmented Diffusion for Language Modeling
May 22, 2026Jean-Marie Lemercier, Tomas Geffner, Karsten Kreis +3Diffusion Language Models
Learned Relay Representations for Forward-Thinking Discrete Diffusion Models
May 21, 2026Benjamin Rozonoyer, Jacopo Minniti, Dhruvesh Patel +4Masked Diffusion Language Models Diffusion Language Models
Learnability-Informed Fine-Tuning of Diffusion Language Models
May 21, 2026Shubham Parashar, Atharv Chagi, Jacob Helwig +5Diffusion Language Models Discrete Tokens
A Tutorial on Diffusion Theory: From Differential Equations to Diffusion Models
May 21, 2026Jiayi Fu, Yuxia WangDiffusion Processes Stochastic Differential Equations
PulseCol: Periodically Refreshed Column-Sparse Attention for Accelerating Diffusion Language Models
May 20, 2026Yanyi Lyu, Letian Chen, Futing Sun +3Dynamic Sparse Attention Diffusion Language Models
Efficient Long-Context Modeling in Diffusion Language Models via Block Approximate Sparse Attention
May 19, 2026Wenhu Zhang, Yiming Wu, Huanyu Wang +6Dynamic Sparse Attention Diffusion Language Models
Drifting Objectives for Refining Discrete Diffusion Language Models
May 19, 2026Daisuke Oba, Hiroki Furuta, Naoaki OkazakiDiffusion Language Models
Continuous Diffusion Scales Competitively with Discrete Diffusion for Language
May 18, 2026Zhihan Yang, Wei Guo, Shuibai Zhang +5Diffusion Language Models Reparameterization
Machine Unlearning for Masked Diffusion Language Models
May 18, 2026Georu Lee, Seungwon Jeong, Hoki Kim +2Masked Diffusion Language Models Diffusion Language Models
Elastic-dLLM: Position Preserving Context Compression and Augmentation of Diffusion LLMs
May 18, 2026Junyi Wu, Tianchen Zhao, Shaoqiu Zhang +3Diffusion Language Models Context Compression
Prompt Compression in Diffusion Large Language Models: Evaluating LLMLingua-2 on LLaDA
May 18, 2026Sterling Huang, Abigayle Brown, Jiyoo Noh +4Large Language Model Compression Diffusion Language Models

Jun 9, 2026 · cs.CLJ/K move · Enter open · S save

Prefilling-dLLM: Predictive Prefilling for Long-Context Inference in Diffusion Language Models

Jing Xiong, Qi Han, Shansan Gong, Yunta Hsieh+4

The University of Hong Kong · University of Michigan, Ann Arbor · LMSYS Org

Diffusion large language models (dLLMs) re-encode the entire prefix at every denoising step, causing recomputation that scales quadratically with context length and becomes prohibitive for long-context scenarios. We propose Prefilling-dLLM, a training-free prefill-decode disaggregation framework for dLLMs that partitions the prefix into N chunks, caches their KV representations once, and selects the top-K most relevant chunks with intra-chunk token sparsity for decoding, showing that sparse prefilling can outperform dense attention while reducing per-step complexity from quadratic in the full sequence length to quadratic only in the decode length. On LongBench and InfiniteBench, Prefilling-dLLM achieves state-of-the-art quality among dLLM acceleration methods, and an attention kernel that parallelizes decoding over the non-contiguously cached chunk KV yields 9.1--28.0x speedup at 8K--32K contexts. We further show that beginning-of-sequence tokens prepended to each chunk act as periodic attention anchors that eliminate the lost-in-the-middle phenomenon. Code is available at https://github.com/menik1126/Prefilling-dLLM.

Diffusion Language Models Prefill LLM Inference Efficiency Large Language Model Pretraining

Open paper Ask about it PDF

Recent momentum

Neighbouring directions

Recent digests

13 new papers

4 new papers

7 new papers

Latest in Diffusion Language Models