Diffusion Language Models

Follow this topic Ask about this topic

Momentum

55 papers in the last four weeks, up 450% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 306

All topics

Representation-Space MMD for Diffusion Language Models
Oct 5, 2026Ilya Drobyshevskiy, Ilia Sudakov, Maksim Semenov +7Diffusion Language Models
Yandex Research · HSE University · T-Tech +3
D-Loop: Looped Diffusion Drafting for Speculative Decoding
Oct 5, 2026Kecheng Chen, Yuyang He, Cheng Gong +8Block-Diffusion Drafter Diffusion Language Models
City University of Hong Kong · The Chinese University of Hong Kong · Huawei Research
Context-Tower Conversion Preserves Generation While Freezing Retains Knowledge: Low-Budget AR-to-Diffusion Conversion of MoE LLMs
Oct 2, 2026Wentao Lu, Tianyu Zhu, Jesse ClarkDiffusion Language Models Frozen Language Model
Celeris, San Francisco, CA
Hierarchical Continuous Diffusion Language Models
Oct 1, 2026Hui Ren, Zihan Li, Chang Liu +2Diffusion Language Models Diffusion Models
University of Illinois Urbana-Champaign · Amazon.com, Inc.
ITC-MoE: Importance-guided Token-aware Compression for MoE Diffusion Language Models
Oct 1, 2026Lianjun Liu, Shipeng Li, You Huang +5Token Compression Localized Lora-Moe
Hainan University, Haikou, China · Xiamen University, Xiamen, China · Zhejiang Normal University, Jinhua, China
Know When to Hold 'em: Correct-Token Retention in Uniform-State Diffusion Language Models
Oct 1, 2026Mojtaba Nafez, James HendersonDiffusion Language Models Diffusion Models
EPFL & Idiap Lausanne, Switzerland · Idiap Research Institute Martigny, Switzerland
Temporally-Resolved Token Attribution Reveals the Generation Dynamics of Diffusion Language Models
Oct 1, 2026Darpan Aswal, Céline HudelotDiffusion Language Models Growth
Université Grenoble Alpes · MICS, CentraleSupélec
Scaling and Distilling Text Embeddings for Better Diffusibility
Oct 1, 2026Zekai Zhang, Yunjie Tian, Yanjin He +4Diffusion Language Models Text Embeddings
University of Michigan
Two Clocks in Diffusion MLLMs: When Answers Stabilize Before Rationales Unfold
Oct 1, 2026Keuntae Kim, Yong Suk ChoiVisual Question Answering Benchmarks Answer
Department of Computer Science, Hanyang University, Seoul, Korea
Clock Diffusion: Efficient Semi-Autoregressive Continuous Diffusion Language Models
Oct 1, 2026Yair Schiff, Omer Belhasin, Roy Uziel +7Diffusion Language Models Diffusion Models
NVIDIA · Cornell University
Exploring More, Reasoning Better: Stepwise Risk-Sensitive GRPO for Diffusion Language Models
Sep 30, 2026Yue YU, Bowen Zuo, David Crandall +2Diffusion Language Models Large Language Model Reinforcement Learning
Indiana University · University of California, Riverside
Distribution Matching Distillation for Continuous Diffusion Language Models
Sep 30, 2026Paul Le Van Kiem, Dario Shariatian, Umut Simsekli +1Diffusion Language Models Diffusion Models
Inria, PSL Research University · Cohere · CMAP, Ecole Polytechnique
Fork-dLLM: Avoiding the Flexibility Trap in Diffusion Language Models
Sep 30, 2026Stipe Frković, Metod Jazbec, Christian A. NaessethDiffusion Language Models Masked Diffusion Language Models
University of Amsterdam · UvA-Bosch Delta Lab, University of Amsterdam
Self-Repulsive Sampling for Diffusion Language Models
Sep 30, 2026Michael Helcig, Martin JaggiDiffusion Language Models Diffusion Sampling
ETH Zürich · EPFL
Blackboard Intelligence Can Surpass Autoregressive on Globally Constrained Problems
Sep 30, 2026Woosang Jeon, Jaeyeon Kim, Sham Kakade +6Constraint Satisfaction Next-Token Prediction
Seoul National University · Harvard University · University of Central Florida
Fine-Tuning Diffusion Language Models with Context Selection and Target Weighting
Sep 29, 2026Loay Mualem, Lluís Pastor-Pérez, Vinh Tong +4Diffusion Language Models Language Modeling
IMPRS-IS · Institute for AI, University of Stuttgart · Bitdefender
Acceleration of Diffusion Language Model through Discrete Average Generator
Sep 29, 2026Yidong Ouyang, Zhengyan Wan, Themis Haris +8Diffusion Language Models Few-Step Generation
University of California, Los Angeles · Google · Work done as a research intern at Google
Time-Anchored Diffusion Language Models: Latent-Space Caching for Fast Generation
Sep 29, 2026Joel Anto Paul, Litu Rout, Aditya Akella +1Diffusion Language Models Dense Temporal Annotation
UT Austin
E-MoE: Enhanced Mixture-of-Experts for Non-Factorized Diffusion Language Models
Sep 29, 2026Arseny Ivanov, Alexander Kolesov, Alexander Korotin +2Masked Diffusion Language Models Diffusion Language Models
AXXX, Moscow, Russia · Applied AI Institute, Moscow, Russia
DScale: Scaling Block-Diffusion Speculative Decoding with Adaptive Verification
Sep 29, 2026Rongjian Chen, Minxian Xu, Zhengxin Fang +2Speculative Decoding Diffusion Language Models
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518052, China · University of Chinese Academy of Sciences, Beijing 100049, China · Victoria University of Wellington, Wellington, New Zealand +1
How Should Diffusion Language Models Edit Code?
Sep 29, 2026Xijia Tao, Ziru Liu, Shansan Gong +7Multimodal Knowledge Editing Diffusion Language Models
The University of Hong Kong · Huawei Technologies Ltd. · City University of Hong Kong +1
HyperZip: Efficient Data Compression through Personalized Diffusion LLMs with Hypernetworks
Sep 28, 2026Thai Nguyen, Khang Tran, NhatHai PhanTask-Aware Compression Autoregressive Decoding
New Jersey Institute of Technology
Why Deterministic PRM Guidance Underperforms in Discrete Diffusion Reasoning
Sep 28, 2026Yan Zhan, Shaobo Liu, Zhijun GaoDiffusion Language Models Process Reward Model
Peking University · Shenzhen University
d-OPD: Future-Aware On-Policy Distillation for Block Diffusion Language Models
Sep 28, 2026Ruitao Liu, Qinghao Hu, Song HanDiffusion Language Models Efficient On-Policy Distillation
Tsinghua University · MIT · NVIDIA
One Readout, Many Repairs: Diffusion-Guided Hierarchical Search for Tool-Agent Repair
Sep 28, 2026Xiang Xia, Cheng Yan, Wuyang Zhang +4Repair Tool Invocation
University of Science and Technology of China, Hefei, China
BV Loss: Block Verification-Aware Loss for Block Diffusion Speculative Decoding
Sep 28, 2026Suyoung Kim, Jahyun Koo, Hyeonjin Kim +5Block-Diffusion Drafter Diffusion Language Models
a2sys (a2sys.ai) · Seoul National University (snu.ac.kr) · University of Wisconsin (wisc.edu)
Before the Token Commits: Trajectory-Level Benchmarking of Visual Hallucinations in Diffusion VLMs
Sep 28, 2026Yadong Wang, Siping Yue, Yu Tian +2Large Language Model Hallucination Visual Hallucinations
Nanjing University of Aeronautics and Astronautics · State Key Laboratory of Ocean Sensing, ZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University, Hangzhou, 311215, China · Security Capability Center
Low-Confidence Remasking Traps Flexibility: Realizing Arbitrary-Order Potential for Diverse Rollouts in Diffusion LLMs
Sep 28, 2026Moongyu Jeon, Dongjae Jeon, Bumjun Kim +2Masked Diffusion Language Models Diffusion Language Models
Yonsei University · KRAFTON AI · Kookmin University
DreamingGoose: Staged Distillation from Autoregressive Transformers to Bidirectional Recurrent Diffusion Language Models
Sep 28, 2026Julian Boesch, Andrew Wee, Alexander StranzlDiffusion Language Models Autoregressive Transformers
Purdue University · Obit Research · State University of New York at Stony Brook.
Simple Diffusion Language Models Are More Effective Few-Step Generators Than Reported
Sep 27, 2026Hasan Amin, Ming Yin, Rajiv KhannaDiffusion Language Models Few-Step Generation
Purdue University
Diffusion Reward Models
Sep 27, 2026Xiangyang Wang, Bingxiang He, Zeyuan Liu +13Progress Reward Modeling Diffusion Language Models
Tsinghua University · University of Illinois Urbana-Champaign
One Latent, Many Tokens: Jointly Learning Compressed Embeddings for Efficient Language Diffusion
Sep 27, 2026Yulin Yuan, Ying Zhang, Xiangming MengDiffusion Language Models Large Language Model Pretraining
Zhejiang University · University of Cambridge · ZJU-UIUC Institute, Zhejiang University
From Position Risks to Block Survival: Faster Generation for Diffusion Language Models
Sep 27, 2026Siwei Chen, Yuxiang Wan, Yifan Yu +1Diffusion Language Models Diffusion Models
University of Illinois Urbana-Champaign
Unmask the State: When Does State Adaptation Matter for Masked Diffusion Language Models
Sep 27, 2026Injin Kong, Sunghwan Choi, Yohan JoMasked Diffusion Language Models Diffusion Language Models
Graduate School of Data Science, Seoul National University
Hesitation-Aware On-Policy Distillation for Diffusion Language Models
Sep 27, 2026Jianguo Huang, Lipeng Wan, Yanchen Deng +1Efficient On-Policy Distillation Diffusion Language Models
Nanyang Technological University, Singapore · Xi’an Jiaotong University, China
ELF-REG: Scaling Continuous Diffusion Language Models to Reasoning Tasks
Sep 24, 2026Zeyu Michael Li, William Xingxu Chen, Bingshuo Qian +2Diffusion Language Models Reasoning Skills
Duke University · Tsinghua University
Towards Efficient Reasoning: Learning Causal Shortcuts for Diffusion Language Models
Sep 23, 2026Dian Jin, Kairong Han, Baohong Li +5Diffusion Language Models Causal Attention
Zhejiang University · Shanghai AI Laboratory
LOCKR: A Hidden-State Trajectory-Guided Planner for Detecting and Repairing Stable-but-Wrong Lock-In in Diffusion Language Models
Sep 23, 2026Guoshenghui Zhao, Tan Yu, Weijie ZhaoDiffusion Language Models Lock-In
Rochester Institute of Technology · NVIDIA Corporation
Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs
Sep 22, 2026Quan Nguyen-Tri, Mukul Ranjan, Zhiqiang ShenLLM Inference Optimization Diffusion Language Models
VILA Lab, MBZUAI Abu Dhabi, UAE
Informed Masking: Structure-Aware Perturbation for Reinforcement Learning in Diffusion Large Language Models
Sep 22, 2026Xiaoyi Yu, Enver Sangineto, Pei Fu +6Diffusion Language Models Large Language Model Reinforcement Learning
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China · MiLM Plus, Xiaomi Inc., Beijing, China · University of Modena and Reggio Emilia, Italy +1
Circuit Hypernetworks for Quantum-Augmented Diffusion Language Models
Sep 21, 2026Xiaoqiang Wang, Mengyang Xiong, Jun Dai +1Large Language Model Quantization Diffusion Language Models
D´epartement d´informatique et de recherche op´erationnelle, Universit´e de Montr´eal, Montr´eal, QC, Canada. · Mila, Quebec AI Institute, Montr´eal, QC, Canada. · McGill University, Montr´eal, QC, Canada.
dQwen3.5: Hybrid-Attention Diffusion Language Models
Sep 17, 2026Anton Xue, Litu Rout, Aditya Akella +3Diffusion Language Models Autoregressive Model
University of Texas at Austin
Parallelism, critical windows, and separations among diffusion language models
Sep 17, 2026Sitan Chen, Liye WangDiffusion Language Models Diffusion Sampling
SEAS, Harvard University · Tsinghua University
Zarya: A Hybrid Autoregressive--Masked Diffusion Language Model with Flexible Training and Dual-Mode Inference
Sep 17, 2026Leonid Sinev, Ilya Koziev, Vladislav LeshchukAutoregressive Language Models Diffusion Language Models
How to Guide Your Language Flow
Sep 16, 2026Rohit Dilip, Tianrong Chen, Yuyang Wang +3Diffusion Language Models Diffusion Models
Apple · Caltech
Block Parallelism For Efficient Distributed Long-Context Diffusion Language Model Training
Sep 16, 2026Tarun Suresh, Pranshu Chaturvedi, Hangoo Kang +4Diffusion Language Models Efficient Long-Context Inference
Stanford University
Early-Bird Decoding: Accelerating Diffusion LLMs with Learnable Block Sizes and Parallel Sampling
Sep 15, 2026Lixuan Wei, Wei Zhou, Jianwen Wu +3Diffusion Language Models Decoding
Harvard University · Georgia Institute of Technology · Purdue University
Register Tokens for Bounded-State Reasoning in Diffusion Language Models
Sep 14, 2026Albert Ge, Chandan Singh, Yufan Zhuang +3Diffusion Language Models Masked Diffusion Language Models
University of Wisconsin–Madison · Microsoft Research · UC San Diego
Discrete Beckmann Transport Models for One-Step Language Modeling and Reasoning
Sep 14, 2026Sophia Tang, Shiyi WangDiffusion Language Models Autoregressive Language Models
University of Pennsylvania · Harvard University · Kempner Institute +1
Temporal Self-Distillation: Faster Inference in Discrete Diffusion Language Models
Sep 14, 2026Shijian Xu, Andrea Miele, Metod Jazbec +3Diffusion Language Models Fast Inference
University of Basel · University of Amsterdam · Johns Hopkins University
DA-DLM: Explicitly Modeling Token Dependencies in Diffusion Language Models
Sep 14, 2026Pengyu Ji, Zichen Zhang, Xiang Hu +1Diffusion Language Models Autoregressive Transformers
‡School of Information Science and Technology, ShanghaiTech University · §Shanghai Engineering Research Center of Intelligent Vision and Imaging · ¶Tencent
Fixed State, Long Reach: What a Constant-Size Cache Buys Block Diffusion at Scale
Sep 14, 2026Vaibhav Singh, Pierre-André Noël, Torsten Scholak +2Diffusion Language Models Cache
Mila · Concordia University · ServiceNow Research
CanvasAnneal: Curriculum Reinforcement Learning for Diffusion Language Models
Sep 14, 2026Blake Olson, Yuhang Song, Emmett McQuinn +1Diffusion Language Models Diffusion-Based Reinforcement Learning Methods
Google DeepMind
Routing by Reasoning Need: Trajectory-Aware Decoding Control for Diffusion Vision-Language Models
Sep 12, 2026Yixiang Liu, Zhongxing Xu, Zhonghua Wang +1Diffusion Language Models Inference-Time Steering
Southern University of Science and Technology · Monash University
Decomposition-Guided Diffusion Language Models for Inertial Confinement Fusion Prediction
Sep 7, 2026Xiang Zhang, Varchas Gopalaswamy, Rahman Ejaz +2Nuclear Fusion Large Language Model Forecasting
Purdue University · University of Rochester
In-Place Instruction Following in Diffusion Language Models
Sep 7, 2026Zheng Nie, Zherui Li, Jiaming Zhang +3Diffusion Language Models Instruction-Tuned Models
National University of Singapore · Nanyang Technological University · Peking University
Diffusion Language Models for Mobile Edge Agentic AI: Foundations, Applications, and Challenges
Sep 7, 2026Chenqi Li, Minghui Min, Dusit Niyato +1Diffusion Language Models Device
School of Information and Control Engineering, China University of Mining and Technology, Xuzhou, 221116, Jiangsu, China · College of Computing and Data Science, Nanyang Technological University, Singapore, 639798, Singapore · School of Engineering, Edith Cowan University, Perth, Western Australia, 6027, Australia
Unlocking Lossless Speedups in LLMs via Discrete Diffusion
Sep 3, 2026Subham Sekhar Sahoo, Lingjie Chen, Khiem Pham +14Diffusion Language Models Speculative Decoding
Institue of Foundation Models · University of Illinois Urbana-Champaign · Cornell Tech +2
Reliability Challenges in Diffusion Vision-Language Models
Sep 1, 2026Md. Atabuzzaman, Chris ThomasDiffusion Language Models Large Language Model Reliability
Department of Computer Science Virginia Tech
Membership Inference in Fine-tuned Diffusion Language Models via Token-level Memorization Asymmetry
Sep 1, 2026Shengfang Zhai, Leo Marchyok, Yuling Shi +4Diffusion Language Models Membership Inference
National University of Singapore · Oregon State University · Shanghai Jiao Tong University +1

Oct 5, 2026, cs.CLJ/K move · Enter open · S save

Representation-Space MMD for Diffusion Language Models

Ilya Drobyshevskiy, Ilia Sudakov, Maksim Semenov, Denis Kuznedelev+6

Yandex Research · HSE University · T-Tech+3

Diffusion Language Models

Open paper AskPDF arXiv

We introduce a post-training method for diffusion language models (DLMs) that minimizes Maximum Mean Discrepancy (MMD) between generated and reference distributions in the feature space of a frozen pretrained DLM. To estimate MMD, we retain contextual features at individual token positions, obtaining multiple observations per sequence from a single extractor pass. We optimize this objective using policy gradients for discrete models and direct differentiation through generated latents for continuous models. In both cases, computing the loss directly from these features enables efficient post-training without full sampling trajectories or jointly trained auxiliary models. Experiments show lower generative perplexity at comparable entropy on OpenWebText and better accuracy-computation trade-offs on GSM8K. On 16B DMax-LLaDA2.0 models with hybrid masked-uniform diffusion, we increase decoding parallelism with similar or higher accuracy on math and code benchmarks.

Momentum

Related topics

Latest papers 306