Continuous Diffusion Language Models

Follow this topic Ask about this topic

Continuous diffusion language models study how to generate text by refining noisy continuous representations rather than choosing each token in a strictly left-to-right sequence. Common approaches use Gaussian diffusion or flow matching in token-embedding space, or learn compressed semantic latents with an encoder and decoder; some combine these with masked diffusion, autoregressive decoding, or distillation for faster sampling. These models are increasingly competitive with discrete diffusion and autoregressive baselines on language modeling, summarization, reasoning, and code benchmarks, with methods targeting generation in fewer denoising steps. Key challenges include reliably converting continuous states into coherent tokens, preventing repetition, and balancing sample quality, long-range coherence, and inference speed.

Latest papers

49 papers18 papers in the last four weeks, up 260% on the four weeks before. 0.1% of all new papers.

Self-Regularized Flow Language Models
Oct 27, 2026Theodoros Kouzelis, Spyros Gidaris, Nikos KomodakisDiffusion Language Model Training Continuous Diffusion Language Models
Archimedes, Athena RC · National Technical University of Athens · University of Crete +1
Spherical Brownian Diffusion for Language Modeling
Oct 26, 2026Andrea Miele, Ilija BogunovicDiffusion Language Models Continuous Diffusion Language Models
University of Basel
LLaCO: Continuous Diffusion for Large Language Models with Competition-Aware Semantic Scheduling
Oct 23, 2026Zijian Jia, Peixuan Zhang, Genying Li +5Diffusion Language Models Continuous Diffusion Language Models
School of Artificial Intelligence, Beijing University of Posts and Telecommunications · Yuanpei College, Peking University · Beijing Key Laboratory of Multimodal Data Intelligent Perception and Governance +3
Diffusion Language Models for Perplexity-Aware Black-Box Attacks
Oct 22, 2026Ali Al-Lawati, Elsayed Eshra, Dongwon Lee +1Adversarial Attacks on LLMs LLM Safety Evaluation
The Pennsylvania State University, USA
Loopholing All the Way: Language Modeling with pure Self-Conditioning
Oct 21, 2026Adrien Deschenaux, Nadezhda Ilieva, Justin Deschenaux +6Diffusion Language Models Continuous Diffusion Language Models
EPFL, Lausanne, Switzerland · University of Basel, Switzerland · ELLIS Institute Tübingen, MPI for Intelligent Systems, Tübingen AI Center, Germany +1
GRAFT: Restoring the Missing Sampling Interface in Continuous Diffusion Language Models
Oct 21, 2026Haoning Yang, Lili PanDiffusion Language Model Decoding Continuous Diffusion Language Models
University of Electronic Science and Technology of China
Designing Transformers for Continuous Diffusion Language Models
Oct 15, 2026Weijian Luo, Yujian Chen, Hao Jiang +2Diffusion Language Models Continuous Diffusion Language Models
Dots Studio, Xiaohongshu · Polyadic Research Group · Peking University +2
A Step in the Right Place: Understanding and Optimizing Timestep Placement in Continuous Diffusion Language Models
Oct 11, 2026Shutian Zheng, Zhihong Yu, Linye Wei +2Continuous Diffusion Language Models Diffusion Timestep Scheduling
College of Engineering, Peking University · Weiyang College, Tsinghua University · Institute for Artificial Intelligence, Peking University +3
On the Design Space of Continuous Diffusion Language Models
Oct 9, 2026Oscar Davis, Daan Roos, Alice Bizeul +5Self-Conditioning Continuous Diffusion Language Models
Apple · University of Amsterdam · University of Oxford
Rethinking the Roles of Diffusion and Decoding: Predictive-State Flows for Parallel Language Generation
Oct 9, 2026Jinjin He, Xinsong Feng, Lucheng Fu +3Text Generation Continuous Diffusion Language Models
Georgia Institute of Technology
Efficient Reasoning with Flow Language Models
Oct 7, 2026Hanru Bai, Faissal Izermine, Oscar Davis +1Efficient Language Model Reasoning Language Modeling
Max Planck Institute for Intelligent Systems · ETH Zurich · ELLIS Institute Tübingen +3
Denoising Blocks, Not Tokens: Efficient Compressed Continuous Diffusion with Branching Token Realization
Oct 7, 2026Xinsong Feng, Peng Du, Zhizhuo Yang +3Diffusion Language Models Diffusion Language Model Inference
Georgia Institute of Technology · Writer AI Research · William & Mary
Noise Your Prompt: Noising Conditioning Tokens in Continuous Diffusion Language Models
Oct 6, 2026Justin JungDiffusion Language Model Training Continuous Diffusion Language Models
Lateral Intelligence
Denoising Hierarchical Representations: Joint Continuous Diffusion for Language Modeling
Oct 6, 2026Mathias Ollu, Nikos KomodakisLanguage Modeling Diffusion Language Models
Ecole Polytechnique · Archimedes, Athena RC University of Crete IACM-Forth
Large Language Continuous Diffusion Models
Oct 2, 2026Zhihan Yang, Wei Guo, Jean-Marie Lemercier +14Continuous Diffusion Language Models Diffusion Language Model Inference
NVIDIA · Cornell University · Georgia Institute of Technology
Hierarchical Continuous Diffusion Language Models
Oct 1, 2026Hui Ren, Zihan Li, Chang Liu +2Structured Reasoning in Language Models Diffusion Language Model Decoding
University of Illinois Urbana-Champaign · Amazon.com, Inc.
Scaling and Distilling Text Embeddings for Better Diffusibility
Oct 1, 2026Zekai Zhang, Yunjie Tian, Yanjin He +4Word Embeddings Language Model Distillation
University of Michigan
Clock Diffusion: Efficient Semi-Autoregressive Continuous Diffusion Language Models
Oct 1, 2026Yair Schiff, Omer Belhasin, Roy Uziel +7Diffusion Language Model Inference Language Modeling
NVIDIA · Cornell University
Distribution Matching Distillation for Continuous Diffusion Language Models
Sep 30, 2026Paul Le Van Kiem, Dario Shariatian, Umut Simsekli +1Continuous Diffusion Language Models Diffusion Language Model Distillation
Inria, PSL Research University · Cohere · CMAP, Ecole Polytechnique
Reasoning with Continuous Latent Diffusion
Sep 28, 2026Xiang ChengNumerical Reasoning in Language Models Code Generation
Duke University, Department of Electrical and Computer Engineering
One Latent, Many Tokens: Jointly Learning Compressed Embeddings for Efficient Language Diffusion
Sep 27, 2026Yulin Yuan, Ying Zhang, Xiangming MengDiffusion Language Model Inference Diffusion Model Compression
Zhejiang University · University of Cambridge · ZJU-UIUC Institute, Zhejiang University
ELF-REG: Scaling Continuous Diffusion Language Models to Reasoning Tasks
Sep 24, 2026Zeyu Michael Li, William Xingxu Chen, Bingshuo Qian +2Code Generation Diffusion Language Models
Duke University · Tsinghua University
Parallelism, critical windows, and separations among diffusion language models
Sep 17, 2026Sitan Chen, Liye WangMasked Diffusion Models Diffusion Sampling
SEAS, Harvard University · Tsinghua University
How to Guide Your Language Flow
Sep 16, 2026Rohit Dilip, Tianrong Chen, Yuyang Wang +3Flow Matching Diffusion Model Guidance
Apple · Caltech
Distilled Continuous Diffusion Language Models Can Write Code in Few Steps---or One
Sep 3, 2026Fred Zhangzhi Peng, Kaiwen Zheng, Anru R. ZhangDiffusion Model Distillation One-Step Generative Modeling
Duke University · Tsinghua University
ConvergeFlow: Language Flow with Provable Convergence to Token Embeddings
Aug 24, 2026Na Li, Yuchen Jiao, Changxiao Cai +1Flow-Based Generative Modeling Continuous Diffusion Language Models
Department of Statistics and Data Science, Chinese University of Hong Kong, Hong Kong · Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor, USA
AURORA-LM: Autoencoding Unified Representation for Continuous-Latent Diffusion Language Modeling
Aug 3, 2026Jiajun Liang, Yucheng Liao, Yukang Cao +12Diffusion Transformer Blockwise Diffusion Language Models
PRLab, Nanjing University · S-Lab, Nanyang Technological University · Imperial College London
Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decoding
Jul 7, 2026Yonggan Fu, Lexington Whalen, Abhinav Garg +23Autoregressive Diffusion Autoregressive Language Modeling
NVIDIA · Georgia Tech · Work done during internships at NVIDIA +3
Low Perplexity is Repetition: A One-Dimensional Self-Conditioning Attractor in Continuous Diffusion LMs
Jul 1, 2026Shuai Zhang, Zijie Chen, Hongliang He +2Repetition in Language Generation Language Model Generation Evaluation
Zhejiang University · Westlake University · Ant Group
Continuous Language Diffusion as a Decoder-Interface Problem
Jun 7, 2026Zhicheng Du, Lan MaLatent Diffusion Models Diffusion Language Model Decoding
Tsinghua Shenzhen International Graduate School, Tsinghua University

Oct 27, 2026J/K move · Enter open · S save

Self-Regularized Flow Language Models

Theodoros Kouzelis, Spyros Gidaris, Nikos Komodakis

Archimedes, Athena RC · National Technical University of Athens · University of Crete+1

Diffusion Language Model Training Continuous Diffusion Language Models Diffusion Language Model Distillation Self-Distillation

Open paper AskPDF arXiv

We introduce SERF (Self-Regularized Flow Language Models), a simple self-supervised framework for improving continuous diffusion language models by exploiting the denoising trajectory itself. While continuous flow-based language models offer a promising alternative to autoregressive generation, their training objectives typically rely on supervision from clean target tokens, leaving the denoising trajectory underexploited as a learning signal. SERF addresses this limitation through noisy-to-clean self-distillation: an exponential moving average (EMA) teacher receives a cleaner version of the student's noisy input along the same interpolation path and provides additional supervision. We investigate two variants: SERF-L, which aligns the student's intermediate representations with deeper teacher representations, and SERF-P, which aligns their output token distributions, either at the student's output or at an early student layer. All variants augment the standard training objective without requiring external pretrained models or modifying the inference procedure. Experiments across three continuous language-model paradigms (one-hot FLM, latent ELF, and hyper-spherical S-FLM) demonstrate substantial improvements in training efficiency, generation quality, and few-step generation. In particular, SERF achieves up to 7.7x wall-clock training acceleration on OpenWebText, improves MAUVE by 24.3% and GSM8K accuracy by up to 28.8%, and matches baseline performance with up to 64x fewer sampling steps. These results establish noisy-to-clean self-distillation as an effective approach to improving continuous diffusion language models.