Blockwise Diffusion Language Models

Follow this topic Ask about this topic

These models generate sequences by denoising groups of masked or noisy tokens, combining parallel refinement within a block with sequential progress across blocks. Research explores fixed and adaptive block boundaries, training objectives that align denoising with inference, and samplers that use confidence, speculation, or reinforcement learning to balance parallelism and coherence. Systems work adds attention caching, sparse or linear attention, and block-level batching to speed up long-context generation and reduce memory use. The central challenge is gaining speed over autoregressive models without losing accuracy or coherent long-range reasoning, especially when training and inference use different block sizes or decoding schedules.

Latest papers

55 papers15 papers in the last four weeks, up 200% on the four weeks before. 0.1% of all new papers.

Block Diffusion for Longitudinal EHR: Accuracy, Efficiency, and Representation Trade-offs
Oct 21, 2026Ashvin Gupta, Joshua Placidi, Brendan Delaney +1Longitudinal EHR Modeling Blockwise Diffusion Language Models
Imperial College London
DStill: Trajectory Consistency Distillation for Accelerated One-Step Drafting in Block Diffusion Speculative Decoding
Oct 19, 2026Shreyas Kulkarni, Uttam Gurram, Madhav Kanda +5Speculative Decoding for Diffusion Language Models LLM Inference Acceleration
AI Foundations, Capital One
ReAnchor: Reusing Rejected Verifier States for Diffusion Drafting
Oct 19, 2026Sanghyun Lee, Wonjin Nam, Seungho Jang +4Speculative Decoding Speculative Decoding for Diffusion Language Models
KAIST · Seoul National University
PV-Decoding: Progressive Verification Decoding with Decode Gating for Diffusion Language Models
Oct 17, 2026Renxi Liu, Yiming Li, Shengyin Sun +8Speculative Decoding for Diffusion Language Models Diffusion Language Model Decoding
Huawei Technologies · City University of Hong Kong
On-Policy Residual Learning for Diffusion Language Models
Oct 14, 2026Yifan Yu, Yuqing Jian, Jisen Li +9Diffusion Language Models Diffusion Language Model Inference
Together AI · University of Illinois Urbana-Champaign · University of California, Berkeley +3
One Block Ahead: Promoting Sampler State in Diffusion Language Models
Oct 14, 2026Yangyi Shen, Pin-Jui Ku, Nicolai Oswald +6Parallel Decoding for Diffusion Language Models Diffusion Language Model Decoding
Stanford University · NVIDIA
A Few More Tokens Ahead: Reinforcing Parallelism in Diffusion LLMs
Oct 13, 2026Jiahao Kuang, Yangfan Gao, Tao Gui +4Parallel Decoding for Diffusion Language Models Parallel Decoding
East China Normal University · Fudan University
HaloBlock: Faster and Better Decoding for Diffusion Large Language Models
Oct 13, 2026Zhaohui Zheng, Chenhang He, Jinrui Zhang +2Diffusion Language Model Inference Diffusion Language Model Decoding
The Hong Kong Polytechnic University · College of Computer Science, Nankai University
Revisiting the 16x Pretraining Compute Gap between Diffusion and Autoregressive Language Models
Oct 13, 2026Guanghan Wang, Bing He, Zhan Shi +3Diffusion Language Models Language Model Pretraining
Cornell University, Cornell Tech · Amazon
Rethinking the Roles of Diffusion and Decoding: Predictive-State Flows for Parallel Language Generation
Oct 9, 2026Jinjin He, Xinsong Feng, Lucheng Fu +3Text Generation Continuous Diffusion Language Models
Georgia Institute of Technology
Towards Unbiased On-Policy Distillation for Block Diffusion Language Models
Oct 4, 2026Zaiquan Yang, Fei Wei, Yong Wang +6Blockwise Diffusion Language Models On-Policy Distillation
City University of Hong Kong · Alibaba Group · Beijing Institute of Technology +2
Mask-Guided KV Cache Eviction in Block Diffusion Language Models
Oct 4, 2026Gleb Molodtsov, Ekaterina Alimaskina, Evgeny Uskov +2KV-Cache Eviction Diffusion Language Model Inference
BRAIn Lab · Yandex Research · Yandex School of Data Analysis
Large Language Continuous Diffusion Models
Oct 2, 2026Zhihan Yang, Wei Guo, Jean-Marie Lemercier +14Continuous Diffusion Language Models Diffusion Language Model Inference
NVIDIA · Cornell University · Georgia Institute of Technology
LongSpark: Efficient speculative decoding with a fixed-cost parallel drafter
Sep 29, 2026Hao-Yuan He, Peng-Fei Liu, Si Shen +1Long-Context Language Model Inference Blockwise Diffusion Language Models
d-OPD: Future-Aware On-Policy Distillation for Block Diffusion Language Models
Sep 28, 2026Ruitao Liu, Qinghao Hu, Song HanDiffusion Model Distillation Blockwise Diffusion Language Models
Tsinghua University · MIT · NVIDIA
Block Parallelism For Efficient Distributed Long-Context Diffusion Language Model Training
Sep 16, 2026Tarun Suresh, Pranshu Chaturvedi, Hangoo Kang +4Communication-Efficient Distributed Training Long-Context Language Modeling
Stanford University
Fixed State, Long Reach: What a Constant-Size Cache Buys Block Diffusion at Scale
Sep 9, 2026Vaibhav Singh, Pierre-André Noël, Torsten Scholak +2Blockwise Diffusion Language Models Diffusion Language Model Inference
Mila · Concordia University · ServiceNow Research
LLaDA-UI: Bringing Block-wise Diffusion to Vision-Language GUI Agents
Sep 9, 2026Zhangxuan Gu, Haoxing Chen, Qi Qin +17GUI Agents Multimodal Agents
AGI Research Center, Inclusion AI · Venus Team · Westlake University
DFlow: Enabling Verifier Information Flow in Block Diffusion Speculative Decoding
Sep 6, 2026Yaojie Zhang, Linfeng Zhang, Bin Cui +1Self-Conditioning Speculative Decoding for Diffusion Language Models
School of Computer Science & Beijing Key Laboratory of Software and Hardware Cooperative Artificial Intelligence Systems, Peking University · University of Electronic Science and Technology of China · Shanghai Jiao Tong University +1
A Ticket from Marginals to Joints: Coupled-Noise Distillation for One-Step Block Generation in Diffusion Language Models
Sep 6, 2026Lin YaoOne-Step Generative Modeling Diffusion Language Models
School of Computer Science, Shanghai Jiao Tong University, Shanghai, 200240, China · Zhongguancun Academy, Beijing, 100097, China
PACE-dLLM: Elastic Block Decoding via Confidence Cliff Estimation for Diffusion Language Models
Aug 13, 2026Xiaocheng Lu, Shuhan Guo, Ziyue Ma +5Diffusion Language Model Inference Blockwise Diffusion Language Models
The Hong Kong University of Science and Technology · Northwestern Polytechnical University · The Hong Kong Polytechnic University
Retrofitting Linear Attention into Diffusion Language Models
Aug 6, 2026Jinha Kim, Younghun Roh, Jaeyeon KimBlockwise Diffusion Language Models Diffusion Language Model Inference
Apple · *Work done at MIT · Google DeepMind +1
AURORA-LM: Autoencoding Unified Representation for Continuous-Latent Diffusion Language Modeling
Aug 3, 2026Jiajun Liang, Yucheng Liao, Yukang Cao +12Diffusion Transformer Blockwise Diffusion Language Models
PRLab, Nanjing University · S-Lab, Nanyang Technological University · Imperial College London
DiffusionGemma Technical Report
Jul 31, 2026DiffusionGemma Team, Adrien Ali Taïga, James Assiene +41Diffusion Language Model Decoding Blockwise Diffusion Language Models
Rethinking the Generation Order of Block Diffusion Language Models
Jul 27, 2026Kai Syun Hou, James KwokDiffusion Model Sampling Blockwise Diffusion Language Models
Department of Computer Science and Engineering The Hong Kong University of Science and Technology
FlowBlock: Wavefront-Parallel Decoding for Self-Correcting Diffusion Language Models
Jul 20, 2026Bing Tian, Haikun Liu, Xiaocheng Zhong +5Blockwise Diffusion Language Models Parallel Decoding
† Huazhong University of Science and Technology · § Xiaohongshu Inc., China
Training Hybrid Block Diffusion Language Models with Partial Bidirectionality
Jul 2, 2026Pranshu Chaturvedi, Parth Shroff, Tarun Suresh +2Blockwise Diffusion Language Models Diffusion Language Models
Stanford University
Diffusion-GR2: Diffusion Generative Reasoning Re-ranker
Jul 1, 2026Zhuoxuan Zhang, Kangqi Ni, Yuhang Chen +12Blockwise Diffusion Language Models Efficient Language Model Reasoning
Yang · 2UNC Chapel Hill · 1Meta AI
Adaptive Block Diffusion: Resolving Training-Inference Mismatch in Diffusion Language Models
Jun 28, 2026Gagan JainDiffusion Language Model Training Blockwise Diffusion Language Models
Work done while at Microsoft AI.
Multi-Block Diffusion Language Models
Jun 28, 2026Yijie Jin, Jiajun Xu, Yuxuan Liu +8Blockwise Diffusion Language Models Diffusion Language Model Decoding
Shanghai Jiao Tong University · Xi’an Jiao Tong University · Huawei

Oct 21, 2026J/K move · Enter open · S save

Block Diffusion for Longitudinal EHR: Accuracy, Efficiency, and Representation Trade-offs

Ashvin Gupta, Joshua Placidi, Brendan Delaney, Alessandra Russo

Imperial College London

Longitudinal EHR Modeling Blockwise Diffusion Language Models Masked Diffusion Models Diffusion Model Inference Acceleration

Open paper AskPDF arXiv

Autoregressive (AR) models have proved effective for zero-shot prediction from longitudinal electronic health records, but impose strictly sequential generation on trajectories containing temporally clustered and interdependent clinical events. We investigate block-based masked diffusion (BD) as an alternative on MIMIC-IV under matched tokenisation, context length, model capacity, and evaluation conditions. Across block sizes, BD exhibits a clear quality--efficiency trade-off, with small-block configurations retaining much of the predictive performance of AR while enabling substantially faster inference. A representative

B{=}8

configuration achieves approximately

1.43\times

faster evaluation. Although generative rollout remains below AR in most settings, frozen linear probes on BD and AR representations achieve near-identical performance across clinical prediction tasks, indicating that BD learns comparably informative clinical representations. Increasing the denoising budget yields diminishing returns, while alternative decoding strategies provide limited gains. Overall, our results show that block diffusion is a viable and computationally attractive approach to longitudinal EHR modelling, with further improvements in iterative decoding offering a promising route toward closing the remaining generative performance gap.