Blockwise Diffusion Language Models

Overview

These models generate sequences by denoising groups of masked or noisy tokens, combining parallel refinement within a block with sequential progress across blocks. Research explores fixed and adaptive block boundaries, training objectives that align denoising with inference, and samplers that use confidence, speculation, or reinforcement learning to balance parallelism and coherence. Systems work adds attention caching, sparse or linear attention, and block-level batching to speed up long-context generation and reduce memory use. The central challenge is gaining speed over autoregressive models without losing accuracy or coherent long-range reasoning, especially when training and inference use different block sizes or decoding schedules.

Latest papers

CardsList

63 papers11 papers in the last four weeks, down 8% on the four weeks before. 0.1% of all new papers.

  1. See Every Block Once: Variance-Reduced Post-Training for Masked Diffusion Language Models

    Nov 13, 2026Yunzhen He, Hiroaki Yamagiwa, Hidetoshi ShimodairaDiffusion Language Model TrainingMasked Diffusion Models

  2. Schedule-Calibrated Commitment Order in Blockwise Masked Diffusion Language Models

    Nov 11, 2026Gaurav Sarkar, Umang Pandey, Pratik Prabhanjan Brahma +2Diffusion Language ModelsMasked Diffusion Models

  3. Blurring the Block Boundaries: Overlapping Block Denoising for Efficient Diffusion Language Models

    Nov 10, 2026Amir Mann, Jonathan Fhima, Shaked Brody +4Diffusion Language Model InferenceBlockwise Diffusion Language Models

  4. DForkTree: Accelerating Speculative Decoding with Fork-Aware Block Diffusion Draft Trees

    Nov 9, 2026Zhicheng Zheng, Cheng Zhang, Chenkai Tang +1Speculative DecodingLLM Inference Acceleration

  5. DUET: Dual-Sparse Inference for Block Diffusion Language Models

    Nov 6, 2026Siyuan Yan, Shen Gu, Xiaoxing Ma +2Diffusion Language Model InferenceDiffusion Model Inference Acceleration

  6. PBPO: Prefix-Block Policy Optimization for Diffusion Language Models

    Nov 5, 2026Katherine Tieu, Dongqi Fu, Si Zhang +6Diffusion Language Model TrainingRL for Diffusion Models

  7. D3D: Decoupled Diffusion Drafting for Asynchronous Speculative Decoding

    Nov 4, 2026Baoyou Chen, Ruiqiao Mei, Guancong Lin +4Speculative DecodingSpeculative Decoding for Diffusion Language Models

  8. On-Policy Residual Learning for Diffusion Language Models

    Oct 14, 2026Yifan Yu, Yuqing Jian, Jisen Li +9Diffusion Language ModelsDiffusion Language Model Inference

  9. A Few More Tokens Ahead: Reinforcing Parallelism in Diffusion LLMs

    Oct 13, 2026Jiahao Kuang, Yangfan Gao, Tao Gui +4Parallel Decoding for Diffusion Language ModelsParallel Decoding

  10. HaloBlock: Faster and Better Decoding for Diffusion Large Language Models

    Oct 13, 2026Zhaohui Zheng, Chenhang He, Jinrui Zhang +2Diffusion Language Model InferenceDiffusion Language Model Decoding

  11. Towards Unbiased On-Policy Distillation for Block Diffusion Language Models

    Oct 4, 2026Zaiquan Yang, Fei Wei, Yong Wang +6Blockwise Diffusion Language ModelsOn-Policy Distillation

  12. Mask-Guided KV Cache Eviction in Block Diffusion Language Models

    Oct 4, 2026Gleb Molodtsov, Ekaterina Alimaskina, Evgeny Uskov +2KV-Cache EvictionDiffusion Language Model Inference

  13. Large Language Continuous Diffusion Models

    Oct 2, 2026Zhihan Yang, Wei Guo, Jean-Marie Lemercier +14Continuous Diffusion Language ModelsDiffusion Language Model Inference

  14. Fixed State, Long Reach: What a Constant-Size Cache Buys Block Diffusion at Scale

    Sep 9, 2026Vaibhav Singh, Pierre-André Noël, Torsten Scholak +2Blockwise Diffusion Language ModelsDiffusion Language Model Inference

  15. LLaDA-UI: Bringing Block-wise Diffusion to Vision-Language GUI Agents

    Sep 9, 2026Zhangxuan Gu, Haoxing Chen, Qi Qin +17GUI AgentsMultimodal Agents

  16. DFlow: Enabling Verifier Information Flow in Block Diffusion Speculative Decoding

    Sep 6, 2026Yaojie Zhang, Linfeng Zhang, Bin Cui +1Self-ConditioningSpeculative Decoding for Diffusion Language Models

  17. A Ticket from Marginals to Joints: Coupled-Noise Distillation for One-Step Block Generation in Diffusion Language Models

    Sep 6, 2026Lin YaoOne-Step Generative ModelingDiffusion Language Models

  18. PACE-dLLM: Elastic Block Decoding via Confidence Cliff Estimation for Diffusion Language Models

    Aug 13, 2026Xiaocheng Lu, Shuhan Guo, Ziyue Ma +5Diffusion Language Model InferenceBlockwise Diffusion Language Models

  19. Retrofitting Linear Attention into Diffusion Language Models

    Aug 6, 2026Jinha Kim, Younghun Roh, Jaeyeon KimBlockwise Diffusion Language ModelsDiffusion Language Model Inference