Blockwise Diffusion Language Models

Overview

These models generate sequences by denoising groups of masked or noisy tokens, combining parallel refinement within a block with sequential progress across blocks. Research explores fixed and adaptive block boundaries, training objectives that align denoising with inference, and samplers that use confidence, speculation, or reinforcement learning to balance parallelism and coherence. Systems work adds attention caching, sparse or linear attention, and block-level batching to speed up long-context generation and reduce memory use. The central challenge is gaining speed over autoregressive models without losing accuracy or coherent long-range reasoning, especially when training and inference use different block sizes or decoding schedules.

Latest papers

CardsList

64 papers11 papers in the last four weeks, down 8% on the four weeks before. 0.1% of all new papers.

  1. Locally Coherent Parallel Decoding in Diffusion Language Models

    Mar 3, 2026Michael Hersche, Nicolas Menet, Ronan Tanios +1Parallel Decoding for Diffusion Language ModelsDiffusion Language Model Decoding

  2. DODO: Discrete OCR Diffusion Models

    Feb 18, 2026Sean Man, Gilad Deutch, Roy Ganz +4Optical Character RecognitionDiffusion Language Model Decoding

  3. FlashBlock: Attention Caching for Efficient Long-Context Block Diffusion

    Feb 5, 2026Zhuokun Chen, Jianfei Cai, Bohan ZhuangVideo Diffusion ModelsSelf-Attention

  4. Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decoding

    Jan 25, 2026Zhongyu Xiao, Zhiwei Hao, Jianyuan Guo +4Adaptive InferenceBlockwise Diffusion Language Models