cs.CLFeb 6, 2026

DAWN: Dependency-Aware Fast Inference for Diffusion LLMs

Authors: Lizhuo Luo, Zhuoran Shi, Jiajun Luo, Zhi Wang, Shen Ren, Wenya Wang, Tianwei Zhang

Organizations: Nanyang Technological University · AUMOVIO-NTU Corporate Lab, Nanyang Technological University · Southern University of Science and Technology · SIGS, Tsinghua University · 

Abstract

Diffusion large language models (dLLMs) have shown advantages in text generation, particularly due to their inherent ability for parallel decoding. However, constrained by the quality--speed trade-off, existing inference solutions adopt conservative parallel strategies, leaving substantial efficiency potential underexplored. A core challenge is that parallel decoding assumes each position can be filled independently, but tokens are often semantically coupled. Thus, the correct choice at one position constrains valid choices at others. Without modeling these positional dependencies, parallel strategies produce deteriorated outputs. Motivated by this insight, we propose DAWN, a training-free, dependency-aware decoding method for fast dLLM inference. DAWN extracts dependency relations and leverages two key observations: (1) masked positions dependent on unmasked certain positions become more reliable; (2) coupled masked positions strongly influence each other's predictions. Given those findings, DAWN leverages a dependency graph to select more reliable unmasking positions at each iteration, achieving high parallelism with negligible loss in generation quality. Extensive experiments across multiple models and datasets demonstrate that DAWN speedups the inference by 1.80-8.06x over baselines while preserving the generation quality. Code is released at https://github.com/lizhuo-luo/DAWN.

Figures & tables

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Roll Out and Roll Back: Diffusion LLMs are Their Own Efficiency Teachers

    May 16, 2026Fanqin Zeng, Feng Hong, Geng Yu +6Diffusion Model DistillationDiffusion Language Model Decoding

  2. PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding

    May 15, 2026Shengyin Sun, Yiming Li, Renxi Liu +7Parallel Speculative DecodingMasked Diffusion Models

  3. DAPD: Dependency-Aware Parallel Decoding via Attention for Diffusion LLMs

    Mar 13, 2026Bumjun Kim, Dongjae Jeon, Moongyu Jeon +1Parallel Decoding for Diffusion Language ModelsParallel Decoding