cs.CLNov 7, 2026

Global Context Guidance for Diffusion Large Language Models

Authors: Jeimin Jeon, Bumsub Ham

Organizations: Yonsei University · Articron Inc.

Abstract

Diffusion large language models (LLMs) have recently emerged as a promising alternative to autoregressive LLMs, enabling parallel generation through iterative denoising. Recent guidance methods improve diffusion LLMs by introducing a weak model to guide the prediction of the base model during denoising. However, the influence of the weak model during denoising remains less clear, and computing its prediction requires an additional forward pass at each denoising step. We present in this paper an in-depth analysis of how diffusion LLMs use contextual information during denoising. Our analysis reveals that diffusion LLMs rely more heavily on local context while often underutilizing global context. Based on this observation, we introduce Global Context Guidance (GCG), a feature-space guidance method that enhances global-context utilization without constructing an explicit weak model. Extensive experiments on LLaDA-8B, Dream-7B, and LLaDA-1.5 across diverse benchmarks demonstrate that GCG consistently outperforms existing guidance methods, while incurring much lower inference overhead.

Figures & tables

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Focus on the Core: Empowering Diffusion Large Language Models by Self-Contrast

    May 2, 2026Jinyuan Feng, Xin Yu, Yiqun Chen +5Diffusion Language Model InferenceDiffusion Language Model Decoding

  2. Unlocking Lossless Speedups in LLMs via Discrete Diffusion

    Sep 3, 2026Subham Sekhar Sahoo, Lingjie Chen, Khiem Pham +14Diffusion Language Model InferenceLLM Inference Acceleration

  3. Beyond Fully Random Masking: Attention-Guided Denoising and Optimization for Diffusion Language Models

    Jun 10, 2026Jia Deng, Junyi Li, Wayne Xin Zhao +3Masked Diffusion ModelsDiffusion Model Fine-Tuning