cs.CLOct 13, 2025

Latent Refinement Decoding: Enhancing Diffusion-Based Language Models by Refining Belief States

Authors: Qinglin Zhu, Yizhen Yao, Runcong Zhao, Yanzheng Xiang, Siya Qi, Amrutha Saseendran, Chen Jin, Philip Teare, 

Organizations: King’s College London, UK · Centre for AI, Data Science & Artificial Intelligence, BioPharmaceuticals R&D, AstraZeneca, UK · The Chinese University of Hong Kong · MoE Lab, CUHK · 

Abstract

Autoregressive (AR) models remain the dominant paradigm for natural language generation, but their strictly sequential decoding process leads to high inference latency. Recent diffusion-inspired language models, such as LLaDA and Dream, alleviate this issue through parallel generation, yet they still face two key limitations: information loss, since predictive distributions over non-finalised tokens are discarded at each step, and unstable commitment dynamics, where local token decisions are not sufficiently coordinated at the global level. We propose Latent Refinement Decoding (LRD), a two-stage decoding framework consisting of Latent Refinement and a Predictive Feedback Loop. In the first stage, LRD keeps masked positions as distributional mixtures of predicted tokens and the mask embedding, enabling the model to form more globally consistent beliefs before committing tokens. In the second stage, LRD progressively finalises confident tokens while preserving uncertain ones for further iterative refinement, with KL-divergence dynamics serving as a stable criterion for convergence and early stopping. Experiments show that LRD improves performance on both coding tasks, including HumanEval by 6.3 and MBPP by 2.6, and reasoning tasks, including GSM8K by 2.9 and MATH500 by 3.8, while achieving up to 10.6x overall speedup. Moreover, LRD is compatible with single-token decoding, multi-token threshold commitment, and system-level accelerators; when combined with Fast-dLLM, it reaches up to 18.4x speedup over vanilla decoding while further improving accuracy.

Figures & tables

Appendix figures & tables31 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Mean-Field Parallel Decoding for Discrete Diffusion Language Models

    Jun 14, 2026Tamim Zoabi, Ameen Ali, Liran Ringel +1Parallel DecodingDiffusion Language Model Decoding

  2. AsyncLane: Decoupling Refinement from Advancement in Diffusion Language Model Decoding

    Jun 7, 2026Yingxuan Ren, Yuxuan Lou, Yong Liu +4LLM Inference SchedulingBlockwise Diffusion Language Models