Overview

Language model decoding is the process that turns a model’s next-token predictions into text, and its choices affect accuracy, factuality, style, and speed. Common approaches include greedy, temperature, top-k and nucleus sampling, beam search, and best-of-many selection; newer methods use uncertainty, reward models, verifiers, or retrieved context to guide choices during generation. Work also explores parallel and diffusion-based decoding, speculative generation, and attention or cache optimizations to reduce latency and memory use. A central challenge is balancing quality and reliability against cost and diversity, since confidence and reward signals do not always identify the best continuation and gains can vary by task and evaluation setting.

Latest papers

CardsList

182 papers19 papers in the last four weeks, down 27% on the four weeks before. 0.1% of all new papers.

  1. Evolutionary Guided Decoding: Iterative Value Refinement for LLMs

    Mar 4, 2025Zhenhua Liu, Lijun Li, Ruizhe Chen +5LLM AlignmentLanguage Model Decoding

  2. DYNASHIELD: A Black-Box Moving Target Defense for LLMs via Dynamic Decoding Customization

    Dec 10, 2024Xiaoqun Liu, Weiming Qi, Qiben YanAdversarial Robustness of Language ModelsLLM Jailbreak Attacks