Overview

Language model decoding is the process that turns a model’s next-token predictions into text, and its choices affect accuracy, factuality, style, and speed. Common approaches include greedy, temperature, top-k and nucleus sampling, beam search, and best-of-many selection; newer methods use uncertainty, reward models, verifiers, or retrieved context to guide choices during generation. Work also explores parallel and diffusion-based decoding, speculative generation, and attention or cache optimizations to reduce latency and memory use. A central challenge is balancing quality and reliability against cost and diversity, since confidence and reward signals do not always identify the best continuation and gains can vary by task and evaluation setting.

Latest papers

CardsList

182 papers19 papers in the last four weeks, down 27% on the four weeks before. 0.1% of all new papers.

  1. Long-Context Generation Is a Sampling Problem (When You Turn Up the Heat)

    Nov 17, 2026Allen G. Roush, Minh Nhat Nguyen, Ravid Shwartz-Ziv +2Language Model DecodingText Generation

  2. EEGHyena2: Fixation-Inspired Joint Lexical-Grammatical Conditional Random Field for Interpretable EEG-to-Text Decoding

    Nov 14, 2026Xiaoshan Zhou, Chenlan LiuEEG-to-Text DecodingLanguage Model Decoding

  3. Expanding the Decision Space of Pretrained Models through Structured Re-Decoding

    Nov 11, 2026Tinglan Wang, Qun Su, Shihang Wang +11Constrained DecodingLanguage Model Decoding

  4. Free Lunch Uncertainty: Sequence-Level Rényi Entropy from Particle Decoding for Adaptive LLM Reasoning

    Nov 10, 2026Erfan Baghaei Potraghloo, Milad Heidari Sorkhehdizaj, Shahab Rahimirad +6LLM Uncertainty EstimationAdaptive Reasoning in Language Models

  5. Beyond Truncation: When Can LLMs Safely Omit Generated Text?

    Nov 6, 2026Mahmoud Alyosify, Mirna Imbabi, Eman Elkhamisi +1Language Model Decoding

  6. LoRazor: Trimming Adapter Execution for Efficient Multi-LoRA Serving

    Nov 5, 2026Xiang Li, Pengcheng Wang, Azam Ikram +1Multi-LoRA InferenceLLM Serving

  7. Template Gap Reconstruction for Shell Output Simulation with Small Language Models

    Nov 4, 2026Jarrod Ragsdale, Rajendra BoppanaLanguage Model DecodingSmall Language Models

  8. Verifier-Guided Self-Consistency: Test-Time Reasoning with Imperfect Generators and Verifiers

    Nov 4, 2026Ekaterina Fadeeva, Xueyan Li, Mrinmaya Sachan +1Language Model DecodingAdaptive Reasoning in Language Models

  9. Confucius4-R2T2: A Stable-Prefix Streaming ASR Model for Real Real-Time Transcription

    Nov 4, 2026Daniel Huang, Qi He, Shaoxiong Duan +3Streaming ASRAutomatic Speech Recognition

  10. AdSteer: Native Advertising in Large Language Model Generation

    Nov 3, 2026Shuhang Lin, Chuhao Zhou, Zhenyu Wang +4Online AdvertisingLanguage Model Steering

  11. Specification-Guided Generation for Large Language Models

    Oct 30, 2026Alessandro Previti, Konstantinos Vandikas, Mario Alviano +5Language Model DecodingConstrained Decoding

  12. Accept or Repair: Preserving Passage-Supported Answers in Contrastive Decoding

    Oct 21, 2026Jungseob Lee, Seungyoon Lee, Hyeonseok Moon +6Evidence-Grounded QALanguage Model Decoding

  13. LaVGuard: Selective Decoding for Contextual Value Drift

    Oct 21, 2026Jiale Dai, Liuxian Ma, Hongcan Deng +5LLM AlignmentLanguage Model Decoding

  14. When Copying Is Hard: Copy-Constrained Decoding for Exact Span Reproduction

    Oct 20, 2026Jinghui Zhang, Lang Gao, Zongfang Liu +5Language Model DecodingConstrained Decoding

  15. Answering While You Type: Optimizing Time-to-Answer with Streaming Generation

    Oct 19, 2026Nikita Dragunov, Matvey Mikhalchuk, Anton RazzhigaevStreaming InferenceLanguage Model Decoding

  16. Keep the Lineages Alive: Diversity-Preserving SMC for Diffusion LLM Decoding

    Oct 19, 2026Minoo Ahmadi, Seyedarmin Azizi, Mehdi Kamal +1Language Model DecodingDiffusion Language Model Decoding

  17. CALM: Consistency-Aware Logit Masking in Decoding Space for Mitigating Overthinking of Large Reasoning Models

    Oct 19, 2026Shengjing Liu, Feiyang Li, Yuxuan Yang +2Overthinking in Language ModelsEfficient Language Model Reasoning

  18. Intent-Triggered Anchoring for Multi-Token Format Units in Parallel Decoding of Diffusion Language Models

    Oct 12, 2026Junhyeok Lee, Kyu Sung ChoiParallel Decoding for Diffusion Language ModelsLanguage Model Decoding

  19. From Candidate Tokens to Final Outcomes: Decoding Relative Long-Horizon Candidate Value from Reasoning Model Representations

    Oct 11, 2026Yixin Yang, Yue Wang, Fandong Meng +1LLM ReasoningLarge Reasoning Models

  20. The Optimal Decoder is Tilted Temperature

    Oct 10, 2026Nikolaos Papagiannis, Ananth Grama, Wojciech SzpankowskiLanguage Model Decoding

  21. SparseDecoding: Decoding-Aware Pruning for Accurate and Efficient LLM Inference

    Oct 8, 2026Qitong Wang, Xinwei Niu, Mingluo Su +3LLM PruningLLM Inference Acceleration

  22. Can Language Models Learn to Reject Their Own Bad Reasoning Steps?

    Oct 5, 2026Siheng Xiong, Xiaoze Liu, Yiqiao Jin +2LLM Self-CorrectionLanguage Model Decoding

  23. Nash Equilibrium Text: A Game-Theoretic Decoding Framework for Text Generation

    Oct 5, 2026Alireza Jafari, Arman Adibi, Mohammad Ghavamzadeh +1Nash EquilibriumNon-Autoregressive Text Generation