Language Model Decoding

Ask about this topic

Momentum

18 papers in the last four weeks, up 200% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 108

All topics
CardsList

  1. SharedKV-BT: Node-Local Typed Decisions for Behavior-Tree Agents

    Oct 5, 2026Naoki Wake, Justin WagleLanguage Model DecodingRobot Task Planning

    Microsoft

  2. Can Language Models Learn to Reject Their Own Bad Reasoning Steps?

    Oct 5, 2026Siheng Xiong, Xiaoze Liu, Yiqiao Jin +2LLM Self-CorrectionLanguage Model Decoding

    Georgia Institute of Technology · Purdue University

  3. Nash Equilibrium Text: A Game-Theoretic Decoding Framework for Text Generation

    Oct 5, 2026Alireza Jafari, Arman Adibi, Mohammad Ghavamzadeh +1Nash EquilibriumNon-Autoregressive Text Generation

    Department of Computer Science University of Virginia Charlottesville, VA, USA · School of Computer and Cyber Sciences Augusta University Augusta, GA, USA · Qualcomm AI Research San Diego, CA, USA

  4. Decoding Looped Transformers Better for (Almost) Free

    Oct 1, 2026Weihao Liu, Huangjie Zheng, Tianrong Chen +5Efficient Transformer InferenceLanguage Model Decoding

    Apple

  5. Gacha Decoding: Eliciting Diverse Generations Through Instruction Following

    Oct 1, 2026Scott Geng, Yufei Zhang, Joseph Lee +3Language Model DecodingDiverse Text Generation

    University of Washington · Meta Superintelligence Labs

  6. AS2^2D: Accelerating On-Demand Audio Understanding on Mobile Devices

    Sep 29, 2026Yunzhe Li, Kyoungjun Park, Hongzi Zhu +1On-Device Language Model InferenceLanguage Model Decoding

    The University of Texas at Austin · Shanghai Jiao Tong University

  7. Twist, Don't Tilt: Trajectory-Exact Constrained Decoding for Masked Diffusion Models

    Sep 28, 2026Aditya Thimmaiah, Lara Marinov, Jayanth Srinivasa +3Constrained DecodingMasked Diffusion Models

    The University of Texas at Austin · Cisco Research

  8. Composable Decoding on the Probability Simplex: Theory and Implementation

    Sep 28, 2026Xiaotong Ji, Ahmed Khaled Khamis, Rasul Tutunov +2Constrained DecodingLanguage Model Decoding

    Huawei Noah’s Ark Lab · UCL Centre for AI

  9. Where Activation Sparsity and KV-Cache Sparsity Cross in LLM Decoding

    Sep 27, 2026Jungseob Lee, Seungyoon Lee, Seongtae Hong +2Language Model DecodingActivation Sparsity

    Korea University · Yonsei University Mirae Campus

  10. Encoded but Not Decoded: Layer-Localized Evidence for a Three-Level Gap in LLM Syntax

    Sep 24, 2026Zhenyan Lu, He Wang, Xiaohui HuangLLM EvaluationLanguage Model Decoding

    College of International Studies, National University of Defense Technology, Nanjing, China

  11. Your Transformer Can Hold Two Thoughts at Once: Evidence of Linear Superposition in LLMs

    Sep 24, 2026Pavel Tikhonov, Anton Korznikov, Matvey Mikhalchuk +6Language Model DecodingDecoder-Only Language Models

  12. NumericJev: Jev-like LLM Numerical Decoding with Multiway Decision Trees

    Sep 23, 2026Weiwei Ye, Hangchen Liu, Renhe JiangNumerical Reasoning in Language ModelsLanguage Model Decoding

    The University of Tokyo.

  13. FLEET: From Logits Entropy to Enhanced Trajectories in Text Generation

    Sep 23, 2026Oleksii Streltsov, Oleksandra VitkoLanguage Model DecodingMemory-Augmented Language Models

    Department of Artificial Intelligence, Kharkiv National University of Radio Electronics, Kharkiv, Ukraine

  14. Greedy Decoding Is Not Precision-Invariant: Cross-Precision Output Divergence in LLM Inference

    Sep 22, 2026Gaoyuan Du, Anam Nawaz Khan, Rex Zhou +4Language Model DecodingLLM Inference

    University of Tennessee, Knoxville · University of Chicago · Amazon +1

  15. On-Demand Attention: Language Models Know When to Recall

    Sep 17, 2026Haibo Feng, Ruiqi Liang, Hanyang Peng +1Language Model DecodingLong-Context Language Model Inference

    Southern University of Science and Technology · Peking University · Peng Cheng Laboratory

  16. Beyond Truncation: Rethinking LLM Decoding as Ensemble Pruning

    Sep 16, 2026Dunyao Xue, Chengshuo Du, Zhengbo Wang +2Language Model DecodingDiverse Text Generation

    Institute of Statistics and Big Data, Renmin University of China, Beijing, China · Big Data and Responsible Artificial Intelligence for National Governance, Renmin University of China, Beijing, China · Center for Applied Statistics, Institute of Statistics and Big Data, Renmin University of China, Beijing, China

  17. Look Before You Leap: Factual Decoding with Internal Attribution Signals

    Sep 14, 2026Hayeong Ryu, JungMin Yun, Byeonggeuk Lim +2Language Model DecodingLanguage Model Error Detection

    Department of Artificial Intelligence, Chung-Ang University · Graduate School of Advanced Imaging Sciences, Multimedia and Film, Chung-Ang University

  18. Temperature Fragility and the Conditional Benefits of Truncation Sampling

    Sep 14, 2026Francesco La RosaLLM EvaluationLanguage Model Decoding

    University of Edinburgh

  19. Chopthin-Consensus Power Sampling: A Diversity-Preserving Approach to LLM Decoding

    Sep 14, 2026Minoo Ahmadi, Seyedarmin Azizi, Erfan Baghaei Potraghloo +2Language Model DecodingLLM Inference

    University of Southern California, Los Angeles, USA

  20. Route, Don't Fix: Regime-Dependent Decoding Correction and a Trajectory-Gated Router for Reliable Clinical LLM Answer Selection

    Sep 13, 2026Zeyu Dong, Benjamin Wang, Joyee W. JinAdaptive InferenceLanguage Model Decoding

    St. Theresa of Lisieux Catholic High School Toronto, Canada · Department of Computer Science University of Toronto Toronto, Canada

  21. Line-Coupled Language Model

    Sep 7, 2026Shiyuan Li, Shaorong Zhang, Zhaorui Yang +3Multi-Token PredictionLanguage Model Decoding

    University of California, Riverside Riverside, CA, USA

  22. Predict, Don't Iterate: Efficient Adaptive-Length Infilling for Diffusion Language Models

    Sep 2, 2026Haobo Xu, Sirui Chen, Yuanchen Bei +5Language Model DecodingDiffusion Language Models

    University of Illinois at Urbana-Champaign · Meta

  23. hLLM: Single Pass Decoding for Generative Reranking

    Sep 1, 2026Emil Laftchiev, Prachi Agrawal, Moe Kayali +7Language Model DecodingLLM Reranking

    Meta Platforms, Inc.

  24. Overfitting Mitigation via Singular Value Decomposition in Minimum Bayes Risk Decoding

    Sep 1, 2026Riza Setiawan Soetedjo, Yusuke Sakai, Hidetaka Kamigaito +2Language Model DecodingMBR Decoding

    Nara Institute of Science and Technology (NAIST) · The University of Tokyo

  25. Hidden in the Request: Explaining Unethical LLM Compliance through Token Relevance

    Aug 24, 2026Or Biton, Tomer Krichli, Itai Allouche +1Language Model DecodingLanguage Model Safety Evaluation

    Faculty of Electrical and Computer Engineering, Technion, Haifa, Israel

  26. Ripple-Pivot Search: Active Parallel Decoding for Diffusion Large Language Models

    Aug 12, 2026Yushi Ye, Xu Chen, Haoyun Jiang +7Language Model DecodingDiffusion Language Model Inference

    Cooperative Medianet Innovation Center, Shanghai Jiao Tong University · Alibaba Group · TMLR Group, Department of Computer Science, Hong Kong Baptist University +2

  27. Decomposition-Induced Context-Memory Conflict: When Fact-Checking Pipelines Contradict Their Own Source Text

    Aug 11, 2026Yu-Feng YenLanguage Model DecodingKnowledge Conflicts in Language Models

    Independent Researcher

  28. FutureBridge: Token Selection Beyond Local Preference in Collaborative Decoding

    Aug 7, 2026Quanquan Li, Hongbo Zhang, Yihe Chi +9Language Model DecodingMulti-LLM Collaboration

    1East China Normal University · 2Peking University · University of Science and Technology of China +3

  29. Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs

    Aug 2, 2026Neil F. Johnson, Frank Yingjie Huo, Bella Xinrui LiLanguage Model DecodingAutoregressive Generation

    Department of Physics, The George Washington University Washington, DC 20052, USA

  30. Instruction-Tuned Language Models Cannot Sample from Distributions They Can Describe

    Jul 28, 2026Chaemin Jang, Dongman Lee, Jihee KimLanguage Model DecodingLLM Prompting

    School of Computing, KAIST · School of Business and Technology Management, KAIST

  31. Understanding Semantic IDs: From Item Representation to Item Selection in Generative Recommendation

    Jul 27, 2026Junting Wang, Xinrui He, Yunzhe Li +1Semantic ID RecommendationLanguage Model Decoding

    University of Illinois Urbana-Champaign Urbana, Illinois, USA

  32. SyRuP: Enhancing System-Prompt Following via Reward-Guided Prediction in LLM Decoding

    Jul 27, 2026Seoyeon Kim, Minjae Kang, Jaehyung KimLanguage Model DecodingLanguage Model Steering

    Yonsei University

  33. Refusal-Gated Decoding: Preserving Refusal Behavior Under High-Temperature Sampling

    Jul 22, 2026Phillip Howard, Xin Su, Allen Roush +3Language Model DecodingLLM Refusal Behavior

    Thoughtworks

  34. Speculative Correction: Draft-then-Refine Decoding for Diffusion Language Models

    Jul 21, 2026Brian K Chen, Chong Wu, Kenji KawaguchiLanguage Model DecodingSpeculative Decoding for Diffusion Language Models

    National University of Singapore · City University of Hong Kong

  35. Bayesian Repetition Penalty: A Principled Adjacent-Conditional Framework for Reversing Attention Collapse in Autoregressive Language Models

    Jul 17, 2026Wenjie Fan, Bin Ma, Dong LiLanguage Model DecodingAutoregressive Language Modeling

    Yotta Labs

  36. Adaptive Multi-Step Lookahead Decoding for Diffusion Language Models

    Jul 17, 2026Yingqian Cui, Wei Deng, Lantao Mei +4Masked Diffusion ModelsLanguage Model Decoding

    Michigan State University · Morgan Stanley · IBM T.J. Watson Research Center

  37. Calibrated e-CUSUM Decoding for Quantized Reasoning Models: Why Token Log-Probability Is the Wrong Observable for Decoding Monitors

    Jul 13, 2026El Hassane Ettifouri, Ayoub Belfatmi, Mahaman Sanoussi Yahaya Alassan +1Language Model DecodingChange-Point Detection

    Novelis Research, Paris, France

  38. Gauge dependence and structured-output corruption in sign-branched repetition penalties: measurements across models, inference stacks, and alternative repetition controls

    Jul 9, 2026Peter HollowsLanguage Model DecodingLLM Inference

  39. Noisy-Channel Minimum Bayes Risk Decoding

    Jul 6, 2026Yusuke Sakai, Hidetaka Kamigaito, Taro WatanabeLanguage Model DecodingMBR Decoding

    Nara Institute of Science and Technology, Nara, Japan.

  40. What Does a Routing Oracle Measure Under Stochastic Decoding? Coupling, Scorer Choice, and Single-Commit Ceilings

    Jul 3, 2026Teng-Ruei ChenLLM EvaluationLanguage Model Decoding

    Institute of Bioinformatics and Systems Biology, National Yang Ming Chiao Tung University, Hsinchu 300, Taiwan · Krixvon AI, Taiwan

  41. Set Diffusion: Interpolating Token Orderings Between Autoregression and Diffusion for Fast and Flexible Decoding

    Jul 2, 2026Marianne Arriola, Volodymyr KuleshovLanguage Model DecodingDiffusion Language Model Decoding

    Department of Computer Science, Cornell University, Ithaca, NY, USA.

  42. Depth Exploration for LLM Decoding

    Jun 28, 2026Weisi Yang, Zipeng Sun, Stephen XiaLanguage Model DecodingLLM Inference Acceleration

    Northwestern University · MILA – Quebec AI Institute · McGill University

  43. When are likely answers right? On Sequence Probability and Correctness in LLMs

    Jun 25, 2026Johannes Zenn, Jonas GeipingLLM EvaluationLanguage Model Decoding

    Max Planck Institute for Intelligent Systems · ELLIS Institute Tübingen · AI Center Tübingen +2

  44. Scheduling Thoughts: Learning the Order of Thought in Diffusion Language Models

    Jun 22, 2026Jiawei Xu, Minghui Liu, Aakriti Agrawal +2Masked Diffusion ModelsLanguage Model Decoding

    University of Maryland, College Park · University of California, Los Angeles.

  45. The Anatomy of the CTC Oracle Gap: Acoustic Exhaustion and Linguistic Recovery

    Jun 22, 2026Ivan NovosadASR EvaluationLanguage Model Decoding

    Faculty of Computer Science, HSE University, Moscow, Russia.

  46. Breaking the Likelihood Trap: Variance-Calibrated Modulation for Large Language Model Decoding

    Jun 21, 2026Yuanhao Ding, Meimingwei Li, Esteban Garces Arias +3Open-Ended GenerationLanguage Model Decoding

    School of Computer and Information Engineering, Henan University · Department of Statistics, LMU Munich · 3Munich Center for Machine Learning (MCML)

  47. Deeper is Not Always Better: Mitigating the Alignment Tax via Confident Layer Decoding

    Jun 20, 2026Xuanming Zhang, Sining Zhoubian, Yuxuan Chen +8LLM AlignmentLanguage Model Decoding

    Qwen Team, Alibaba Inc. · Tsinghua University · Nanyang Technological University

  48. From Awareness to Adherence: Bridging the Context Gap in Spoken Dialogue Systems via Context-Aware Decoding

    Jun 15, 2026Che Hyun Lee, Heeseung Kim, Sungroh YoonLanguage Model DecodingSpoken Dialogue Systems

    ECE and 2 IPAI, Seoul National University, Seoul 08826, Korea · Department of AI, University of Seoul, Seoul 02504, Korea

  49. Tyler: Typed Latent Reasoning for Language Models -- When to Think, What to Compute, and How Much to Allocate

    Jun 15, 2026Hanyu Lin, Min Cai, Jiawei Wen +1Language Model DecodingEfficient Language Model Reasoning

    1Shenzhen University · University of Alberta

  50. Adaptive Nucleus Truncation for Long-Form Reasoning

    Jun 12, 2026Ousmane Amadou DiaLanguage Model DecodingAdaptive Sampling

  51. ReSET: Accurate Latency-Critical NVFP4 Reasoning via Step-Aware Temperature Scaling

    Jun 11, 2026Sihwa Lee, Janghwan Lee, Donghoon Yoo +4LLM QuantizationLanguage Model Decoding

    Hanyang University, Seoul, Republic of Korea · Xenoscube Korean Inc., Seoul, Republic of Korea

  52. A Controlled Study of Decoding-Time Truthfulness Methods on Instruction-Tuned LLMs

    Jun 10, 2026Ao SunLLM EvaluationLanguage Model Decoding

    Independent Researcher

  53. CLP: Collocation-Length Prediction for Zero-Loss Adaptive Multi-Token Inference

    Jun 9, 2026Xuezhen Xie, Zhiqiang ZhouMulti-Token PredictionLanguage Model Decoding

  54. K-Forcing: Joint Next-K-Token Decoding via Push-Forward Language Modeling

    Jun 9, 2026Zhiwei Tang, Yuanyu He, Yizheng Han +4LLM ServingLanguage Model Decoding

    1DAMO Academy, Alibaba Group · 2Hupan Lab · 3Zhejiang University +1

  55. STORM: Stepwise Token Optimization with Reward-Guided Beam Search

    Jun 9, 2026Arthur Satouf, Giulio D'Erasmo, Yuxuan Zong +3Language Model DecodingLexical Retrieval

    MILA – Quebec AI Institute & ILLS, Canada · 5Sapienza, University of Rome, Italy · 4Sorbonne Université & ISIR & CNRS, France +2

  56. Gradient-Guided Reward Optimization for Inference-time Alignment

    Jun 8, 2026Hankun Lin, Ruqi ZhangReward HackingLLM Alignment

    Department of Computer Science, Purdue University

  57. Steganography Without Modification: Hidden Communication via LLM Seeds

    Jun 8, 2026Felix Mächtle, Jonas Sander, Sebastian Berndt +3Language Model DecodingLLM Inference

    Institute for IT Security, University of Lübeck, Lübeck, Germany · Technische Hochschule Lübeck, Lübeck, Germany

  58. The Token Not Taken: Sampling, State, and the Stochasticity of AI Agents

    Jun 8, 2026Muhammad Zia Hydari, Raja IqbalLanguage Model DecodingLLM Agents

    University of Pittsburgh · Ejento AI Inc.

  59. Inside the LLM Word Factory

    Jun 7, 2026Benzi Busigin, Yuval PinterLanguage Model DecodingLLM Interpretability

    Stein Faculty of Computer and Information Science Ben-Gurion University of the Negev Beer Sheva, Israel

  60. Don't Pause: Streaming Video-Language Synchrony for Online Video Understanding

    Jun 5, 2026Zhenyu Yang, Kairui Zhang, Shengsheng Qian +2Streaming Video UnderstandingLanguage Model Decoding