Entropy-Based Decoding

Latest papers

8 papers0 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Entropogram-guided Visual Realignment Decoding: Capturing the Rhythm of Multimodal Reasoning
Nov 4, 2026Jiaqi Deng, Zhe Fu, Yakun Chen +5Entropy-Based Decoding Multimodal Hallucination
University of Technology Sydney · Alibaba Cloud Computing · Stony Brook University +3
FLEET: From Logits Entropy to Enhanced Trajectories in Text Generation
Sep 23, 2026Oleksii Streltsov, Oleksandra VitkoLanguage Model Decoding Memory-Augmented Language Models
Department of Artificial Intelligence, Kharkiv National University of Radio Electronics, Kharkiv, Ukraine
Entropy Can Flow, or It Can Guide. Be Entropy. LEDFlow: Introducing Entropy-guided Generation Order into Uniform Discrete Flow
Sep 20, 2026Tung Sum Thomas Kwok, Yidong Ouyang, Yingjia Wan +3Entropy-Based Decoding Flow-Based Generative Modeling
University of California, Los Angeles · Shanghai Jiao Tong University
Entropy-informed Decoding: Adaptive Information-Driven Branching
May 10, 2026Benjamin Patrick Evans, Sumitra Ganesh, Leo ArdonLanguage Model Decoding LLM Inference Acceleration
JP Morgan AI Research, London, UK · JP Morgan AI Research, New York, USA
Confidence-Based Decoding is Provably Efficient for Diffusion Language Models
Mar 23, 2026Changxiao Cai, Gen LiEntropy-Based Decoding Diffusion Language Model Decoding
Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor, USA · Department of Statistics and Data Science, The Chinese University of Hong Kong, Hong Kong
Think Twice Before You Write -- an Entropy-based Decoding Strategy to Enhance LLM Reasoning
Mar 10, 2026Jiashu He, Meizhu Liu, Olaitan P Olaleye +9Entropy-Based Decoding Language Model Decoding
Oracle AI Science · University of Pennsylvania
Thinking in Uncertainty: Mitigating Hallucinations in MLRMs with Latent Entropy-Aware Decoding
Mar 9, 2026Zhongxing Xu, Zhonghua Wang, Zhe Qian +10Entropy-Based Decoding VLM Hallucination Mitigation
Monash University · Georgia Tech · Northeastern University +3
Entropy-Aware Token Rejection for Improving Speculative Decoding
Dec 29, 2025Tiancheng Su, Meicong Zhang, Guoxiu HeSpeculative Decoding LLM Inference Acceleration
School of Economics and Management East China Normal University

Nov 4, 2026J/K move · Enter open · S save

Entropogram-guided Visual Realignment Decoding: Capturing the Rhythm of Multimodal Reasoning

Jiaqi Deng, Zhe Fu, Yakun Chen, Dichang Zhang+4

University of Technology Sydney · Alibaba Cloud Computing · Stony Brook University+3

Entropy-Based Decoding Multimodal Hallucination MLLM Reasoning VLM Hallucination Mitigation Adaptive Reasoning in Language Models

Open paper AskPDF arXiv

Multimodal large language models (MLLMs) have demonstrated remarkable progress on cross-modal reasoning tasks, yet they continue to suffer from hallucinations in long-chain reasoning processes. In this work, we revisit hallucination in MLLMs from the perspective of reasoning generation dynamics. We argue that entropy is not merely a static measure of uncertainty but a structured signal that reflects how modality information is integrated throughout decoding. Through systematic analysis, we identify two characteristic entropy patterns associated with hallucination behaviors: (1) a generally high-confidence regime corresponds to premature convergence to inappropriate multimodal interactions; and (2) local abrupt entropy transitions are associated with reasoning instability and hallucination onset. Building on these observations, we propose an entropy-aware self-introspective framework for MLLM reasoning. Our method continuously tracks entropy dynamics during decoding and performs targeted visual re-alignment when suspicious hallucination states are detected, encouraging the model to re-examine salient visual evidence at critical reasoning stages. Extensive experiments across six multimodal reasoning benchmarks spanning nine distinct evaluation settings demonstrate that our approach delivers robust improvements in reasoning accuracy and hallucination robustness, while providing new insights into the generation dynamics underlying MLLM failures. The code is available at: https://github.com/jden4524/SelfRetrospectiveDecoding.