cs.CVNov 19, 2026

Support-Guided Triadic Decoding for Hallucination Mitigation in Multimodal Dialogue with LVLMs

Authors: Yiheng Li, Wenhao Wang, Xingyi Li, Zhilin Tang, Zichang Tan, Zhen Lei, Yang Yang

Organizations: School of Artificial Intelligence, University of Chinese Academy of Sciences · MAIS, Institute of Automation, Chinese Academy of Sciences · Vast Intelligence Lab · Institute of Information Engineering, Chinese Academy of Sciences · 

Abstract

In multimodal multi-turn dialogue, dialogue history plays a dual role: reliable responses provide useful context for subsequent turns, whereas hallucinated responses can propagate errors across turns. Existing training-free methods globally adjust the effect of dialogue history on generation, making it difficult to suppress misleading information without weakening useful context. To address this trade-off, we introduce Support-Guided Triadic Decoding (SGTD), a training-free framework that selectively controls the influence of unreliable history and fuses complementary predictions during decoding. Specifically, Historical Answer Support Estimation measures the visual support of historical responses against type-matched alternatives and identifies low-support content that may mislead later generation. Support-Targeted Attention Intervention then weakens the influence of this content while preserving the surrounding dialogue, producing a support-refined prediction. Triadic Decoding combines this prediction with the full-history prediction and answer-free visual evidence, while the Support-Adaptive Plausibility Constraint filters implausible tokens using visual agreement and historical support. Experiments on MMHalSnowball and VisDiaHalBench across representative LVLMs show that SGTD consistently achieves the highest accuracy under hallucinated histories while preserving the benefits of factual context.

Figures & tables

Explore similar work

CardsList
  1. Decoding by Perturbation: Mitigating MLLM Hallucinations via Dynamic Textual Perturbation

    Apr 14, 2026Sihang Jia, Shuliang Liu, Songbo Yang +3VLM Hallucination MitigationMultimodal Large Language Models

  2. Do We Really Need External Tools to Mitigate Hallucinations? SIRA: Shared-Prefix Internal Reconstruction of Attribution

    May 14, 2026Tian Qin, Junzhe Chen, Yuqing Shi +3Efficient VLM InferenceHallucination in Language Models