Contrastive Decoding

Follow this topic Ask about this topic

Contrastive decoding studies how comparing a model’s predictions under different or counterfactual conditions can strengthen useful evidence and suppress unwanted priors. Common approaches contrast positive and negative prompts, visual or audio perturbations, model layers, or internal decoding paths, often with adaptive gating that intervenes only on uncertain tokens. The methods are applied to hallucination reduction, grounded vision and speech understanding, factual question answering, reasoning, and behavior control, usually without retraining. Gains depend on the model, task, and kind of error, so a central challenge is improving grounding without overcorrecting or mistaking benchmark improvements for genuine reliability.

Latest papers

56 papers7 papers in the last four weeks, up 17% on the four weeks before. 0.0% of all new papers.

MAGIC: Margin-gated Contrastive Debiasing with Evidence Restoration for VLM Test-time Robustness Scaling
Oct 23, 2026Shunshun Gu, Enze Zhang, Ming Pei +2VLM Robustness Test-Time Scaling for VLMs
University of Wisconsin–Madison · University of North Carolina at Chapel Hill
Accept or Repair: Preserving Passage-Supported Answers in Contrastive Decoding
Oct 21, 2026Jungseob Lee, Seungyoon Lee, Hyeonseok Moon +6Evidence-Grounded QA Language Model Decoding
Korea University · Sookmyung Women’s University · Yonsei University Mirae Campus +2
Mitigating Query Shortcuts in Chain-of-Thought with Context-Aware Decoding
Oct 21, 2026Shin'ya YamaguchiCoT Reasoning Faithfulness of Language Model Reasoning
NTT
Mitigating Video Hallucinations via Attention-Guided Representation Perturbation
Oct 14, 2026Zhixin Cai, Ruoqi Yuan, Xinchen Yang +4VLM Hallucination Mitigation Hallucination Mitigation
School of Computer Science and Engineering, Beihang University · Beijing Institute for General Artificial Intelligence, BIGAI · Sino-French Engineer School, Beihang University
DoubtLess: Training-Free Contrastive Decoding for Efficient Reasoning
Oct 13, 2026Minghao Luo, Liang ChenEfficient Language Model Reasoning CoT Compression
The Chinese University of Hong Kong · EPFL
Lost in Contrast: Instruction-Preserving Decoding for Reliable Visual Question Answering
Oct 12, 2026Haonan Zhang, Ting Jin, Wenzhong Tang +1Visual Question Answering Contrastive Decoding
School of Computer Science and Engineering, Beihang University Beijing 100191, China
Decoding Looped Transformers Better for (Almost) Free
Oct 1, 2026Weihao Liu, Huangjie Zheng, Tianrong Chen +5Efficient Transformer Inference Language Model Decoding
Apple
SelfCue: Making a 3D CT Report Generator Say What It Already Knows
Sep 25, 2026Renjie Liang, Yang Yang, Jinqian Pan +3CT Report Generation Radiology Report Generation
University of Florida, Gainesville, FL, USA
Recovering Agentic Sovereignty: Mitigating the Consensus Paradox via Contrastive Epistemic Decoding
Sep 22, 2026Dahlia Shehata, Ming LiLLM Sycophancy Adversarial Attacks on LLMs
University of Waterloo Canada
LoopCD: Loop-wise Contrastive Decoding for Improving Reasoning in Looped Language Models
Sep 21, 2026Byeongho Yu, Junhyuk So, Eunhyeok ParkContrastive Decoding Latent Reasoning
Department of Computer Science and Engineering Pohang University of Science and Technology (POSTECH) · Graduate School of Artificial Intelligence Pohang University of Science and Technology (POSTECH)
What Do Hallucinations Reveal About Multimodal Reasoning? Diagnosing Visual Grounding Failures via Contrastive Decoding Probes
Sep 15, 2026Zhipeng Zhao, Wenxu Wang, Peishun Liu +1Hallucination Detection in VLMs Multimodal Grounding
1Ocean University of China
TAD: Token-Adaptive Contrastive Decoding with Confidence-Guided Gating for Hallucination Mitigation in Large Audio-Language Models
Sep 7, 2026Heyu Chang, Nianwen Si, Hao Zhang +2Audio QA Audio-Language Model Hallucination Detection
Information Engineering University, China
CopyShield: A Cross-Level Benchmark of Copyright Defenses in LLMs
Sep 1, 2026Maryam Alshehyari, Dushyant Singh Chauhan, Samuele Poppi +3LLM Safety Benchmarks Direct Preference Optimization
Computing and Mathematical Science Division Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)
Attention-Guided Reliability Scaling for Contrastive Decoding in Robust Audio-Visual Speech Recognition
Aug 26, 2026YoungChae Kim, Da-Hee Yang, Joon-Hyuk ChangAudio-Visual Speech Recognition Contrastive Decoding
Dept. Artificial Intelligence, Hanyang University, Seoul, Republic of Korea · Dept. Electronic and Electrical Engineering, Hanyang University, Seoul, Republic of Korea
Training-Free VLM Personalization via Calibrated Residual Decoding
Aug 23, 2026Jiaao Yu, Yujian Ma, Xianming Hu +2Vision-Language Models VLM Adaptation
East China Normal University, Shanghai, China · Chang’an University, Shanxi, China
Counterfactual Anatomy-guided Spatial-Temporal Decoding for Annotation-Free Hallucination Mitigation in Medical VLMs
Aug 18, 2026Yifan Lu, Adinath Dukre, Abhijit Das +4VLM Hallucination Mitigation Medical VLMs
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE
Decoupled Contrastive Decoding via Expert-Aligned Drafting
Aug 13, 2026Zhixuan Liu, Zhichen Dong, Yuanfu Wang +1Speculative Decoding LLM Inference Acceleration
Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory
Experience-Calibrated Contrastive Decoding for Mitigating Hallucinations in LM-Based Text-to-Speech
Aug 1, 2026Chenlin Liu, Minghui Fang, Zhonghao Bi +3TTS Synthesis Contrastive Decoding
Harbin Institute of Technology, China · Zhejiang University, China
TPCD: Tone-Pressure Contrastive Decoding and the Label-Free Gating Bottleneck in Vision-Language Models
Jul 29, 2026Jinkun Zhao, Kui Zhang, Wenjun WuVLM Robustness Adversarial Attacks on VLMs
Beihang University
Rethinking CD: A Reproducibility Study and Extension on the Ineffectiveness of Contrastive Decoding at Mitigating Object Hallucinations in MLLMs
Jul 28, 2026Arnav Bendre, Guneesh Gupta, Kavish Grover +2Object Hallucination in VLMs Multimodal Large Language Models
Department of Hydro and Renewable Energy Indian Institute of Technology, Roorkee · Department of Electrical Engineering Indian Institute of Technology, Roorkee · Department of Mechanical Engineering Indian Institute of Technology, Roorkee +2
Attention-Guided Layer Selection for Contrastive Decoding in Large Language Models
Jul 25, 2026Yusuke Sakai, Natthawut Kertkeidkachorn, Kiyoaki ShiraiFactual Knowledge in Language Models Contrastive Decoding
Japan Advanced Institute of Science and Technology
Dual-Confidence Contrastive Decoding for Retrieval-Augmented Generation
Jul 1, 2026Raymond Li, Md Tawkat Islam Khondaker, Amirhossein Abaskohi +3Retrieval-Augmented Generation Knowledge Conflicts in Language Models
ServiceNow Research · University of British Columbia
Adaptive Perturbation Selection for Contrastive Audio Decoding
Jun 30, 2026Aaron Isidore Grace, Zhouyuan Huo, Weiran WangAudio-Language Model Evaluation Audio-Language Model Hallucination Detection
Department of Computer Science University of Iowa Iowa City, IA, USA · Google Mountain View, CA, USA
IHDec: Divergence-Steered Contrastive Decoding for Securing Multi-Turn Instruction Hierarchies
Jun 29, 2026Nicole Geumheon Liu, Haeun Jang, Yonghyun Jun +1Instruction Following Prompt Injection Defense
Chung-Ang University, Seoul, Korea
From Context-Aware to Conflict-Aware: Generalizing Contrastive Decoding for Knowledge Conflict in LLMs
Jun 9, 2026Runze Jiang, Taiqiang Wu, Yan Wang +2LLM Evaluation Knowledge Conflicts in Language Models
1Peking University · 2Alibaba Group · 3The University of Hong Kong
FIDES: Faithful Inference via Deep Evidence Signals for Retrieval-Memory Conflict in RAG
Jun 4, 2026Zhe Yu, Wenpeng Xing, Tiancheng Zhao +3RAG Citation Faithfulness Retrieval-Augmented Generation
Binjiang Institute of Zhejiang University · Zhejiang University · Guangzhou University
Beyond Text Following: Repairable Arbitration Reversals in Audio-Language Models
Jun 3, 2026Yichen Gao, Yiqun Zhang, Zijing Wang +7Audio-Language Model Evaluation Audio Understanding
Northeastern University, China · Shanghai Artificial Intelligence Laboratory, China
YARD: Y-Architecture Register Decoding for Efficient Hallucination Mitigation in Large Vision-Language Models
May 29, 2026Ting Chen, Geng Li, Guohao Chen +5Efficient VLM Inference Object Hallucination in VLMs
Guangdong University of Technology · Nanyang Technological University · Shenzhen TENCLASS Technology Co., Ltd.
Reading the Finetuning Prior: Verbatim Content Recovery via Contrastive Decoding Diffing
May 25, 2026Michał Brzozowski, Zuzanna Dubanowska, Enrico Cassano +1Training Data Extraction Language Model Fingerprinting
Samsung AI Center, Warsaw, Poland · University of Turin, Italy · University of Warsaw, Poland
CHASD: Language Increment-Calibrated Contrastive Decoding against Hallucination in LVLMs
May 22, 2026Xiaoyi Huang, Kejia Zhang, Zhiming LuoHallucination Detection in VLMs Efficient VLM Inference
Institute of Artificial Intelligence, Xiamen University · Department of Artificial Intelligence, Xiamen University

Oct 23, 2026J/K move · Enter open · S save

MAGIC: Margin-gated Contrastive Debiasing with Evidence Restoration for VLM Test-time Robustness Scaling

Shunshun Gu, Enze Zhang, Ming Pei, Hang Zhou+1

University of Wisconsin–Madison · University of North Carolina at Chapel Hill

VLM Robustness Test-Time Scaling for VLMs Vision-Language Models Contrastive Decoding Visual Evidence Sensitivity in VLMs

Open paper AskPDF arXiv

This paper studies the problem of test-time robustness scaling for large vision-language models (LVLMs), which aims to mitigate language bias during inference. Existing approaches typically adopt test-time scaling techniques such as contrastive decoding to improve inference performance. Despite the progress, the performance of these approaches is far from satisfactory due to intentional evidence removal during perturbations and redundant inference cost. Towards this end, we propose a novel approach named Margin-gated Contrastive Debiasing with Evidence Restoration (MAGIC). The core of our MAGIC is to evaluate the decision margins of LVLMs to selectively activate textual and visual perturbation branches for efficient debiased inference. In particular, our MAGIC first measures the prediction margins of crucial tokens, and applies a gating function to identify low-confidence responses that are susceptible to language biases. For selected samples, we integrate the token logits from multiple textual and image perturbation branches with the original ones for debiased decision. More importantly, we selectively restore the most informative corrupted views with sharp prediction distributions for ambiguous samples after debiasing, enabling evidence recovery for discriminative decision-making. We further utilize a calibration dataset to actively identify and prune ineffective branches during contrastive debiasing for cost reduction. Extensive experiments on benchmark datasets validate the effectiveness of the proposed MAGIC in comparison with extensive baselines.