Sycophancy in VLMs

VLM: Vision-Language Model

Follow this topic Ask about this topic

Latest papers

11 papers1 paper in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Seeing Through the Forgery Is Not Enough: Regressive Sycophancy under Fabricated Evidence in Medical Vision-Language Models
Oct 13, 2026Sadif Ahmed, Soumya Swagata Biswas, Rifat Shahriyar +1Sycophancy in VLMs Medical VLMs
Bangladesh University of Engineering and Technology
Geometric Parrots: Rule Absence vs. Rule Fragility in Spatial and Physical Reasoning for Vision-Language Models
Oct 12, 2026Bilal Jawaid, Hamza Hanif, Muhammad Maaz Uddin +8Sycophancy in VLMs VLM Reasoning
Independent Researcher · Simon Fraser University, Canada · Salim Habib University, Karachi, Pakistan +7
PRISM-VLM: A Multi-Axis Discriminative Benchmark for Compact Vision-Language Models
Sep 23, 2026Sanghee Park, Kee-Eung KimVLM Evaluation VLM Robustness
NAVER Cloud AI · KAIST AI
Caved or Convinced: Temporal Sampling Gates Claim Deference in Video Large Language Models
Aug 4, 2026Yuxin Cao, Wei Song, Jingling Xue +1Selective Prediction Video-Language Model Evaluation
National University of Singapore, Singapore · University of New South Wales, Australia
Sycophancy Undermines Epistemic Vigilance in Cooperative Vision-Language Tasks
Jul 31, 2026Rupak Sarkar, Neha Srikanth, Saloni Gupta +3VLM Evaluation Vision-Language Models
University of Maryland, College Park · Army Research Lab
Position, Not Provenance: Separating Reasoning Mediation from Sycophancy in Medical Vision-Language Models
Jul 29, 2026Supratik Bhowal, Subhrajyoti Basu, Aritra Gir Mahanta +1Medical VQA Medical VLMs
Department of Computer Science (AIML) IEM Kolkata, School of UEMK Kolkata, India · Department of Computer Science and Business System Heritage Institute of Technology Kolkata, India · Department of Computer Science and Engineering Indian Institute of Information Technology, Kalyani Kalyani, India
SycoPhantasy: Quantifying Sycophancy and Hallucination in Small Open Weight VLMs for Vision-Language Scoring of Fantasy Characters
Apr 27, 2026Arya Shah, Deepali Mishra, Chaklam SilpasuwanchaiVLM Evaluation Vision-Language Models
IIT Gandhinagar Gandhinagar, India · IIT Kanpur Kanpur, India · Asian Institute of Technology Bangkok, Thailand
Spatiotemporal Sycophancy: Negation-Based Gaslighting in Video Large Language Models
Apr 20, 2026Ziyao Tang, Pengkun Jiao, Bin Zhu +3VLM Evaluation VLM Robustness
Institute of Trustworthy Embodied AI, Fudan University · Shanghai Key Laboratory of Multimodal Embodied AI · Singapore Management University
Gaslight, Gatekeep, V1-V3: Early Visual Cortex Alignment Shields Vision-Language Models from Sycophantic Manipulation
Apr 15, 2026Arya Shah, Vaibhav Tripathi, Mayank Singh +1Sycophancy in VLMs VLM Robustness
Indian Institute of Technology Gandhinagar Gandhinagar, India · Asian Institute of Technology Bangkok, Thailand
To Agree or To Be Right? The Grounding-Sycophancy Tradeoff in Medical Vision-Language Models
Mar 23, 2026OFM Riaz Rahman Aranya, Kevin DesaiSycophancy in VLMs Medical VLMs
Department of Computer Science, The University of Texas at San Antonio
To See or To Please: Uncovering Visual Sycophancy and Split Beliefs in VLMs
Mar 19, 2026Rui Hong, Shuxue QuanSelective Prediction for VLMs Sycophancy in VLMs
George Mason University, Fairfax, VA, USA · Independent Researcher

Oct 13, 2026J/K move · Enter open · S save

Seeing Through the Forgery Is Not Enough: Regressive Sycophancy under Fabricated Evidence in Medical Vision-Language Models

Sadif Ahmed, Soumya Swagata Biswas, Rifat Shahriyar, Chowdhury Mohammad Rakin Haider

Bangladesh University of Engineering and Technology

Sycophancy in VLMs Medical VLMs VLM Evaluation Medical VQA

Open paper AskPDF arXiv

A medical vision-language model may analyze a given CT scan and correctly detect both kidneys in view. Continuing this conversation, if a user provides a prescription with an opposite direction, medical VLMs often sycophantically choose to realign their diagnosis. In this work, we study regressive sycophancy under fabricated evidence, where a model abandons a correct answer when pushback comes with a fake document. We test four open VLMs on 400 VQA-RAD yes/no questions in 19,200 multi-turn conversations, with and without a fabricated prescription. Without a disclosure that documents are synthetic, the models abandon 80.1% of 6,488 correct answers. Notably, under one script each, the same prescription raises the flip rate of Qwen2.5-VL-7B from 45.9% to 89.6% and lowers that of Qwen3-VL-8B from 77.7% to 50.9%. A model rejected the prescription while still preserving the correct answer in 915 of 3,232 conversations with the prescription (28.3%), and 40% of these flipped in later turns. System-prompt sentences that disclose the documents as synthetic lower one model's flip rate from 77.8% to 47.3%. Thus, we conclude that awareness about a forgery does not prevent a model from adopting its answer.