cs.CVNov 9, 2026

Evaluating Cross-Signal Coherence in Mammography Vision-Language Models

Authors: Rebeca Nam, Chase M. Fensore, Christian A. Fauria-Robinson, Judy Gichoya, Joyce C. Ho

Organizations: Yonsei University, 50 Yonsei-ro, Seodaemun-gu, 03722, Seoul, Republic of Korea. · Department of Computer Science, Emory University, 400 Dowman Drive, Atlanta, 30322, GA, USA. · Department of Radiology and Imaging Sciences, Emory University, 100 Woodruff Circle, Atlanta, 30322, GA, USA.

Abstract

As vision-language models (VLMs) move toward interpreting medical images directly, their outputs increasingly include not just a label but a stated confidence and a written justification. Yet the field still predominantly evaluates VLMs using just accuracy. We evaluate the coherence between answer, confidence, and explanation across four VLMs on breast density and BI-RADS categorization, using two mammography datasets. We perform controlled perturbations of the question, multiple choice option orders, and the images and assess changes in the three outputs. Our results suggest confidence does not decrease even as the mammography image is changed. Shuffling the options also changes most VLMs' answers. Notably, there are cases where a model's own stated reasoning contradicts its final answer, a distinction accuracy alone cannot capture but one that governs whether an AI-generated answer can be trusted.

Figures & tables

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Answer or Defer? Testing VLMs' Stated Confidence on Brain MRI

    Nov 9, 2026Amir Sabbaghziarani, Parsa Ghazvinian, Sepehr Salem +1Radiology VLMsSelective Prediction for VLMs

  2. Confident but Unreliable: A Behavioral Safety Audit of Vision-Language Models on Brain MRI

    Aug 3, 2026Amir Sabbaghziarani, Mohammadsajad Abavisani, Sergey PlisVLM EvaluationVLM Hallucination

  3. Attention Without Grounding: Causal Evaluation of Visual Explanations in Medical VLMs

    Jul 20, 2026Binesh Sadanandan, Vahid BehzadanVLM EvaluationVLM Interpretability