cs.CVNov 19, 2026

Bilateral Evidence Selection and Reasoning for Mammography Vision--Language Models

Authors: Zhengbo Zhou, Dooman Arefan, Shandong Wu

Organizations: Intelligent Systems Program University of Pittsburgh, Pittsburgh, PA, USA · Department of Radiology University of Pittsburgh, Pittsburgh, PA, USA · Department of Biomedical Informatics and Department of Bioengineering University of Pittsburgh, Pittsburgh, PA, USA

Abstract

Medical vision--language models (VLMs) often need to identify small, subtle lesions within high-resolution images containing extensive normal anatomy and redundant visual information. Visual-token compression improves efficiency, but preserving salient regions alone can discard the context needed for diagnostic interpretation. This context can extend across images: in bilateral mammography, the opposite breast provides a patient-specific anatomical reference. Yet retaining such evidence does not ensure that the model uses it effectively. We propose Bilateral Evidence Selection and Reasoning (BESR), which connects evidence preservation with comparative assessment through a bilateral-guided token selector and a bilateral reward. Under a shared token budget, the selector jointly prioritizes lesion appearance and its local and bilateral context, while the reward encourages assessments that benefit from the preserved comparison. Experiments on a bilateral dataset constructed from CBIS-DDSM and on VinDr-Mammo demonstrate improved evidence retention and diagnostic performance under constrained visual-token budgets. Inference benchmarking further shows reduced prefill latency.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Evaluating Cross-Signal Coherence in Mammography Vision-Language Models

    Nov 9, 2026Rebeca Nam, Chase M. Fensore, Christian A. Fauria-Robinson +2VLM EvaluationFaithfulness in VLMs

  2. Region-Grounded Vision-Language Learning for Detection-Guided Mammographic Lesion Classification

    Jul 17, 2026Zhengbo Zhou, Jiren Li, Dooman Arefan +2Radiology VLMsVision-Language Models

  3. Towards Responsible Multimodal Medical Reasoning via Context-Aligned Vision-Language Models

    Apr 9, 2026Sumra Khan, Sagar Chhabriya, Aizan Zafar +5VLM Hallucination MitigationMedical VLMs