cs.CVOct 20, 2026

Beyond the Evidence Boundary: Behavioral Mirage Under Progressive Degradation in Medical VLMs

Authors: Edric Castel C. Hao, Rez Samantha Z. Floresca

Organizations: Ateneo de Manila Senior High School

Abstract

VLMs for medical image interpretation are typically evaluated on clean inputs, despite the progressive evidence loss that can occur in high stakes clinical settings. Existing robustness studies show that performance can degrade while confidence remains high, but do not establish when a previously reliable model-case pair ceases to behave reliably. We introduce a baseline-anchored framework that traces baseline-correct cases through progressive image degradation to identify an empirical evidence boundary, and define mirage as continued commitment at this boundary, regardless of correctness. Across three VLMs on three medical datasets, only MedGemma provides a sufficiently large baseline-correct pool for trajectory analysis. On VQA-RAD, the only dataset in which the question text varies, language-only answerability is associated with baseline correctness (AUROC 0.677) and with commitment at the boundary (AUROC 0.653 to 0.670). MedGemma's post-boundary behavior is heterogeneous, including direct mirage, correct recovery, mirage dissolution, consistent abstention, and oscillation, with mirage rates of 16.0 to 100.0% across dataset-modality pairs. Despite this behavioral heterogeneity, residual-stream probes decode commit-versus-abstain behavior at the boundary with AUROC of 0.895 to 0.994 under nested cross-validation. These results show that progressive evidence loss can produce persistent commitment that output entropy and abstention behavior do not consistently reflect, motivating boundary-aware evaluation, option-order controls, and improved abstention mechanisms for reliable medical foundation models operating under partial observation.

Figures & tables

Explore similar work

CardsList
  1. MedVIGIL: Evaluating Trustworthy Medical VLMs Under Broken Visual Evidence

    May 8, 2026Hanqi Jiang, Junhao Chen, Mingyu Kang +12VLM EvaluationVLM Robustness

  2. MedObvious: Exposing the Medical Moravec's Paradox in VLMs via Clinical Triage

    Mar 24, 2026Ufaq Khan, Umair Nawaz, L D M S S Teja +5VLM EvaluationMedical VLMs

  3. Med-R2: An Adversarial Benchmark for Evidence-Grounded Reasoning in Medical VLMs

    May 23, 2026Wen Ma, Fucheng Niu, Zhiting Fan +3Adversarial Attacks on VLMsHealthcare