eess.IVNov 11, 2026

Supervision-Aware Evaluation of MedGemma for Reference-Assisted Radiotherapy Contour Ranking

Authors: Jaden Moon, Caiwen Jiang, Songlin Zhao, Wei Liu

Organizations: Dartmouth College, Hanover, NH, USA · Mayo Clinic, Phoenix, AZ, USA

Abstract

Radiotherapy contour ranking can support quality assurance, but vision-language model (VLM) gains may reflect supervision, score construction, available inputs, or representation access. We evaluated frozen MedGemma 1.5 in a single-institution study, a 12-patient held-out HaN-Seg test, five-fold cross-fitting on 30 non-test HaN-Seg patients, and transfer without refitting to 120 SegRap2023 patients. On held-out HaN-Seg, a supervised readout of the final prompt state exceeded a supervised readout of the model's yes/no output probabilities by +0.3047 pooled average precision (AP; 95% patient-bootstrap interval [+0.2822, +0.3269]); the advantage persisted against a parameter-count-matched nonlinear control. A readout of projected visual tokens exceeded the final-state readout in cross-fitting and transfer (+0.0706 AP averaged equally across patients on SegRap; 95% interval [+0.0607, +0.0804]), while a structured panel-feature readout remained competitive with visual tokens. The anatomy-conditioned readout did not demonstrate an advantage over the native-feature comparator. These results motivate visual-token and structured-feature controls in medical-imaging VLM evaluations; outcomes were computational or synthetic, not clinical acceptability.

Figures & tables

Explore similar work

CardsList
  1. Attention Without Grounding: Causal Evaluation of Visual Explanations in Medical VLMs

    Jul 20, 2026Binesh Sadanandan, Vahid BehzadanVLM EvaluationVLM Interpretability

  2. MedGemma 1.5 Technical Report

    Apr 6, 2026Andrew Sellergren, Chufan Gao, Fereshteh Mahvar +39Medical Imaging Foundation Models3D Medical Imaging

  3. Thinking with Gaze: Sequential Eye-Tracking as Visual Reasoning Supervision for Medical VLMs

    Mar 5, 2026Yiwei Li, Yifan Zhou, Huaqin Zhao +6Radiology VLMsMedical VLMs