While deep learning has achieved remarkable success in medical imaging, the "black-box" nature of backpropagation-based models remains a significant barrier to clinical adoption. To bridge this gap, we propose Aristotelian Rapid Object Modeling (A-ROM), a framework built upon the Platonic Representation Hypothesis (PRH). This hypothesis posits that models trained on vast, diverse datasets converge toward a universal and objective representation of reality. By leveraging the generalizable metric space of pretrained Vision Transformers (ViTs), A-ROM enables the rapid modeling of novel medical concepts without the computational burden or opacity of further gradient-based fine-tuning. We replace traditional, opaque decision layers with a human-readable concept dictionary and a k-Nearest Neighbors (kNN) classifier to ensure the model's logic remains interpretable. Experiments on the MedMNIST v2 suite demonstrate that A-ROM delivers performance competitive with standard benchmarks while providing a simple and scalable, "few-shot" solution that meets the rigorous transparency demands of modern clinical environments.
Figures & tables
Figure 1: Visual overview of the MedMNIST v2 benchmark, featuring sample images from each of the 12 constituent 2D datasets [ 36 ] .
Figure 2: Layer-wise classification performance across the 25 transformer blocks of the DINOv2-L/14 backbone. Each box plot aggregates the accuracy variance for a specific depth across 11 MedMNIST v2 datasets, reflecting the interplay between different Alphabet and Vocabulary parameter combinations. Peak accuracies for each layer are indicated by orange markers.
Figure 3: Performance heatmap of A-ROM versus established MedMNIST v2 benchmarks [ 36 ] for the 11 considered datasets. The comparison evaluates the optimized configuration. Values representing the global maximum for each dataset are bolded and underlined. Tha A-ROM framework achieved the highest average accuracy of 83.7% and matched the highest benchmark average AUC of 0.940.
Figure 4: Classification performance across 11 MedMNIST v2 datasets as a function of training samples per class ( n=8 to 512 ). The rightmost column denotes the retention rate, representing the percentage of peak accuracy maintained at the 512-sample threshold.
Figure 5: Case-Based Evidence and Manifold Visualization. (Left) A spiral plot illustrating the 10 nearest neighbors to the query sample, ranked by proximity for the organAMNIST dataset. (Right) A t-SNE projection of the latent manifold, situating the test sample and its k -nearest neighbors within the global context of the supervised concept dictionary for the organAMNIST dataset.
While medical Vision-Language models (VLMs) achieve strong performance on tasks such as tumor or organ segmentation and diagnosis prediction, their opaque latent representations limit clinical trust and the ability to explain predictions. Interpretability of these multimodal representations are therefore essential for the trustworthy clinical deployment of pretrained medical VLMs. However, current interpretability methods, such as gradient- or attention-based visualizations, are often limited to specific tasks such as classification. Moreover, they do not provide concept-level explanations derived from shared pretrained representations that can be reused across downstream tasks. We introduce MedConcept, a framework that uncovers latent medical concepts in a fully unsupervised manner and grounds them in clinically verifiable textual semantics. MedConcept identifies sparse neuron-level concept activations from pretrained VLM representations and translates them into pseudo-report-style summaries, enabling physician-level inspection of internal model reasoning. To address the lack of quantitative evaluation in concept-based interpretability, we introduce a quantitative semantic verification protocol that leverages an independent pretrained medical LLM as a frozen external evaluator to assess concept alignment with radiology reports. We define three concept scores, Aligned, Unaligned, and Uncertain, to quantify semantic support, contradiction, or ambiguity relative to radiology reports and use them exclusively for post hoc evaluation. These scores provide a quantitative baseline for assessing interpretability in medical VLMs. All codes, prompt and data to be released on acceptance. Ke
Md Rakibul Haque, KM Arefeen Sultan, Tushar Kataria +1
Kahlert School of Computing, University of Utah · Scientific Computing and Imaging Institute, University of Utah
Medical imaging models often operate as black boxes, limiting interpretability and systematic debugging. We introduce an easy-to-use, plug-and-play framework for concept-based interpretation and model refinement. By aligning a single-modality encoder to BioMedCLIP, we construct a Concept Bottleneck Model (CBM) that enables concept-level interventions. These interventions allow us to isolate causal versus spuriously correlated concepts, validate insights with domain experts, and generate counterfactual samples for targeted fine-tuning. We evaluate our framework on a Mayo Clinic ultrasound dataset and the CheXpert 5x200 chest X-ray dataset. Results demonstrate that concept intervention enables reliable model diagnosis while maintaining, and occasionally improving predictive performance via guided fine-tuning. Our findings highlight the practical value of this framework for controlled, interpretable refinement of clinical deep learning models.
Samrajya Thapa, Daniel J. Quest, Timothy L. Kline +3
Iowa State University, Ames IA, USA · Mayo Clinic, Rochester MN, USA
Concept Bottleneck Models (CBMs) are a prominent framework for interpretable AI that map learned visual features to a set of meaningful concepts for task-specific downstream predictions. Their sequential structure enhances transparency by connecting model predictions to the underlying concepts that support them. In medical imaging, where transparency is essential, CBMs offer an appealing foundation for explainable model design. However, discrete concept representations often overlook broader clinical context such as diagnostic guidelines and expert heuristics, reducing reliability in complex cases. We propose MedCBR, a concept-based reasoning framework that integrates clinical guidelines with vision-language and reasoning models. Labeled clinical descriptors are transformed into guideline-conformant text, and a concept-based model is trained with a multitask objective combining multimodal contrastive alignment, concept supervision, and diagnostic classification to jointly ground image features, concepts, and pathology. A reasoning model then converts these predictions into structured clinical narratives that explain the diagnosis, emulating expert reasoning based on established guidelines. MedCBR achieves superior diagnostic and concept-level performance, with AUROCs of 94.2% on ultrasound and 84.0% on mammography. Further experiments on non-medical datasets achieve 86.1% accuracy. Our framework enhances interpretability and forms an end-to-end bridge from medical image analysis to decision-making.
Mohamed Harmanani, Bining Long, Zhuoxin Guo +6
Queen’s University · Vector Institute · McMaster University +1