Predicting Ethanol Drinking Status from Nonhuman Primate Liver Histology: A Proof of Concept for Parameter-efficient Fine-tuning in Few-Shot Vision-Language Modeling
Organizations: Department of Electrical and Computer Engineering, Texas A&M University · Oregon Health & Science University · Department of Computer Science, Baylor University
Abstract
Combining vision-language models (VLMs) with parameter-efficient fine-tuning (PEFT) offers a promising approach to few-shot whole-slide image (WSI) classification in computational pathology. This study evaluates this approach for predicting ethanol drinking status from nonhuman primate liver histology. Adapters are evaluated against non-PEFT configurations on a subset of the Monkey Alcohol Tissue Research Resource (MATRR). All evaluated PEFT configurations achieve higher mean balanced accuracy than the non-PEFT baselines. For example, in the four-shot setting, attention-based multiple instance learning (ABMIL) alone achieves 43.5% mean balanced accuracy, compared with 78.1% after adding an adapter. Qualitative attention analysis shows that textual pathway adaptations produce localized attention heatmaps, while visual pathway adaptations produce more diffuse ones. A pathologist's review identified similar histological features across adaptations.
Figures & tables
| 2-shot | 4-shot | |||||
|---|---|---|---|---|---|---|
| Method | BACC | AUC | F1 score | BACC | AUC | F1 score |
| Max Pool | ||||||
| ABMIL | ||||||
| ABMIL + Adapter | ||||||
| ABMIL + Adapter | ||||||
| ABMIL + TaskRes | ||||||