cs.CVNov 17, 2026

Efficient Self-Supervised Adaptation of 3D Abdominal Vision-Language Model for Institution-Specific HCC Classification via Full Fine-Tuning and PEFT

Authors: Febryan Putra Kartika, Cheng-Yu Ma, Ying-Jia Lin, Chi-Tung Cheng, Kuan-Fu Chen

Organizations: Department of Emergency Medicine, Chang Gung Memorial Hospital, Keelung, Taiwan · Institute of Health Data Science, College of Intelligent Computing, Chang Gung University, Taoyuan, Taiwan · Department of Artificial Intelligence, College of Intelligent Computing, Chang Gung University, Taoyuan, Taiwan · Artificial Intelligence Research Center, Chang Gung University, Taoyuan, Taiwan · 

Abstract

Medical vision-language models (VLMs) capture cross-modal relationships between image and text, yet their adaptation to institution-specific clinical tasks remains underexplored. In this study, we fine-tuned a pretrained 3D medical VLM for hepatocellular carcinoma (HCC) classification using paired abdominal CT scans and radiology reports from a different institution with acquisition characteristics that differ from the model's original pretraining corpus. We compared two adaptation strategies: full fine-tuning and parameter-efficient fine-tuning (PEFT), motivated by the common use of PEFT to reduce computational cost and enable adaptation under limited-data constraints. Both approaches achieve strong downstream HCC classification performance despite the cross-institutional domain shift, with PEFT reaching an AUC of 0.79 and F1 of 0.70, and full fine-tuning achieving an AUC of 0.82 and F1 of 0.71. Although these results remain below the performance reported by supervised HCC classification approaches, our method achieves improvements over the pretrained model through CT-report contrastive learning without requiring lesion-level annotations or HCC classification labels as training targets. Full fine-tuning converges rapidly but overfits within a few epochs, whereas PEFT (ConvLoRA for the image encoder and LoRA for the text encoder) attains comparable performance while updating approximately 1% of the model parameters, although requiring more training steps. We also examine the role of contrastive temperature, observing that temperature initialization significantly affects classification performance. This study demonstrates that a 3D medical VLM can be efficiently adapted to institution-specific HCC classification using self-supervised CT-report contrastive learning, while highlighting the practical trade-offs between full fine-tuning and parameter-efficient fine-tuning.

Figures & tables

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. When Can Test-Time Adaptation Help Zero-Shot CT Vision-Language Models?

    Jul 17, 2026Ailar Mahdizadeh, Puria Azadi Moghadam, Xiangteng He +1Zero-Shot LearningRadiology VLMs

  2. Revisiting LLM Adaptation for 3D CT Report Generation: A Study of Scaling and Diagnostic Priors

    Jun 15, 2026Vanshali Sharma, Andrea M. Bejar, Halil Ertugrul Aktas +43D Medical ImagingMedical VLMs

  3. Disease-Centric Vision-Language Pretraining with Hybrid Visual Encoding for 3D Computed Tomography

    Jun 24, 2026Bowen Shi, Weiwei Cao, Ruifeng Yuan +5Contrastive Learning3D Medical Imaging