Parameter Efficient Fine-tuning for Domain-specific Gastrointestinal Disease Recognition

Sanjaya Poudel; Nikita Kunwor; Raj Simkhada; Mustafa Munir; Manish Dhakal; Khem Poudel

cs.CVApr 12, 2026

Parameter Efficient Fine-tuning for Domain-specific Gastrointestinal Disease Recognition

Authors: Sanjaya Poudel, Nikita Kunwor, Raj Simkhada, Mustafa Munir, Manish Dhakal, Khem Poudel

Organizations: Auburn University · Tribhuvan University · The University of Texas at Austin · Georgia State University · Middle Tennessee State University

Abstract

Despite recent advancements in the field of medical image analysis with the use of pretrained foundation models, the issue of distribution shifts between cross-source images largely remains adamant. To circumvent that issue, investigators generally train a separate model for each source. However, this method becomes expensive when we fully fine-tune pretrained large models for a single dataset, as we must store multiple copies of those models. Thus, in this work, we propose using a low-rank adaptation (LoRA) module for fine-tuning downstream classification tasks. LoRAs learn lightweight task-specific low-rank matrices that perturb pretrained weights to optimize those downstream tasks. For gastrointestinal tract diseases, they exhibit significantly better results than end-to-end finetuning with improved parameter efficiency. Code is available at: github.com/sanjay931/peft-gi-recognition.

Figures & tables

Train \ Test	GastroVision	Kvasir-Capsule
GastroVision [ 11 ]	86.31	8.04
Kvasir-Capsule [ 16 ]	10.12	99.05

Table 1: Cross-domain evaluation of ConvNeXt-V2-Tiny [ 22 ] : Cross-domain (Non-diagonal) accuracy demonstrates performance drops due to distribution shifts.

Figure 1: System Architecture of PEFT using ConvNeXt-Base.

Figure 2: LoRA-based weight updates, where we only train $A$ and $B$ meanwhile keeping the whole pretrained model frozen.

		GastroVision					Kvasir-Capsule
Method	#Params.	Prec.	Rec.	F1	Acc.	MCC	Prec.	Rec.	F1	Acc.	MCC
MobileNetV2 [ 15 ]	3.5M	74.74	76.67	74.53	76.67	73.88	95.34	95.49	95.22	95.49	90.51
Inception V3 [ 18 ]	25.4M	79.87	80.44	79.31	80.44	78.17	97.27	97.41	97.32	97.41	94.57
DenseNet121 [ 9 ]	7.5M	82.57	83.17	82.12	83.17	81.14	97.42	97.59	97.47	97.59	94.93
DenseNet201 [ 9 ]	19.1M	83.14	82.91	81.82	82.91	80.92	98.62	98.60	98.57	98.60	97.07
ConvNeXtV2 [ 21 ] + Swin-T [ 13 ]	28.3M	73.92	74.77	73.87	74.77	71.75	95.92	95.97	95.80	95.97	91.51

Table 2: Quantitative comparison of different models on the GastroVision and Kvasir-Capsule datasets.

Panel of Figure 3 — Figure 3: Saliency maps comparison between our method (ConvNeXtV2 + LoRA) and other baseline methods. Our method is more coherent and focuses on the semantics of the image, which a clinician uses to make the classification of the disease.

Explore similar work

WaRA: Wavelet Low-Rank Adaptation for Medical Image Classification
Jun 25, 2025Moein Heidari, Yijin Huang, Yasamin Medghalchi +3Vision Foundation Model Adaptation Low-Rank Adaptation
School of Biomedical Engineering, The University of British Columbia, Vancouver, BC, Canada · Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen, China · Department of Electrical and Computer Engineering, The University of British Columbia, Vancouver, BC, Canada +2
Analogical Reasoning as a Doctor: A Foundation Model for Gastrointestinal Endoscopy Diagnosis
Apr 7, 2026Peixi Peng, Housheng Xie, Yanling Wei +5Medical Imaging Foundation Models Medical Diagnosis
Institute of Medical Robotics, School of Biomedical Engineering, Shanghai Jiao Tong University · Institute of Medical Robotics, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China. · Daping Hospital, Army Medical University +3
GLT-PEFT: Gated Lie-Tucker Parameter-Efficient Fine-Tuning for Alzheimer's Disease Diagnosis with Hippocampal Segmentation Pretraining
May 16, 2026Guanghua He, Hancan Zhu, Gaohang Yu +1Medical Diagnosis Low-Rank Adaptation
Department of Mathematics, Hangzhou Dianzi University, Hangzhou, 310018, China · School of Mathematics, Physics and Information, Shaoxing2026 University, Shaoxing, 312000, China · Department of Mathematics, Zhejiang University of Science and Technology, Hangzhou, 310023, China

Jun 25, 2025J/K move · Enter open · S save

WaRA: Wavelet Low-Rank Adaptation for Medical Image Classification

Moein Heidari, Yijin Huang, Yasamin Medghalchi, Alireza Rafiei+2

School of Biomedical Engineering, The University of British Columbia, Vancouver, BC, Canada · Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen, China · Department of Electrical and Computer Engineering, The University of British Columbia, Vancouver, BC, Canada+2

Vision Foundation Model Adaptation Low-Rank Adaptation Medical Image Classification Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

Adapting large pretrained vision models to medical image classification is often limited by memory, computation, and task-specific specializations. Parameter-efficient fine-tuning (PEFT) methods like LoRA reduce this cost by learning low-rank updates, but operating directly in feature space can struggle to capture the localized, multi-scale features common in medical imaging. We propose WaRA, a wavelet-structured adaptation module that performs low-rank adaptation in a wavelet domain. WaRA reshapes patch tokens into a spatial grid, applies a fixed discrete wavelet transform, updates subband coefficients using a shared low-rank adapter, and reconstructs the additive update through an inverse wavelet transform. This design provides a compact trainable interface while biasing the update toward both coarse structure and fine detail. For extremely low-resource settings, we introduce Tiny-WaRA, which further reduces trainable parameters by learning only a small set of coefficients in a fixed basis derived from the pretrained weights through a truncated SVD. Experiments on medical image classification across four modalities and datasets demonstrate that WaRA consistently improves performance over strong PEFT baselines, while retaining a favorable efficiency profile. Our code is publicly available at https://github.com/moeinheidari7829/WaRA.

Abstract

Explore similar work

WaRA: Wavelet Low-Rank Adaptation for Medical Image Classification

Analogical Reasoning as a Doctor: A Foundation Model for Gastrointestinal Endoscopy Diagnosis

GLT-PEFT: Gated Lie-Tucker Parameter-Efficient Fine-Tuning for Alzheimer's Disease Diagnosis with Hippocampal Segmentation Pretraining