AdaLoRA-QAT: Adaptive Low-Rank and Quantization-Aware Segmentation

Prantik Deb; Srimanth Dhondy; N. Ramakrishna; Anu Kapoor; Raju S. Bapi; Tapabrata Chakraborti

eess.IVApr 1, 2026

AdaLoRA-QAT: Adaptive Low-Rank and Quantization-Aware Segmentation

Authors: Prantik Deb, Srimanth Dhondy, N. Ramakrishna, Anu Kapoor, Raju S. Bapi, Tapabrata Chakraborti

Organizations: International Institute of Information Technology (IIIT-H), Hyderabad, India · Nizam’s Institute of Medical Sciences (NIMS), Hyderabad, India · The Alan Turing Institute, London and University College London, United Kingdom

Abstract

Chest X-ray (CXR) segmentation is an important step in computer-aided diagnosis, yet deploying large foundation models in clinical settings remains challenging due to computational constraints. We propose AdaLoRA-QAT, a two-stage fine-tuning framework that combines adaptive low-rank encoder adaptation with full quantization-aware training. Adaptive rank allocation improves parameter efficiency, while selective mixed-precision INT8 quantization preserves structural fidelity crucial for clinical reliability. Evaluated across large-scale CXR datasets, AdaLoRA-QAT achieves 95.6% Dice, matching full-precision SAM decoder fine-tuning while reducing trainable parameters by 16.6\times and yielding 2.24\times model compression. A Wilcoxon signed-rank test confirms that quantization does not significantly degrade segmentation accuracy. These results demonstrate that AdaLoRA-QAT effectively balances accuracy, efficiency, and structural trust-worthiness, enabling compact and deployable foundation models for medical image segmentation. Code and pretrained models are available at: https://prantik-pdeb.github.io/adaloraqat.github.io/

Figures & tables

Figure 1: Two-stage training pipeline of the proposed AdaLoRA-QAT framework.

Model	DSC	IOU	NSD
nnU-Net	0.9485 $\pm$ 0.119	0.9184 $\pm$ 0.148	0.8118 $\pm$ 0.233
DeepLabv3+	0.9536 $\pm$ 0.117	0.9269 $\pm$ 0.142	0.8405 $\pm$ 0.213
SegFormer	0.9228 $\pm$ 0.174	0.8890 $\pm$ 0.202	0.7622 $\pm$ 0.237
SAM Decoder	0.9555 $\pm$ 0.0664	0.9210 $\pm$ 0.0998	0.9279 $\pm$ 0.0956
SAM Ada-LoRA + D-QAT ∗	0.9563 $\pm$ 0.0651	0.9224 $\pm$ 0.0984	0.9293 $\pm$ 0.0947
SAM Ada-LoRA + Full QAT †	0.9559 $\pm$ 0.0647	0.9216 $\pm$ 0.0983	0.9286 $\pm$ 0.0938

Table 1: Quantitative comparison of segmentation performance across baseline models and SAM Ada-LoRA + Full QAT model.

Model	Total	Trainable	Reduction	DSC
	(M)	(M)		(%)
nnU-Net	30.0	30.0	–	94.85
DeepLabV3+	41.0	41.0	–	95.36
SegFormer	84.6	84.6	–	92.28
SAM Decoder FT	89.7	3.8	23.6×	95.55
Ours (SAM Ada-LoRA)	89.7	5.4	16.6×	95.60

Table 2: Parameter efficiency and segmentation performance comparison.

Figure 2: Structural Similarity Index (SSIM) heatmap comparison of lung segmentation. From left: input CXR, ground truth, baseline SAM SSIM map, proposed AdaLoRA+ Full QAT SSIM map, and $\Delta$ SSIM (QAT – Baseline). Bright regions indicate higher structural agreement. Green in the difference map denotes localized improvements, while red marks degradations.

Figure 3: Quantization error analysis of AdaLoRA-QAT: (a) zero-mean Gaussian noise distribution, (b) FP32-–INT8 correlation, (c) stable error across weight amplitudes, and (d) Q–Q validation of normality.

Method	Encoder	Decoder	Train (%)	Dice (%)
Decoder-only FT	Frozen	Full	4.2	95.55
Enc.-only LoRA (r=8)	LoRA	Frozen	1.9	70.86
Hybrid (Stage 1)	AdaLoRA	Full	6.1	95.60
Hybrid + D-QAT (Stage 2)	AdaLoRA	INT8	6.1	95.63
Hybrid + Full QAT	AdaLoRA and INT8	INT8	6.1	95.59

Table 3: Component-wise ablation of the SAM-base model with AdaLoRA (rank = 32).

Medical Image Segmentation

Medical image segmentation delineates organs, lesions, and other structures in scans to support measurement, diagnosis, treatment planning, and research.

See all 788 papers

Parameter-Efficient Fine-Tuning of Foundation Models for Liver Tumor Segmentation in CT
Sep 12, 2026Ramtin Mojtahedi, Mohammad Hamghalam, Jacob J. Peoples +2Medical Image Segmentation Parameter-Efficient Fine-Tuning
School of Computing, Queen’s University, Kingston, ON, Canada · Toronto General Research Institute, University Health Network, Toronto, ON, Canada · Department of Electrical Engineering, Qa.C., Islamic Azad University, Qazvin, Iran +3
A data- and compute-efficient chest X-ray foundation model beyond aggressive scaling
Feb 26, 2026Chong Wang, Yabin Zhang, Yunhe Gao +9Medical Imaging Foundation Models Medical VLMs
Stanford Center for Artificial Intelligence in Medicine and Imaging, Stanford University, Palo Alto, CA, USA. · Department of Radiology, Stanford University, Stanford, CA, USA. · Department of Computer Science, Stanford University, Stanford, CA, USA. +3
A Few Cases Are All You Need: An Empirical Study of Annotation-Efficient LoRA Fine-Tuning of MedSAM3
Aug 19, 2026Sachin Dudda Nagaraju, Bendik Skarre Abrahamsen, Ashkan Moradi +1Few-Shot Medical Image Segmentation 3D Medical Image Segmentation
Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, Trondheim, Norway · Central Staff, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway

Sep 12, 2026J/K move · Enter open · S save

Parameter-Efficient Fine-Tuning of Foundation Models for Liver Tumor Segmentation in CT

Ramtin Mojtahedi, Mohammad Hamghalam, Jacob J. Peoples, Richard K. G. Do+1

School of Computing, Queen’s University, Kingston, ON, Canada · Toronto General Research Institute, University Health Network, Toronto, ON, Canada · Department of Electrical Engineering, Qa.C., Islamic Azad University, Qazvin, Iran+3

Medical Image Segmentation Parameter-Efficient Fine-Tuning Vision Foundation Model Adaptation Medical Imaging Foundation Models Promptable Image Segmentation

Open paper AskPDF arXiv

We evaluated parameter-efficient fine-tuning (PEFT) of the Segment Anything Model (SAM) for liver tumor segmentation in abdominal CT of colorectal liver metastases. We compared Low-Rank Adaptation (LoRA), 4-bit Quantized LoRA (QLoRA), a convolutional adapter (Conv-Adapter), and our Directional Spectral Top-K adapter (DiSCo), training only adapters while freezing the SAM backbone. DiSCo derives spectral bases from singular value decomposition of row-normalized weights and learns rank-gated spectral coefficients, per-output magnitude offsets, and a spectral gain, with optional Top-K rank selection at inference and 0.14 M trainable parameters. We benchmarked five prompting regimes: no prompt, single-point, multi-point, and bounding boxes at intersection over union 0.50 and 0.75. Conv-Adapter and LoRA achieved the highest accuracy (overall Dice 0.793 and 0.792; single-point Dice 0.795 and 0.792; 95th-percentile Hausdorff distance (HD95) 32 mm). QLoRA was close (overall Dice 0.766; single-point Dice 0.768; HD95 36.41 mm), with 0.91 M trainable parameters, 120 ms latency, and 4.9 GB peak memory. DiSCo achieved the highest Dice per million trainable parameters (4.66), with overall Dice 0.653, single-point Dice 0.698, and HD95 49.53 mm. These results show an accuracy-efficiency trade-off and support PEFT for liver tumor segmentation with reduced adaptation costs when compute and labeled data are limited. Code: https://github.com/Ramtin-Mojtahedi/PEFT-SAM-Liver-CT

Abstract

Medical Image Segmentation

Explore similar work

Parameter-Efficient Fine-Tuning of Foundation Models for Liver Tumor Segmentation in CT

A data- and compute-efficient chest X-ray foundation model beyond aggressive scaling

A Few Cases Are All You Need: An Empirical Study of Annotation-Efficient LoRA Fine-Tuning of MedSAM3