Parameter-Efficient Fine-Tuning of Foundation Models for Liver Tumor Segmentation in CT

Ramtin Mojtahedi; Mohammad Hamghalam; Jacob J. Peoples; Richard K. G. Do; Amber L. Simpson

cs.CVSep 12, 2026

Parameter-Efficient Fine-Tuning of Foundation Models for Liver Tumor Segmentation in CT

Authors: Ramtin Mojtahedi, Mohammad Hamghalam, Jacob J. Peoples, Richard K. G. Do, Amber L. Simpson

Organizations: School of Computing, Queen’s University, Kingston, ON, Canada · Toronto General Research Institute, University Health Network, Toronto, ON, Canada · Department of Electrical Engineering, Qa.C., Islamic Azad University, Qazvin, Iran · Department of Radiology, Memorial Sloan Kettering Cancer Center, New York, NY, USA · Department of Radiology and Diagnostic Imaging, University of Alberta, Edmonton, AB, Canada · Alberta Machine Intelligence Institute, Edmonton, AB, Canada

Abstract

We evaluated parameter-efficient fine-tuning (PEFT) of the Segment Anything Model (SAM) for liver tumor segmentation in abdominal CT of colorectal liver metastases. We compared Low-Rank Adaptation (LoRA), 4-bit Quantized LoRA (QLoRA), a convolutional adapter (Conv-Adapter), and our Directional Spectral Top-K adapter (DiSCo), training only adapters while freezing the SAM backbone. DiSCo derives spectral bases from singular value decomposition of row-normalized weights and learns rank-gated spectral coefficients, per-output magnitude offsets, and a spectral gain, with optional Top-K rank selection at inference and 0.14 M trainable parameters. We benchmarked five prompting regimes: no prompt, single-point, multi-point, and bounding boxes at intersection over union 0.50 and 0.75. Conv-Adapter and LoRA achieved the highest accuracy (overall Dice 0.793 and 0.792; single-point Dice 0.795 and 0.792; 95th-percentile Hausdorff distance (HD95) 32 mm). QLoRA was close (overall Dice 0.766; single-point Dice 0.768; HD95 36.41 mm), with 0.91 M trainable parameters, 120 ms latency, and 4.9 GB peak memory. DiSCo achieved the highest Dice per million trainable parameters (4.66), with overall Dice 0.653, single-point Dice 0.698, and HD95 49.53 mm. These results show an accuracy-efficiency trade-off and support PEFT for liver tumor segmentation with reduced adaptation costs when compute and labeled data are limited. Code: https://github.com/Ramtin-Mojtahedi/PEFT-SAM-Liver-CT

Figures & tables

Figure 1: PEFT on SAM for liver tumor segmentation in CRLM CT: we froze SAM encoders, trained only Conv-Adapter, LoRA, QLoRA, and SVD-based DiSCo adapters in the encoder or decoder while the mask decoder fused point or box prompts with image embeddings, and we evaluated multiple prompts using Dice, HD95, and compute metrics.

Figure 2: Adapter accuracy–efficiency summary. (a) Mean Dice: Conv-Adapter/LoRA highest; QLoRA lower; DiSCo lowest. (b) Accuracy vs. latency: QLoRA fastest; DiSCo highlighted. (c) Accuracy vs. trainable parameters (log): Conv-Adapter/LoRA at higher capacity; QLoRA cuts parameters with moderate Dice drop; DiSCo in the extreme low-parameter regime. (d) Dice per million trainable parameters: DiSCo shows the best capacity efficiency.

Adapter	Dice $\uparrow$	Single-pt $\uparrow$	HD95 (mm) $\downarrow$	Lat. (ms) $\downarrow$	Mem (GB) $\downarrow$	Train. (M) $\downarrow$	FLOPs (TFLOPs/img) $\downarrow$	Dice/M $\uparrow$
DiSCo	0.6530	0.6983	49.53	143.93	5.47	0.14	0.2807	4.657
Conv-Adapter	0.7932	0.7953	32.01	131.95	5.34	8.72	0.1860	0.091
LoRA	0.7919	0.7918	31.74	145.38	6.09	11.31	0.9000	0.070
QLoRA	0.7655	0.7680	36.41	119.97	4.94	0.91	0.0020	0.842

Table 1: Combined performance and resource summary on adapters. Dice and HD95 are patient-level means averaged across all prompt regimes; Single-pt reports the single-point regime. Resource metrics include inference latency, peak memory, trainable parameters, and FLOPs per image (TFLOPs/img); Dice/M reports Dice per million trainable parameters.

Explore similar work

Spectral Adapters for Segment Anything Model-based Segmentation of Colorectal Liver Metastases in Computed Tomography
Sep 10, 2026Ramtin Mojtahedi, Mohammad Hamghalam, Jacob J. Peoples +6Promptable Image Segmentation Vision Foundation Model Adaptation
School of Computing, Queen’s University, Kingston, ON, Canada · Toronto General Hospital Research Institute, University Health Network, 200 Elizabeth Street, Toronto, ON M5G 2C4, Canada · Department of Electrical Engineering, Qazvin Branch, Islamic Azad University, Qazvin, Iran +6
BaT-SAM: Introducing Inductive Bias into SAM via a Mixture of Convolutional Experts for Brain Tumor Segmentation
Oct 11, 2026Sumiaya Jannat, Md Mahedi Hasan, Md Shamimul Islam +1Medical Image Segmentation Brain Tumor Segmentation
Mymensingh Medical College Mymensingh, Bangladesh · Department of LCSEE West Virginia University Morgantown, WV, USA · Department of CS Florida Polytechnic University Lakeland, Florida, USA +1
Parameter-Efficient Adaptation of SAM3 for Prompt-Driven Surgical Concept Segmentation
Jul 26, 2026Changjing Liu, Yiming Huang, Beilei Cui +5Promptable Image Segmentation Vision Foundation Model Adaptation
Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China · XGEN Labs

Sep 10, 2026J/K move · Enter open · S save

Spectral Adapters for Segment Anything Model-based Segmentation of Colorectal Liver Metastases in Computed Tomography

Ramtin Mojtahedi, Mohammad Hamghalam, Jacob J. Peoples, Natalie Gangai+5

School of Computing, Queen’s University, Kingston, ON, Canada · Toronto General Hospital Research Institute, University Health Network, 200 Elizabeth Street, Toronto, ON M5G 2C4, Canada · Department of Electrical Engineering, Qazvin Branch, Islamic Azad University, Qazvin, Iran+6

Promptable Image Segmentation Vision Foundation Model Adaptation Medical Image Segmentation

Open paper AskPDF arXiv

Accurate segmentation of colorectal liver metastases (CRLM) in contrast-enhanced computed tomography (CT) is important for response assessment, surgical planning, and follow-up. We propose two parameter-efficient spectral adapters for the Segment Anything Model (SAM): the Directional Spectral Adapter (DiSECT) and Spectral Instance-Guided Adapter (SiGA). DiSECT uses singular value decomposition of frozen weights to constrain residual updates to leading spectral directions, while SiGA adds global and input-conditioned gating through a multilayer perceptron. We evaluate these methods on 446 contrast-enhanced CT volumes (355 training, 91 testing) and compare them with LoRA, QLoRA, convolutional adapters (CAD), and a 3D nnU-Net baseline. Experiments consider single-point, three-point, bounding-box, and no-prompt regimes. SiGA achieves the best single-point performance with a Dice score of 0.77, IoU of 0.69, and HD95 of 35.39 mm. Under no-prompt inference, SiGA reaches 0.76 Dice, 0.68 IoU, and 46.76 mm HD95, comparable to the nnU-Net baseline (0.758 Dice). DiSECT uses only 0.14 million trainable parameters. These results show that spectral adapters can efficiently adapt SAM for CRLM segmentation while retaining strong accuracy with limited trainable parameters.

Abstract

Explore similar work

Spectral Adapters for Segment Anything Model-based Segmentation of Colorectal Liver Metastases in Computed Tomography

BaT-SAM: Introducing Inductive Bias into SAM via a Mixture of Convolutional Experts for Brain Tumor Segmentation

Parameter-Efficient Adaptation of SAM3 for Prompt-Driven Surgical Concept Segmentation