When Adaptation Hurts: Connecting Representational Drift to OOD Failures in MedSAM Fine-Tuning
Authors: Marko Haralović, Sounic Akkaraju, Carlo Baretta, Vasil Zapryanov, Alexia Briassouli
Organizations: University of Zagreb, Faculty of Electrical Engineering and Computing, Zagreb, Croatia · University of Twente, Enschede, The Netherlands
Foundation models for medical image segmentation, like prompt-based MedSAM, generalize well across domains and modalities, often in zero or few-shot setups. However, their performance depends on the quality of prompts and the adaptation of the models to custom datasets. This work systematically examines how MedSAM generalizes across diverse medical imaging benchmarks, with six adaptation strategies: full-model and encoder-only LoRA, shallow and deep visual prompt tuning (VPT), and decoder-only and full fine-tuning. Models are trained on the International Skin Imaging Collaboration Challenge (ISIC 2018) dataset and evaluated under clean and increasingly noisy prompts on IN and Out-of-Distribution (OOD) datasets: close-OOD PH2 (dermoscopy), far-OOD BUSI (Breast Ultrasound Images Dataset) and CBIS-DDSM (Curated Breast Imaging Subset of the Digital Database for Screening Mammography). We show that adaptation improves performance on IN and close-OOD data but often reduces performance on far-OOD data. Full fine-tuning provides the best tradeoff, while encoder-only LoRA is the strongest parameter-efficient alternative, outperforming standard LoRA and VPT under far-OOD shifts. Using Centered Kernel Alignment (CKA), we show that far-OOD degradation is strongly associated with drift in decoder representations, whereas encoder similarity alone does not explain robustness. This suggests encoder-only LoRA provides stronger robustness than standard LoRA by adapting the encoder to distribution shift in visual features, while preserving the decoder pathway. We further show that random 0-100 pixel jitter on prompts produces more robust and better performing models. We thus conclude that robust MedSAM adaptation requires the combined consideration of prompt noise exposure, domain shift, and representation preservation. We release our code: https://github.com/ImSounic/medsam-vpt
Figures & tables
Figure 1: Prompt perturbation examples.
Figure 2: Robustness to bounding-box perturbation under variable training jitter. Dice is reported across evaluation jitter levels for all adaptation strategies on ID, near-OOD, and far-OOD datasets.
Regime
Method
0
20
r100
Mean
Best train
Best method
ID
Dec. only
-0.016
-0.007
- 0.014
−0.003±0.002
r100
VPT-S
-0.008
-0.001
- 0.009
−0.000±0.003
r100
VPT-D
-0.010
-0.005
- 0.013
−0.001±0.002
r100
LoRA
- 0.012
- 0.016
- 0.035
0.021±0.001
r100
LoRA
Enc-LoRA
-0.027
-0.028
- 0.053
−0.001±0.002
r100
Full FT
-0.006
-0.005
- 0.034
0.007±0.002
r100
Table 1: Mean Dice gain over zero-shot by regime, method, and training jitter. Values are averaged over three runs; the Mean column reports mean ± standard deviation. Bold and underline indicate the best and second-best methods.
CKA feature
ID (ISIC 2018)
Close-OOD (PH 2 )
Far-OOD (BUSI/CBIS)
IoU token layer
0.097 / 0.106
-0.018 / -0.021
0.760 ∗∗∗ / 0.692 ∗∗∗
Output layer
0.060 / 0.026
0.004 / -0.017
0.755 ∗∗∗ / 0.718 ∗∗∗
Decoder layers
-0.075 / -0.107
-0.022 / -0.079
0.726 ∗∗∗ / 0.687 ∗∗∗
Upscaled emb.
0.195 / 0.050
0.294 ∗ / 0.213
0.713 ∗∗∗ / 0.791 ∗∗∗
Encoder layers
-0.282 ∗ / -0.299 ∗
-0.281 ∗ / -0.352 ∗∗
0.091 / 0.059
Table 2: Spearman/Pearson correlations between CKA and Dice gain over zero-shot. Stars denote false discovery rate corrected significance across 30 tests: ∗ q<0.05 , ∗∗ q<0.01 , ∗∗∗ q<0.001 . Total sample size is n=3000 .
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
pm=20
pm=50
pm=100
pm=200
ISIC2018-test
11.70%
30.51%
66.40%
152.26%
PH2
6.57%
16.84%
35.11%
72.06%
BUSI
18.71%
51.36%
113.75%
270.96%
CBIS-DDSM
64.06%
193.49%
485.19%
1327.90%
Overall
17.61%
48.85%
112.67%
280.14%
Appendix
Table 3: Relative increase in tight bounding-box area under expansion due to evaluation jitter. Values report the mean area increase with respect to the original tight bounding box for each dataset and perturbation level.
Method
ISIC 2018 (ID)
PH 2 (near-OOD)
BUSI (far-OOD)
CBIS-DDSM (far-OOD)
Zero-shot
6 [0, 15]
7 [1, 16]
14 [9, 25]
12 [7, 17]
Dec. only
0 [0, 3]
0 [0, 3]
4 [2, 17]
4 [3, 6]
VPT-S
0 [0, 3]
0 [0, 4]
21 [5, 387]
14 [8, 37]
VPT-D
0 [0, 3]
0 [0, 4]
25 [6, 394]
18 [10, 359]
LoRA
0 [0, 1]
0 [0, 1]
26 [6, 402]
47 [13, 438]
Enc-LoRA
0 [0, 0]
0 [0, 0]
3 [1, 11]
5 [3, 7]
Appendix
Table 4: HD95 at zero evaluation jitter. Values are median [interquartile range] in pixels at 1024×1024 ; lower is better.
Figure 3: HD95 across evaluation jitter levels for each adaptation method and dataset. Values are shown on a logarithmic scale in pixels at 1024×1024 ; lower is better.
Figure 4: CKA-performance relationship by method.
Figure 5: Overview of the relationship between CKA similarity and segmentation performance.
Figure 6: CKA similarity across datasets and jitters using perfect bounding boxes.
Figure 7: Dice degradation with increasing bounding-box jitter by dataset and method.
Figure 8: Heatmap of the degradation in Dice with increasing bounding-box jitter, by datasets across models
Dataset
Training
Method
Dice@0
mDice
Dice@200
CKA@0
mCKA
CKA@200
ISIC 2018 ID
0
Zero-shot
0.9072
0.9019
0.7673
1.0000
1.0000
1.0000
Decoder-only
0.9487
0.8798
0.7075
0.9462
0.9432
0.9330
VPT-shallow
0.9457
0.8858
0.7306
0.8845
0.8908
0.8913
VPT-deep
0.9472
0.8863
0.7256
0.8767
0.8791
0.8718
LoRA
0.9545
0.9015
0.7978
0.5685
0.5794
0.5771
LoRA Encoder-only
0.9580
0.8435
0.6450
0.6484
0.6462
0.6379
Appendix
Table 5: Performance and CKA under bounding-box jitter. Dice performance and CKA are reported at zero jitter, averaged across evaluation jitters, and at 200 px jitter. For performance columns, underline marks the best method within each dataset and training-jitter block, while bold marks the best value for each dataset and evaluation column across all training-jitter settings. For CKA columns, bold and underline mark the best and second-best adapted methods, respectively, within each dataset and training-jitter block
Dataset
Train
Method
E0
E20
E50
E100
E200
ISIC 2018 trained on
pm=0
Zero-shot
0.9072
0.9206
0.9144
0.8708
0.7673
Decoder-only
0.9487
0.9349
0.8900
0.8176
0.7102
VPT-shallow
0.9449
0.9327
0.8952
0.8335
0.7352
VPT-deep
0.9470
0.9343
0.8944
0.8281
0.7251
LoRA
0.9556
0.9416
0.9040
0.8522
0.7867
LoRA Encoder-only
0.9580
0.9364
0.8832
0.7951
0.6450
Appendix
Table 6: Dice under bounding-box jitter. Models are trained with 0 px, 20 px, or random 0–100 px jitter and evaluated at 0–200 px jitter. Values are means over 3 seeds; underline denotes the best method within each training block and bold the best per dataset/evaluation jitter.
Medical image segmentation models built on Segment Anything Model (SAM) achieve strong performance on clean benchmarks, yet their reliability often degrades under realistic image corruptions such as noise, blur, motion artifacts, and modality-specific distortions. Existing approaches address either medical-domain adaptation or corruption robustness, but not both jointly. In SAM, we find that these capabilities are concentrated in complementary modules: the image encoder preserves medical priors, while the mask decoder governs corruption robustness. Motivated by this observation, we propose RobustMedSAM, which adopts module-wise checkpoint fusion by initializing the image encoder from MedSAM and the mask decoder from RobustSAM under a shared ViT-B architecture. We then fine-tune only the mask decoder on 35 medical datasets from MedSegBench, spanning six imaging modalities and 12 corruption types, while freezing the remaining components to preserve pretrained medical representations. We additionally investigate an SVD-based parameter-efficient variant for limited encoder adaptation. Experiments on both in-distribution and out-of-distribution benchmarks show that RobustMedSAM improves degraded-image Dice from 0.613 to 0.719 (+0.106) over SAM, demonstrating that structured fusion of complementary pretrained models is an effective and practical approach for robust medical image segmentation.
The advent of vision foundation models, notably the Segment Anything Model (SAM), has catalyzed significant advancements in natural image segmentation. However, their direct transfer to medical imaging remains severely bottlenecked by profound domain gaps, such as cross-modality and cross-center shifts. Existing Parameter-Efficient Fine-Tuning (PEFT) methods facilitate the adaptation of SAM to medical domains; nevertheless, they frequently suffer from performance degradation under severe distribution shifts. This vulnerability primarily stems from the implicit entanglement of heterogeneous frequency components within a shared low-rank subspace, which directly exacerbates sub-optimal structural alignment and localized boundary blurring. To overcome this representational bottleneck, we propose the Fourier-Adaptive Nonlinear Low-Rank Adaptor (FAN-LoRA), a novel frequency-decoupled fine-tuning architecture. FAN-LoRA explicitly separates the optimization space by employing a B-spline-driven low-pass branch for global structural alignment, synergistically coupled with a discrete Fourier high-pass branch for local textural compensation. Extensive experiments across three challenging cross-modality and cross-center benchmarks demonstrate that FAN-LoRA consistently outperforms state-of-the-art PEFT baselines. Compared to the strongest competitors, our method achieves consistent improvements in average Dice scores and notable reductions in boundary errors, while maintaining a compact module size without compromising computational efficiency.
Ziquan Liu, Zhewei Zhu, Xuyang Shi
School of Information and Control Engineering, Southwest University of Science and Technology Mianyang, China
Segmentation models such as Segment Anything Model (SAM) and SAM2 achieve strong prompt-driven zero-shot performance. However, their training on natural images limits domain transfer to medical data. Consequently, accurate segmentation typically requires extensive fine-tuning and expert-designed prompts. We propose DiffuSAM, a diffusion-based adaptation of SAM2 for prompt-free medical image segmentation. Our framework synthesizes SAM2-compatible segmentation mask-like embeddings via a lightweight diffusion-prior from off-the-shelf frozen SAM2 image features. The generated embeddings are integrated into SAM2's mask decoder to produce accurate segmentations, thereby eliminating the need for user prompts. The diffusion prior is further conditioned on previously segmented slices, enforcing spatial consistency across volumes. Evaluated on the BTCV and CHAOS datasets for CT and MRI under Source-Free Unsupervised Domain Adaptation (SF-UDA) and Few-Shot settings, DiffuSAM achieves competitive performance with efficient training and inference. Code is available upon request from the corresponding author.
Tal Grossman, Noa Cahan, Lev Ayzenberg +1
School of Electrical Engineering, Tel Aviv University, Israel · School of Biomedical Engineering, Tel Aviv University, Israel