Organizations: Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, Trondheim, Norway · Central Staff, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
Medical image segmentation is essential for clinical workflows such as treatment planning and disease assessment. While specialist tools like TotalSegmentator and MRSegmentator achieve strong performance, they require large annotated datasets for training. Medical foundation models offer a promising alternative through large-scale pretraining that reduces the annotation burden for new tasks, but zero-shot performance remains limited. Parameter-efficient adaptation via Low-Rank Adaptation (LoRA) enables efficient specialization with few trainable parameters, but a key question remains: how many expert-annotated cases are needed to achieve clinically useful segmentation performance? We address this by adapting MedSAM3 with LoRA for five abdominal organs (liver, kidneys, spleen, gallbladder, and pancreas) in CT and MRI using only 1, 2, 5, and 10 annotated cases, evaluating on AMOS22 dataset. With just 10 cases, models achieve performance competitive with specialist systems trained on orders of magnitude more data. Notably, this includes reliable gallbladder segmentation (Dice 0.68 CT, 0.59 MRI) where existing tools fail almost completely (Dice 0.0004), while remaining within 5--10% of MRSegmentator for liver, kidneys, and spleen using over 100 times fewer annotations. Furthermore, external validation on the Whole Heart Segmentation dataset shows that the approach extends to cardiac segmentation, a use case beyond the scope of TotalSegmentator (MRI) and MRSegmentator, achieving competitive left ventricle (LV) performance with only 10 annotated cases. Training requires only3--5,hours per organ on a single GPU, approximately 2--3 times faster than nnU-Net. These findings suggest that ten annotated cases are sufficient for clinically useful segmentation, effectively reducing bottlenecks for both image annotation and training time.
Figures & tables
Figure 1: Phase 1 development analysis. (a) Rank comparison ( r∈{4,8,16} ) for kidneys CT-trained, evaluated on CT test cases. (b) Rank comparison for kidneys MRI-trained, evaluated on MRI test cases. (c) Training curves for rank r=16 , 10-shot, across three organs and both modalities (solid lines: CT-trained evaluated on CT; dashed lines: MRI-trained evaluated on MRI), confirming convergence by epoch 30 across all six organ/modality combinations.
Figure 2: Annotation efficiency comparison on AMOS22 test set. Top row: CT-trained models evaluated on CT test cases. Bottom row: MRI-trained models evaluated on MRI test cases. Dashed curves show MedSAM3+LoRA Dice at 1, 2, 5, and 10 annotated cases. Solid curves show nnU-Net Dice at 10, 50, 100, and full annotation budgets for CT (746, 652, 704, 362, and 491 cases for liver, kidney, spleen, pancreas, and gallbladder respectively) and 10, 50, 100, and full budgets for MRI (193, 175, 180, 146, and 103 cases respectively).
Specialist / Zero-shot (no fine-tuning)
MedSAM3+LoRA (10-shot)
Organ
Mod.
SAM3 \textsczs
MedSAM3 \textsczs
TotalSeg
MRSeg
CT-trained
MRI-trained
Liver
CT
0.262
0.545
0.951
0.959
0.928±0.008
0.881±0.081
MRI
0.453
0.331
0.921
0.956
0.918±0.008
0.927±0.012
Kidney
CT
0.102
0.242
0.903
0.925
0.898±0.006
0.864±0.005
MRI
0.122
0.140
0.875
0.944
0.852±0.022
0.884±0.019
Spleen
CT
0.057
0.249
0.924
0.952
0.898±0.026
0.786±0.005
Table 1: Dice similarity coefficient on the AMOS22 test set. Each row shows results on CT or MRI test data; columns show CT-trained and MRI-trained models, enabling both same-modality and cross-modality comparison. MedSAM3+LoRA results are mean ± std over three independent 10-shot runs. Bold: best result per row.
Method
Train data
CT LV
CT RV
MRI LV
MRI RV
TotalSeg [ 14 ]
large (CT)
0.896
0.915
N/A
N/A
MRSeg [ 3 ]
large
N/A
N/A
N/A
N/A
MedSAM3+LoRA (CT)
10 cases
0.909
0.785
N/A
N/A
MedSAM3+LoRA (MRI)
10 cases
N/A
N/A
0.820
0.637
CARE-WHS best [ 17 ]
86 cases
0.943
0.928
0.935
0.914
Table 2: Cross-center cardiac segmentation on WHS. 3D Dice for LV and RV under same-modality 10-shot settings, compared against TotalSegmentator (CT only) and CARE-WHS 2025 challenge top entries trained on 86 fully annotated cases. N/A indicates the model does not support the modality or task.
Medical image segmentation is vital for clinical diagnosis and quantitative analysis, yet remains challenging due to the heterogeneity of imaging modalities and the high cost of pixel-level annotations. Although general interactive segmentation models like SAM have achieved remarkable progress, their transfer to medical imaging still faces two key bottlenecks: (i) the lack of adaptive mechanisms for modality- and anatomy-specific tasks, which limits generalization in out-of-distribution medical scenarios; and (ii) current medical adaptation methods fine-tune on large, heterogeneous datasets without selection, leading to noisy supervision, higher cost, and negative transfer. To address these issues, we propose SegMoTE, an efficient and adaptive framework for medical image segmentation. SegMoTE preserves SAM's original prompt interface, efficient inference, and zero-shot generalization while introducing only a small number of learnable parameters to dynamically adapt across modalities and tasks. In addition, we design a progressive prompt tokenization mechanism that enables fully automatic segmentation, significantly reducing annotation dependence. Trained on MedSeg-HQ, a curated dataset less than 1% of existing large-scale datasets, SegMoTE achieves SOTA performance across diverse imaging modalities and anatomical tasks. It represents the first efficient, robust, and scalable adaptation of general segmentation models to the medical domain under extremely low annotation cost, advancing the practical deployment of foundation vision models in clinical applications.
Yujie Lu, Jingwen Li, Sibo Ju +5
Sichuan University · Xinjiang University · Fuzhou University +1
Transforming foundation segmentation models from human-prompted tools into auto-promptable annotators is critical for scalable medical data annotation. Current methods commonly depend on external feature matchers or auxiliary networks to automate geometric prompting, but introducing architectural overhead and limiting performance scalability. Although SAM3 natively supports concept segmentation via reusable text prompts, its direct use in medical imaging is hindered by a lack of fine-grained clinical knowledge and the ambiguity of human-written descriptions. In this work, we propose Mask to Concept (M2C), an efficient framework that adapts SAM3 for medical few-shot annotation without external modules, parameter retraining, or manual text engineering. Using only a few labeled images, M2C enables SAM3 to automatically search for transferable visual concepts entirely within its frozen architecture: it initializes a learnable concept embedding, uses it to prompt segmentation, and updates the embedding by gradients of minimizing the concept segmentation error. We further introduce a Hybrid Uncertainty Estimation (HUE) module that calculates the prediction entropy and maps concept predictions back to the box prompts, measuring concept-geometry prompting inconsistency. Highly uncertain samples are flagged actively for human correction, and the corrected masks are then fed back to M2C to continuously search for more precise concept embeddings, forming a self-enhancing annotation loop with minimal expert effort. Experiments on medical segmentation benchmarks show that our method achieves SOTA few-shot segmentation performance and outstanding annotation efficiency, offering a practical and efficient pathway toward scalable medical image labeling. Codes are at https://github.com/Huster-Hq/M2C.
Quan Zhou, Shaoqing Zhai, Qiang Hu +3
Wuhan University of Technology · Huazhong University of Science and Technology · Changzhou United lmaging Healthcare Surgical Technology Co. Ltd.
The Segment Anything Model with Concepts (SAM3) heralds a new paradigm for open-vocabulary segmentation through natural language interaction, offering significant potential for medical image analysis. However, effectively adapting such a powerful vision-language model to the diverse and nuanced domain of medical imaging remains a key challenge. Naive fine-tuning is parameter-inefficient, while standard Mixture-of-Experts (MoE) methods introduce prohibitive computational overhead, limiting their clinical applicability. To address this, we propose Dual-Adaptive SAM3 (DA-SAM3), a novel framework that achieves both high segmentation accuracy and extreme parameter efficiency via a dual-adaptive specialization mechanism. Our first adaptation is task-aware: a Dynamic Expert Router (DER) that sparsely activates the most relevant experts by jointly reasoning about the visual input and the textual concept prompt, mimicking a clinical consultation process. Our second adaptation is parameter-aware: a Decomposed Parameterized Experts (DPE) design that represents each expert as a shared frozen base (inherited from the pretrained SAM3) and a lightweight trainable low-rank delta, reducing MoE parameter overhead by over 80%. Extensive experiments on multiple public medical segmentation benchmarks demonstrate that Dual-Adaptive SAM3 not only matches or exceeds the accuracy of fully fine-tuned SAM3 and standard MoE baselines, but also achieves a notable 5% gain over current state-of-the-art methods, with interpretable results validating its effectiveness. The code is available at: https://github.com/Reconsider80/DA-SAM3.
Ying Chen, Jinyue Li, Kun Wang +2
Shenzhen Research Institute, The Chinese University of Hong Kong · University of Science and Technology of China · Nanyang Technological University +1