cs.CVMar 5, 2026
SaveTell2Adapt: A Unified Framework for Source Free Unsupervised Domain Adaptation via Vision Foundation Model
Organizations: College of Medicine and Biological Information Engineering, Northeastern University · Engineering Research Center of Medical Imaging and Intelligent Analysis, Ministry of Education · Key Laboratory of Medical Image Computing, Ministry of Education, Northeastern University
Abstract
Source Free Unsupervised Domain Adaptation (SFUDA) is critical for deploying deep learning models across diverse clinical settings. However, existing methods are typically designed for low-gap, specific domain shifts and cannot generalize into a unified, multi-modalities, and multi-target framework, which presents a major barrier to real-world application. To overcome this issue, we introduce Tell2Adapt, a novel SFUDA framework that harnesses the vast, generalizable knowledge of the Vision Foundation Model (VFM). Our approach ensures high-fidelity VFM prompts through Context-Aware Prompts Regularization (CAPR), which robustly translates varied text prompts into canonical instructions. This enables the generation of high-quality pseudo-labels for efficiently adapting the lightweight student model to target domain. To guarantee clinical reliability, the framework incorporates Visual Plausibility Refinement (VPR), which leverages the VFM's anatomical knowledge to re-ground the adapted model's predictions in target image's low-level visual features, effectively removing noise and false positives. We conduct one of the most extensive SFUDA evaluations to date, validating our framework across 10 domain adaptation directions and 22 anatomical targets, including brain, cardiac, polyp, and abdominal targets. Our results demonstrate that Tell2Adapt consistently outperforms existing approaches, achieving SOTA for a unified SFUDA framework in medical image segmentation. Code are avaliable at https://github.com/derekshiii/Tell2Adapt.
Figures & tables
Figure 1: Distribution shift across imaging modalities in BraTS. Kolmogorov-Smirnov statistics quantify the divergence between intensity distributions of the four different imaging modalities.
Figure 2: Generalization performance of Tell2Adapt across diverse adaptation tasks. The outermost ring displays the adaptation direction, the middle ring shows Dice score (DICE), while the innermost ring represents the Average Surface Distance (ASD).
Figure 3: Overview of Tell2Adapt, a unified SFUDA framework. The workflow decouples pseudo-label generation from the source model by leveraging VFM guided by text prompts. We introduce CAPR to standardize these prompts, enabling the VFM to generate high-fidelity pseudo-labels. This knowledge is then distilled into the lightweight source model via self-training and refined using VPR.
| Original Prompts | Regularized Prompts |
|---|---|
| Liverr [SEP] Spleen CT [SEP] Abdomen duodenum | Liver in abdomen CT [SEP] Spleen in abdomen CT [SEP] Duodenum in abdomen CT |
| Right ventricle [SEP] Myocardium MRI [SEP] Left ventricle | Right ventricle in cardiac MRI [SEP] Myocardium in cardiac MRI [SEP] Left ventricle in cardiac MRI |
Table 1: Comparison of prompts before and after regularization
Figure 4: Qualitative results of Tell2Adapt demonstrate its strong generalization capability across diverse and challenging SFUDA directions. The figure presents segmentation outputs from Tell2Adapt adapted to various target domains. Specifically, the first and second rows show MR-CT adaptation for abdominal targets in AMOS, while the third row illustrates extreme-gap adaptation in cardiac targets (MR-US), single target adaptation in polyp (Kvasir-CVCDB), and multi-sequence adaptation in brain targets (T1n-T2w and T1c-T2f).
| MR CT | |||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Methods | VFM | Spleen | R. K | L. K | Gallb | Esoph | Liver | Stom | Aortic | V-Cava | Pancr | R. Adre | L. Adre | Duod | Bladder | Pros/Uter | mDICE |
| Baseline | 65.1 30.4 | 71.2 26.9 | 63.2 29.6 | 27.2 27.7 | 42.2 22.1 | 82.2 9.9 | 46.9 28.6 | 72.0 20.3 | 61.7 23.7 | 55.7 23.3 | 39.7 22.8 | 44.7 24.2 | 38.5 22.4 | 0.0 0.0 | 0.0 0.0 | 47.4 24.7 | |
| Supervised | 97.2 1.7 | 96.4 1.7 | 94.7 11.7 | 86.0 13.6 | 84.1 10.0 | 97.6 1.0 | 93.1 5.1 | 95.3 2.8 | 90.2 5.7 | 83.7 12.9 | 77.8 11.0 | 78.9 11.7 | 81.6 10.9 | 83.3 19.3 | 85.5 7.0 | 88.4 7.5 | |
| UPL [ 21 ] | 23.9 2.0 | 7.7 0.2 | 30.2 26.9 | 0.0 0.0 | 12.5 12.7 | 23.6 30.9 | 10.7 12.2 | 9.3 10.0 | 0.7 1.4 | 8.6 8.3 | 0.0 0.0 | 0.0 0.0 | 2.0 7.2 | 0.0 0.0 | 0.0 0.0 | 8.6 9.4 | |
| ProtoContra [ 24 ] | 5.6 6.3 | 11.9 2.3 | 7.3 2.6 | 4.3 3.1 | 37.5 24.1 | 40.1 24.8 | 25.1 21.3 | 38.6 27.8 | 29.2 18.9 | 39.1 20.1 | 0.0 0.0 | 0.0 0.0 | 17.6 16.2 | 0.0 0.0 | 0.0 0.0 | 17.1 15.9 | |
| IAPC [ 3 ] | 8.7 4.5 | 4.5 2.3 | 1.9 1.4 | 1.3 1.5 | 4.5 11.3 | 15.2 2.1 | 14.3 5.6 | 9.7 5.0 | 4.6 1.1 | 2.0 0.6 | 0.0 0.0 | 0.0 0.0 | 2.2 7.4 | 19.8 16.8 | 7.2 6.8 | 6.4 6.1 | |
Table 2: DICE (%, mean std) of segmentation results on abdominal targets. ASD for all anatomical targets is provided in Appendix.
| Methods | VFM | T1n T2w | T2w T1n | ||||||||||||||
| DICE (%, mean std) | ASD (mm, mean std) | DICE (%, mean std) | ASD (mm, mean std) | ||||||||||||||
| TC | SNFH | ET | RC | TC | SNFH | ET | RC | TC | SNFH | ET | RC | TC | SNFH | ET | RC | ||
| Baseline | 19.7 29.1 | 7.5 7.4 | 4.7 5.6 | 30.0 24.9 | 40.6 17.8 | 21.2 8.6 | 40.2 18.8 | 58.9 25.7 | 4.6 4.8 | 13.9 10.5 | 3.2 4.9 | 25.9 38.1 | 48.4 22.9 | 19.4 7.6 | 38.1 15.9 | 57.1 20.6 | |
| Supervised | 61.9 20.8 | 84.3 7.3 | 68.0 14.8 | 86.6 10.9 | 2.9 5.1 | 1.3 0.5 | 1.8 1.9 | 1.0 3.0 | 62.8 20.7 | 83.6 6.7 | 71.6 14.9 | 83.9 14.3 | 2.3 2.5 | 1.4 0.6 | 1.8 2.4 | 1.2 2.9 | |
| UPL [ 21 ] | 47.2 23.4 | 36.9 22.5 | 61.7 48.6 | 46.7 23.5 | 33.6 22.8 | 33.5 20.8 | 6.8 6.1 | 29.5 23.1 | 46.5 34.1 | 10.7 12.1 | 60.3 48.9 | 38.7 24.6 | 16.5 12.8 | 56.9 22.1 | 23.2 14.3 | 22.9 17.0 | |
| ProtoContra [ 24 ] | 0.7 2.3 | 4.9 12.3 | 6.1 1.9 | 15.9 17.8 | 19.7 14.3 | 52.2 16.4 | 30.6 5.6 | 35.0 16.8 | 0.7 1.6 | 11.6 15.1 | 0.4 3.6 | 0.7 0.6 | 55.8 17.5 | 61.4 22.7 | 36.2 21.7 | 52.7 31.6 | |
Table 3: DICE and ASD of segmentation results on brain targets.
| Methods | VFM | MR US | US MR | ||||||
| DICE (%, mean std) | ASD (mm, mean std) | DICE (%, mean std) | ASD (mm, mean std) | ||||||
| LV | MYO | LV | MYO | LV | MYO | LV | MYO | ||
| Baseline | 1.7 0.7 | 3.1 0.3 | 56.2 37.2 | 66.9 42.3 | 3.2 0.6 | 1.8 0.3 | 60.5 34.9 | 56.8 27.5 | |
| Supervised | 94.4 2.7 | 88.1 4.6 | 3.2 1.6 | 4.1 1.3 | 95.8 3.8 | 91.2 2.1 | 0.3 0.1 | 0.3 0.1 | |
| UPL [ 21 ] | 2.8 0.1 | 0.0 0.0 | 90.5 3.2 | 0.0 0.0 | 0.5 0.4 | 2.9 2.1 | 88.9 9.6 | 85.6 11.6 | |
| ProtoContra [ 24 ] | 1.2 1.2 | 17.0 7.2 | 54.9 14.2 | 56.4 13.9 | 5.2 5.8 | 4.9 3.2 | 72.9 12.4 | 74.8 10.5 | |
Table 4: DICE and ASD of segmentation results on cardiac targets.
| Methods | VFM | Kvasir CVCDB | CVCDB Kvasir | ||
|---|---|---|---|---|---|
| DICE (%, mean std) | ASD (mm, mean std) | DICE (%, mean std) | ASD (mm, mean std) | ||
| Baseline | 65.1 21.5 | 5.8 11.1 | 60.2 27.0 | 23.7 20.4 | |
| Supervised | 99.8 0.1 | 0.36 0.1 | 97.4 1.3 | 0.9 1.1 | |
| UPL [ 21 ] | 67.1 14.6 | 2.2 0.9 | 60.2 31.3 | 11.4 17.2 | |
| ProtoContra [ 24 ] | 78.0 12.1 | 0.9 0.6 | 68.9 26.4 | 9.5 15.0 | |
| IAPC [ 3 ] | 73.3 29.6 | 4.4 7.9 | 61.2 26.3 | 13.7 13.2 | |
Table 5: DICE and ASD of segmentation results on polyp.
| MR CT | ||||||||||||||||
| Methods | Spleen | R. K | L. K | Gallb | Esoph | Liver | Stom | Aortic | V-Cava | Pancr | R. Adre | L. Adre | Duod | Bladder | Pros/Uter | mDICE |
| Baseline | 65.1 30.4 | 71.2 26.9 | 63.2 29.6 | 27.2 27.7 | 42.2 22.1 | 82.2 9.9 | 46.9 28.6 | 72.0 20.3 | 61.7 23.7 | 55.7 23.3 | 39.7 22.8 | 44.7 24.2 | 38.5 22.4 | 0.0 0.0 | 0.0 0.0 | 47.4 24.7 |
| Ours w/o CAPR (Chaotic Prompts) | 85.2 1.5 | 0.0 0.0 | 88.3 2.5 | 84.4 10.0 | 84.7 9.4 | 0.0 0.0 | 0.0 0.0 | 85.4 2.1 | 69.7 13.7 | 0.0 0.0 | 70.5 7.6 | 0.0 0.0 | 81.7 6.1 | 0.0 0.0 | 83.7 8.2 | 48.9 37.1 |
| Ours w/o CAPR (Normal Prompts) | 91.1 2.2 | 92.0 4.7 | 89.9 3.3 | 84.6 7.2 | 87.5 2.1 | 92.4 3.5 | 85.8 4.9 | 88.4 5.2 | 90.2 5.7 | 81.1 6.8 | 75.4 9.8 | 73.9 3.7 | 78.2 5.4 | 83.4 6.0 | 88.3 7.2 | 85.5 6.7 |
| Ours w/o VPR (Normal Prompts) | 90.1 2.3 | 91.8 3.7 | 92.5 4.4 | 80.2 12.1 | 83.2 6.1 | 94.9 3.3 | 90.0 7.6 | 91.8 4.1 | 90.1 5.8 | 80.6 11.6 | 72.7 8.0 | 74.6 9.7 | 75.4 16.8 | 82.3 14.4 | 84.9 7.0 | 85.0 7.4 |
| Ours (Chaotic Prompts) | 96.9 1.4 | 96.0 1.3 | 95.7 1.5 | 85.4 10.2 | 85.2 6.6 | 96.8 7.8 | 90.5 10.8 | 93.3 2.1 | 91.5 4.0 | 85.6 5.6 | 77.3 9.9 | 78.4 11.3 | 80.8 6.1 | 83.1 11.0 | 49.2 7.6 | 85.7 12.6 |
Table 6: DICE (%, mean std) of segmentation results in the ablation study on abdominal targets. The corresponding ASD results for all anatomical targets are provided in the Appendix, along with detailed descriptions of the chaotic prompts and normal prompts.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
| MR CT | |||||||||||||||||
| Methods | VFM | Spleen | R. K | L. K | Gallb | Esoph | Liver | Stom | Aortic | V-Cava | Pancr | R. Adre | L. Adre | Duod | Bladder | Pros/Uter | mASD |
| Baseline | 17.5 19.2 | 17.2 18.5 | 15.5 17.8 | 23.1 30.7 | 7.8 9.6 | 11.4 7.2 | 15.3 14.6 | 8.9 9.9 | 6.9 11.2 | 9.1 10.7 | 4.7 3.4 | 4.8 4.2 | 10.3 9.7 | N/A | N/A | 13.0 4.8 | |
| Supervised | 1.1 0.3 | 1.1 0.8 | 1.2 1.4 | 1.2 2.4 | 0.8 0.7 | 2.3 0.5 | 1.9 1.6 | 1.1 1.3 | 1.0 0.4 | 1.4 1.0 | 0.7 0.7 | 0.7 0.7 | 1.6 1.1 | 1.9 1.9 | 2.7 7.7 | 1.4 0.6 | |
| UPL [ 21 ] | 79.0 19.9 | 87.2 15.2 | 72.4 17.6 | N/A | 14.8 17.6 | 37.0 20.6 | 31.6 21.5 | 23.7 12.8 | 50.2 15.4 | 29.3 16.7 | N/A | N/A | 54.4 23.4 | N/A | N/A | 48.0 23.6 | |
| ProtoContra [ 24 ] | 99.5 25.5 | 72.6 16.3 | 67.9 18.4 | 94.1 27.6 | 34.4 26.9 | 75.6 30.4 | 59.4 19.0 | 24.8 16.3 | 37.1 15.1 | 30.8 15.1 | N/A | N/A | 45.8 18.7 | N/A | N/A | 58.4 24.6 | |
| IAPC [ 3 ] | 92.3 80.2 | 65.2 19.9 | 69.8 21.2 | 82.0 37.9 | 25.8 14.7 | 54.4 31.0 | 77.6 47.8 | 29.9 20.0 | 27.7 15.5 | 54.2 32.7 | N/A | N/A | 45.3 25.0 | 31.1 18.2 | 31.2 19.7 | 54.9 22.2 | |
Table 7: ASD (mm, mean std) of segmentation results on abdominal targets.
| MR CT | ||||||||||||||||
| Methods | Spleen | R. K | L. K | Gallb | Esoph | Liver | Stom | Aortic | V-Cava | Pancr | R. Adre | L. Adre | Duod | Bladder | Pros/Uter | mASD |
| Baseline | 17.5 19.2 | 17.2 18.5 | 15.5 17.8 | 23.1 30.7 | 7.8 9.6 | 11.4 7.2 | 15.3 14.6 | 8.9 9.9 | 6.9 11.2 | 9.1 10.7 | N/A | N/A | 10.3 9.7 | N/A | N/A | 13.0 4.8 |
| Ours w/o CAPR (Chaotic Prompts) | 1.7 1.1 | N/A | 2.5 0.5 | 2.6 1.4 | 2.6 0.8 | N/A | N/A | 2.4 0.7 | 5.4 9.3 | N/A | 1.7 0.4 | N/A | 3.9 2.4 | N/A | 2.6 2.5 | 2.8 1.1 |
| Ours w/o CAPR (Normal Prompts) | 1.5 0.9 | 2.2 0.4 | 2.1 1.5 | 2.6 0.8 | 1.7 0.9 | 3.3 1.5 | 2.7 0.4 | 2.0 1.9 | 4.7 2.6 | 1.9 1.1 | 1.5 0.4 | 2.1 1.4 | 2.2 1.9 | 2.9 1.1 | 2.4 0.7 | 2.4 8 |
| Ours w/o VPR (Normal Prompts) | 2.1 1.5 | 2.4 1.3 | 2.4 2.1 | 2.9 1.8 | 2.5 0.7 | 3.5 3.0 | 3.7 2.9 | 2.2 1.7 | 5.3 4.8 | 2.6 1.8 | 1.5 1.2 | 2.4 0.9 | 3.1 1.7 | 3.4 2.7 | 2.6 3.3 | 2.8 0.9 |
| Ours (Chaotic Prompts) | 1.2 0.4 | 2.3 0.1 | 1.3 0.4 | 2.8 2.4 | 1.5 0.5 | 2.3 0.2 | 2.1 5.5 | 1.4 0.9 | 1.5 0.2 | 0.9 0.4 | 0.9 4.2 | 1.0 4.4 | 3.8 0.9 | 1.7 2.0 | 1.8 1.5 | 1.8 0.8 |
Table 8: ASD (mm, mean std) of segmentation results in the ablation study on abdominal targets.
| Target | Chaos Prompt in MR CT | Chaos Prompt in CT MR |
|---|---|---|
| Spleen | abdominal in CT Spleen. | pSleen MR in abdomianl. |
| Right Kidney | Right idney in abdominal CT. | MR kidey in Right abdominal. |
| Left Kidney | Left abdominl kdne in CT. | Lewt kdiney in abdominl MR. |
| Gallbladder | Gallbladedr in abdominal CT. | Gallbladder in abdomital MR. |
| Esophagus | in CT abdominal Esophagus. | abdominal in MR Esophagus. |
| Liver | Liiver in abdominal CT. | abdminal in MR Liver. |
Table 9: Chaos prompt for ablation study in Tell2Adapt.
| Target | Prompt in MR CT | Prompt in CT MR |
|---|---|---|
| Spleen | Spleen in abdominal CT. | Spleen in abdominal MR. |
| Right Kidney | Right kidney in abdominal CT. | Right kidney in abdominal MR. |
| Left Kidney | Left kidney in abdominal CT. | Left kidney in abdominal MR. |
| Gallbladder | Gallbladder in abdominal CT. | Gallbladder in abdominal MR. |
| Esophagus | Esophagus in abdominal CT. | Esophagus in abdominal MR. |
| Liver | Liver in abdominal CT. | Liver in abdominal MR. |
Table 10: Prompts for abdominal targets in Tell2Adapt.
| Target | Prompt in T1n T2w | Prompt in T2w T1n | Prompt in T1c T2f | Prompt in T2f T1c |
|---|---|---|---|---|
| TC | Non-enhancing tumor core in head T2 weighted MR. | Non-enhancing tumor core in head MR naive T1. | Non-enhancing tumor core in head MR T2 FLAIR. | Non-enhancing tumor core in head post-contrast T1 MR. |
| SNFH | Surrounding non-enhancing FLAIR hyperintensity in head T2 weighted MR | Surrounding non-enhancing FLAIR hyperintensity in head MR naive T1. | Surrounding non-enhancing FLAIR hyperintensity in head MR T2 FLAIR. | Surrounding non-enhancing FLAIR hyperintensity in head post-contrast T1 MR. |
| ET | Enhancing tissue in head T2 weighted MR. | Enhancing tissue in head MR naive T1. | Enhancing tissue in head MR T2 FLAIR. | Enhancing tissue in head post-contrast T1 MR. |
| RC | Resection cavity in head T2 weighted MR. | Resection cavity in head MR naive T1. | Resection cavity in head MR T2 FLAIR. | Resection cavity in head post-contrast T1 MR. |
Table 11: Prompts for brain targets in Tell2Adapt.
| Target | Prompt in MR US | Prompt in US MR |
|---|---|---|
| Left Ventricle | Left ventricle in heart ultrasound. | Left ventricle in cardiac MRI. |
| Myocardium | Myocardium in heart ultrasound. | Myocardium in cardiac MRI. |
| Target | Prompt in Kvasir CVCDB | Prompt in CVCDB Kvasir |
| Polyp | Polyp in colon endoscopes. | Polyp in colon endoscopes. |
Table 12: Prompts for cardiac targets and polyp in Tell2Adapt.