Multiscale Switch for Semi-Supervised and Contrastive Learning in Medical Ultrasound Image Segmentation

Jingguo Qu; Xinyang Han; Yao Pu; Man-Lik Chui; Simon Takadiyi Gunda; Ziman Chen; Jing Qin; Ann Dorothy King

cs.CVMar 19, 2026

Multiscale Switch for Semi-Supervised and Contrastive Learning in Medical Ultrasound Image Segmentation

Authors: Jingguo Qu, Xinyang Han, Yao Pu, Man-Lik Chui, Simon Takadiyi Gunda, Ziman Chen, Jing Qin, Ann Dorothy King,

Organizations: Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Hong Kong, China · Centre for Smart Health and School of Nursing, The Hong Kong Polytechnic University, Hong Kong, China · Department of Imaging and Interventional Radiology, The Chinese University of Hong Kong, Hong Kong, China

Abstract

Medical ultrasound image segmentation faces significant challenges due to limited labeled data and characteristic imaging artifacts including speckle noise and low-contrast boundaries. While semi-supervised learning (SSL) approaches have emerged to address data scarcity, existing methods suffer from suboptimal unlabeled data utilization and lack robust feature representation mechanisms. In this paper, we propose Switch, a novel SSL framework with two key innovations: (1) Multiscale Switch (MSS) strategy that employs hierarchical patch mixing to achieve uniform spatial coverage; (2) Frequency Domain Switch (FDS) with contrastive learning that performs amplitude switching in Fourier space for robust feature representations. Our framework integrates these components within a teacher-student architecture to effectively leverage both labeled and unlabeled data. Comprehensive evaluation across six diverse ultrasound datasets (lymph nodes, breast lesions, thyroid nodules, and prostate) demonstrates consistent superiority over state-of-the-art methods. At 5% labeling ratio, Switch achieves remarkable improvements: 80.04% Dice on LN-INT, 85.52% Dice on DDTI, and 83.48% Dice on Prostate datasets, with our semi-supervised approach even exceeding fully supervised baselines. The method maintains parameter efficiency (1.8M parameters) while delivering superior performance, validating its effectiveness for resource-constrained medical imaging applications. The source code is publicly available at https://github.com/jinggqu/Switch

Figures & tables

Fig. 1: A comparison of model performance between previous semi-supervised SOTAs and the proposed Switch method on the lymph node dataset (LN-INT). Our method achieved the best results across five data labeling ratios (i.e. models were trained using only part of the ground truth).

Fig. 2: Overview of our proposed method. $f_{\mathrm{s}}$ and $f_{\mathrm{t}}$ are the student and teacher network, respectively. Stage 1 : The system splits labeled data into two subsets ( $x_{1}$ , $x_{2}$ ) and unlabeled data into two subsets ( $u_{1}$ , $u_{2}$ ), generates pseudo-labels for unlabeled images using $f_{\mathrm{t}}$ , creates spatial masks for patch-based mixing, and combines unlabeled base images with labeled patches to create mixed training samples for semi-supervised learning. Stage 2 : The system performs amplitude switching between labeled and unlabeled image batches in the frequency domain, creates new mixed images using the same spatial masks from MSS, and applies CL to align feature representations between original and frequency-switched mixed images, while enforcing consistency regularization between their model predictions to enhance semi-supervised learning robustness. More details of MSS and FDS modules can be found in Figure 3 and Figure 4 .

Fig. 3: Overview of multiscale switch mechanism. $\mathcal{M}$ represents the binary mask with $p$ coarse patches and $q$ fine patches, while $\mathbin{\sim}\mathcal{M}$ refers to its complement. The white areas in both $\mathcal{M}$ and $\mathbin{\sim}\mathcal{M}$ stand for True, and the pixel at the corresponding position will be retained after multiplication. Vice versa for black regions.

Fig. 4: Overview of frequency domain switch approach. $x$ and $u$ are decomposed into amplitude ( $A_{x},A_{u}$ ) and phase ( $P_{x},P_{u}$ ) domain by fast Fourier transform ( $\mathcal{F}$ ). The frequency region ( $\mathcal{R}$ ) in amplitude domain is switched between $A_{x}$ and $A_{u}$ , and then the pair of $x^{r}$ and $u^{r}$ are reconstructed by inverse fast Fourier transform ( $\mathcal{F}^{-1}$ ). The relative difference of ( $x,x^{r}$ ) and ( $u,u^{r}$ ) made by FDS is shown in the dotted box.

Type	Operation	Explanation
None	Identity	No augmentation
Weak	Resize and crop	Randomly resize the image by [0.8, 1.2]
	Horizontal flip	Horizontally flips the image
	Vertical flip	Vertically flips the image
Strong	Auto contrast	Maximizes (normalize) the image contrast
	Gaussian Blur	Blurs the image with a Gaussian kernel

TABLE I: Summary of augmentations.

Dataset	Training	Validation	Testing	Total
LN-INT	1,034	129	129	1,292
LN-EXT	0	0	374	374
BUSI [ 23 ]	518	65	64	647
DDTI [ 24 ]	510	64	63	637
TN3K [ 25 ]	2,304	575	614	3,493
Prostate [ 26 ]	1,937	215	758	2,910

TABLE II: Overview of datasets and partitions.

Method (Venue)	Params	#Label	LN-INT				LN-EXT				BUSI
Method (Venue)	Params	#Label	Dice $\uparrow$	IoU $\uparrow$	HD95 $\downarrow$	ASD $\downarrow$	Dice $\uparrow$	IoU $\uparrow$	HD95 $\downarrow$	ASD $\downarrow$	Dice $\uparrow$	IoU $\uparrow$	HD95 $\downarrow$	ASD $\downarrow$
U-Net (MICCAI’15)	1.8 M	5%	59.32	49.55	52.98	17.82	48.92	39.26	62.76	21.64	40.28	30.28	70.27	27.77
MT (NIPS’17)	1.8 M		57.45	47.13	56.84	19.98	48.01	37.84	76.50	29.24	41.06	30.71	76.75	29.52
UA-MT (MICCAI’19)	1.8 M		55.42	45.35	58.75	20.60	50.22	40.34	68.24	24.91	40.30	29.91	74.17	29.58
CCT (CVPR’20)	3.7 M		58.42	48.67	50.37	16.35	47.93	37.38	78.52	30.76	40.72	30.06	75.69	29.12
MC-Net (MICCAI’21)	3.8 M		61.01	51.11	51.70	17.96	50.30	39.62	68.85	25.56	40.12	29.83	73.23	28.05

TABLE III: Experimental results of LN-INT, LN-EXT and BUSI datasets, averaged over three runs. ‘Params’ denotes the number of trainable parameters, and ‘#Label’ refers to the labeled training ratio. The best and second best results are highlighted, with

\uparrow

\downarrow

indicating whether higher or lower values are better.

Method (Venue)	Params	#Label	DDTI				TN3K				Prostate
Method (Venue)	Params	#Label	Dice $\uparrow$	IoU $\uparrow$	HD95 $\downarrow$	ASD $\downarrow$	Dice $\uparrow$	IoU $\uparrow$	HD95 $\downarrow$	ASD $\downarrow$	Dice $\uparrow$	IoU $\uparrow$	HD95 $\downarrow$	ASD $\downarrow$
U-Net (MICCAI’15)	1.8 M	5%	76.31	62.51	39.49	15.64	54.99	43.77	52.31	20.70	79.84	73.40	36.38	11.28
MT (NIPS’17)	1.8 M		75.74	61.83	41.05	15.89	55.50	44.03	53.10	20.30	78.99	72.06	38.48	12.05
UA-MT (MICCAI’19)	1.8 M		76.17	62.39	42.08	16.23	56.86	45.16	52.82	20.24	78.83	71.61	38.45	12.16
CCT (CVPR’20)	3.7 M		77.50	64.25	45.57	18.79	59.79	48.36	47.54	18.14	79.60	72.73	33.78	10.25
MC-Net (MICCAI’21)	3.8 M		80.88	68.49	32.89	13.16	58.18	46.78	48.81	18.34	79.43	72.94	28.10	8.29

TABLE IV: Experimental results on the DDTI, TN3K, and Prostate datasets, averaged over three runs. ‘Params’ denotes the number of trainable parameters, and ‘#Label’ refers to the labeled training ratio. The best and second best results are highlighted, with

\uparrow

\downarrow

indicating whether higher or lower values are better.

Fig. 5: A comparison of IoU values of the proposed method and SOTAs on six datasets with five data labeling ratios. The maximum and minimum IoU values obtained by these methods for each dataset are marked in italics. All labeling ratios are displayed in the five corresponding corners of the radar charts.

Fig. 6: A comparison of IoU values of the proposed method and SOTAs on six datasets with 50% labeled data. IoU values of all methods are displayed above the corresponding bar charts.

Fig. 7: Visualization of the proposed method and SOTAs on LN-INT, LN-EXT, BUSI [ 23 ] , DDTI [ 24 ] , TN3K [ 25 ] and Prostate [ 26 ] datasets with 5% labeled data. The first column shows the original images and Full-sup. refers to the results of U-Net trained in a fully supervised manner ( i.e. , 100% labeled). Regions in red, green and yellow indicate the ground truth, false positive and true positive, respectively.

Fig. 8: UMAP visualization of embedding spaces for projection head and segmentation head using 20 LN-INT test samples. In the projection head, dots of the same color represent feature pairs from original and FDS-reconstructed images, demonstrating their proximity after CL. In the segmentation embedding space, yellow and blue dots denote foreground and background features, respectively.

Exp.	MSS	FDS	Augs.	Cont.	Consist.	Dice $\uparrow$	IoU $\uparrow$	HD95 $\downarrow$	ASD $\downarrow$
(a)						59.52	49.86	55.57	18.82
(b)	✓					75.15	66.47	35.98	11.99
(c)	✓	✓		✓		74.17	65.26	36.54	11.64
(d)	✓	✓			✓	75.65	66.91	35.67	11.80
(e)	✓		✓			76.79	68.74	28.42	9.55
(f)	✓	✓	✓	✓		78.24	70.00	29.52	10.30

TABLE V: Ablation study of different modules on the LN dataset with 5% labeled data. Augs., Cont. and Consist. refer to augmentation, contrastive and consistency, respectively.

Fig. 9: Visual comparison of ablation studies on LN-INT dataset with 5% labeled data. The first column shows the original images and (a)-(h) refer to the experiments in Table V . Regions in red, green and yellow indicate the ground truth, false positive and true positive, respectively.

$p$ coarse patches	$q$ fine patches	Dice $\uparrow$	IoU $\uparrow$	HD95 $\downarrow$	ASD $\downarrow$	Max. area
1	2	78.32	70.24	27.86	8.79	9/32
1	4	78.32	70.14	29.10	9.28	10/32
2	2	80.04	71.98	26.51	8.35	17/32
2	4	79.70	71.20	29.87	9.52	18/32
3	2	79.20	71.13	29.96	9.34	25/32
3	4	78.76	70.40	33.74	11.00	26/32

TABLE VI: Ablation study of the amount of coarse and fine patches on the LN dataset with 5% labeled data. The Max. area column indicates the maximum ratio of the total image area occupied by all patches without overlapping.

Weak	Strong	Dice $\uparrow$	IoU $\uparrow$	HD95 $\downarrow$	ASD $\downarrow$
		77.38	69.51	30.32	9.42
✓		79.30	71.32	27.94	9.21
	✓	77.14	68.30	35.04	11.33
✓	✓	80.04	71.98	26.51	8.35

TABLE VII: Ablation study of different augmentations on the LN dataset with 5% labeled data.

Weight	Value	Dice $\uparrow$	IoU $\uparrow$	HD95 $\downarrow$	ASD $\downarrow$
Contrastive	0.0	77.85	69.70	30.40	10.04
	0.05	79.81	71.51	30.91	9.86
	0.1	80.04	71.98	26.51	8.35
	0.2	79.55	70.96	31.37	10.22
	0.5	79.54	71.29	30.29	9.50
Consistency	0.0	78.24	70.00	29.52	10.30

TABLE VIII: Ablation study of contrastive and consistency weights on the LN dataset with 5% labeled data.

Method	LN-INT (10)		LN-EXT		TN3K (23)		Prostate (19)
Method	Dice $\uparrow$	HD95 $\downarrow$	Dice $\uparrow$	HD95 $\downarrow$	Dice $\uparrow$	HD95 $\downarrow$	Dice $\uparrow$	HD95 $\downarrow$
U-Net	34.68	79.49	29.03	85.52	38.24	75.52	75.83	52.20
BCP	50.49	68.96	39.13	66.44	56.80	46.63	81.63	26.47
ABD	53.10	51.85	41.93	56.28	65.07	36.99	79.80	26.49
$\beta$ -FFT	52.94	64.62	38.88	60.48	57.86	47.15	78.81	34.53
Switch	66.62	42.04	60.49	42.34	61.84	37.86	82.44	22.64

TABLE IX: Experimental results under the 1% labeled data scenario. Numbers in parentheses refer to the quantity of labeled data.

Fig. 10: Quantitative comparison between BCP and MSS switching strategies. Top : Switching probability heatmaps (10,000 iterations) reveal geometric bias of BCP versus uniform distribution of MSS. Bottom : Gradient magnitude analysis shows MSS ( $p$ =2, $q$ =2) achieves 54.86% lower variance, indicating smoother mixing transitions, while MSS ( $p$ =2, $q$ =10, similar area compares to BCP 2/3 strategy) also gains 8.64% improvement. Results demonstrate MSS superiority in sample uniformity.

Semi-Supervised Medical Image Segmentation

Semi-supervised medical image segmentation uses labeled and unlabeled scans to train models that delineate organs, lesions, and tissues with fewer annotations.

See all 49 papers

SSS: Semi-Supervised SAM-2 with Efficient Prompting for Medical Imaging Segmentation
Jun 10, 2025Hongjie Zhu, Xiwei Liu, Rundong Xue +5Promptable Image Segmentation Semi-Supervised Medical Image Segmentation
SWUN · MBZUAI · XJTU +2
A Semi-Supervised Framework for Breast Ultrasound Segmentation with Training-Free Pseudo-Label Generation and Label Refinement
Mar 6, 2026Ruili Li, Jiayi Ding, Ruiyu Li +7Ultrasound Image Segmentation Semi-Supervised Medical Image Segmentation
Tohoku University Graduate School of Medicine, Sendai 980-8575, Japan · State Key Laboratory of Oncology in South China, Sun Yat-sen University Cancer Center, Guangzhou 510060, China · Southeast University, School of Cyber Science and Engineering Nanjing, Jiangsu, China +4
Quality-Guided Semi-Supervised Learning for Medical Image Segmentation
Jun 1, 2026Kumar Abhishek, Ghassan HamarnehSemi-Supervised Medical Image Segmentation Medical Image Segmentation
School of Computing Science, Simon Fraser University, Canada

Jun 10, 2025J/K move · Enter open · S save

SSS: Semi-Supervised SAM-2 with Efficient Prompting for Medical Imaging Segmentation

Hongjie Zhu, Xiwei Liu, Rundong Xue, Zeyu Zhang+4

SWUN · MBZUAI · XJTU+2

Promptable Image Segmentation Semi-Supervised Medical Image Segmentation Medical Image Segmentation Semi-Supervised Learning Segment Anything Models

Open paper AskPDF arXiv

In the era of information explosion, efficiently leveraging large-scale unlabeled data while minimizing the reliance on high-quality pixel-level annotations remains a critical challenge in the field of medical imaging. Semi-supervised learning (SSL) enhances the utilization of unlabeled data by facilitating knowledge transfer, significantly improving the performance of fully supervised models and emerging as a highly promising research direction in medical image analysis. Inspired by the ability of Vision Foundation Models (e.g., SAM-2) to provide rich prior knowledge, we propose SSS (Semi-Supervised SAM-2), a novel approach that leverages SAM-2's robust feature extraction capabilities to uncover latent knowledge in unlabeled medical images, thus effectively enhancing feature support for fully supervised medical image segmentation. Specifically, building upon the single-stream "weak-to-strong" consistency regularization framework, this paper introduces a Discriminative Feature Enhancement (DFE) mechanism to further explore the feature discrepancies introduced by various data augmentation strategies across multiple views. By leveraging feature similarity and dissimilarity across multi-scale augmentation techniques, the method reconstructs and models the features, thereby effectively optimizing the salient regions. Furthermore, a prompt generator is developed that integrates Physical Constraints with a Sliding Window (PCSW) mechanism to generate input prompts for unlabeled data, fulfilling SAM-2's requirement for additional prompts. Extensive experiments demonstrate the superiority of the proposed method for semi-supervised medical image segmentation on two multi-label datasets, i.e., ACDC and BHSD. Notably, SSS achieves an average Dice score of 53.15 on BHSD, surpassing the previous state-of-the-art method by +3.65 Dice. Code will be available at https://github.com/AIGeeksGroup/SSS.

Abstract

Semi-Supervised Medical Image Segmentation

Explore similar work

SSS: Semi-Supervised SAM-2 with Efficient Prompting for Medical Imaging Segmentation

A Semi-Supervised Framework for Breast Ultrasound Segmentation with Training-Free Pseudo-Label Generation and Label Refinement

Quality-Guided Semi-Supervised Learning for Medical Image Segmentation