We present CSSL, a contrastive semi-supervised method for medical image segmentation. CSSL samples class-wise patches from labelled images and from confident pseudo-labels, separates their embeddings with a supervised contrastive loss with mixed hard negatives, and uses the Silhouette coefficient of the embedding clusters to set the strength of a feature-perturbation consistency loss. On MoNuSeg, patch embeddings are obtained by masked pooling of the encoder features of the unmasked image, nucleus rims form an additional contrastive class and a memory bank stores past embeddings; on CHAOS, patches are pixel-masked and no memory bank is used. The CSSL losses are added to a weak-to-strong pseudo-labelling recipe. We follow the protocols of published baselines: Self-Loop on MoNuSeg and DAFNet on CHAOS. On MoNuSeg with 50% labels, CSSL reaches 80.87 ± 0.17 nuclei F1 over three label splits, compared with 79.10 for Self-Loop and 78.73 for the same recipe without CSSL. On CHAOS T2 with 25% labels, CSSL reaches 89.01 Dice, compared with 82.0 for DAFNet and 88.94 for the recipe alone. Ablations show that the contrastive branch and, on MoNuSeg, the boundary class and feature-level pooling account for most of the improvement. Code: https://github.com/prinshul/CSSL
Figures & tables
Figure 1: CSSL on top of the recipe. Gray: the weak-to-strong recipe, identical in every semi-supervised run of this paper. Blue: CSSL’s components. Class-wise patches from labels and confident pseudo-labels are embedded by masked pooling of encoder features. A supervised contrastive loss with mixed hard negatives and a memory bank separates them. The Silhouette coefficient of the embedding clusters sets the noise level λ of a feature-perturbation consistency loss. The diagram shows the MoNuSeg configuration. On CHAOS, embeddings come from pixel-masked patches, no memory bank is used, and the boundary class applies to binary tasks only.
Figure 2: What the contrastive branch sees (synthetic illustration). (a) A 64×64 window. (b) Pixel-level masking: a nucleus patch and a background patch with the pixels of the other class set to zero. (c) Feature-level masking encodes the unmasked image and pools encoder features over the cells covered by the class. The cell shading shows the pooling weight mˉ . (d) The boundary class: nucleus pixels within w=2 pixels of the background (orange) form a separate contrastive class.
Figure 3: Main results. Dots are individual label splits (MoNuSeg seeds; DAFNet’s CHAOS splits). Horizontal ticks (MoNuSeg) and diamonds (CHAOS) are means. Dashed lines are the published baselines. CHAOS uses the best-validation checkpoint (DAFNet’s rule).
Table 1: MoNuSeg, 50% labels (15 of 30 training images), nuclei F1 (%) on the 14 test images, final weights. Published numbers are single runs on an unspecified labelled subset. Ours are three random labelled subsets with the same ResUNet-18.
Semi-supervised medical image segmentation uses labeled and unlabeled scans to train models that delineate organs, lesions, and tissues with fewer annotations.
Image segmentation relies on large annotated datasets, which are expensive and slow to produce. Silver-standard (AI-generated) labels are easier to obtain, but they risk introducing bias. Self-supervised learning, needing only images, has become key for pre-training. Recent work combining contrastive learning with counterfactual generation improves representation learning for classification but does not readily extend to pixel-level tasks. We propose a pipeline combining counterfactual generation with dense contrastive learning via Dual-View (DVD-CL) and Multi-View (MVD-CL) methods, along with supervised variants that utilize available silver-standard annotations. A new visualisation algorithm, the Color-coded High Resolution Overlay map (CHRO-map) is also introduced. Experiments show annotation-free DVD-CL outperforms other dense contrastive learning methods, while supervised variants using silver-standard labels outperform training on the silver-standard labeled data directly, achieving ∼94% DSC on challenging data. These results highlight that pixel-level contrastive learning, enhanced by counterfactuals and silver-standard annotations, improves robustness to acquisition and pathological variations.
Marceau Lafargue-Hauret, Raghav Mehta, Fabio De Sousa Ribeiro +2
Department of Computing, Imperial College London, UK
Training accurate medical image segmentation models requires large amounts of densely annotated data, which is costly and time-consuming to obtain. Semi-supervised learning (SSL) alleviates this by learning from both abundant unlabeled data and limited labeled data. However, most modern SSL methods rely on pseudolabels for unlabeled data, and typically assess their reliability through model confidence or uncertainty, measures that are self-referential and lack explicit grounding in segmentation quality. Instead, we propose a quality-guided SSL framework that trains a dedicated network to estimate segmentation quality from image-mask pairs. The predictor is trained on variable-quality masks generated through synthetic corruptions augmented with imperfect outputs from partially trained segmentation models, capturing realistic error patterns encountered during training. We integrate the quality predictor into SSL through two complementary mechanisms: a quality-aware regularization loss and a quality-based pseudolabel sample reweighting scheme. We show that our method serves as a drop-in enhancement to existing SSL frameworks. Extensive experiments across five datasets and multiple architectures demonstrate consistent improvements over competing SSL methods, advancing the state-of-the-art in semi-supervised medical image segmentation.
Kumar Abhishek, Ghassan Hamarneh
School of Computing Science, Simon Fraser University, Canada
In the era of information explosion, efficiently leveraging large-scale unlabeled data while minimizing the reliance on high-quality pixel-level annotations remains a critical challenge in the field of medical imaging. Semi-supervised learning (SSL) enhances the utilization of unlabeled data by facilitating knowledge transfer, significantly improving the performance of fully supervised models and emerging as a highly promising research direction in medical image analysis. Inspired by the ability of Vision Foundation Models (e.g., SAM-2) to provide rich prior knowledge, we propose SSS (Semi-Supervised SAM-2), a novel approach that leverages SAM-2's robust feature extraction capabilities to uncover latent knowledge in unlabeled medical images, thus effectively enhancing feature support for fully supervised medical image segmentation. Specifically, building upon the single-stream "weak-to-strong" consistency regularization framework, this paper introduces a Discriminative Feature Enhancement (DFE) mechanism to further explore the feature discrepancies introduced by various data augmentation strategies across multiple views. By leveraging feature similarity and dissimilarity across multi-scale augmentation techniques, the method reconstructs and models the features, thereby effectively optimizing the salient regions. Furthermore, a prompt generator is developed that integrates Physical Constraints with a Sliding Window (PCSW) mechanism to generate input prompts for unlabeled data, fulfilling SAM-2's requirement for additional prompts. Extensive experiments demonstrate the superiority of the proposed method for semi-supervised medical image segmentation on two multi-label datasets, i.e., ACDC and BHSD. Notably, SSS achieves an average Dice score of 53.15 on BHSD, surpassing the previous state-of-the-art method by +3.65 Dice. Code will be available at https://github.com/AIGeeksGroup/SSS.