Pixel-level Counterfactual Contrastive Learning for Medical Image Segmentation

Marceau Lafargue-Hauret; Raghav Mehta; Fabio De Sousa Ribeiro; Mélanie Roschewitz; Ben Glocker

cs.CVMar 17, 2026

Pixel-level Counterfactual Contrastive Learning for Medical Image Segmentation

Authors: Marceau Lafargue-Hauret, Raghav Mehta, Fabio De Sousa Ribeiro, Mélanie Roschewitz, Ben Glocker

Organizations: Department of Computing, Imperial College London, UK

Abstract

Image segmentation relies on large annotated datasets, which are expensive and slow to produce. Silver-standard (AI-generated) labels are easier to obtain, but they risk introducing bias. Self-supervised learning, needing only images, has become key for pre-training. Recent work combining contrastive learning with counterfactual generation improves representation learning for classification but does not readily extend to pixel-level tasks. We propose a pipeline combining counterfactual generation with dense contrastive learning via Dual-View (DVD-CL) and Multi-View (MVD-CL) methods, along with supervised variants that utilize available silver-standard annotations. A new visualisation algorithm, the Color-coded High Resolution Overlay map (CHRO-map) is also introduced. Experiments show annotation-free DVD-CL outperforms other dense contrastive learning methods, while supervised variants using silver-standard labels outperform training on the silver-standard labeled data directly, achieving $\sim$ 94% DSC on challenging data. These results highlight that pixel-level contrastive learning, enhanced by counterfactuals and silver-standard annotations, improves robustness to acquisition and pathological variations.

Figures & tables

Figure 1: Overview of the proposed methods. Views are formed through scanner (SC) and pleural effusion (PE) counterfactuals, in combination with a traditional augmentation pipeline. (S-)DVD-CL computes three dense similarity computations between the anchor view and each of the target views, and average results. (S-)MVD-CL computes similarity between all views at once.

Panel of Figure 2 — Figure 2: Output CHRO-maps of the four different methods. Similar colors indicate similar encodings. We observe that MVD-CL fails to capture meaningful representations, whereas DVD-CL effectively encodes pixels based on their relative position to the spine, which can be further fine-tuned for segmentation. S-DVD-CL and S-MVD-CL manage to sharply distinguish both lungs.

	d/ $\sigma$ ( $\uparrow$ )	K-means-purity ( $\uparrow$ )
DVD-CL	0.83	0.6979
MVD-CL	1.39	0.6391
S-DVD-CL	17.07	0.9581
S-MVD-CL	12.16	0.9705

Table 1: Evaluation of the learned embedding using the proposed contrastive learning methods before fine-tuning.

	Pre-training Method	DSC ( $\uparrow$ )	$\text{DSC}_{\text{NF}}(\uparrow)$	$\text{DSC}_{\text{PE}}(\uparrow)$	$\text{HD}_{\text{95}}(\downarrow)$	$\text{ASD}(\downarrow)$
	None	87.05 $\pm$ 4.34	93.59 $\pm$ 1.28	84.76 $\pm$ 1.28	21.9 $\pm$ 8.0	5.39 $\pm$ 2.05
Unsupervised	SimCLR [ 3 ]	90.90 $\pm$ 2.94	96.71 $\pm$ 0.28	88.61 $\pm$ 4.15	11.9 $\pm$ 5.3	3.64 $\pm$ 1.63
	VADeR [ 5 ]	92.26 $\pm$ 2.66	97.22 $\pm$ 0.16	90.33 $\pm$ 3.68	10.4 $\pm$ 3.7	2.79 $\pm$ 1.14
	DVD-CL ( ours )	92.76 $\pm$ 2.23	97.36 $\pm$ 0.14	91.08 $\pm$ 3.18	10.5 $\pm$ 3.6	2.86 $\pm$ 1.04
	MVD-CL ( ours )	92.27 $\pm$ 2.73	97.18 $\pm$ 0.25	90.36 $\pm$ 3.79	9.1 $\pm$ 4.9	2.61 $\pm$ 1.16
Supervised	CheXMask	93.65 $\pm$ 2.33	97.81 $\pm$ 0.20	92.02 $\pm$ 3.23	7.3 $\pm$ 1.8	2.46 $\pm$ 1.26

Table 2: Performance (mean

\pm

std ) of the various contrastive pre-training methods after fine-tuning across five folds using Dice Scores (%), 95% Hausdorff Distance (

\text{HD}_{\text{95}}

), and Average Surface Distance (ASD). For Dice Scores, we report the overall performance (DSC), specifically for healthy patients (

\text{DSC}_{\text{NF}}

) and for patients with Pleural Effusion (

\text{DSC}_{\text{PE}}

). Overall best results are highlighted in bold . Best results for both unsupervised and supervised methods are also highlighted.

Figure 3: Qualitative results for lung segmentation using various pre-training methods on PE image. Supervised methods (top row) show better performance compared to unsupervised methods (bottom row). We also observe that the proposed methods outperform their counterpart baselines.

Explore similar work

SPARC: SuperPixel-Aware Region Contrastive Learning for Self-Supervised Dense Prediction
Sep 16, 2026David Szczecina, Yuanpei Xiang, Jitao Hu +4Self-Supervised Visual Representation Learning Contrastive Learning
Systems Design Engineering University of Waterloo Waterloo, Ontario, Canada
ConPro: Contrast Projection Pretraining for Label-Efficient Vessel Segmentation in DSA Sequences
Sep 24, 2026Xinge Guo, Yuanhao Wang, Liqi Shu +2Medical Imaging Semi-Supervised Medical Image Segmentation
Duke University, Durham, NC, USA · Carnegie Mellon University, Pittsburgh, PA, USA · University of Pittsburgh Medical Center, Pittsburgh, PA, USA
Semantic Class Distribution Learning for Debiasing Semi-Supervised Medical Image Segmentation
Mar 5, 2026Yingxue Su, Yiheng Zhong, Keying Zhu +7Semi-Supervised Medical Image Segmentation Class-Imbalanced Learning
Carnegie Mellon University, United States · Yale University, United States · Xi’an Jiaotong-Liverpool University, China +2

Sep 16, 2026J/K move · Enter open · S save

SPARC: SuperPixel-Aware Region Contrastive Learning for Self-Supervised Dense Prediction

David Szczecina, Yuanpei Xiang, Jitao Hu, David Clausi+3

Systems Design Engineering University of Waterloo Waterloo, Ontario, Canada

Self-Supervised Visual Representation Learning Contrastive Learning Self-Supervised Pre-Training Dense Prediction

Open paper AskPDF arXiv

Self-supervised learning (SSL) has become an effective approach for learning visual representations without manual annotations. Among SSL approaches, contrastive learning has been widely used for visual representation learning. However, existing contrastive SSL methods have focused primarily on image-level or pixel-level representation learning, while region-level representation learning remains less explored. We propose SPARC, a region-level contrastive learning framework that leverages superpixels to establish explicit correspondence between augmented image views. SPARC introduces a region contrastive branch that performs superpixel-based feature pooling and optimizes a region-level contrastive objective jointly with a global image-level objective. Under identical settings, SPARC consistently outperforms previous methods such as MoCo-v2 and DenseCL, achieving improvements of up to +9.79 mIoU for semantic segmentation and +4.88 AP for object detection. Ablation studies further demonstrate that region-level objectives produce the strongest performance. Thus, region-level contrastive learning is an effective approach for improving self-supervised visual pretraining for dense prediction tasks. Code repository can be accessed at https://github.com/xRIPEIx/SPARC.

Abstract

Explore similar work

SPARC: SuperPixel-Aware Region Contrastive Learning for Self-Supervised Dense Prediction

ConPro: Contrast Projection Pretraining for Label-Efficient Vessel Segmentation in DSA Sequences

Semantic Class Distribution Learning for Debiasing Semi-Supervised Medical Image Segmentation