cs.CVNov 3, 2026

Mitigating Spurious Invariance in Contrastive Learning: A Probabilistic Self-Supervised Learning Framework for Medical Image Analysis

Authors: Wei Wei, Yading Yuan

Organizations: Columbia University Irving Medical Center, New York, NY 10032, USA · the Data Science Institute, Columbia University, New York, NY 10027, USA

Abstract

Owing to the prohibitive cost of manual annotation for enormous medical images, self-supervised learning (SSL) has gained substantial attention and shown promise in various medical imaging tasks. Among SSL approaches, contrastive learning has emerged as a prominent one, encouraging models to encode semantic information that remains invariant between different augmented views. However, this invariance can be spurious for medical images, as subtle but critical medical features can be suppressed during augmentations. In this paper, we mitigate this issue by encoding the learned feature embeddings as probability distributions rather than fixed vectors, thus capturing a richer and more nuanced spectrum of information. Furthermore, we propose a unified probabilistic framework that seamlessly integrates multiple SSL approaches. Our method achieves superior performance over the state-of-the-art SSL methods on various downstream tasks across multiple datasets, including medical image segmentation and classification. We further validate the method through additional analyses of robustness to stronger augmentations, label efficiency under limited fine-tuning labels, and computational trade-offs. The code is released at https://github.com/wwzjer/ProbSSL.

Figures & tables

Explore similar work

CardsList
  1. Task-Aligned Self-Supervised Learning for Medical Image Analysis: A Task-Oriented Review with Practical Design Guidelines

    May 18, 2026Chathura Wimalasiri, Kishor Nandakishor, Marimuthu PalaniswamiContrastive LearningSelf-Supervised Learning

  2. Chaos-SSL: An Attention-Based Self-Supervised Learning Framework with Chaotic Transformation for Medical Image Classification

    May 26, 2026Joao Batista FlorindoContrastive LearningData Augmentation

  3. Pixel-level Counterfactual Contrastive Learning for Medical Image Segmentation

    Mar 17, 2026Marceau Lafargue-Hauret, Raghav Mehta, Fabio De Sousa Ribeiro +2Medical Image SegmentationContrastive Learning