Medical Image Representation Learning

Ask about this topic

Overview

The field studies how models can encode useful structure from images such as CT, MRI, X-rays, ultrasound, retinal scans, and pathology slides, especially when expert labels are scarce. Common approaches include contrastive learning, masked image modeling and joint-embedding prediction, as well as vision-language alignment and anatomical or temporal supervision. Researchers test whether these representations transfer through linear probes, few-shot learning, segmentation, retrieval, and evaluation across hospitals or acquisition protocols. A central tension is that specialized pretraining can help on closely matched tasks, but performance is uneven across modalities and domains, and global features may miss small lesions or clinically important spatial detail.

Latest papers

CardsList

260 papers32 papers in the last four weeks, up 45% on the four weeks before. 0.2% of all new papers.

  1. Sparse Autoencoders for Interpretable Medical Image Representation Learning

    Mar 24, 2026Philipp Wesp, Robbie Holland, Vasiliki Sideri-Lampretsa +1Medical Image RetrievalExplainable Medical Image Analysis

    Stanford Center for Artificial Intelligence in Medicine and Imaging, Stanford University, Stanford, CA, USA · Department of Radiology, Stanford University, Stanford, CA, USA · Chair of AI in Healthcare and Medicine, Technical University of Munich, Munich, Germany +1

  2. CoRe: Joint Optimization with Contrastive Learning for Medical Image Registration

    Mar 24, 2026Eytan Kats, Christoph Grossbroehmer, Ziad Al-Haj Hemidi +3Medical Image RegistrationDeformable Image Registration

    Insitute of Medical Informatics, University of Luebeck, Luebeck, Germany

  3. FDIF: Formula-Driven supervised Learning with Implicit Functions for 3D Medical Image Segmentation

    Mar 24, 2026Yukinori Yamamoto, Kazuya Nishimura, Tsukasa Fukusato +33D Medical Image SegmentationMedical Image Representation Learning

    Waseda University · National Institute of Advanced Industrial Science and Technology · The University of Osaka +1

  4. Pretext Matters: An Empirical Study of SSL Methods in Medical Imaging

    Mar 23, 2026Vedrana Ivezić, Mara Pleasure, Ashwath Radhachandran +7Self-Supervised LearningMedical Image Representation Learning

    Biomedical AI Research Lab, University of California, Los Angeles, Los Angeles, CA, USA · Department of Radiology, University of California, Los Angeles, Los Angeles, CA, USA · Department of Bioengineering, University of California, Los Angeles, Los Angeles, CA, USA +4

  5. Chronological Contrastive Learning: Few-Shot Progression Assessment in Irreversible Diseases

    Mar 23, 2026Clemens Watzenböck, Daniel Aletaha, Michaël Deman +9Medical Image AnalysisMedical Image Representation Learning

    Computational Imaging Research Lab, Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Vienna, Austria · Comprehensive Center for Artificial Intelligence in Medicine, Medical University of Vienna, Vienna, Austria · Division of Rheumatology, Department of Medicine III, Medical University of Vienna, Vienna, Austria +3

  6. LoFi: Location-Aware Fine-Grained Representation Learning for Chest X-ray

    Mar 19, 2026Myeongkyun Kang, Yanting Yang, Xiaoxiao LiMedical Image Representation LearningMedical VLMs

    The University of British Columbia, Vancouver, BC V6T 1Z4, Canada · Vector Institute, Toronto, ON M5G 0C6, Canada

  7. Towards Interpretable Foundation Models for Retinal Fundus Images

    Mar 19, 2026Samuel Ofosu Mensah, Camila Roa, Kerol Djoumessi +1Fundus ImagingMedical Imaging Foundation Models

    Hertie Institute for AI in Brain Health, University of Tübingen, Germany · Tübingen AI Center, University of Tübingen, Germany

  8. Multimodal Model for Computational Pathology:Representation Learning and Image Compression

    Mar 19, 2026Peihang Wu, Zehong Chen, Lijian XuComputational PathologyWhole-Slide Image Analysis

    Shenzhen University of Advanced Technology

  9. Pixel-level Counterfactual Contrastive Learning for Medical Image Segmentation

    Mar 17, 2026Marceau Lafargue-Hauret, Raghav Mehta, Fabio De Sousa Ribeiro +2Medical Image SegmentationContrastive Learning

    Department of Computing, Imperial College London, UK

  10. IConE: Batch Independent Collapse Prevention for Self-Supervised Representation Learning

    Mar 16, 2026Konstantinos Almpanakis, Anna KreshukSelf-Supervised LearningRepresentation Collapse

    European Molecular Biology Laboratory, Heidelberg, Germany

  11. TopoCL: Topological Contrastive Learning for Medical Imaging

    Mar 15, 2026Guangyu Meng, Pengfei Gu, Peixian Liang +3Medical Image Representation LearningContrastive Learning

    Dept. of Computer Science and Engineering, University of Notre Dame · The University of Texas Rio Grande Valley · Dept. of IT, Analytics, and Operations, University of Notre Dame

  12. Effective Feature Learning for 3D Medical Registration via Domain-Specialized DINO Pretraining

    Mar 14, 2026Eytan Kats, Mattias P. HeinrichMedical Image RegistrationDeformable Image Registration

    Institute of Medical Informatics, University of Lübeck, Germany

  13. EyeWorld: A Generative World Model of Ocular State and Dynamics

    Mar 14, 2026Ziyu Gao, Xinyuan Wu, Xiaolan Chen +10Ophthalmic Image AnalysisMedical Image Analysis

    School of Optometry, The Hong Kong Polytechnic University, Hong Kong. · Eye Center of Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, 310009, Zhejiang, China · Zhejiang Provincial Key Laboratory of Ophthalmology, Zhejiang Provincial Clinical Research Center for Eye Diseases, Zhejiang Provincial Engineering Institute On Eye Diseases, Hangzhou, 310009, Zhejiang, China +5

  14. Learning Generalizable 3D Medical Image Representations from Mask-Guided Self-Supervision

    Mar 14, 2026Yunhe Gao, Yabin Zhang, Chong Wang +5Medical Image Representation LearningSelf-Supervised Pre-Training

    Stanford University

  15. No Image, No Problem: End-to-End Multi-Task Cardiac Analysis from Undersampled k-Space

    Mar 10, 2026Yundi Zhang, Sevgi Gokce Kafali, Niklas Bubeck +2Medical Image Representation LearningCardiac MRI

    Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital, Munich, Germany · Department of Computing, Imperial College London, United Kingdom · Munich Center for Machine Learning, Technical University of Munich, Germany

  16. VIVID-Med: LLM-Supervised Structured Pretraining for Deployable Medical ViTs

    Mar 10, 2026Xiyao Wang, Xiaoyu Tan, Yang Dai +3Medical Image Representation LearningVision-Language Pretraining

    Shanghai University of Engineering Science, China · National University of Singapore, Singapore · Case Western Reserve University, United States

  17. PDD: Manifold-Prior Diverse Distillation for Medical Anomaly Detection

    Mar 7, 2026Xijun Lu, Hongying Liu, Fanhua Shang +2Medical Image Anomaly DetectionKnowledge Distillation

    Medical School, Tianjin University, Tianjin, China · College of Intelligence and Computing, Tianjin University, Tianjin, China

  18. Efficient Chest X-ray Representation Learning via Semantic-Partitioned Contrastive Learning

    Mar 7, 2026Wangyu Feng, Shawn Young, Lijian XuMedical Image Representation LearningSelf-Supervised Pre-Training

    Shenzhen University of Advanced Technology, Shenzhen, China

  19. ICHOR: A Robust Representation Learning Approach for ASL CBF Maps with Self-Supervised Masked Autoencoders

    Mar 5, 2026Xavier Beltran-Urbano, Yiran Li, Xinglin Zeng +10Medical Image Representation LearningSelf-Supervised Pre-Training

    University of Pennsylvania, Philadelphia, USA · University of Maryland School of Medicine, Baltimore, USA

  20. Semantic Class Distribution Learning for Debiasing Semi-Supervised Medical Image Segmentation

    Mar 5, 2026Yingxue Su, Yiheng Zhong, Keying Zhu +7Semi-Supervised Medical Image SegmentationClass-Imbalanced Learning

    Carnegie Mellon University, United States · Yale University, United States · Xi’an Jiaotong-Liverpool University, China +2

  21. Location-Aware Pretraining for Medical Difference Visual Question Answering

    Mar 5, 2026Denis Musinguzi, Caren Han, Prasenjit MitraMedical VQAVision-Language Pretraining

    Department of Electrical and Computer Engineering, Carnegie Mellon University, Kigali, Rwanda · University of Melbourne, Melbourne, Australia.

  22. Understanding Sources of Demographic Predictability in Brain MRI via Disentangling Anatomy and Contrast

    Mar 4, 2026Mehmet Yigit Avci, Akshit Achara, Andrew King +1Medical Image AnalysisAlgorithmic Bias

    and for the Alzheimer's Disease Neuroimaging Initiative · King’s College London, UK

  23. ProFound: A moderate-sized vision foundation model for multi-task prostate imaging

    Mar 4, 2026Yipei Wang, Yinsong Xu, Weixi Yi +11Medical Imaging Foundation ModelsMedical Image Representation Learning

    UCL Hawkes Institute, University College London, London, UK · Department of Medical Physics and Biomedical Engineering, University College London, London, UK · School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, China +11

  24. Differential privacy representation geometry for medical image analysis

    Mar 1, 2026Soroosh Tayebi Arasteh, Marziyeh Mohammadi, Sven Nebelung +1Neural Representation GeometryHealthcare

    Lab for AI in Medicine, RWTH Aachen University, Aachen, Germany · Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany · INDA Institute, RWTH Aachen University, Aachen, Germany

  25. Stroke outcome and evolution prediction from CT brain using a spatiotemporal diffusion autoencoder

    Feb 28, 2026Adam Marcus, Paul Bentley, Daniel RueckertDiffusion AutoencoderClinical Outcome Prediction

    Imperial College London, London, UK · Technische Universität München, München, Germany

  26. Seeking Necessary and Sufficient Information from Multimodal Medical Data

    Feb 27, 2026Boyu Chen, Weiye Bao, Junjie Liu +5Medical Image Representation LearningCausal Reasoning

    University College London, London, UK · Imperial College London, London, UK · Mingdu Tech, China +1

  27. GLIDE-Reg: Global-to-Local Deformable Registration Using Co-Optimized Foundation and Handcrafted Features

    Feb 27, 2026Yunzheng Zhu, Aichi Chien, Kimaya kulkarni +5Medical Image RegistrationDeformable Image Registration

    Medical & Imaging Informatics, Department of Radiological Sciences, David Geffen School of Medicine at UCLA, USA · Magnetic Resonance Research Labs, Department of Radiological Sciences, David Geffen School of Medicine at UCLA, USA · Department of Radiation Oncology, David Geffen School of Medicine at UCLA, Los Angeles, USA

  28. SegReg: Latent Space Regularization for Improved Medical Image Segmentation

    Feb 26, 2026Puru Vaish, Amin Ranem, Felix Meister +3Latent-Space RegularizationMedical Image Segmentation

    Department of Applied Mathematics, Technical Medical Centre, University of Twente, The Netherlands · Digital Technology and Innovation, Siemens Healthineers, Erlangen, Germany

  29. Mixed Magnification Aggregation for Generalizable Region-Level Representations in Computational Pathology

    Feb 25, 2026Eric Zimmermann, Julian Viret, Michal Zelechowski +7Computational PathologyMedical Image Representation Learning

    Microsoft Research, Cambridge, MA, United States · Paige, NYC, NY, United States

  30. SigVLP: Sigmoid Volume-Language Pre-Training for Self-Supervised CT-Volume Adaptive Representation Learning

    Feb 25, 2026Jiayi Wang, Hadrien Reynaud, Ibrahim Ethem Hamamci +4Medical VLMsMedical Image Representation Learning

    Friedrich-Alexander University Erlangen-Nürnberg, Germany · Department of Quantitative Biomedicine, University of Zurich, Switzerland · ETH AI Center, ETH Zurich, Switzerland +2