Medical Image Representation Learning

Ask about this topic

Overview

The field studies how models can encode useful structure from images such as CT, MRI, X-rays, ultrasound, retinal scans, and pathology slides, especially when expert labels are scarce. Common approaches include contrastive learning, masked image modeling and joint-embedding prediction, as well as vision-language alignment and anatomical or temporal supervision. Researchers test whether these representations transfer through linear probes, few-shot learning, segmentation, retrieval, and evaluation across hospitals or acquisition protocols. A central tension is that specialized pretraining can help on closely matched tasks, but performance is uneven across modalities and domains, and global features may miss small lesions or clinically important spatial detail.

Latest papers

CardsList

260 papers32 papers in the last four weeks, up 45% on the four weeks before. 0.2% of all new papers.

  1. Systematic Morphometric Brain Autoencoding for Representation Learning in Neurodegenerative Diseases

    Oct 27, 2026Nuria Balbás-Esteban, Albert Belenguer-Llorens, Elaheh Moradi +3Medical Image Representation LearningSelf-Supervised Pre-Training

    Department of Signal Processing and Communications, Universidad Carlos III de Madrid, Leganés, 28911, Madrid, Spain · Department of Health Sciences and Technology (DHEST), ETH Zurich, 8092, Zürich, Switzerland · SIB Swiss Institute of Bioinformatics, Lausanne, Switzerland +2

  2. Dense Representation Learning and Differentiable Shape Modeling for Spine Landmark Detection

    Oct 27, 2026Augustin Daridon, Sofiane Sifaoui, Ayman Assi +4Anatomical Landmark DetectionMedical Image Representation Learning

    LTCI, Télécom Paris, Institut Polytechnique de Paris, Palaiseau, France · Clinical Biomechanics of the Spine Research Fund (CBS), Paris, France · Laboratory of Biomechanics and Medical Imaging, Saint-Joseph University, Beirut, Lebanon

  3. MedVTok: A General-Purpose Medical Visual Tokenizer

    Oct 26, 2026Chenglong Ma, Yuanfeng Ji, Junzhi Ning +18Image TokenizationVisual Tokenization

    Fudan University, China · Shanghai Innovation Institute, China · Stanford University School of Medicine, United States +4

  4. Low-rank MERA Decomposition of 3D OCT Image Volume for Compression, Denoising, Feature Extraction and Diagnosis of Retinal Diseases

    Oct 26, 2026Ivica Kopriva, Fei Shi, Marija Štanfel +4Optical Coherence TomographyMedical Image Representation Learning

    Ivica Kopriva is with Division of Computing and Data Science, Rudjer Boskovich Institute, Zagreb,Croatia · Fei Shi and Gaowei Li are with the School of Electronics and Information Engineering, Soochow University, Suzhou, China, and the Jiangsu Key Laboratory of Artificial Intelligence and Medical Imaging, Soochow University, Suzhou, China · Marija Štanfel is with Department of Ophthalmology, University Hospital Center Zagreb, Zagreb, Croatia +2

  5. VTRL-Med: Progressive Visual Refinement with Topology-Guided Regularization for Medical Image Clustering

    Oct 23, 2026Guangyu Meng, Pengfei Gu, Peixian Liang +3Deep ClusteringMedical Image Analysis

    Dept. of Computer Science and Engineering, University of Notre Dame · Dept. of Computer Science, The University of Texas Rio Grande Valley · Dept. of IT, Analytics, and Operations, University of Notre Dame

  6. Learning Pathology-Sensitive Representations for Unsupervised Retinal OCT Anomaly Detection

    Oct 23, 2026Tania Haghighi, Sina Gholami, Behafarin Emam +5Medical Image Anomaly DetectionUnsupervised Anomaly Detection

    University of North Carolina at Charlotte, Charlotte, North Carolina, USA · University of Illinois Chicago, Chicago, IL, USA · Byers Eye Institute at Stanford, Stanford University School of Medicine, Stanford, CA, USA +1

  7. K-CLUE: A Segmentation-to-Clustering Framework for Ki67 Cell Typing

    Oct 23, 2026Ana Beatriz Vieira, Sara P. Oliveira, Diana Montezuma +3Histopathology Image ClassificationMedical Image Representation Learning

    Instituto Superior Técnico, Universidade de Lisboa, Portugal · INESC-ID, Lisboa, Portugal · The Netherlands Cancer Institute Amsterdam, The Netherlands +3

  8. UNIVERS: Learning UNIversal Voxel-wise Embedding via Randomized Synthesis for generalizable one-shot landmark localization

    Oct 21, 2026Derong Yu, Qingsong Zhao, Yanbing Yue +3Anatomical Landmark DetectionMedical Image Representation Learning

    Institute of Medical Robotics, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China. · Department of Cardiology, Shanghai General Hospital (originally named Shanghai First People’s Hospital), Shanghai Jiao Tong University School of Medicine, Shanghai 200080, China.

  9. ReSeg: Retraining-free segmentation of abdominal organs via generalizable semantic representation learning

    Oct 21, 2026Derong Yu, Guoyan ZhengZero-Shot Medical Image SegmentationDomain Generalization for Semantic Segmentation

    Institute of Medical Robotics, School of Biomedical Engineering, Shanghai Jiao Tong University, No.800 Dongchuan Road, Shanghai, 200240, China

  10. Brain2Vec: A Self-Supervised 3D Foundation Model for Structural Brain MRI

    Oct 20, 2026Spiros Maggioros, Guray Erus, Gareth Harman +4Medical Imaging Foundation ModelsMedical Image Representation Learning

    University of Pennsylvania

  11. TAN: A Size- and Form-Adaptive Masked-Convolution Encoder for Tissue Graphs

    Oct 16, 2026Jan Herschel, Anselm von GladissMedical Image Representation LearningComputational Pathology

    Institute for Computer Science, University of Koblenz, Germany · MTI Mittelrhein, University of Koblenz, Germany

  12. L2GSSL: Label-Efficient Local-to-Global Self-Supervised Learning for Volumetric Healthcare Data

    Oct 15, 2026Po-Cheng Cheng, Hung-Hsuan ChenLung Nodule DetectionMedical Image Representation Learning

    Computer Science and Information Engineering, National Central University

  13. Beyond the Encoder: Object-Centric Representations Emerging from Segmentation Foundation Models

    Oct 13, 2026Michalis Pistos, Bartłomiej W. PapieżCancer Subtype ClassificationObject-Centric Representation Learning

    Big Data Institute, Li Ka Shing Centre for Health Information and Discovery, Nuffield Department of Population Health, University of Oxford, Oxford, OX3 7LF, United Kingdom

  14. LiFUSE: Frequency-Spatial Unified Self-Supervised Pretraining for Liver Disease Analysis on Non-contrast CT

    Oct 13, 2026Hao Zhang, Yi Huang, Wei Liu +6Medical Image AnalysisMedical Image Representation Learning

    College of Biomedical Engineering, Fudan University · DAMO Academy, Alibaba Group · Hupan Lab, HangZhou +2

  15. dMRI-F: A Foundation Model for Generalizable Diffusion MRI Analysis from Sparse Acquisitions

    Oct 13, 2026Xi Zhu, Yijie Li, Xiaofan Wang +4Diffusion MRIMedical Imaging Foundation Models

    University of Electronic Science and Technology of China.

  16. Where to Look Matters: Rethinking Sub-Volume Sampling in 3D Medical Self-supervised Learning

    Oct 9, 2026Junkai Liu, Le ZhangMedical Image Representation LearningSelf-Supervised Pre-Training

    School of Engineering, University of Birmingham

  17. Exploring the Latent Structure of AMD Severity Using Retinal Foundation Models

    Oct 9, 2026Niccolò Fineschi, Jovana Panic, Atsushi Watanabe +7Medical Image Representation LearningMedical Imaging Foundation Models

    Department of Electronics and Telecommunications, Polytechnic of Turin, Turin, Italy. · Premium Research Institute for Human Metaverse Medicine (WPI-PRIMe), The University of Osaka, Osaka, Japan. · D3 Center, The University of Osaka, Osaka, Japan. +2

  18. EchoTab: Learning Measurement-Grounded Representations for Echocardiographic Diagnosis through Tabular Reconstruction

    Oct 9, 2026Michelle Espranita Liman, Ioana Ionac, Alexander Müller +3Medical Image Representation LearningRepresentation Learning

    Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital, Munich, Germany · Department of Cardiology, TUM University Hospital, Munich, Germany · Department of Computing, Imperial College London, London, UK +1

  19. The Predictability Bias of Joint-Embedding Predictive Learning in Medical Imaging

    Oct 9, 2026Fabian Drexel, Bernhard Kainz, Jakob N. Kather +4Medical Image Representation LearningJoint-Embedding Predictive Architecture

    Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital, Munich, Germany · Technical University of Munich, School of Medicine and Health, TUM University Hospital, Klinikum rechts der Isar, Department of Diagnostic and Interventional Radiology, Munich, Germany · Dept. AIBE, Friedrich-Alexander University Erlangen-Nürnberg, Erlangen, Germany +7

  20. Dataset-Specific Encoding of Image Content Underlies Cross-Dataset Failure in Brain MRI Foundation Models

    Oct 9, 2026Minh Sao Khue Luu, Bair N. TuchinovMedical Imaging Foundation ModelsMedical Image Representation Learning

    The Artificial Intelligence Research Center of Novosibirsk State University, Novosibirsk 630090, Russia

  21. Learning Features, Not Layouts: Feature-Axis Attention as a Transfer Prior for Medical Imaging

    Oct 9, 2026Aon Safdar, Mohamed SaadeldinMedical Image Representation LearningVision Transformer

    University College Dublin, Ireland

  22. ΔΔRepresentation: Geometry Supervised Representation Learning of Phenotypes via Counterfactual Reasoning for Medical VLMs

    Oct 7, 2026Hao Wang, Qiwei Zeng, Jinghao Lin +6Medical Image Representation LearningMedical VLMs

    The University of Sydney · Shanghai Jiao Tong University · Northeastern University +1

  23. Geometry-Supervised Visual Representation Learning for Multi-Phenotype Lesion Interpretation in Medical VLMs

    Oct 7, 2026Hao Wang, Qiwei Zeng, Shuchang Ye +5Visual Representation LearningMedical Image Representation Learning

    The University of Sydney · Shanghai Jiao Tong University · Northeastern University

  24. Masked Feature Encoding for Large-Scale Whole Slide Image Representation

    Oct 7, 2026Haoyu He, Basile Tessier-Cloutier, Yang Wang +1Whole-Slide Image AnalysisMedical Image Representation Learning

    Department of Computer Science and Software Engineering (CSSE), Concordia University, Montreal, Canada · McGill University, Montreal, Canada · Mila – Quebec AI Institute, Montreal, Canada

  25. NeuroCBIR: A Fast and Accurate Image Retrieval System for Whole-Brain and Region-Specific MRI

    Oct 5, 2026Felix Nieto-del-Amor, Jingru Fu, J. -Sebastian Muehlboeck +3Image RetrievalMedical Image Representation Learning

  26. EchoDino: A pediatric foundation model for transferable echocardiographic analysis across the lifespan

    Oct 4, 2026Sheng Cheng, Donnchadh M. O'Sullivan, Daniel J. Penny +3Medical Imaging Foundation ModelsSelf-Supervised Learning

    Department of Computer Science, Rice University, Houston, TX, USA · Division of Pediatric Cardiology, Baylor College of Medicine, Houston, TX, USA · Texas Children’s Hospital, Houston, TX, USA

  27. MAGEFormer: Learning Metric-Consistent Representations for Anisotropic CT Segmentation

    Oct 2, 2026Jiaying Li, Paolo RemagninoMedical Image Segmentation3D Medical Image Segmentation

    Department of Computer Science, Durham University, Durham, UK

  28. Learning Subject-Specific Anatomical Representations via Manifold Expansion: Application to Accelerated Multi-Contrast MRI

    Oct 2, 2026Ruimin Feng, Wanyu Bian, Albert Jang +2Accelerated MRI ReconstructionMagnetic Resonance Imaging

    Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, Massachusetts, United States · Harvard Medical School, Boston, Massachusetts, United States · Department of Radiology, Massachusetts General Hospital, Charlestown, Massachusetts, United States

  29. FiberGeoText: A Vision-Language Model for Population- Level Organization of Superficial White Matter

    Oct 2, 2026Yuqian Chen, R. Jarrett Rushmore, Guikun Chen +5NeuroimagingMedical Image Representation Learning

    Department of Radiology, Mass General Brigham, Boston MA, USA · Harvard Medical School, Boston MA, USA · Department of Anatomy and Neurobiology, Boston University Chobanian & Avedisian School of Medicine, Boston MA, USA +2

  30. STRIDE: Spatial-Temporal Representation for Interval-conditioned Disease Evolution in Longitudinal Glioblastoma MRI

    Oct 2, 2026Wenhao Guo, Changchang Yin, Pierre Giglio +3Longitudinal Medical Image AnalysisMedical Image Representation Learning

    Department of Computer Science and Engineering, The Ohio State University, Columbus, OH, USA · Department of Biomedical Informatics, The Ohio State University, Columbus, OH, USA · Department of Neurology, The Ohio State University, Columbus, OH, USA