Zero-Shot Medical Image Segmentation

Follow this topic Ask about this topic

Zero-shot medical image segmentation studies how pretrained models can delineate organs, lesions, cells, and other structures in medical images without labeled masks for the target task. Common approaches use promptable segmentation models and vision-language models guided by text, points, or boxes, alongside training-free refinement, test-time augmentation, and lightweight adapters. Methods are tested across imaging types and settings, including 2D and 3D scans, histology, ultrasound, and transfer between devices or datasets. A central challenge is making predictions reliable when prompts, anatomy, acquisition conditions, or modalities differ from those seen during pretraining.

Latest papers

18 papers3 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

ReSeg: Retraining-free segmentation of abdominal organs via generalizable semantic representation learning
Oct 21, 2026Derong Yu, Guoyan ZhengZero-Shot Medical Image Segmentation Domain Generalization for Semantic Segmentation
Institute of Medical Robotics, School of Biomedical Engineering, Shanghai Jiao Tong University, No.800 Dongchuan Road, Shanghai, 200240, China
Performance at What Cost? A Sustainability-Aware Performance Index for Cell and Nucleus Instance Segmentation
Oct 7, 2026Eiram Mahera Sheikh, Alaa Tharwat, Wolfram SchenckInstance Segmentation Cost-Aware Inference
Department of Engineering and Mathematics, Bielefeld University of Applied Sciences and Arts, Interaktion 1, 33619 Bielefeld
SegBanana: Steering Unified Multimodal Models into Medical Segmenters
Sep 28, 2026Xiaoye Liang, Ye Yan, Mingze Yin +5Unified Multimodal Models Medical Image Segmentation
Beihang University · Zhongguancun Academy · Zhejiang University
ARCOS: Zero-shot Boundary Localization for Corneal Layer Segmentation Across Optical Coherence Tomography Devices
Sep 3, 2026Nuno Vivas Brás, Benjamin Memmi, Maëlle Bouhassane +4Domain Generalization Optical Coherence Tomography
Laboratoire d’Optique et Biosciences, CNRS, INSERM, ´Ecole polytechnique, Institut Polytechnique de Paris, Palaiseau, France · GRC 32, Transplantation et Th´erapies Innovantes de la Corn´ee, Sorbonne Universit´e, Centre Hospitalier National d’Ophtalmologie des Quinze-Vingts, Paris, France
AlphaRAD: Grounded Zero-Shot Classification in Chest Radiology via $α$ -Corrected Binary Cross Entropy and Factorized Latent Supervision
Sep 1, 2026Jianzhong You, Yuan Gao, Chris McIntoshChest X-Ray Classification Medical Image Classification
Peter Munk Cardiac Centre, University Health Network (UHN), Toronto, Canada · Ted Rogers Centre for Heart Research, Toronto, Canada · Vector Institute, Toronto, Canada +5
GazeRefine: Expert Gaze as a Test-Time Prompt for Training-Free Medical Image Segmentation
Sep 1, 2026Mohammed Oussama Benyahia, Marouane Tliba, Mohamed Amine Kerkouri +10Interactive Medical Image Segmentation Image Segmentation
Université Sorbonne Paris Nord · Northwestern University · VSB-Technical University of Ostrava
Zero-Shot Adaptation of Medical Vision Foundation Models for High-Frequency Micro-Ultrasound Prostate Segmentation
Aug 14, 2026Ayusha Abbas, Saram Abbas, Kabita AdhikariUltrasound Image Segmentation Medical Image Segmentation
Newcastle University Newcastle Upon Tyne, UK
Fine-Grained Vision-Language Pretraining with Organ-Conditioned Pattern Tokens for CT Understanding
Jul 15, 2026Guoliang You, Xiaomeng ChuMedical Image Classification Medical VLMs
University of Pennsylvania · Yale University
Disease-Centric Vision-Language Pretraining with Hybrid Visual Encoding for 3D Computed Tomography
Jun 24, 2026Bowen Shi, Weiwei Cao, Ruifeng Yuan +5Contrastive Learning 3D Medical Imaging
DAMO Academy, Alibaba Group · Hupan Lab, 310023, Hangzhou, China · Shanghai Jiao Tong University, China +2
Vision-Language Models as Zero-Annotation Oracles in Histopathology
Jun 15, 2026Vishal Jain, Giorgio Buzzanca, Sarah Cechnicka +6Computational Pathology Histopathology Image Segmentation
Imperial College London, London, UK · Leiden University Medical Center, Leiden, The Netherlands · KU Leuven, Leuven, Belgium +3
GLINT: Sparsely Gated Vision-Language Alignment for Fine-Grained Radiology Representations
Jun 2, 2026Jonggwon Park, Seongeun Lee, Junhyun Park +6Radiology VLMs Medical Image Segmentation
DEEPNOID Inc., Seoul, South Korea · Department of Artificial Intelligence and Data Science, Sejong University, Seoul, South Korea
Exploring Prompt Alignment with Clinical Factors in Zero-Shot Segmentation VLMs for NSCLC Tumor Segmentation
May 2, 2026Suraj Pai, Thibault Heintz, Cosmin Ciausu +3VLM Robustness Image Segmentation
Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, USA
SegTTA: Training-Free Test-Time Augmentation for Zero-Shot Medical Imaging Segmentation
Apr 19, 2026Yihong Yao, Chunlei Li, Canxuan Gang +4Image Segmentation Medical Imaging Foundation Models
AI Geeks · Qingdao Municipal Hospital · University of Chinese Academy of Sciences
T-Gated Adapter: A Lightweight Temporal Adapter for Vision-Language Medical Segmentation
Apr 9, 2026Pranjal KhadkaMedical Image Segmentation 3D Medical Image Segmentation
Independent Researcher
Automatic Segmentation of 3D CT scans with SAM2 using a zero-shot approach
Mar 24, 2026Miquel Lopez Escoriza, Pau Amargant Alvarez3D Medical Image Segmentation Zero-Shot Medical Image Segmentation
Department of Computer Science, EPFL, Switzerland
Concept-to-Pixel: Prompt-Free Universal Medical Image Segmentation
Mar 18, 2026Haoyun Chen, Fenghe Tang, Wenxin Ma +1Medical Image Segmentation Cross-Modality Medical Image Segmentation
School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China (USTC), Hefei, Anhui 230026, China · Center for Medical Imaging, Robotics, Analytic Computing & Learning (MIRACLE), Suzhou Institute for Advanced Research, USTC, Suzhou, Jiangsu 215123, China · Jiangsu Provincial Key Laboratory of Multimodal Digital Twin Technology, Suzhou Jiangsu, 215123, China +1
Developing Foundation Models for Universal Segmentation from 3D Whole-Body Positron Emission Tomography
Mar 12, 2026Yichi Zhang, Le Xue, Wenbo Zhang +16Medical Imaging Foundation Models 3D Medical Image Segmentation
Artificial Intelligence Innovation and Incubation Institute, Fudan University, Shanghai, China. · Shanghai Academy of Artificial Intelligence for Science, Shanghai, China. · Department of Nuclear Medicine and PET Center, Huashan Hospital, Fudan University, Shanghai, China. +6
Prompt Group-Aware Training for Robust Text-Guided Nuclei Segmentation
Mar 6, 2026Yonghuang Wu, Zhenyang Liang, Wenwen Zeng +2Text-Guided Image Segmentation Medical Image Segmentation
Fudan University, Shanghai, China

Oct 21, 2026J/K move · Enter open · S save

ReSeg: Retraining-free segmentation of abdominal organs via generalizable semantic representation learning

Derong Yu, Guoyan Zheng

Institute of Medical Robotics, School of Biomedical Engineering, Shanghai Jiao Tong University, No.800 Dongchuan Road, Shanghai, 200240, China

Zero-Shot Medical Image Segmentation Domain Generalization for Semantic Segmentation Multi-Organ Segmentation Medical Image Representation Learning Domain Randomization

Open paper AskPDF arXiv

Abdominal organ segmentation is essential in modern medical applications. While existing deep learning approaches demonstrate promising performance, their heavy reliance on large-scale image data raises concerns over generalization capability and privacy risks. This motivates the development of generalizable segmentation training paradigms based on domain-randomized image synthesis, facilitating generalization across unseen real image domains. However, existing paradigms either suffer from limited segmentation performance when applied to abdominal organs, or require dataset-specific finetuning. To address these limitations, we propose ReSeg, a novel two-stage retraining-free framework for domain-agnostic zero-shot abdominal organ segmentation. At the first stage, our organ-agnostic synthetic data-driven pretraining approach exploits semantically and visually diverse synthetic images generated by a dedicated data engine to optimize a Generalizable Semantic Representation Network (GSRNet). The network is thereby capable of learning semantic features robust against variations in anatomical structure, imaging contrast, resolution and spatial context, which provide universal representation capabilities that serve as a foundation for segmentation learning. Subsequently, in the second stage, we continue to train an Organ-specific Encoder-decoder Segmentation Network (OESNet), whose encoder is initialized with the pretrained weights. This training process is facilitated by another dedicated data engine, which generates diversified visual images while strictly retaining the inherent spatial layout of abdominal organs. Comprehensive evaluation on seven abdominal organ segmentation datasets across three different modalities including CT, MRI, and PET demonstrates that ReSeg consistently outperforms existing state-of-the-art (SOTA) methods while eliminating the need for real training images.