Promptable Image Segmentation

Ask about this topic

Overview

Promptable image segmentation studies how a model can turn a user's or another model's cue—such as a click, box, scribble, text description, or example mask—into a precise region. Much work builds on Segment Anything models, adapting them with lightweight adapters or fine-tuning, while other methods generate better prompts automatically or use retrieval and visual examples to guide frozen models. These approaches support interactive correction and open-vocabulary segmentation across settings such as medical imaging, remote sensing, microscopy, and video. A central challenge is that results can be sensitive to prompt choice and domain shift, so evaluations increasingly test realistic human prompts, robustness, uncertainty, and the trade-off between accuracy, interaction effort, and speed.

Latest papers

CardsList

152 papers11 papers in the last four weeks, down 27% on the four weeks before. 0.1% of all new papers.

  1. SCISSR: Scribble-Conditioned Interactive Surgical Segmentation and Refinement

    Mar 19, 2026Haonan Ping, Jian Jiang, Cheng Yuan +3Interactive Medical Image SegmentationMedical Image Segmentation

    Global College, Shanghai Jiao Tong University, Shanghai, China

  2. Eye image segmentation using visual and concept prompts with Segment Anything Model 3 (SAM3)

    Mar 18, 2026Diederick C. Niehorster, Marcus NyströmMedical Image SegmentationPromptable Image Segmentation

    Lund University Humanities Lab & Dept. of Psychology, Lund University, Lund, Sweden · Lund University Humanities Lab, Lund, Sweden

  3. SSP-SAM: SAM with Semantic-Spatial Prompt for Referring Expression Segmentation

    Mar 18, 2026Wei Tang, Xuejing Liu, Yanpeng Sun +1Referring Expression SegmentationText-Guided Image Segmentation

    School of Computer Science and Engineering, Nanjing University of Science and Technology, No. 200 Xiaolingwei Road, Nanjing 210094, China. · Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China. · NExT++ Lab, School of Computing, National University of Singapore, Singapore 119077, Singapore.

  4. Exclusivity-Guided Mask Learning for Semi-Supervised Crowd Instance Segmentation and Counting

    Mar 17, 2026Jiyang Huang, Hongru Chen, Wei Lin +2Crowd CountingInstance Segmentation

    Harbin Institute of Technology, Shenzhen, China · City University of Hong Kong, Kowloon Tong, Hong Kong

  5. EviATTA: Evidential Active Test-Time Adaptation for Medical Segment Anything Models

    Mar 15, 2026Jiayi Chen, Yasmeen George, Winston Chong +1Test-Time AdaptationVision Foundation Model Adaptation

    Department of Data Science & AI, Faculty of Information Technology, Monash University, VIC 3800, Australia · Alfred Health, VIC 3004, Australia

  6. SAIF: A Stability-Aware Inference Framework for Medical Image Segmentation with Segment Anything Model

    Mar 13, 2026Ke Wu, Shiqi Chen, Yiheng Zhong +5Medical Image SegmentationPromptable Image Segmentation

    Xi’an Jiaotong-Liverpool University, Suzhou, China · University College London, London, United Kingdom · Anhui Jianzhu University, Hefei, China

  7. SAP: Segment Any 4K Panorama

    Mar 13, 2026Lutao Jiang, Zidong Cao, Weikai Chen +14Instance SegmentationVision Foundation Models

    The Hong Kong University of Science and Technology (Guangzhou) · LIGHTSPEED · The University of Hong Kong +1

  8. HFP-SAM: Hierarchical Frequency Prompted SAM for Efficient Marine Animal Segmentation

    Mar 13, 2026Pingping Zhang, Tianyu Yan, Yuhao Wang +7Image SegmentationPromptable Image Segmentation

    School of Future Technology, Dalian University of Technology. · Central Hospital of Dalian University of Technology. · Affiliated Zhongshan Hospital of Dalian University. +1

  9. Prompt-Driven Lightweight Foundation Model for Instance Segmentation-Based Fault Detection in Freight Trains

    Mar 13, 2026Guodong Sun, Qihang Liang, Xingyu Pan +2Fault DetectionInstance Segmentation

    the School of Mechanical Engineering, Hubei University of Technology, Wuhan 430068, China · The Hubei Key Laboratory of Modern Manufacturing Quality Engineering, Hubei University of Technology, Wuhan 430068, China · the School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan 430074, China

  10. PicoSAM3: Real-Time In-Sensor Region-of-Interest Segmentation

    Mar 12, 2026Pietro Bonazzi, Nicola Farronato, Stefan Zihlmann +2In-Sensor ComputingPromptable Image Segmentation

    Equal contribution · ETH Zürich, Zürich, Switzerland. · IBM Research, Zürich, Switzerland.

  11. Developing Foundation Models for Universal Segmentation from 3D Whole-Body Positron Emission Tomography

    Mar 12, 2026Yichi Zhang, Le Xue, Wenbo Zhang +16Medical Imaging Foundation Models3D Medical Image Segmentation

    Artificial Intelligence Innovation and Incubation Institute, Fudan University, Shanghai, China. · Shanghai Academy of Artificial Intelligence for Science, Shanghai, China. · Department of Nuclear Medicine and PET Center, Huashan Hospital, Fudan University, Shanghai, China. +6

  12. BALD-SAM: Disagreement-based Active Prompting in Interactive Segmentation

    Mar 11, 2026Prithwijit Chowdhury, Mohit Prabhushankar, Ghassan AlRegibInteractive Image SegmentationPromptable Image Segmentation

    OLIVES at the Georgia Institute of Technology

  13. Prompting with the human-touch: evaluating model-sensitivity of foundation models for musculoskeletal CT segmentation

    Mar 11, 2026Caroline Magg, Maaike A. ter Wee, Johannes G. G. Dobbe +4Medical Image SegmentationMedical Imaging Foundation Models

    Quantitative Healthcare Analysis (QurAI) Group, University of Amsterdam, Science Park 900, Amsterdam, 1098 XH, The Netherlands · Department Biomedical Engineering and Physics, Amsterdam UMC, Meibergdreef 9, Amsterdam, 1105 AZ, The Netherlands · Department Orthopaedics, Amsterdam UMC, Meibergdreef 9, Amsterdam, 1105 AZ, The Netherlands

  14. OilSAM2: Memory-Augmented SAM2 for Scalable SAR Oil Spill Detection

    Mar 10, 2026Shuaiyu Chen, Ming Yin, Peng Ren +2Synthetic Aperture RadarRemote Sensing Image Segmentation

    Multimodal Intelligence Lab, Department of Computer Science, University of Exeter, United Kingdom · College of Oceanography and Space Informatics, China University of Petroleum (East China), China. · Department of Computer Science, University of Exeter, United Kingdom

  15. Disentangling Prompt Dependence to Evaluate Segmentation Reliability in Gynecological MRI

    Mar 9, 2026Elodie Germani, Krystel Nyangoh-Timoh, Pierre Jannin +1Prompt SensitivityPromptable Image Segmentation

    UR, LTSI · Laboratoire Traitement du Signal et de l’Image (LTSI, INSERM UMR 1099), Université de Rennes, Rennes, France · LTSI +1

  16. StructSAM: Structure- and Spectrum-Preserving Token Merging for Segment Anything Models

    Mar 7, 2026Duy M. H. Nguyen, Tuan A. Tran, Duong Nguyen +17Token MergingImage Segmentation

    University of Stuttgart · DFKI · Max Planck Research School +10

  17. Unify the Views: View-Consistent Prototype Learning for Few-Shot Segmentation

    Mar 6, 2026Hongli Liu, Yu Wang, Shengjie ZhaoFew-Shot Semantic SegmentationPromptable Image Segmentation

    School of Computer Science and Technology, Tongji University

  18. Zero-Shot and Supervised Bird Image Segmentation Using Foundation Models: A Dual-Pipeline Approach with Grounding DINO1.5, YOLOv11, and SAM2.1

    Feb 26, 2026Abhinav MunagalaZero-Shot Referring Image SegmentationImage Segmentation

  19. Understanding Annotation Error Propagation and Learning an Adaptive Policy for Expert Intervention in Barrett's Video Segmentation

    Feb 25, 2026Lokesha Rasanjalee, Jin Lin Tan, Dileepa Pitawela +2Interactive Medical Image SegmentationHuman-in-the-Loop Annotation

    School of Computer Science and Information Technology, Adelaide University, Australia · The Australian Institute for Machine Learning (AIML), Australia · Adelaide Medical School, Faculty of Health and Medical Sciences, Adelaide University, Australia +1

  20. CAD-Prompted SAM3: Geometry-Conditioned Instance Segmentation for Industrial Objects

    Feb 24, 2026Zhenran Tang, Rohan Nagabhirava, Changliu LiuImage SegmentationInstance Segmentation

    the Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA

  21. Discover, Segment, and Select: A Progressive Mechanism for Zero-shot Camouflaged Object Segmentation

    Feb 23, 2026Yilong Yang, Jianxin Tian, Shengchuan Zhang +1Camouflaged Object DetectionPromptable Image Segmentation

    Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China.

  22. Prefer-DAS: Learning from Local Preferences and Sparse Prompts for Domain Adaptive Segmentation of Electron Microscopy

    Feb 23, 2026Jiabao Chen, Shan Xiong, Jialin PengDomain AdaptationUnsupervised Domain Adaptation

    College of Computer Science and Technology, Huaqiao University, Xiamen 361021, China

  23. SegMoTE: Token-Level Mixture of Experts for Medical Image Segmentation

    Feb 22, 2026Yujie Lu, Jingwen Li, Sibo Ju +5Medical Image SegmentationVision Foundation Model Adaptation

    Sichuan University · Xinjiang University · Fuzhou University +1

  24. Promptable segmentation with region exploration enables minimal-effort expert-level prostate cancer delineation

    Feb 19, 2026Junqing Yang, Natasha Thorley, Ahmed Nadeem Abbasi +4Medical Image SegmentationInteractive Medical Image Segmentation

    UCL Hawkes Institute; Department of Medical Physics and Biomedical Engineering, University College London, UK. · Centre of Medical Imaging, University College London, UK. · Department of Oncology, Aga Khan University Hospital, Pakistan. +1

  25. Enabling Training-Free Text-Based Remote Sensing Segmentation

    Feb 19, 2026Jose Sosa, Danila Rukhovich, Anis Kacem +1Text-Guided Image SegmentationRemote Sensing Image Segmentation

    SnT, University of Luxembourg

  26. Atlas is Your Perfect Context: One-Shot Customization for Generalizable Foundational Medical Image Segmentation

    Dec 20, 2025Ziyu Zhang, Yi Yu, Simeng Zhu +4Image SegmentationMedical Imaging Foundation Models

    Nanjing University · The Ohio State University · Stanford University

  27. UnSAMv2: Self-Supervised Learning Enables Segment Anything at Any Granularity

    Nov 17, 2025Junwei Yu, Trevor Darrell, XuDong WangImage SegmentationPromptable Image Segmentation

    UC Berkeley

  28. Confidence-Calibrating Regularization for Robust Brain MRI Segmentation Under Domain Shift

    Sep 27, 2025Behraj Khan, Tahir Qasim Syed, Syed Ahmad Chan BukhariImage SegmentationPromptable Image Segmentation

    St. John’s University New York, USA · Institute of Business Administration Karachi Pakistan

  29. Single Point, Full Mask: Velocity-Guided Level Set Evolution for End-to-End Amodal Segmentation

    Aug 3, 2025Zhixuan Li, Yujia Liu, Chen Hui +2Promptable Image SegmentationAmodal Segmentation

    College of Computing and Data Science, Nanyang Technological University, Singapore · School of Computer Science, Peking University, Beijing, China · National Key Laboratory for Multimedia Information Processing, Peking University, Beijing, China +2

  30. SSS: Semi-Supervised SAM-2 with Efficient Prompting for Medical Imaging Segmentation

    Jun 10, 2025Hongjie Zhu, Xiwei Liu, Rundong Xue +5Promptable Image SegmentationSemi-Supervised Medical Image Segmentation

    SWUN · MBZUAI · XJTU +2