Promptable Image Segmentation

Ask about this topic

Overview

Promptable image segmentation studies how a model can turn a user's or another model's cue—such as a click, box, scribble, text description, or example mask—into a precise region. Much work builds on Segment Anything models, adapting them with lightweight adapters or fine-tuning, while other methods generate better prompts automatically or use retrieval and visual examples to guide frozen models. These approaches support interactive correction and open-vocabulary segmentation across settings such as medical imaging, remote sensing, microscopy, and video. A central challenge is that results can be sensitive to prompt choice and domain shift, so evaluations increasingly test realistic human prompts, robustness, uncertainty, and the trade-off between accuracy, interaction effort, and speed.

Latest papers

CardsList

152 papers11 papers in the last four weeks, down 27% on the four weeks before. 0.1% of all new papers.

  1. GarmentSegAgent: Autonomous Garment Segmentation for Web Fashion Images via Textual Self-Refinement

    Oct 27, 2026Jia Chen, Chao Huang, Xinrong Hu +3Text-Guided Image SegmentationSemantic Segmentation

    School of Computer Science and Artificial Intelligence, Wuhan Textile University, Wuhan, China · Engineering Research Center of Hubei Province for Clothing Information, Wuhan Textile University, Wuhan, China · Australian Institute for Machine Learning (AIML), Adelaide University, Adelaide, Australia

  2. GuidedSAM-Plume: An Expert-Guided Segmentation Framework for Volcanic Plumes

    Oct 21, 2026Taddeo D'Adamo, Simon Thivet, Catherine Pothier +6Remote Sensing Image SegmentationInteractive Image Segmentation

    LIRIS, imagine · UNIGE

  3. Efficient SAM3 Adaptation for Multi-Class Semantic Segmentation via Dense Competitive Representations

    Oct 20, 2026Wenbin Liao, Hao Zhu, Yike Ma +2Promptable Image SegmentationSemantic Segmentation

    Institute of Computing Technology, Chinese Academy of Science · School of Computer Science and Technology, University of Chinese Academy of Sciences

  4. BaT-SAM: Introducing Inductive Bias into SAM via a Mixture of Convolutional Experts for Brain Tumor Segmentation

    Oct 11, 2026Sumiaya Jannat, Md Mahedi Hasan, Md Shamimul Islam +1Medical Image SegmentationBrain Tumor Segmentation

    Mymensingh Medical College Mymensingh, Bangladesh · Department of LCSEE West Virginia University Morgantown, WV, USA · Department of CS Florida Polytechnic University Lakeland, Florida, USA +1

  5. Text-Guided Medical Image Segmentation via Residual Aggregation and Dense Alignment Representations

    Oct 10, 2026Jie Gui, Hang Tu, Yunfeng Dou +3Text-Guided Image SegmentationPromptable Image Segmentation

    Key Laboratory of Intelligent Computing and Signal Processing, Ministry of Education, Anhui University, Hefei, China · School of Artificial Intelligence, Anhui University, Hefei, China · School of Computer Science and Technology, Anhui University, Hefei, China +1

  6. TAG: Calibrating SAM3 for Universal Image Segmentation without Dense Supervision

    Oct 10, 2026Hao Zhu, Heyu Sun, Shuo Jin +2Promptable Image SegmentationOpen-Vocabulary Semantic Segmentation

  7. One-Shot Adaptive Segmentation For Scientific Images

    Oct 7, 2026Tejaswi V. Panchagnula, Allison M. Davis, Fengqing ZhuImage SegmentationVision Foundation Model Adaptation

    Elmore Family School of Electrical and Computer Engineering Purdue University West Lafayette, Indiana, USA 47907

  8. Topology-Informed Prompt-Conditioned Universal Segmentation of Uterine Structures from Ultrasound and MRI

    Oct 5, 2026Yongheng Sun, Yuexi Gu, Jingwen Sun +2Ultrasound Image SegmentationCross-Modality Medical Image Segmentation

    Department of Radiology and BRIC, UNC at Chapel Hill, Chapel Hill, NC 27599, USA

  9. Seeing as Humans Do: Learning from Motion to Segment Anything Without Supervision

    Sep 30, 2026Weijian Jian, Xiaoyue Zhang, Bin Xiao +5Image SegmentationSelf-Supervised Learning

    AI Research · Independent Researcher · University of Ottawa +1

  10. Universal Cross-Prompt Adversarial Attacks on Promptable Concept Segmentation

    Sep 30, 2026Ziqi Zhou, Yifan Hu, Yufei Song +5Adversarial AttacksPromptable Image Segmentation

    College of Computer Science, Chongqing University · School of Cyber Science and Engineering, Huazhong University of Science and Technology · School of Computer Science and Technology, Huazhong University of Science and Technology +2

  11. S4VY: Segment Anything in Feed-Forward 4D Visual Geometry

    Sep 29, 2026Jingdong Zhang, Xin Li, Jan Kautz +2Instance SegmentationPromptable Image Segmentation

    Texas A&M University, College Station · NVIDIA

  12. XFlow: A Workflow Model for Instruction-Guided Lesion Segmentation in Chest X-rays

    Sep 28, 2026Geon Choi, Hangyul Yoon, Hyunki Park +2Promptable Image SegmentationMedical Image Segmentation

    KAIST · Korea University Guro Hospital · Samsung Medical Center

  13. ReG-SAM: Reference Graph-Driven SAM for 2D Foundational Vessel Segmentation

    Sep 25, 2026Donghang Lyu, Zichen Zhang, Oleh Dzyubachyk +1Medical Image SegmentationPromptable Image Segmentation

    Division of Image Processing, Department of Radiology, Leiden University Medical Center, Leiden, the Netherlands

  14. Can Frozen Hyperspherical Features Guide the Selection of Pseudo Masks?

    Sep 24, 2026Xinge Guo, Fengyang Xiao, Dingming Zhang +6Promptable Image SegmentationPseudo-Labeling

    Duke University · Carnegie Mellon University · University of Alabama at Birmingham

  15. SRPR-Net: Semantic and Relational Prompt Refinement for Automated SAM-based Instance Segmentation

    Sep 21, 2026Lufei Liu, Guojie Li, Suncheng Xiang +1Instance SegmentationPromptable Image Segmentation

    Ruijin-XJTLU Intelligent Medicine Institute, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China; and Xi’an Jiaotong-Liverpool University, Suzhou, China · School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China

  16. P3^3-SAM: SAM with Perceptual Parallel Prompt for Few-Shot Strip Steel Surface Defect Segmentation

    Sep 18, 2026Qian Xu, Hang Xiong, Anpeng Wang +3Image SegmentationFew-Shot Semantic Segmentation

    Shandong University Jinan, China · Alibaba International Digital Commercial Group Beijing, China · Lingnan University Hong Kong, China

  17. SetPlanner: A Lightweight Plug-in Point-Set Planner for Frozen SAM

    Sep 16, 2026Dawei Yan, Yuezhe Yang, Menglan Ruan +2Promptable Image SegmentationMedical Image Segmentation

    School of Computer Science and Engineering, Southeast Univ., China · School of Computer Science, Univ. of Sydney, Australia

  18. PSMP-CLIP: Patch-Prompt SAM and Multi-Semantic Prompting for CLIP-Based Zero-Shot Anomaly Detection

    Sep 15, 2026Xuezhi Xiang, Guanghao Wu, Heqi Xiang +4Promptable Image SegmentationAnomaly Localization

    Information and Communication Engineering, Harbin Engineering University, Harbin, China · Key Laboratory of Advanced Marine Communication and Information Technology, Harbin, China · Department of Computer Science, University of Toronto, Toronto, ON M5S 2E4, Canada +1

  19. Parameter-Efficient Fine-Tuning of Foundation Models for Liver Tumor Segmentation in CT

    Sep 12, 2026Ramtin Mojtahedi, Mohammad Hamghalam, Jacob J. Peoples +2Medical Image SegmentationParameter-Efficient Fine-Tuning

    School of Computing, Queen’s University, Kingston, ON, Canada · Toronto General Research Institute, University Health Network, Toronto, ON, Canada · Department of Electrical Engineering, Qa.C., Islamic Azad University, Qazvin, Iran +3

  20. Adapting Open-Weight MLLMs to Generate Point Prompts for Electron Microscopy Segmentation

    Sep 12, 2026Samia Mohinta, Albert CardonaElectron MicroscopyVLM Adaptation

    University of Cambridge, Cambridge, UK · MRC Laboratory of Molecular Biology, Cambridge, UK

  21. Spectral Adapters for Segment Anything Model-based Segmentation of Colorectal Liver Metastases in Computed Tomography

    Sep 10, 2026Ramtin Mojtahedi, Mohammad Hamghalam, Jacob J. Peoples +6Promptable Image SegmentationVision Foundation Model Adaptation

    School of Computing, Queen’s University, Kingston, ON, Canada · Toronto General Hospital Research Institute, University Health Network, 200 Elizabeth Street, Toronto, ON M5G 2C4, Canada · Department of Electrical Engineering, Qazvin Branch, Islamic Azad University, Qazvin, Iran +6

  22. Beyond Weak Labels: Prompt-Guided Local Refinement for Weakly Supervised Water Segmentation in High-Resolution Multispectral Imagery

    Sep 9, 2026Muhammad Farhan Humayun, Mohammad Imangholiloo, Afifah Shah +2Remote Sensing Image SegmentationPromptable Image Segmentation

    Department of Computing, University of Turku, Finland · Department of Geoinformatics and Cartography, Finnish Geospatial Research Institute, National Land Survey of Finland, Espoo, Finland

  23. SeGDeP: Semantic- and Geometric-Aware Decoupled Prompts for Reasoning Segmentation

    Sep 8, 2026Linnan Zhao, Xu Liu, Lingling Li +3Text-Guided Image SegmentationPromptable Image Segmentation

    Xidian University Xi’an, China

  24. CoRe-SAM3: Conditional Semantic--Visual Reconciliation for SAM3 Crack Segmentation

    Sep 5, 2026Shipeng Liu, Liang Zhao, Dengfeng ChenCrack SegmentationDomain Adaptation

    School of Mechanical and Electrical Engineering, Xi’an University of Architecture and Technology, Xi’an, 710055, Shaanxi, China · School of Artificial Intelligence and Robotics, Xi’an University of Architecture and Technology, Xi’an, 710055, Shaanxi, China · School of Building Equipment Science and Engineering, Xi’an University of Architecture and Technology, Xi’an, 710055, Shaanxi, China

  25. DropClick: Semi-Automated One-Click Segmentation for Agricultural Robotic Data

    Sep 3, 2026Patrick Zimmer, Michael Halstead, Chris McCoolAgricultural Image AnalysisPromptable Image Segmentation

    University of Bonn, Germany · CSIRO, Australia

  26. GazeRefine: Expert Gaze as a Test-Time Prompt for Training-Free Medical Image Segmentation

    Sep 1, 2026Mohammed Oussama Benyahia, Marouane Tliba, Mohamed Amine Kerkouri +10Interactive Medical Image SegmentationImage Segmentation

    Université Sorbonne Paris Nord · Northwestern University · VSB-Technical University of Ostrava

  27. SAM3-LoRA: Parameter-Efficient Adaptation of a Concept-Promptable Foundation Model for Multi-Class Structural Defect Segmentation

    Aug 31, 2026P. Malaisree, S. Youwai, S. Janrungautai +3Promptable Image SegmentationVision Foundation Model Adaptation

    MAA Consultants Co., Ltd. · AI Research Group, Department of Civil Engineering, Faculty of Engineering, King Mongkut´s University of Technology Thonburi (KMUTT). · School of Engineering, University of Phayao. +2

  28. FoundYou: A Unified Model for Personalized Segmentation and Retrieval

    Aug 30, 2026Gabriele Trivigno, Marcos Alfaro, Claudia Cuttano +3Image SegmentationInteractive Image Segmentation

    Politecnico di Torino · Miguel Hernández University of Elche · Valencian Graduate School and Research Network of Artificial Intelligence

  29. Anatomy-Aware Promptable Segmentation with Online Interactive Training for AUTOPET V

    Aug 28, 2026Pablo Lozano-Jimenez, Sergio Romero-Tapiador, Ruben TolosanaMedical Image SegmentationPET/CT Tumor Segmentation

    University of Amsterdam, Faculty of Science, Amsterdam, The Netherlands · BiometricsAI, Universidad Autónoma de Madrid, 28049 Madrid, Spain

  30. Prompt-Guided Interactive Segmentation of Interstitial Lung Disease in Thoracic CT

    Aug 28, 2026Vasilis Dedousis, Lubnaa Abdur Rahman, Lorenzo Brigatο +8Medical Image SegmentationInteractive Medical Image Segmentation

    University of Bern, Bern, Switzerland · Graduate School for Cellular and Biomedical Sciences, Bern, Switzerland · Department of Diagnostic, Interventional, and Pediatric Radiology, Bern University Hospital, Bern, Switzerland +2