Promptable Image Segmentation

Follow this topic Ask about this topic

Momentum

10 papers in the last four weeks, up 100% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 91

All topics

CLIP-Guided SAM: Parameter-Efficient Semantic Conditioning for Promptable Segmentation
May 24, 2026Shayan Jalilian, Abdul BaisVLM Adaptation Text-Guided Image Segmentation
University of Regina, Regina, SK, Canada
Training-Free Fine-Grained Semantic Segmentations in Low Data Regimes: A FungiTastic Baseline
May 21, 2026Sebastian Cavada, Francesco Pelosin, Lapo FaggiPrototype-Based Classification Image Segmentation
Covision Lab, Bressanone, South Tyrol, Italy
ConceptSeg-R1: Segment Any Concept via Meta-Reinforcement Learning
May 19, 2026Yuan Zhao, Youwei Pang, Jiaming Zuo +10Image Segmentation Promptable Image Segmentation
1Dalian University of Technology, China · 3Nanyang Technological University, Singapore · 2X3000 Inspection Co., Ltd +3
Vision Harnessing Agent for Open Ad-hoc Segmentation
May 19, 2026Zilin Wang, Stella X. YuImage Segmentation Promptable Image Segmentation
University of Michigan
Rad-VLSM: A Cross-Modal Framework with Semantics-Assisted Prompting for Medical Segmentation and Diagnosis
May 18, 2026Fengyi Zhang, Xujie Zeng, Mohan Liu +2Ultrasound Image Segmentation Cross-Modality Medical Image Segmentation
SparseSAM: Structured Sparsification of Activations in Segment Anything Models
May 17, 2026Hoai-Chau Tran, Chi H. Nguyen, Duy M. H. Nguyen +3Efficient Transformer Inference Image Segmentation
University of Illinois at Urbana-Champaign · VinUni-Illinois Smart Health Center, VinUniversity · College of Engineering & Computer Science, VinUniversity +3
SegRAG: Retrieval Augmented Spatial Prompting for Open Vocabulary Semantic Segmentation
May 17, 2026Abderrahmene Boudiaf, Irfan Hussain, Sajid JavedImage Segmentation Promptable Image Segmentation
Khalifa University, United Arab Emirates.
Seg-Agent: Test-Time Multimodal Reasoning for Training-Free Language-Guided Segmentation
May 13, 2026Chao Hao, Jun Xu, Ji Du +6Text-Guided Image Segmentation Promptable Image Segmentation
School of Computing and Information Technology, Great Bay University, Dongguan, Guangdong, 523000, China · Hangzhou International Innovation Institute, Beihang University, Hangzhou, 311115, China · Department of Computing, The Hong Kong Polytechnic University, Hong Kong
Segment Anything with Robust Uncertainty-Accuracy Correlation
May 11, 2026Hongyou Zhou, Marc Toussaint, Ling Shao +1Image Segmentation Promptable Image Segmentation
Learning and Intelligent Systems, Technical University of Berlin, Berlin, Germany. · UCAS-Terminus AI Lab, University of Chinese Academy of Sciences, Beijing, China.
From Pixels to Concepts: Do Segmentation Models Understand What They Segment?
May 10, 2026Shuang Liang, Zeqing Wang, Yuxian Li +2VLM Evaluation Image Segmentation
Department of Electrical and Computer Engineering, The University of Hong Kong · CASIC · School of Computer Science and Engineering, Sun Yat-sen University +1
Prompt-Free and Efficient SAM2 Adaptation for Biomedical Semantic Segmentation via Dual Adapters
May 7, 2026Hinako Mitsuoka, Kazuhiro HottaPromptable Image Segmentation Vision Foundation Model Adaptation
Meijo University, 1-501 Shiogamaguchi, Tempaku-ku, Nagoya, 468-8502, Japan
FlowDIS: Language-Guided Dichotomous Image Segmentation with Flow Matching
May 6, 2026Andranik Sargsyan, Shant NavasardyanImage Segmentation Text-Guided Image Segmentation
Picsart AI Research (PAIR)
Approaching human parity in the quality of automated organoid image segmentation
May 4, 2026Chase Cartwright, Gongbo Guo, Sai Teja Pusuluri +3Image Segmentation Promptable Image Segmentation
Department of Physics and Astronomy and Nanoscale and Quantum Phenomena Institute, Ohio University, Athens, Ohio 45701, USA · The Steve and Cindy Rasmussen Institute for Genomic Medicine, Abigail Wexner Research Institute at Nationwide Children’s Hospital, Columbus, OH, USA. · Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati Children’s Hospital Medical Center, 3333 Burnet Avenue, Cincinnati, OH 45229, USA; Division of Developmental Biology, Cincinnati Children’s Hospital Medical Center, 3333 Burnet Avenue, Cincinnati, OH 45229, USA; Center for Stem Cell and Organoid Medicine, Cincinnati Children’s Hospital Medical Center, 3333 Burnet Avenue, Cincinnati, OH 45229, USA. +1
Remote SAMsing: From Segment Anything to Segment Everything
Apr 30, 2026Osmar Luiz Ferreira de Carvalho, Osmar Abílio de Carvalho Júnior, Anesmar Olino de Albuquerque +1Image Segmentation Remote Sensing Image Segmentation
Department of Electrical Engineering, University of Brasília, Brasília, Brazil · Department of Geography, University of Brasília, Brasília, Brazil
Training-Free Tunnel Defect Inspection and Engineering Interpretation via Visual Recalibration and Entity Reconstruction
Apr 30, 2026Shipeng Liu, Liang Zhao, Dengfeng Chen +1Open-Vocabulary Object Detection Promptable Image Segmentation
Xi’an University of Architecture and Technology, School of mechanical and electrical engineering, Xi’an, 710055, Shaanxi, China · Xi’an University of Architecture and Technology, School of Information and Control Engineering, Xi’an, 710055, Shaanxi, China · Xi’an University of Architecture and Technology, School of Civil Engineering, Xi’an, 710055, Shaanxi, China
X2SAM: Any Segmentation in Images and Videos
Apr 27, 2026Hao Wang, Limeng Qiao, Chi Zhang +4Image Segmentation Video Object Segmentation
Sun Yat-Sen University · 2 Peng Cheng Laboratory · 3 Meituan Inc
Learning from Noisy Prompts: Saliency-Guided Prompt Distillation for Robust Segmentation with SAM
Apr 25, 2026Jingxuan Kang, Ziqi Zhang, Shaoming Zheng +9Medical Imaging Foundation Models Promptable Image Segmentation
Imperial College London · Beihang University · National Health Service +1
CoCo-SAM3: Harnessing Concept Conflict in Open-Vocabulary Semantic Segmentation
Apr 21, 2026Yanhui Chen, Baoyao Yang, Siqi Liu +1Promptable Image Segmentation Open-Vocabulary Semantic Segmentation
School of Computers, Guangdong University of Technology · Shenzhen Research Institute of Big Data · Peking University
Is SAM3 ready for pathology segmentation?
Apr 20, 2026Qiuyu Kong, Shakiba Sharifi, Yiming Wang +2Computational Pathology Image Segmentation
Sapienza University of Rome · University of Verona · Fondazione Bruno Kessler
Prompting Foundation Models for Zero-Shot Ship Instance Segmentation in SAR Imagery
Apr 20, 2026Islam Mansour, Francescopaolo Sica, Michael SchmittInstance Segmentation Promptable Image Segmentation
Department of Aerospace Engineering, University of the Bundeswehr Munich, Neubiberg, Germany
SGP-SAM: Self-Gated Prompting for Transferring 3D Segment Anything Models to Lesion Segmentation
Apr 19, 2026Zequan Yao, Zixuan Tang, Jielin Ma +1Image Segmentation Medical Imaging Foundation Models
School of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen China.
From Boundaries to Semantics: Prompt-Guided Multi-Task Learning for Petrographic Thin-section Segmentation
Apr 16, 2026Yili Ren, Shiqi Wen, Li Hou +8Image Segmentation Promptable Image Segmentation
Research Institute of Petroleum Exploration and Development (RIPED) · The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology +1
MedP-CLIP: Medical CLIP with Region-Aware Prompt Integration
Apr 13, 2026Jiahui Peng, He Yao, Jingwen Li +9Promptable Image Segmentation Multimodal Pretraining
aSichuan University, Chengdu, 610065, China · bXinjiang University, Urumqi, 830046, China · cAlibaba Group, Hangzhou, 310023, China +3
ET-SAM: Efficient Point Prompt Prediction in SAM for Unified Scene Text Detection and Layout Analysis
Mar 26, 2026Xike Zhang, Maoyuan Ye, Juhua Liu +1Efficient Neural Network Inference Promptable Image Segmentation
School of Computer Science, National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence, and Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University, China
StructSAM: Structure- and Spectrum-Preserving Token Merging for Segment Anything Models
Mar 7, 2026Duy M. H. Nguyen, Tuan A. Tran, Duong Nguyen +17Token Merging Image Segmentation
University of Stuttgart · DFKI · Max Planck Research School +10
Atlas is Your Perfect Context: One-Shot Customization for Generalizable Foundational Medical Image Segmentation
Dec 20, 2025Ziyu Zhang, Yi Yu, Simeng Zhu +4Image Segmentation Medical Imaging Foundation Models
Nanjing University · The Ohio State University · Stanford University
Single Point, Full Mask: Velocity-Guided Level Set Evolution for End-to-End Amodal Segmentation
Aug 3, 2025Zhixuan Li, Yujia Liu, Chen Hui +2Promptable Image Segmentation
College of Computing and Data Science, Nanyang Technological University, Singapore · School of Computer Science, Peking University, Beijing, China · National Key Laboratory for Multimedia Information Processing, Peking University, Beijing, China +2
LlamaSeg: Image Segmentation via Autoregressive Mask Generation
May 26, 2025Jiru Deng, Tengjin Weng, Tianyu Yang +3Image Segmentation Text-Guided Image Segmentation
Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) · Tsinghua University · Shenzhen University +2
Segment Anything for Dendrites from Electron Microscopy
Nov 4, 2024Zewen Zhuo, Ilya Belevich, Ville Leinonen +4Image Segmentation Vision Foundation Models
A.I. Virtanen Institute for Molecular Sciences University of Eastern Finland Kuopio, Finland · Electron Microscopy Unit Institute of Biotechnology University of Helsinki Helsinki, Finland · Department of Medicine Faculty of Health Sciences University of Eastern Finland Kuopio, Finland
SSS: Semi-Supervised SAM-2 with Efficient Prompting for Medical Imaging Segmentation
Date pendingHongjie Zhu, Xiwei Liu, Rundong Xue +5Promptable Image Segmentation Semi-Supervised Medical Image Segmentation
SWUN · MBZUAI · XJTU +2

May 24, 2026, cs.CVJ/K move · Enter open · S save

CLIP-Guided SAM: Parameter-Efficient Semantic Conditioning for Promptable Segmentation

Shayan Jalilian, Abdul Bais

University of Regina, Regina, SK, Canada

VLM Adaptation Text-Guided Image Segmentation Promptable Image Segmentation Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

Promptable foundation models such as the Segment Anything Model (SAM) produce high-quality masks but remain semantically blind, relying on external prompts to specify categories. Existing vision-language approaches address this limitation by using external prompt coupling, where a vision-language model generates spatial prompts for SAM as a separate stage. We propose CLIP-Guided SAM, a parameter-efficient segmentation framework built on internal semantic conditioning. Instead of using semantic signals only to generate prompts, we inject CLIP-derived text, vision, and similarity features directly into SAM's image encoder through lightweight multi-modal semantic adapters. These adapters condition SAM's internal feature representations, allowing semantic information to influence mask prediction while preserving SAM's original promptable interface. Our framework is designed for low labeled-data settings and applies to both general-domain benchmarks and specialized downstream tasks. It supports two operating modes: Manual mode, for interactive segmentation with both text and spatial prompts, and Semi-Automatic text-only mode, for applications that require concept-specific segmentation using only textual input. We show that robustness depends on aligning training with the type of prompts used at inference, making train-test prompt consistency an important design principle. Through extensive experiments and ablations, we evaluate our method against SAM+PEFT baselines without semantic conditioning, vision-language + SAM pipelines, SAM 3, and strong semi-supervised segmentation methods that rely on large amounts of unlabeled data. Across these settings, CLIP-Guided SAM consistently achieves superior or competitive performance while remaining parameter-efficient in both training and deployment.

Momentum

Related topics

Latest papers 91