Text-Guided Image Segmentation

Ask about this topic

Overview

Text-guided image segmentation studies how to turn class names, referring expressions, clinical reports, and instructions into precise pixel-level masks. Common approaches align language and visual features with contrastive learning or cross-attention, condition segmentation models such as Segment Anything with text prompts or lightweight adapters, and use generative models or multimodal reasoning to propose and refine masks. These methods support settings from open-vocabulary recognition to medical and remote-sensing segmentation, with training-free, few-shot, and supervised approaches. A central challenge is grounding varied or ambiguous language in accurate boundaries across domains while balancing robustness, reliability, and computational cost.

Latest papers

CardsList

96 papers10 papers in the last four weeks, up 25% on the four weeks before. 0.1% of all new papers.

  1. FlowDIS: Language-Guided Dichotomous Image Segmentation with Flow Matching

    May 6, 2026Andranik Sargsyan, Shant NavasardyanImage SegmentationText-Guided Image Segmentation

    Picsart AI Research (PAIR)

  2. From Diffusion to Rectified Flow: Rethinking Text-Based Segmentation

    May 6, 2026Zishen Qu, Xuesong Li, Haijian Gu +5Image SegmentationText-Guided Image Segmentation

    Zhejiang University · bytedance · Southeast University

  3. ESICA: A Scalable Framework for Text-Guided 3D Medical Image Segmentation

    Apr 27, 2026Yu Xin, Gorkem Can Ates, Jun Ma +5Image SegmentationText-Guided Image Segmentation

    University of Florida · University of Toronto · NVIDIA Corporation

  4. Diffusion Model as a Generalist Segmentation Learner

    Apr 27, 2026Haoxiao Wang, Antao Xiang, Haiyang Sun +8Image SegmentationText-Guided Image Segmentation

    Zhejiang University · South China University of Technology · Nanjing University +1

  5. Open-Vocabulary Semantic Segmentation Network Integrating Object-Level Label and Scene-Level Semantic Features for Multimodal Remote Sensing Images

    Apr 27, 2026Jinkun Dai, Yuanxin Ye, Peng Tang +4Remote Sensing Image SegmentationText-Guided Image Segmentation

    Faculty of Geosciences and Engineering, Southwest Jiaotong University, Chengdu 611756, China · State-Province Joint Engineering Laboratory of Spatial Information Technology for High-Speed Railway Safety, Southwest Jiaotong University, Chengdu 611756, China · School of Artificial Intelligence, Wuhan University, Wuhan 430072, China +1

  6. Align then Refine: Text-Guided 3D Prostate Lesion Segmentation

    Apr 20, 2026Cuiling Sun, Linkai Peng, Adam Murphy +9Cross-Modality Medical Image SegmentationText-Guided Image Segmentation

    Department of Radiology, Northwestern University, Chicago, USA · Department of Urology, Northwestern University, Chicago, USA · Center for Cancer Research, National Cancer Institute, Bethesda, USA

  7. AnchorSeg: Language Grounded Query Banks for Reasoning Segmentation

    Apr 20, 2026Rui Qian, Chuanhang Deng, Qiang Huang +6Image SegmentationText-Guided Image Segmentation

    College of Computer Science and Artificial Intelligence, Fudan University · 2BEDI Cloud

  8. RADA: Region-Aware Dual-encoder Auxiliary learning for Barely-supervised Medical Image Segmentation

    Apr 13, 2026Shuang Zeng, Boxu Xie, Lei Zhu +6Medical Image Segmentation3D Medical Image Segmentation

  9. TAMISeg: Text-Aligned Multi-scale Medical Image Segmentation with Semantic Encoder Distillation

    Apr 13, 2026Qiang Gao, Yi Wang, Yong Zhang +4Image SegmentationMedical Image Segmentation

    Department of Data Science and AI, Monash University, Melbourne, Australia · College of Computer Science, Chongqing University, Chongqing, China · Chongqing University Central Hospital, School of Medicine, Chongqing, China

  10. MedVeriSeg: Teaching LISA-Like Medical Segmentation Models to Verify Query Validity Without Extra Training

    Apr 11, 2026Qinyue Tong, Xiaozhen Wang, Ziqian Lu +3Medical Image SegmentationHallucination Detection in VLMs

    School of Aeronautics and Astronautics, Zhejiang University, Hangzhou 310027, China · Southern Medical University, Guangzhou 510515, Guangdong, China · School of Computer Science and Technology (School of Artificial Intelligence), Zhejiang Sci-Tech University, Hangzhou 310018, Zhejiang, China +1

  11. SwinTextUNet: Integrating CLIP-Based Text Guidance into Swin Transformer U-Nets for Medical Image Segmentation

    Apr 11, 2026Ashfak Yeafi, Parthaw Goswami, Md Khairul Islam +1Medical Image SegmentationText-Guided Image Segmentation

    Department of Electrical and Electronic Engineering Khulna University of Engineering & Technology Khulna-9203, Bangladesh · Department of Electronics and Communication Engineering Khulna University of Engineering & Technology Khulna-9203, Bangladesh · Department of Mathematics and Computer Science Hobart and William Smith Colleges Geneva, NY, USA

  12. Tarot-SAM3: Training-free SAM3 for Any Referring Expression Segmentation

    Apr 9, 2026Weiming Zhang, Dingwen Xiao, Songyue Guo +5Referring Expression SegmentationText-Guided Image Segmentation

    The Hong Kong University of Science and Technology (Guangzhou) · Nanyang Technological University

  13. Selective Aggregation of Attention Maps Improves Diffusion-Based Visual Interpretation

    Apr 7, 2026Jungwon Park, Jungmin Ko, Dongnam Byun +1T2I Diffusion ModelsCross-Attention

    Department of Intelligence and Information, Seoul National University, Seoul, Republic of Korea · Interdisciplinary Program in Artificial Intelligence, Seoul National University, Seoul, Republic of Korea

  14. Semantic-Topological Graph Reasoning for Language-Guided Pulmonary Screening

    Apr 7, 2026Chenyu Xue, Yiran Liu, Mian Zhou +2Text-Guided Image SegmentationMedical Image Segmentation

    Xi’an Jiaotong-Liverpool University, Jiangsu, China · University College London, London, United Kingdom · University of Liverpool, Liverpool, United Kingdom

  15. An Explainable Vision-Language Model Framework with Adaptive PID-Tversky Loss for Lumbar Spinal Stenosis Diagnosis

    Apr 2, 2026Md. Sajeebul Islam Sk., Md. Mehedi Hasan Shawon, Md. Golam Rabiul AlamRadiology VLMsMedical Diagnosis

    Department of Computer Science and Engineering, , Dhaka, 1212, , Bangladesh. · Department of Electrical and Computer Engineering, , College Park, 20742, Maryland, USA.

  16. Efficient Reasoning via Thought Compression for Language Segmentation

    Apr 2, 2026Qing Zhou, Shiyu Zhang, Yuyu Jia +4Reasoning SegmentationCoT Compression

    Northwestern Polytechnical University, China · Northwest Institute of Nuclear Technology, China.

  17. Adapting Segment Anything Model 3 for Concept-Driven Lesion Segmentation in Medical Images: An Experimental Study

    Mar 26, 2026Guoping Xu, Jayaram K. Udupa, Yubing Tong +5Medical Image SegmentationPromptable Image Segmentation

    The Medical Artificial Intelligence and Automation (MAIA) Laboratory, Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA · Medical Image Processing Group, Department of Radiology, University of Pennsylvania, Philadelphia, PA 19104, USA · Division of Digestive and Liver Diseases, Department of Internal Medicine, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA

  18. PEARL: Geometry Aligns Semantics for Training-Free Open-Vocabulary Semantic Segmentation

    Mar 23, 2026Gensheng Pei, Xiruo Jiang, Xinhao Cai +3Graph-Based Label PropagationOpen-Vocabulary Semantic Segmentation

    Department of Electrical and Computer Engineering, Sungkyunkwan University · School of Computing and Artificial Intelligence, Southwest Jiaotong University · School of Computer Science and Engineering, Nanjing University of Science and Technology

  19. DGRNet: Disagreement-Guided Refinement for Uncertainty-Aware Brain Tumor Segmentation

    Mar 22, 2026Bahram Mohammadi, Yanqiu Wu, Vu Minh Hieu Phan +6Medical Image SegmentationBrain Tumor Segmentation

    Macquarie University, Sydney, NSW, Australia · Adelaide University, Adelaide, SA, Australia · Flinders University, Adelaide, SA, Australia +1

  20. Hierarchical Text-Guided Brain Tumor Segmentation via Sub-Region-Aware Prompts

    Mar 22, 2026Bahram Mohammadi, Ta Duc Huy, Afrouz Sheikholeslami +8Medical Image SegmentationBrain Tumor Segmentation

    Macquarie University, Sydney, NSW, Australia · Adelaide University, Adelaide, SA, Australia · Flinders University, Adelaide, SA, Australia +1

  21. Eye image segmentation using visual and concept prompts with Segment Anything Model 3 (SAM3)

    Mar 18, 2026Diederick C. Niehorster, Marcus NyströmMedical Image SegmentationPromptable Image Segmentation

    Lund University Humanities Lab & Dept. of Psychology, Lund University, Lund, Sweden · Lund University Humanities Lab, Lund, Sweden

  22. LoGSAM: Parameter-Efficient Cross-Modal Grounding for MRI Segmentation

    Mar 18, 2026Mohammad Robaitul Islam Bhuiyan, Sheethal Bhat, Melika Qahqaie +4Image SegmentationMedical Imaging Foundation Models

    Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany · Siemens Healthineers, Erlangen, Germany

  23. SSP-SAM: SAM with Semantic-Spatial Prompt for Referring Expression Segmentation

    Mar 18, 2026Wei Tang, Xuejing Liu, Yanpeng Sun +1Referring Expression SegmentationText-Guided Image Segmentation

    School of Computer Science and Engineering, Nanjing University of Science and Technology, No. 200 Xiaolingwei Road, Nanjing 210094, China. · Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China. · NExT++ Lab, School of Computing, National University of Singapore, Singapore 119077, Singapore.

  24. DCP-CLIP:A Coarse-to-Fine Framework for Open-Vocabulary Semantic Segmentation with Dual Interaction

    Mar 14, 2026Jing Wang, Huimin Shi, Quan Zhou +3Open-Vocabulary Semantic SegmentationText-Guided Image Segmentation

    National Engineering Research Center of Communications and Networking, Nanjing University of Posts & Telecommunications, Nanjing 210003, P. R. China. · Institute for Advanced Ocean Research (Nantong), Southeast University, Nantong 226334, P. R. China. · Department of Internet of Things, Nanjing University of Posts & Telecommunications, Nanjing 210003, P. R. China +1

  25. Evidential learning driven Breast Tumor Segmentation with Stage-divided Vision-Language Interaction

    Mar 11, 2026Jingxing Zhong, Qingtao Pan, Xuchang Zhou +2Medical Image SegmentationText-Guided Image Segmentation

    Center for Precision Medicine, The First Affiliated Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, China; Department of Laboratory Medicine, The First Affiliated Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, 361003, China · Maynooth International Engineering College, Fuzhou University, Fuzhou, 350108, China · Department of Computer and Data Science, Case Western Reserve University, Cleveland, Ohio, USA, Cleveland, 44106, Ohio, USA +2

  26. A Guideline-Aware AI Agent for Zero-Shot Target Volume Auto-Delineation

    Mar 10, 2026Yoon Jo Kim, Wonyoung Cho, Jongmin Lee +7Text-Guided Image SegmentationMedical Image Segmentation

    Oncosoft Inc., Seoul, South Korea · Department of Radiation Oncology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, South Korea

  27. Prompt Group-Aware Training for Robust Text-Guided Nuclei Segmentation

    Mar 6, 2026Yonghuang Wu, Zhenyang Liang, Wenwen Zeng +2Text-Guided Image SegmentationMedical Image Segmentation

    Fudan University, Shanghai, China

  28. Making Training-Free Diffusion Segmentors Scale with the Generative Power

    Mar 6, 2026Benyuan Meng, Qianqian Xu, Zitai Wang +3Text-Guided Image SegmentationOpen-Vocabulary Semantic Segmentation

    Institute of Information Engineering, CAS · School of Cyber Security, University of Chinese Academy of Sciences · State Key Laboratory of AI Safety, Institute of Computing Technology, CAS +5

  29. Vision-Language Semantic Grounding for Multi-Domain Crop-Weed Segmentation

    Feb 27, 2026Nazia Hossain, Xintong Jiang, Yu Tian +3Domain Generalization for Semantic SegmentationSemantic Segmentation

    Department of Bioresource Engineering, McGill University, Sainte-Anne-de-Bellevue, QC, Canada · Department of Plant Science, McGill University, Sainte-Anne-de-Bellevue, QC, Canada

  30. MedCLIPSeg: Probabilistic Vision-Language Adaptation for Data-Efficient and Generalizable Medical Image Segmentation

    Feb 23, 2026Taha Koleilat, Hojat Asgariandehkordi, Omid Nejati Manzari +3Medical Image SegmentationText-Guided Image Segmentation

    Concordia University, Montreal, Canada