Text-Guided Image Segmentation

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, up 75% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 60

All topics

SAM Meets VLM: Parameter-Decoupled Full-Parameter Training for Unified Medical Reasoning and Segmentation
Sep 29, 2026Xuyang Cao, Enyou Liu, Jun Zhao +3Interactive Medical Image Segmentation Text-Guided Image Segmentation
JDH Algo, JD Health International Inc.
Revisit to Segment: Working Memory Distillation for Reasoning Segmentation
Sep 28, 2026Cilin Yan, Yilun Qiu, Wanyang Zhang +4Text-Guided Image Segmentation Multimodal Large Language Models
Xiaohongshu · NTU · PKU
VL-AcneSeg: A Vision-Language Framework for Region-Aware Acne Lesion Segmentation
Sep 28, 2026Sukju Oh, Soo Ick Cho, Dae Hun Suh +1Text-Guided Image Segmentation Medical VLMs
Department of Computer Science and Artificial Intelligence, Dongguk University, Seoul 04620, Republic of Korea · InSkin Lab Inc., Seoul 06193, Republic of Korea · Department of Dermatology, Seoul National University College of Medicine, and also with the Department of Dermatology, Seoul National University Hospital, Seoul 03080, Republic of Korea
Multimodal Routing and Region Refinement for Language-Guided Medical Image Segmentation
Sep 24, 2026Md Maklachur Rahman, Md Hasan Al Banna, Saraf Anjum +2Text-Guided Image Segmentation Medical Image Segmentation
Texas A&M University, College Station, TX 77843, USA · Independent Researcher
PANORAMA: Panoptic Grounded Captioning via Mask Proposal Selection
Sep 16, 2026Sara Pieri, Evangelos Kazakos, Shizhe Chen +2Image Captioning Panoptic Segmentation
Inria, École normale supérieure, CNRS, PSL Research University · Czech Institute of Informatics, Robotics and Cybernetics, Czech Technical University in Prague
ViCo-SAM3: Vision-Conditioned Alignment for Open-Vocabulary Camouflaged Object Segmentation
Sep 14, 2026Qiangqiang Zhou, Wenjun Tang, Yong Chen +2Text-Guided Image Segmentation Open-Vocabulary Semantic Segmentation
School of Artificial Intelligence, Jiangxi Normal University · Institute of AI Education, East China Normal University
SeGDeP: Semantic- and Geometric-Aware Decoupled Prompts for Reasoning Segmentation
Sep 8, 2026Linnan Zhao, Xu Liu, Lingling Li +3Text-Guided Image Segmentation Promptable Image Segmentation
Xidian University Xi’an, China
InstEditSeg: Instruction-Driven Image Editing for Polyp and Skin Lesion Segmentation
Sep 2, 2026Ziquan Liu, Zhewei Zhu, Xuyang ShiText-Guided Image Segmentation Polyp Segmentation
School of Information and Control Engineering, Southwest University of Science and Technology, Mianyang 621010, China
Vision-Language-Guided Pseudo-Labels for Unsupervised Domain Adaptation in Semantic Segmentation for Waste Sorting
Sep 1, 2026Udo Schlegel, Shubhangi, Gabriel Dax +3Unsupervised Domain Adaptation Text-Guided Image Segmentation
LMU Munich, Munich, Germany · Munich Center for Machine Learning (MCML), Munich, Germany · Fraunhofer Institute for Integrated Circuits IIS, Nuremberg, Germany
Dual-Domain Cross-Modal Decoding for Clinical Text-Guided Medical Image Segmentation
Aug 11, 2026Md Maklachur Rahman, Tracy HammondText-Guided Image Segmentation Medical Image Segmentation
Texas A&M University, College Station, TX 77843, USA
Bootstrapping Vision-Language Model for Hysteroscopic Surgical Scene Segmentation
Aug 10, 2026Jun Huang, Meiyi Chen, Zijie Yue +6Text-Guided Image Segmentation Medical VLMs
College of Electronic and Information Engineering, Tongji University, Shanghai, China. · Department of Obstetrics and Gynecology, Tongji Hospital of Tongji University, Shanghai, China. · School of Medicine, Tongji University, Shanghai, China. +1
Toward Mask Annotation-Free Surgical Instrument Segmentation from Endoscopic Images Using Text-Prompted Segment Anything Model 3 (SAM3)
Aug 9, 2026Nakul Poudel, Richard Simon, Cristian A. LinteImage Segmentation Text-Guided Image Segmentation
Center for Imaging Science, Rochester Institute of Technology, Rochester, NY 14623, USA · Biomedical Engineering, Rochester Institute of Technology, Rochester, NY 14623, USA
LAD-COD: Language-Aligned Dense Perception for Camouflaged Object Detection
Aug 8, 2026Shangye Song, Tianzhi Zhu, Syed Ariff Syed Hesham +2Text-Guided Image Segmentation Vision-Language Alignment
School of Computer Science, The University of Auckland · VCIP, College of Computer Science, Nankai University · School of Electrical and Electronics Engineering, Nanyang Technological University, Singapore +4
EliSeg: Verified Target Construction for Report-Grounded Abnormality Segmentation
Aug 7, 2026Chengyi Peng, Haoyu Yang, Meixing Shi +2Text-Guided Image Segmentation Medical Image Segmentation
1Zhejiang University · 2Shanghai Artificial Intelligence Laboratory
Text-Guided Refinement of Multi-sequence Glioma Subregion Segmentation with a Vision-Language Foundation Model
Aug 5, 2026Zach Eidex, Yu-nong Lin, Mojtaba Safari +4Interactive Medical Image Segmentation Medical Imaging Foundation Models
Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, GA, USA · Department of Radiation and Cellular Oncology, The University of Chicago, Chicago, IL, USA · Department of Radiation Oncology, Winship Cancer Institute, Emory University School of Medicine, Atlanta, GA, USA
UniEvo-RS: Omni-Prompt Unified Remote Sensing Segmentation with Representative Exemplar-Driven Prototype Evolution
Aug 4, 2026Kunquan Zhang, Peilang Li, Xikun Hu +4Remote Sensing Image Segmentation Test-Time Adaptation
Sun Yat-sen University · National University of Defense Technology
PixVL: Self-Supervised Training of Pixel-Level MLLMs via a Unified Mask--Text Consistency Cycle
Aug 2, 2026Yicheng Xiao, Haoxuan Ma, Caorui Li +7Self-Supervised Learning VLM Adaptation
Institute of Automation, Chinese Academy of Sciences · Nanjing University · Southeast University +3
UltraSAM3: A Concept-Driven Foundation Model for Universal Ultrasound Image Segmentation
Jul 31, 2026Bo Xu, Quanhao Zhu, Rui Lin +5Ultrasound Image Segmentation Medical Imaging Foundation Models
School of Software Dalian University of Technology · School of Computing Technologies RMIT University · Dalian University of Technology Affiliated Center Hospital
Open-Ended CT Volume Segmentation with Weak Supervision from Language
Jul 28, 2026Sanjay Subramanian, Junwei Yu, Zirui Wang +5Weakly Supervised Image Segmentation Text-Guided Image Segmentation
Department of Electrical Engineering and Computer Science, UC Berkeley · Voio · Department of Radiology and Biomedical Imaging, UC San Francisco +1
Food Image Segmentation with LLM-Derived Ingredient Labels and Multimodal Fusion
Jul 28, 2026Jui-Feng Chi, Wei-Ta Chu, Sheng-Long LinImage Segmentation Text-Guided Image Segmentation
National Cheng Kung University, Tainan, Taiwan
Text-conditioned Segmentation for Tomato Phenotyping via Procedural Synthetic Data
Jul 20, 2026Samy Mounir, Mikolaj Cieslak, Najmeddine Dhieb +8Synthetic-to-Real Domain Adaptation Plant Phenotyping
Computer, Electrical and Mathematical Sciences and Engineering Division, KAUST, KSA · Department of Computer Science, University of Calgary, Canada · Department of Computer Science, Kiel University, Germany +2
Memory-Supported Synergistic Adaptation for Training-Free Test-Time Medical Image Segmentation
Jul 20, 2026Lingrui Li, Nan Pu, Dong Zhao +4Memory-Augmented VLMs VLM Adaptation
School of Computer Science, University of Nottingham, UK · School of Computer Science and Information Engineering, Hefei University of Technology, China · Information Systems Technology and Design Pillar, Singapore University of Technology and Design, Singapore
UniMedSeg: Unified In-Context Learning for Multi-Paradigm 2D/3D Medical Image Segmentation
Jul 14, 2026Yunzhou Li, Jiesi Hu, Yanwu Yang +5Interactive Medical Image Segmentation Medical Imaging Foundation Models
Harbin Institute of Technology at Shenzhen, Shenzhen, China · Peng Cheng Laboratory, Shenzhen, China · University Hospital Tübingen, Tübingen, Germany +2
Decoupling Language Guidance from Backbones for Text-Guided Medical Segmentation
Jul 10, 2026Yungeng Liu, Xuanzi Fang, Haijin Zeng +2VLM Adaptation Text-Guided Image Segmentation
Harbin Institute of Technology (Shenzhen) · Shenzhen, China · NingBo No.2 Hospital +1
DGSeg: Dynamic Gating of Semantic-Spatial Guided Predictions for Reasoning Segmentation
Jul 6, 2026Ruizhe Zeng, Siyu Cao, Lu Zhang +1Text-Guided Image Segmentation VLM Reasoning
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · Nanjing Artificial Intelligence Research of IA, Nanjing, 211100, China
GEAR-Seg: A Grounded Explainable Agent for Reasoning Segmentation and Data Engine
Jul 1, 2026Yanan Wang, Wen Li, Yibin Ying +1Text-Guided Image Segmentation VLM Reasoning
College of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou 310058, China · ZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University, Hangzhou 311215, China
Dual-Adaptive SAM3: Hierarchical Routing over Low-Rank Expert Layers for Parameter-Efficient Medical Image Segmentation
Jun 30, 2026Ying Chen, Jinyue Li, Kun Wang +2VLM Adaptation Text-Guided Image Segmentation
Shenzhen Research Institute, The Chinese University of Hong Kong · University of Science and Technology of China · Nanyang Technological University +1
Toward Robust In-Context Segmentation via Concept Guidance
Jun 26, 2026Zhigang Chen, Xiawu Zheng, Rongrong JiText-Guided Image Segmentation Promptable Image Segmentation
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China
Text as Illumination: Spatial Contrastive Retinex Learning for Language-guided Medical Image Segmentation
Jun 26, 2026Jian Shi, Cheng Zhen, Pingping Zhang +6Text-Guided Image Segmentation Medical Image Segmentation
School of Software Technology & DUT-RU International School of Information Science and Engineering, Dalian University of Technology, Dalian, China · School of Future Technology, Dalian University of Technology, Dalian, China · Central Hospital of Dalian University of Technology, Dalian, China +2
RSGPNet: Geometric Prompting for Remote Sensing Open-Vocabulary Semantic Segmentation
Jun 25, 2026Shanwen Wang, Xin Sun, Sirui Wang +1Remote Sensing Image Segmentation Text-Guided Image Segmentation
Faculty of Data Science, City University of Macau, 999078, SAR Macao, China · Chair of Data Science in Earth Observation, Technical University of Munich (TUM), Germany · Munich Center for Machine Learning, Munich, Germany

Sep 29, 2026, cs.CVJ/K move · Enter open · S save

SAM Meets VLM: Parameter-Decoupled Full-Parameter Training for Unified Medical Reasoning and Segmentation

Xuyang Cao, Enyou Liu, Jun Zhao, Zhuoyun Liu+2

JDH Algo, JD Health International Inc.

Interactive Medical Image Segmentation Text-Guided Image Segmentation Medical VLMs VLM Reasoning

Open paper AskPDF arXiv

Medical multimodal large language models (MLLMs) are increasingly expected not only to answer clinical questions, but also to localize the visual evidence behind their predictions. A common strategy connects a vision--language model (VLM) with SAM-style segmentation through a special <SEG> token, yet full-parameter training of this unified architecture is difficult because image-level reasoning and pixel-level segmentation impose different requirements on the shared representation space. To address this issue, we propose a parameter-decoupled training framework for unified medical reasoning and segmentation. The framework treats the <SEG> hidden state as a semantic-to-spatial prompt for the mask decoder and encourages it to become separable from generic language states, reducing ambiguous segmentation prompts and potential disruption to reasoning representations. It first performs medical shallow alignment to adapt visual features to clinical language without disturbing the LLM; then controlled instruction tuning shapes separable <SEG> prompt states, monitored by the Davies--Bouldin Index (DBI), while scaling segmentation gradients entering the language backbone; finally, the SAM branch is specialized with the VLM frozen to improve mask precision without altering reasoning parameters. Experiments on medical referring segmentation, grounding, visual QA, and textual QA benchmarks show that our framework achieves strong language-conditioned segmentation while preserving competitive reasoning ability. Ablations show that two-phase instruction tuning, gradient scaling, and segmentation specialization all contribute to the model.

Momentum

Related topics

Latest papers 60