Referring Image Segmentation

Follow this topic Ask about this topic

Referring image segmentation studies how to identify and delineate the image region described by language, including expressions that distinguish one object from similar instances. Common approaches fuse vision and language features in segmentation networks, first ground a phrase to a box or region and then refine a mask, or adapt and prompt foundation models such as vision-language models and Segment Anything. Systems are commonly evaluated on RefCOCO, RefCOCO+, and RefCOCOg, with work also extending to video, remote sensing, and medical images. A central challenge is combining robust semantic and relational understanding with precise boundaries, while generalizing to unfamiliar domains and keeping inference and annotation costs manageable.

Latest papers

45 papers4 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Cross-Scale Sparse Attention for Referring Segmentation in Remote Sensing
Oct 24, 2026Yuxi Li, Lu Si, Xia Liao +4Referring Image Segmentation Remote Sensing Image Segmentation
Westlake Institute for Advanced Study, Hangzhou, Zhejiang, China · School of Engineering, Westlake University, Hangzhou, Zhejiang, China · Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University, Beijing, China +2
LLMSeg: Training-Free Text-Prompted Image Segmentation with B-Splines
Oct 21, 2026Abdullah HamdiImage Segmentation Text-Guided Image Segmentation
King Abdullah University of Science and Technology (KAUST)
Xiaomi-Lens-0 Technical Report
Oct 16, 2026Pei Fu, Yike Liu, Yue Sun +13Large Vision-Language Models Referring Image Segmentation
Xiaoai Plus, SeerRay Team
ContourVLA: A Closed-Loop Perception-Action Contour Policy for Generalized Referring Expression Segmentation
Oct 8, 2026Ruicheng Zhang, Kaiwen Shen, Jiaqi Hou +6Referring Expression Segmentation Referring Image Segmentation
Sun Yat-sen University · Tsinghua University · The Chinese University of Hong Kong, Shenzhen +1
0.5%>100%: Bidirectional Reciprocal Learning for Referring Image Segmentation
Sep 21, 2026Xiaoqiang Lu, Licheng Jiao, Lingling Li +5VLM Adaptation Vision Foundation Model Adaptation
Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education, International Research Center for Intelligent Perception and Computation, School of Artificial Intelligence, Xidian University, Xi’an, Shaanxi Province, 710071, China
Cost-efficient Active Learning for Referring Image Segmentation and Grounding
Aug 31, 2026Junbeom Hong, Seonghoon Yu, Hyung Rok Jung +2Active Learning Referring Image Segmentation
1Meissa · 2KAIST · 3GIST +1
GeoRefer-Bench: A Benchmark from Referring Pixels to Verifiable Geospatial Reasoning
Aug 25, 2026Shuaishuai Cao, Min Huang, Meng Tang +3Spatial Reasoning Benchmarks Geospatial Reasoning
Central South University · Jiangxi Normal University · Hohai University +2
SOS! : A Streamlined Object-Conditional Transformer for Model-free Segmentation
Aug 15, 2026Jiaqi Hu, Junwen Huang, Hongli Xu +4Image Segmentation Referring Image Segmentation
Technical University of Munich Munich, Germany · Siemens AG Munich, Germany · Munich Center for Machine Learning Munich, Germany +1
FIRM: Fine-Grained Intra-Token Representation of Masks for Remote Sensing Reasoning Segmentation
Aug 14, 2026Weidong Tang, Kaiyu Li, Yikai Wang +4Remote Sensing Image Segmentation Referring Image Segmentation
Xi’an Jiaotong University, Xi’an, China · China Agricultural University, Beijing, China · Renmin University of China, Beijing, China +1
DiCoR: Decoupled Referent Disambiguation and Contour Recalibration for Efficient Referring Remote Sensing Image Segmentation
Aug 13, 2026Ziyang Gao, Zhizhuo Jiang, Jingjing Chang +5Remote Sensing Image Segmentation Referring Image Segmentation
School of Integrated Circuits, School of Information Science and Electronic Engineering, Shanghai Jiao Tong University, Shanghai 200240, China · College of Computer Science, Nankai University, Tianjin 300350, China · Department of Electronic Engineering, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China +1
AeroReformer2: Spoken-Query Referring Segmentation for Aerial Images
Aug 9, 2026Rui Li, Chenxi Duan, Haoyang YangRemote Sensing Image Segmentation Remote Sensing
EgoAfford: Task-Oriented Affordance Grounding via Egocentric Referring Segmentation
Aug 5, 2026Xinyuan Guan, Feifan Chen, Xinyu Zhan +3Multimodal Grounding Large Language Model-Based Robot Planning
Shanghai Jiao Tong University · Shanghai Innovation Institute · Noematrix Ltd
CROSS: Cascaded Distillation and Dual-Constraint Grounding for Remote Sensing Referring Segmentation
Aug 4, 2026Tingzhang Luo, Ruizhong Liu, Yichao Liu +3Referring Image Segmentation
City University of Hong Kong · The Hong Kong University of Science and Technology (Guangzhou) · Nankai University +1
Better, Stronger, Faster, and Broader: Structured All-Mask Prediction for MLLM-Based Segmentation
Aug 3, 2026Jiazhen Liu, Mingkuan Feng, Long ChenInstance Segmentation Image Segmentation
The Hong Kong University of Science and Technology (HKUST), Hong Kong.
VespaSeg: A Resource-Aware Ground-then-Segment Pipeline for Referring Expression Segmentation
Aug 2, 2026Savindu Dilshan WickramasinghePromptable Image Segmentation Efficient VLM Inference
University of Moratuwa · Department of Electronic and Telecommunication Engineering, University of Moratuwa Moratuwa, Sri Lanka
GFR-SAM: Training-Free Referring Camouflaged Object Segmentation via Cross-Image Prompting
Jul 13, 2026Yilong Yang, Jianxin Tian, Shengchuan Zhang +1Image Segmentation Promptable Image Segmentation
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University
Actor as Its Own Critic: Unifying Region Understanding and Localization via CycleGRPO
Jul 13, 2026Xin Zhang, Haochen Wang, Yikang Zhou +3Group Relative Policy Optimization Multimodal Large Language Models
National University of Singapore · ByteDance
Attribute Retrieving for Open-Vocabulary Endoscopic Compositional Referring Segmentation
Jul 9, 2026Shun Liu, Nan Xi, Yang Liu +3Endoscopy Medical Imaging
Virginia Commonwealth University · King’s College London · University at Buffalo +1
Seek to Segment: Active Perception for Panoramic Referring Segmentation
Jul 2, 2026Song Tang, Shuming Hu, Xincheng Shuai +2Active Perception Referring Image Segmentation
Fudan University, China
Venice-H1: Failure-Aware Query Re-Ranking with Multi-Scale Grid Signatures for Referring Image Segmentation
Jun 21, 2026Nicolò SavioliLearning to Rank Referring Image Segmentation
OdaxAI Research
An Open-Source Benchmark and Baseline for Multi-temporal Referring Segmentation
May 31, 2026Bingyu Li, Da Zhang, Tao Huo +3Large Vision-Language Models Referring Image Segmentation
University of Science and Technology of China, Hefei, China · Institute of Artificial Intelligence (TeleAI), China Telecom, China · School of Artificial Intelligence, Optics and Electronics (iOPEN), Northwestern Polytechnical University, Xi’an 710072, China
FlowSeg: Dynamic Semantic Guidance for LLM-Conditioned Segmentation
May 28, 2026Zekang Zhang, Guangyu Gao, Youyun Tang +7Text-Guided Image Segmentation Referring Image Segmentation
School of Computer Science, Beijing Institute of Technology · School of Computer Science, University of Birmingham · WEI Lab, Institute of Information Science, Beijing Jiaotong University +1
PinPoint: Prompting with Informative Interior Points
May 26, 2026Pouya Sadeghi, Shawn He, Pedro Pablo Guerrero Vela +3Image Segmentation Promptable Image Segmentation
University of Waterloo · 2Critical ML, University of Waterloo · 3Apple
Image-Conditioned Instance Prompt Network for Referring Remote Sensing Image Segmentation
May 23, 2026Biaoyu Ren, Qingsheng Wang, Cun Xu +2Remote Sensing Image Segmentation Referring Image Segmentation
School of Computer Science, Northwestern Polytechnical University, Xi'an, China · School of Computer Science, Northwestern Polytechnical University, Xi’an, China · College of Intelligent Robotics and Advanced Manufacturing, Fudan University, Shanghai, China +1
B-GRTO: Bootstrapped Group Relative Tool Optimization for Referring Segmentation
May 22, 2026Mario Markov, Stefan Maria Ailuro, Mohammad Mahdi +2Image Segmentation Reinforcement Learning
INSAIT, Sofia University "St. Kliment Ohridski" · INSAIT, Sofia University “St. Kliment Ohridski”
SetCon: Towards Open-Ended Referring Segmentation via Set-Level Concept Prediction
May 19, 2026Zhixiong Zhang, Yizhuo Li, Shuangrui Ding +6Image Segmentation Referring Video Object Segmentation
Shanghai Jiao Tong University · Shanghai Innovation Institute · The Chinese University of Hong Kong +3
Semi-MedRef: Semi-Supervised Medical Referring Image Segmentation with Cross-Modal Alignment
May 15, 2026Yuchen Li, Zhen Zhao, Yi Liu +1Interactive Medical Image Segmentation Cross-Modal Alignment
The University of Sydney, Sydney, NSW 2006, Australia · Shanghai Artificial Intelligence Laboratory, Shanghai, China · Changzhou University, Changzhou, China
Early Semantic Grounding in Image Editing Models for Zero-Shot Referring Image Segmentation
May 13, 2026Jingxuan He, Xiyu Wang, Yunke Wang +2Image Segmentation Referring Image Segmentation
The University of Sydney
ChartREG++: Towards Benchmarking and Improving Chart Referring Expression Grounding under Diverse referring clues and Multi-Target Referring
May 8, 2026Tianhao Niu, Ziyu Han, Xuan Dong +2Referring Image Segmentation Visual Grounding
Research Center for Social Computing and Interactive Robotics Harbin Institute of Technology, China
X2SAM: Any Segmentation in Images and Videos
Apr 27, 2026Hao Wang, Limeng Qiao, Chi Zhang +4Image Segmentation Video Object Segmentation
Sun Yat-Sen University · 2 Peng Cheng Laboratory · 3 Meituan Inc

Oct 24, 2026J/K move · Enter open · S save

Cross-Scale Sparse Attention for Referring Segmentation in Remote Sensing

Yuxi Li, Lu Si, Xia Liao, Yidong Shen+3

Westlake Institute for Advanced Study, Hangzhou, Zhejiang, China · School of Engineering, Westlake University, Hangzhou, Zhejiang, China · Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University, Beijing, China+2

Referring Image Segmentation Remote Sensing Image Segmentation Multi-Scale Attention Sparse Attention Compositional Reasoning in VLMs

Open paper AskPDF arXiv

High-resolution remote sensing (RS) imagery provides fine spatial detail over large areas, making it essential for fine-grained scene understanding. Referring Remote Sensing Image Segmentation (RRSIS) aims to segment regions described by natural language, yet many existing methods rely on sequential alignment--fusion pipelines with limited joint modeling of cross-scale visual hierarchies and linguistic dependencies. Moreover, dense attention mechanisms struggle with the sparsity and wide spatial dispersion of RS targets, while current datasets offer limited structured evaluation of multi-target references, multi-hop relational reasoning, and multi-granularity descriptions, limiting both model development and evaluation. To address these challenges, we propose a Cross-Scale Sparse Attention Model (CSSAM), which aligns multiscale visual features with complex referring expressions through two key modules. The Cross-scale Modality-aligned Fusion Module (CMFM) performs unified cross-scale cross-modal fusion, while the Dynamic Multimodal Sparse Hybrid Attention (DMSHA) selectively attends to informative visual--textual tokens, reducing dense visual interactions. Furthermore, we introduce M

^{3}

RRSIS-D, a RRSIS dataset explicitly designed around three core reasoning dimensions: multi-target (M

^{1}

), reference-dependent multi-hop (M

^{2}

), and multi-granularity (M

^{3}

) expressions. Extensive experiments on M

^{3}

RRSIS-D and two public RRSIS benchmarks demonstrate that CSSAM achieves competitive performance across accuracy, fine-grained reasoning, and high-IoU localization metrics, validating its effectiveness for complex semantic reasoning in realistic RS scenarios. Code, dataset, and model weights will be released upon acceptance.