Cross-Scale Sparse Attention for Referring Segmentation in Remote Sensing

Referring Remote Sensing Image Segmentation (RRSIS) has achieved significant progress through the integration of VLMs and the Segment Anything Model (SAM). However, this progress largely relies on strong pre-trained capabilities, while leaving two fundamental limitations insufficiently addressed: (1) Architectural Weak-Coupling, where the unidirectional flow forces reliance on coarse VLM prompts and wastes SAM's pixel-level structural guidance, causing localization drift; and (2) Object-Centric Semantic Bias, where models overemphasize dominant object semantics while remaining insensitive to spatial reasoning crucial for RRSIS. Motivated by these observations, we propose CROSS, a tightly integrated paradigm for RRSIS. First, we introduce Linguistic-Guided Cascaded Distillation (LGCD) to bridge the architectural gap, which distills SAM's geometric affinities as soft regularizers into VLM intermediate layers, injecting dense structural priors to refine localization. Second, Perspective-Spatial Contrastive Learning (PSCL) imposes cross-anchored constraints by mining mask-filtered deceptive distractors and spatial-linguistic counterfactuals as hard negatives, explicitly shattering semantic shortcuts to enforce genuine logical consistency. Extensive experiments on RRSIS benchmarks demonstrate that CROSS achieves state-of-the-art performance and maintains precise localization even under severe spatial description perturbations, standing as a robust new paradigm for RRSIS.

Referring remote sensing image segmentation (RRSIS) is a challenging multimodal task that requires accurate localization and pixel-level segmentation of targets specified by natural-language expressions. Existing RRSIS datasets remain limited in image resolution, scene diversity, and category coverage, which restricts the generalization ability and practical applicability of current methods. To advance this field, we introduce NWPU-Refer, the largest and most diverse bilingual RRSIS dataset to date. It contains 15,003 high-resolution images (1024--2048 px) collected from more than 30 countries across five continents, together with 49,745 annotated targets covering single-object, multi-object, and no-target scenarios. We further propose the Multi-scale Referring Segmentation Network (MRSNet), a framework tailored to the characteristics of remote sensing imagery. Specifically, MRSNet incorporates an Intra-scale Feature Interaction Module (IFIM) to refine fine-grained features within each encoder stage and a Hierarchical Feature Interaction Module (HFIM) to integrate cross-scale information while preserving spatial structure. Extensive experiments on NWPU-Refer and RRSIS-D demonstrate that MRSNet achieves strong and competitive performance across multiple metrics, especially under strict IoU thresholds and IoU-based evaluation, validating the effectiveness of the proposed dataset and model. The dataset and code are publicly available at https://github.com/CVer-Yang/NWPU-Refer.

Referring Remote Sensing Image Segmentation (RRSIS) seeks to localize and segment the target object or region specified by a natural language expression in a remote sensing image. While existing RRSIS models have benefited from large-scale foundation models, they predominantly rely on full fine-tuning. These approaches are computationally intensive and may weaken the generalization ability of pre-trained models, as extensive fine-tuning on significantly smaller downstream datasets can distort the well-structured feature representations learned during large-scale pre-training. Although Parameter-Efficient Tuning (PET) offers a potential alternative, existing PET frameworks primarily focus on single-modal optimization, failing to capture the complex cross-modal dependencies required for multimodal reasoning, while simultaneously struggling to bridge the substantial domain gap between natural scenes and aerial imagery. To address these limitations, we propose a novel framework, Semantic-driven Scale and Spatial Selection for Efficient Cross-modal Alignment (S4ECA), which enables effective and efficient cross-modal interaction through parameter-efficient adaptation. Specifically, we design a dual-encoder adapter architecture. The textual adapter employs learnable queries to distill highly semantic language proxies from word-level embeddings, facilitating early grounding. Simultaneously, the visual adapter refines hierarchical feature representations through a multi-scale dense extractor, followed by a language-guided scale and spatial selection mechanism that dynamically emphasizes relevant visual contexts, ensuring precise cross-modal alignment. By updating only 2.4% of the backbone parameters, our proposed model achieves state-of-the-art performance on the RRSIS-D and RefSegRS datasets, demonstrating superior efficiency and precision in complex aerial scenarios.