GeoSeg: Training-Free Reasoning-Driven Segmentation in Remote Sensing Imagery

Lifan Jiang; Yuhang Pei; oxi Wu; Yan Zhao; Tianrun Wu; Shulong Yu; Lihui Zhang; Deng Cai

cs.CVMar 4, 2026

GeoSeg: Training-Free Reasoning-Driven Segmentation in Remote Sensing Imagery

Authors: Lifan Jiang, Yuhang Pei, oxi Wu, Yan Zhao, Tianrun Wu, Shulong Yu, Lihui Zhang, Deng Cai

Organizations: State Key lab of CAD&CG, Zhejiang University · UniTTEC Co. Ltd. · Wuchan Zhongda Chengtou (Ningbo) Holdings. Ltd.

Abstract

Recent advances in MLLMs are reframing segmentation from fixed-category prediction to instruction-grounded localization. While reasoning based segmentation has progressed rapidly in natural scenes, remote sensing lacks a generalizable solution due to the prohibitive cost of reasoning-oriented data and domain-specific challenges like overhead viewpoints. We present GeoSeg, a zero-shot, training-free framework that bypasses the supervision bottleneck for reasoning-driven remote sensing segmentation. GeoSeg couples MLLM reasoning with precise localization via: (i) bias-aware coordinate refinement to correct systematic grounding shifts and (ii) a dual-route prompting mechanism to fuse semantic intent with fine-grained spatial cues. We also introduce GeoSeg-Bench, a diagnostic benchmark of 810 image--query pairs with hierarchical difficulty levels. Experiments show that GeoSeg consistently outperforms all baselines, with extensive ablations confirming the effectiveness and necessity of each component.

Figures & tables

Figure 1: Performance across reasoning difficulty levels. We visualize our proposed GeoSeg’s results on image–query pairs across three difficulty levels. The visualized masks illustrate that GeoSeg can handle varying instruction complexity and remain effective in challenging scenarios. Please refer to Appendix B for more results.

Figure 2: Overview of the GeoSeg pipeline. Given a remote sensing image $I$ and a natural language query $q$ , the pipeline operates in three stages: (1) Reasoning-Driven Grounding : The MLLM ( $\mathcal{L}$ ) generates a coarse bounding box $b$ and extracts the object prompt $p$ . (2) Bias-Aware Coordinate Refinement : To mitigate grounding bias, the box is adjusted via asymmetric expansion ( $\alpha,\beta$ ) to yield a refined RoI $I_{b^{\prime}}$ . (3) Dual-Route Segmentation & Fusion : Within the RoI, we perform parallel segmentation using Route A (Visual Cues via CLIP Surgery) and Route B (Semantic Cues via SAM3 with prompt $p$ ). The final prediction $\hat{M}$ is obtained by integrating both paths via Intersection-First Fusion.

Figure 3: Quantification of domain-specific grounding drift. We analyze coordinate offsets on a held-out calibration set comprising 1,000 images randomly sampled from LoveDA, NWPU-VHR-10, and DIOR datasets. The KDE visualization reveals a systematic bottom-right shift inherent to pre-trained MLLMs under overhead views, necessitating our statistically derived asymmetric expansion ( $\alpha=0.2,\beta=0.1$ ).

Figure 4: Overview of GeoSeg-Bench. (a) Representative Domains : We showcase samples from the four domains defined in our scenario taxonomy: Urban , Rural , Traffic , and Nature . (b) Hierarchical Difficulty Design : Using the Traffic domain as a case study, we illustrate the progression across three levels: Basic (Level 1), Description (Level 2), and Reasoning (Level 3), corresponding to increasing reasoning requirements.

Figure 5: Distribution statistics of our GeoSeg-Bench. Left: Proportional breakdown of four scenario categories (Urban, Traffic, Rural, Nature). Right: Fixed ratio composition of three hierarchical difficulty levels (Basic, Description, Reasoning).

Figure 6: Qualitative comparison with multiple baselines. This figure demonstrates the superiority of our approach over three major categories of baseline models: generalist segmentation, reasoning segmentation, and open-source MLLMs. As shown, most baseline methods struggle to comprehend the query intent, resulting in segmentation failures or excessive noise, whereas our method successfully generates accurate masks. More examples are in Appendix B .

Figure 7: Ablation study on component effectiveness. We validate the contribution of the Bias-Aware Coordinate Refinement ( Box Refine ) and the Dual-Route strategy. Route A represents the Point-Prompt path (Visual Cues), and Route B denotes the Text-Prompt path (Semantic Cues). Removing any module significantly degrades performance, validating the necessity of our full pipeline. More examples in Appendix D .

Generalist segmentation
Method	GeoSeg-Bench ( $\uparrow$ )								SegEarth-R2 training dataset ( $\uparrow$ )
Method	IoU	Dice	Acc.	Prec.	Rec.	Spec.	BF	FPS	IoU	Dice	Acc.	Prec.	Rec.	Spec.	BF
SAM3 ( Carion et al., 2025 )	19.9	26.8	89.3	21.7	58.6	91.8	12.9	-	10.5	14.3	77.0	11.2	47.3	78.0	10.9
Grounded SAM ( Ren et al., 2024a )	18.6	25.1	64.2	20.0	73.1	63.9	10.9	-	11.3	15.1	61.9	13.1	55.2	61.8	11.1
CLIP Surgery + SAM3 ( Li et al., 2023 )	24.7	31.4	73.9	26.8	69.8	74.2	10.4	-	14.5	19.0	69.0	15.5	55.7	69.3	10.1
Reasoning segmentation

Table 1: Quantitative comparison on GeoSeg-Bench and the SegEarth-R2 training dataset. We benchmark GeoSeg against representative methods from three categories: (1) Generalist segmentation , (2) Reasoning segmentation , and (3) Open-source MLLMs . We report seven pixel-level metrics (%) and inference speed (FPS). Best and second-best results are highlighted in bold and underlined , respectively. Methods that fail to segment any target (IoU=0) are excluded from Accuracy and Specificity rankings. Inference speed is evaluated on GeoSeg-Bench with a single A100 GPU.

Generalist segmentation
Method	GeoSeg-Bench ( $\uparrow$ )									SegEarth-R2 training dataset ( $\uparrow$ )
	Qwen-8B			Qwen-32B			User			Qwen-8B			Qwen-32B			User
	F.	L.	R.	F.	L.	R.	F.	L.	R.	F.	L.	R.	F.	L.	R.	F.	L.	R.
SAM3 ( Carion et al., 2025 )	1.78	1.72	1.81	2.03	1.74	2.00	2.75	2.30	2.68	1.39	1.36	1.40	1.50	1.37	1.46	2.00	1.87	1.99
Grounded SAM ( Ren et al., 2024a )	1.71	1.65	1.75	1.92	1.64	1.86	2.63	2.17	2.50	1.42	1.39	1.44	1.55	1.41	1.50	2.05	1.88	2.03
CLIP Surgery ( Li et al., 2023 )	1.98	1.91	2.02	2.15	1.88	2.05	2.86	2.51	2.78	1.57	1.52	1.61	1.74	1.54	1.66	2.35	2.05	2.25

Table 2: Model-based and human evaluation on GeoSeg-Bench and the SegEarth-R2 training dataset. We assess reasoning-driven segmentation quality using MLLM judges (Qwen3-VL-8B/32B) and human evaluators (User). Predictions are rated on a 1–5 scale and averaged. Metrics: Faithfulness (F.; compliance with constraints), Localization (L.; correctness of mask boundaries), and Robustness (R.; avoidance of distractors). Best and second-best are bold and underlined , respectively.

Components			GeoSeg-Bench ( $\uparrow$ )							SegEarth-R2 training dataset ( $\uparrow$ )
Box Refine	Route A	Route B	IoU	Dice	Acc.	Prec.	Rec.	Spec.	BF	IoU	Dice	Acc.	Prec.	Rec.	Spec.	BF
✗	✓	✓	51.1	59.9	96.5	62.7	64.3	97.8	21.7	13.4	17.7	93.9	22.9	21.1	96.2	9.7
✓	✗	✓	52.9	61.4	95.8	59.6	71.5	96.6	24.3	16.0	20.6	93.1	23.6	27.9	95.0	11.6
✓	✓	✗	43.2	49.0	96.5	52.4	48.9	99.0	21.2	15.2	19.1	95.1	22.7	22.6	97.6	11.4
✓	✓	✓	56.4	64.2	96.8	69.0	65.7	98.3	26.6	17.4	22.0	94.7	29.2	25.2	97.0	13.2

Table 3: Ablation study of component effectiveness on GeoSeg-Bench and the SegEarth-R2 training dataset. We evaluate the contribution of the Box Refinement module and the two parallel paths in our Dual-Route mechanism. Route A denotes the Point-Prompt path, and Route B denotes the Text-Prompt path. Removing any component degrades overall performance. Best and second-best are bold and underlined , respectively.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 8 — Figure 8: Extended qualitative comparison with multiple baselines. We present five additional challenging cases. Compared to generalist segmentation, reasoning segmentation, and open-source MLLMs, our method consistently interprets complex queries accurately and outputs precise masks. (Figure continued on next page.)

Panel of Figure 9 — Figure 9: Extended qualitative comparison with multiple baselines. (Continued from previous page.)

Panel of Figure 10 — Figure 10: Extended qualitative comparison with multiple baselines. (Continued from previous page.)

Panel of Figure 11 — Figure 11: Extended qualitative ablation study. We showcase three additional cases comparing the full pipeline with its variants. Removing Route A , Route B , or the Box Refine module results in noticeable performance degradation, such as semantic misunderstanding or coarse boundaries, verifying the necessity of our complete architecture.

Remote Sensing Image Segmentation

Remote sensing image segmentation labels land cover and objects in satellite, aerial, radar, and multispectral imagery.

See all 213 papers

GeoReason: Bridging Logical Reasoning and Spatial Fidelity in Remote Sensing Segmentation
Oct 16, 2026Cheng Fan, Han Wu, Tingzhang Luo +3Reasoning Segmentation Remote Sensing Image Segmentation
City University of Hong Kong · School of Computer Science, Peking University · Samsung
RemoteZero: Geospatial Reasoning with Zero Human Annotations
May 6, 2026Liang Yao, Fan Liu, Shengxiang Xu +4Self-Training Remote Sensing
1Hohai University · 2Southeast University
Enabling Training-Free Text-Based Remote Sensing Segmentation
Feb 19, 2026Jose Sosa, Danila Rukhovich, Anis Kacem +1Text-Guided Image Segmentation Remote Sensing Image Segmentation
SnT, University of Luxembourg

Oct 16, 2026J/K move · Enter open · S save

GeoReason: Bridging Logical Reasoning and Spatial Fidelity in Remote Sensing Segmentation

Cheng Fan, Han Wu, Tingzhang Luo, Minjing Dong+2

City University of Hong Kong · School of Computer Science, Peking University · Samsung

Reasoning Segmentation Remote Sensing Image Segmentation Remote Sensing VLMs Spatial Reasoning in Language Models

Open paper AskPDF arXiv

Multimodal Large Language Models (MLLMs) have significantly advanced instruction-driven perception through the "Embedding-as-Mask" paradigm. However, extending this capability to Remote Sensing (RS) reasoning segmentation remains exceptionally challenging under cluttered geographical contexts and extreme scale variations, often leading to Spatial Fidelity Degradation and Target Misidentification. We identify a critical yet underexplored bottleneck: the sequential alienation of the segmentation token [SEG] from its initial visual grounding within the autoregressive generation paradigm. Produced only at the end of long reasoning chains, [SEG] relies on visual evidence that has been progressively diluted by linguistic abstraction and autoregressive noise. This information decay is particularly catastrophic for RS targets with minuscule pixel footprints, where even marginal fidelity loss leads to localization failure. To address these bottlenecks, we propose GeoReason, a novel framework designed to fortify logical deduction and preserve spatial integrity. First, we introduce an Anticipatory Prior mechanism that injects the [SEG] token directly into the initial user query, shifting it from a terminal output to a primary condition. This paradigm shift transforms the token into a persistent spatial anchor, preventing information decay during long-form linguistic inference. Second, we enhance visual reasoning via a Location-aware Chain-of-Thought (CoT) that enforces coarse-to-fine localization using bounding boxes, coupled with a Latent Patch-to-Token Alignment (PTA) Loss that explicitly aligns the latent segmentation token with physical RS textures. Extensive experiments demonstrate that GeoReason consistently outperforms state-of-the-art methods on the Earthreason and RRSIS-D benchmarks. Code is available at https://github.com/Fanc0126/GeoReason.

Abstract

Remote Sensing Image Segmentation

Explore similar work

GeoReason: Bridging Logical Reasoning and Spatial Fidelity in Remote Sensing Segmentation

RemoteZero: Geospatial Reasoning with Zero Human Annotations

Enabling Training-Free Text-Based Remote Sensing Segmentation