Think and Answer ME: Benchmarking and Exploring Multi-Entity Reasoning Grounding in Remote Sensing

Shuchang Lyu; Haiquan Wen; Guangliang Cheng; Meng Li; Zheng Zhou; You Zhou; Dingding Yao; Zhenwei Shi

cs.CVMar 13, 2026

Think and Answer ME: Benchmarking and Exploring Multi-Entity Reasoning Grounding in Remote Sensing

Authors: Shuchang Lyu, Haiquan Wen, Guangliang Cheng, Meng Li, Zheng Zhou, You Zhou, Dingding Yao, Zhenwei Shi

Organizations: Beihang University, Beijing, China. · University of Liverpool, Liverpool, UK. · Institute of Acoustics, Chinese Academy of Sciences, Beijing, China

Abstract

Recent advances in reasoning language models and reinforcement learning with verifiable rewards have significantly enhanced multi-step reasoning capabilities. This progress motivates the extension of reasoning paradigms to remote sensing visual grounding task. However, existing remote sensing grounding methods remain largely confined to perception-level matching and single-entity formulations, limiting the role of explicit reasoning and inter-entity modeling. To address this challenge, we introduce a new benchmark dataset for Multi-Entity Reasoning Grounding in Remote Sensing (ME-RSRG). Based on ME-RSRG, we reformulate remote sensing grounding as a multi-entity reasoning task and propose an Entity-Aware Reasoning (EAR) framework built upon visual-linguistic foundation models. EAR generates structured reasoning traces and subject-object grounding outputs. It adopts supervised fine-tuning for cold-start initialization and is further optimized via entity-aware reward-driven Group Relative Policy Optimization (GRPO). Extensive experiments on ME-RSRG demonstrate the challenges of multi-entity reasoning and verify the effectiveness of our proposed EAR framework. Our dataset, code, and models will be available at https://github.com/CV-ShuchangLyu/ME-RSRG.

Figures & tables

Figure 1: Paradigm comparison between (a) matching-based visual grounding and (b) multi-entity reasoning grounding in remote sensing.

Figure 2: Overview of ME-RSRG dataset. It includes (a) Dataset production, (b) Dataset statistics, and (c) Data case.

Dataset	Im./Ins. Num.	Entity	Im. Size	Sources	Tasks
Remote Sensing Visual Grounding Datasets
RSVG [ 31 ]	4,239/7,933	Single	$1024\times 1024$	Google Earth	VG
RSVG-HR [ 16 ]	4,239/2,650	Single	$1024\times 1024$	Google Earth	VG
DIOR-RSVG [ 39 ]	17,402/38,320	Single	$800\times 800$	DIOR [ 19 ]	VG
OPT-RSVG [ 20 ]	25,452/48,952	Single	$\sim$ 152 2 - $\sim$ 10569 2	DIOR [ 19 ] , SPCD [ 3 ] , HRRSD [ 40 ]	VG
VRSBench-Ref [ 21 ]	29,614/52,472	Single	$512\times 512$	DIOR [ 19 ] , DOTA-v2 [ 7 ]	VG

Table 1: Comparison between ME-RSRG and other related datasets. In the table, “Im.”, and “Ins.” respectively denote the image and instance (image-text pair).

Figure 3: Overview of EAR framework. We adopt a two-stage optimization strategy: SFT initialization and entity-aware reward-driven GRPO refinement.

Methods (policy models)	Baseline			SFT			EAR
	sub.	obj.	mean	sub.	obj.	mean	sub.	obj.	mean
Qwen-VL series
Qwen2-VL-2B [ 34 ]	1.40	1.15	1.27	23.18	23.08	23.13	27.55	22.93	25.16
Qwen2.5-VL-3B [ 2 ]	7.45	10.81	9.18	27.04	30.43	28.79	38.63	42.08	40.41
Qwen2.5-VL-7B [ 2 ]	11.31	9.44	10.35	31.75	34.84	33.34	44.40	48.90	46.72
Qwen3-VL-4B [ 1 ]	20.95	13.80	17.25	36.39	35.89	36.13	38.52	36.83	37.65

Table 2: Performance of representative methods on the ME-RSRG dataset. Results for “sub.”, “obj.”, and “mean” denote [email protected]_sub (%), [email protected]_obj (%), and [email protected] (%), respectively. The best-performing results are highlighted in bold.

Table 6

Figure 4: Qualitative Results Visualization.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Prompt design for instance selection and CoT generation. Red words indicate the required inputs for the two steps. Best viewed in color.

SFT		GRPO
Parameters	Values	Parameters	Values
Learning rate	1e-4	Learning rate	1e-5
Warmup ratio	0.05	Warmup ratio	0.01
Max model length	2048	Max model length	2048
LoRA rank	8	LoRA rank	8
LoRA $\alpha$	32	LoRA $\alpha$	32

Appendix

Table 5: Optimization details for SFT and GRPO.

Figure 8: Supplementary qualitative visualization results.

Figure 9: Supplementary failure cases visualization.

Remote Sensing Visual Grounding

Remote sensing visual grounding links language descriptions to the corresponding objects or regions in satellite and aerial imagery.

See all 20 papers

ReDiS: Reasoning-Diagnosis Self-Evolution for Multimodal Reasoning in Remote Sensing
Oct 13, 2026Yi Liu, Wentao Jiang, Shuo Ni +5Remote Sensing VLMs Multimodal Reasoning
School of Computer Science, Wuhan University · Zhongguancun Academy · National Key Laboratory of Science and Technology on Space-Born Intelligent Information Processing, Beijing Institute of Technology
GeoSolver: Scaling Test-Time Reasoning in Remote Sensing with Fine-Grained Process Supervision
Mar 10, 2026Lang Sun, Ronghao Fu, Zhuoran Duan +3VLM Reasoning Process Reward Models
College of Computer Science and Technology, Jilin University, Changchun 130012, China · Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin University
GeoSearcher: Anchor-Guided Progressive Reasoning for Remote Sensing Visual Grounding with Process Supervision
Jul 1, 2026Dianyu Wang, Peirong Zhang, Xuyang Li +2Vision-Language Grounding VLM Reasoning
Aerospace Information Research Institute, the Key Laboratory of Network Information System Technology (NIST), and the Key Laboratory of Target Cognition and Application Technology (TCAT), Chinese Academy of Sciences, Beijing 100190, China · School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 100190, China

Oct 13, 2026J/K move · Enter open · S save

ReDiS: Reasoning-Diagnosis Self-Evolution for Multimodal Reasoning in Remote Sensing

Yi Liu, Wentao Jiang, Shuo Ni, Zhiming Luo+4

School of Computer Science, Wuhan University · Zhongguancun Academy · National Key Laboratory of Science and Technology on Space-Born Intelligent Information Processing, Beijing Institute of Technology

Remote Sensing VLMs Multimodal Reasoning Multimodal Large Language Models Remote Sensing Image Understanding

Open paper AskPDF arXiv

Multimodal large language models (MLLMs) have shown strong potential for remote sensing understanding, yet reliable reasoning remains challenging when tasks require fine-grained visual grounding and multi-step inference. Existing approaches typically rely on either costly pre-constructed reasoning trajectories or self-generated signals such as response consistency, which can reinforce shared errors and provide unreliable supervision. We introduce Reasoning--Diagnosis Self-Evolution (ReDiS), a framework that separates reasoning exploration from candidate evaluation while allowing the two processes to evolve jointly. ReDiS instantiates two role-specific experts from a shared multimodal backbone: a Reasoner that explores diverse visual evidence--answer candidates and a Diagnoser that jointly evaluates the complete candidate set and provides credibility-aware feedback. As the Reasoner evolves, it exposes new candidate distributions and failure modes; the Diagnoser adapts to these changes and, in turn, reshapes the learning signal for subsequent exploration. To handle groups in which all sampled answers are incorrect and group-relative advantages vanish, we further introduce Failure-Triggered Recovery (FTR), which provides minimal outcome-level correction without prescribing intermediate reasoning trajectories. Experiments on LHRS-Bench, MME-RS, and CHOICE show consistent improvements over Qwen3.5 backbones at 0.8B, 2B, and 4B scales. ReDiS improves category-averaged accuracy by 6.47 percentage points at 0.8B, while ReDiS-4B achieves the highest average performance among the compared remote sensing MLLMs.

Abstract

Remote Sensing Visual Grounding

Explore similar work

ReDiS: Reasoning-Diagnosis Self-Evolution for Multimodal Reasoning in Remote Sensing

GeoSolver: Scaling Test-Time Reasoning in Remote Sensing with Fine-Grained Process Supervision

GeoSearcher: Anchor-Guided Progressive Reasoning for Remote Sensing Visual Grounding with Process Supervision