Knowing Whether, Not Just Where: Anchor-Probing Distillation for Open-Set Remote Sensing Visual Grounding

Zhihao Chang; Lei Zhai; Shuyuan Yang; Zhixi Feng

cs.CVOct 9, 2026

Knowing Whether, Not Just Where: Anchor-Probing Distillation for Open-Set Remote Sensing Visual Grounding

Authors: Zhihao Chang, Lei Zhai, Shuyuan Yang, Zhixi Feng

Abstract

Open-set visual grounding in remote sensing requires models to determine not only where a queried object is located, but also whether the image provides sufficient evidence for the query at all. Existing closed-set grounding models, however, tend to return plausible boxes even for absent objects, blurring localization confidence with genuine Visual Support. We introduce AnchorProbe-Distill (APD), a teacher-student framework that recasts open-set grounding as Visual Support learning. APD builds on the insight that supported queries should induce coherent spatial anchors, whereas unsupported or mismatched queries tend to produce unstable or misleading localization patterns. APD-T therefore uses the Original Query as a Reference Anchor to interpret query perturbations: Semantic-Preserving Queries retain the annotated target, Near-Category Ambiguous Queries are resolved according to whether they form a reliable alternative anchor or collapse onto the Reference Anchor, and Far-Category Counterfactual Queries provide rejection supervision. This reference-aware reasoning is distilled into APD-S, enabling unified localization and rejection from a single image--query pair. Experiments show that APD enables the resulting model to reject absent targets while also improving its localization of present targets.

Figures & tables

Figure 1: Hallucinated localization. Both models localize the supported airplane query; for the absent ship query, a vanilla MLLM predicts a box while APD returns <absent> .

Figure 2: Stochastic predictions from (a) Qwen3-VL-2B-FT and (b) GeoGround. Green and blue boxes denote the baseball-diamond and harbor queries, respectively; dashed red is ground truth.

Figure 3: AnchorProbe-Distill. APD-T converts type-aware stochastic probes into localization or rejection targets, which APD-S distills for single-pass open-set grounding.

Figure 4: Internal Stability of paired supported and unsupported queries for (a) Qwen3-VL-2B-FT and (b) GeoGround. Color shows pair count on a shared log scale; zero scores are omitted.

Method	AVVG		DIOR-RSVG		GeoChat		RRSISD		RSVG		VRSBench		Overall
	A	RR	A	RR	A	RR	A	RR	A	RR	A	RR	A	RR
General models
Qwen3-VL-2B	14.90	44.31	49.23	22.27	44.62	19.68	51.54	21.32	19.72	16.06	45.82	10.99	38.65	23.24
LLaVA-7B	1.07	60.72	6.40	52.45	6.37	57.22	8.04	54.22	0.16	44.82	5.88	53.86	4.90	55.36
Fine-tuned models
Qwen3-VL-2B §	27.19	0.20	79.98	0.00	70.90	0.00	83.55	0.00	42.30	0.00	70.92	0.00	62.48	0.05

Table 1: Open-set grounding on RefGeo (%). A : [email protected]; RR : Reject Ratio on the paired Far-Category set. Bold marks the best reported value per column.

VLM	Metric	$K{=}5$	$K{=}10$	$K{=}25$	$K{=}50$
GeoGround	[email protected]	58.66	58.84	58.82	58.87
GeoGround	RR	80.37	81.68	82.17	82.05
LLaVA-7B	[email protected]	5.66	6.29	7.75	8.69
LLaVA-7B	RR	99.05	99.38	99.36	99.32
LLaVA-7B §	[email protected]	14.87	15.37	16.38	16.75
LLaVA-7B §	RR	96.73	98.01	97.73	97.31

Table 2: APD-T probe-budget ablation on paired RefGeo/Far-Category sets (%). Bold marks the best value per VLM and metric; § denotes fine-tuned.

	Qwen3-VL-2B §		GeoGround		LLaVA-7B §
$\tau_{cluster}$	[email protected]	$R_{cons}$	[email protected]	$R_{cons}$	[email protected]	$R_{cons}$
0.1	68.32	0.872	58.93	0.931	16.71	0.626
0.2	68.38	0.866	58.96	0.924	16.75	0.590
0.3	68.40	0.862	58.98	0.913	16.89	0.548
0.4	68.41	0.857	59.00	0.901	16.85	0.499
0.5	68.50	0.849	59.00	0.882	16.63	0.439

Table 3: Consensus Anchor’s performance versus

\tau_{cluster}

Figure 5: APD-T RR versus

\eta

. Dots mark the largest value with at least 80% score feasibility. The selected per-VLM value is reused for Near-Category routing.

Figure 6: APD-T supervision: (a) Original, “The tiny vehicle on the top”; (b) Semantic-Preserving, “a small vehicle on the top of the stadium”; (c) accepted Near-Category, “a basketball court in the bottom”; (d) Far-Category, “a storage tank in the middle of the field.” Panels (a)–(c) yield box targets; (d) yields <absent> . Red/green mark the Reference/APD target; insets enlarge target regions.

Figure 7: Near-Category routing ( $S_{0}=0.836$ , $\eta=1.04$ , $\tau=0.869$ ). (a) “a basketball court in the bottom”: $S_{stab}=0.929$ , $R_{cons}=1.0$ , disjoint positive. (b) “a tennis court in the bottom”: $S_{stab}=0.841$ , $R_{cons}=1.0$ , below threshold and unused. Red/green mark the Reference/candidate anchor.

Figure 8: Query-conditioned box density before/after APD-S: (a,b) supported “1 gray vehicle at the bottom of the image”; (c,d) absent “Locate the golf course”; (a,c) before and (b,d) after training. Red/blue indicate high/low density.

VLM / training query sets	[email protected]	RR
Qwen3-VL-2B §	62.48	0.05
$\mathcal{Q}_{o}+\mathcal{Q}_{s}$	64.28	0.00
$\mathcal{Q}_{o}+\mathcal{Q}_{s}+\mathcal{Q}_{n}$	65.64	2.23
$\mathcal{Q}_{o}+\mathcal{Q}_{s}+\mathcal{Q}_{n}+\mathcal{Q}_{f}$	66.30	97.81

Table 4: Performance under different APD-S supervision.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Method	AVVG	DIOR-RSVG	GeoChat	RRSISD	RSVG	VRSBench	Overall
Published, non-unified protocols
GeoChat ( Kuckreja et al. 2024 )	0.28	24.05	22.74	–	2.04	11.52	–
EarthGPT ( Zhang et al. 2024a )	–	76.65	–	–	–	–	–
SkySenseGPT ( Luo et al. 2024 )	26.50	60.80	–	69.00	39.50	63.50	–
GeoViS ( Zhang et al. 2025 )	–	79.80	23.70	–	–	68.50	–
SkyMoE ( Liu et al. 2026 )	36.50	68.60	–	72.50	42.50	66.10	–

Appendix

Table 5: Localization under published and unified protocols (%). Upper: paper-reported [email protected]/[email protected]; lower: our RefGeo evaluation. The blocks are not directly comparable.

Figure 9: Repeated predictions on one DIOR-RSVG image: (a,c) Qwen3-VL-2B-FT; (b,d) GeoGround; (a,b) airplane query; (c,d) dam query. Blue: 50 samples; red: Reference Anchor; green: first prediction.

Figure 10: Supported-query outputs: (a) Qwen3-VL-2B, (b) LLaVA-7B, (c,d) their fine-tuned variants, (e) GeoGround, (f) Geo-R1-3B, (g) RemoteReasoner, (h) GeoPixel, (i) Text4Seg, (j) Qwen3-VL-2B APD-T, (k) GeoGround APD-T, and (l) Qwen3-VL-2B APD-S. Dashed red is ground truth; solid green predictions have IoU $>0.5$ .

Figure 11: Far-Category outputs; panels follow Figure 10 . Methods (a)–(i) return boxes, whereas APD variants (j)–(l) return <absent> .

Visual Grounding

Visual grounding connects language to the image regions, objects, or pixels that support a description, question, or action.

See all 144 papers

GeoSearcher: Anchor-Guided Progressive Reasoning for Remote Sensing Visual Grounding with Process Supervision
Jul 1, 2026Dianyu Wang, Peirong Zhang, Xuyang Li +2Vision-Language Grounding VLM Reasoning
Aerospace Information Research Institute, the Key Laboratory of Network Information System Technology (NIST), and the Key Laboratory of Target Cognition and Application Technology (TCAT), Chinese Academy of Sciences, Beijing 100190, China · School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 100190, China
Benchmarking Answer-Grounding Consistency in Remote Sensing Visual Question Answering
Oct 9, 2026Ping Guo, Zhiqi Huang, Xinran Li +3Remote Sensing VQA VLM Evaluation
Graduate School of Information, Production and Systems, Waseda University, Kitakyushu, Japan · School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen, China · School of Software, Dalian University of Technology, Dalian, China +1
Pro-Bench: Prompt-Robust Open-Vocabulary Visual Grounding Across Real-World Heterogeneous Environments
Sep 22, 2026Linus Nwankwo, Muslim Alaran, Christian Rauch +2Vision-Language Grounding Visual Grounding
Chair of Cyber-Physical Systems, Montanuniversität Leoben, Austria. · Department of Design Engineering and Mathematics, Middlesex University, London, UK.

Jul 1, 2026J/K move · Enter open · S save

GeoSearcher: Anchor-Guided Progressive Reasoning for Remote Sensing Visual Grounding with Process Supervision

Dianyu Wang, Peirong Zhang, Xuyang Li, Xiaoxuan Liu+1

Aerospace Information Research Institute, the Key Laboratory of Network Information System Technology (NIST), and the Key Laboratory of Target Cognition and Application Technology (TCAT), Chinese Academy of Sciences, Beijing 100190, China · School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 100190, China

Vision-Language Grounding VLM Reasoning Process Supervision Visually Grounded Reasoning Visual Grounding Remote Sensing Visual Grounding

Open paper AskPDF arXiv

Recent multimodal large language models (MLLMs) have shown strong cross-modal understanding and coordinate generation abilities in visual grounding. However, transferring these abilities to remote sensing visual grounding (RSVG) remains challenging. High-resolution remote sensing images usually cover large-scale scenes, where targets are often extremely small and surrounded by numerous visually similar distractors. Meanwhile, queries often contain multiple clues, such as reference objects, spatial relations, and target attributes. Existing MLLM-based methods usually formulate RSVG as one-step coordinate generation, which may lead to unstable predictions for small-object localization and complex queries. To address these challenges, we propose GeoSearcher, which reformulates RSVG as an anchor-guided progressive reasoning process and realizes it through two coupled stages: Anchor-Centric Reasoning Supervised Fine-Tuning (ACR-SFT) and Process-Faithful Group Relative Policy Optimization (PF-GRPO). In ACR-SFT, anchor-centric reasoning data are used to teach the model to represent key visual clues as anchors and progressively integrate location, relational, and attribute clues around them. In PF-GRPO, Process-Aware Reward (PAR) and Reasoning-Informative Sample Selector (RISS) further optimize this reasoning behavior by jointly evaluating key reasoning steps and target localization, while focusing training on samples that are more beneficial for improving progressive reasoning. Through this design, GeoSearcher transforms large-scale visual search into a more constrained local reasoning process. Extensive experiments on DIOR-RSVG, OPT-RSVG, and VRS-Bench show that GeoSearcher outperforms existing state-of-the-art methods. The project will be released at https://github.com/wangdianyu954-xixi/GeoSearcher.

Abstract

Visual Grounding

Explore similar work

GeoSearcher: Anchor-Guided Progressive Reasoning for Remote Sensing Visual Grounding with Process Supervision

Benchmarking Answer-Grounding Consistency in Remote Sensing Visual Question Answering

Pro-Bench: Prompt-Robust Open-Vocabulary Visual Grounding Across Real-World Heterogeneous Environments