GeoBrowse: A Geolocation Benchmark for Agentic Tool Use with Expert-Annotated Reasoning Traces

Xinyu Geng; Yanjing Xiao; Yuyang Zhang; Hanwen Wang; Xinyan Liu; Rui Min; Tianqing Fang; Yi R. Fung

cs.CLApr 5, 2026

GeoBrowse: A Geolocation Benchmark for Agentic Tool Use with Expert-Annotated Reasoning Traces

Authors: Xinyu Geng, Yanjing Xiao, Yuyang Zhang, Hanwen Wang, Xinyan Liu, Rui Min, Tianqing Fang, Yi R. Fung

Organizations: Hong Kong University of Science and Technology

Abstract

Deep research agents integrate fragmented evidence through multi-step tool use. BrowseComp offers a text-only testbed for such agents, but existing multimodal benchmarks rarely require both weak visual cues composition and BrowseComp-style multi-hop verification. Geolocation is a natural testbed because answers depend on combining multiple ambiguous visual cues and validating them with open-web evidence. Thus, we introduce GeoBrowse, a geolocation benchmark that combines visual reasoning with knowledge-intensive multi-hop queries. Level 1 tests extracting and composing fragmented visual cues, and Level 2 increases query difficulty by injecting long-tail knowledge and obfuscating key entities. To support evaluation, we provide an agentic workflow GATE with five think-with-image tools and four knowledge-intensive tools, and release expert-annotated stepwise traces grounded in verifiable evidence for trajectory-level analysis. Experiments show that GATE outperforms direct inference and open-source agents, indicating that no-tool, search-only or image-only setups are insufficient. Gains come from coherent, level-specific tool-use plans rather than more tool calls, as they more reliably reach annotated key evidence steps and make fewer errors when integrating into the final decision. The GeoBrowse bernchmark and codes are provided in https://github.com/ornamentt/GeoBrowse

Figures & tables

Figure 1: GeoBrowse couples a tool-use framework with a geolocation benchmark: Level 1 emphasizes visual cue composition, while Level 2 contains BrowseComp-style queries, all paired with expert-annotated stepwise traces.

Dataset	Geographic Coverage	Human Annotation	Reasonable Localizability	CoT	Agentic Tool-use
MP-16 [ 22 ]	Local	✗	✗	✗	✗
GLDv2 [ 50 ]	Local	✗	✗	✗	✗
VIGOR [ 59 ]	Local	✗	✗	✗	✗
Google-WS [ 8 ]	Global	✗	✗	✗	✗
OSV [ 2 ]	Global	✗	✗	✗	✗
GeoComp [ 43 ]	Global	✓	❍	✗	✗

Table 1: Comparison of existing geolocation datasets with GeoBrowse. “Local” denotes region-level coverage, whereas “Global” spans multiple continents. ✓ and ✗ indicate the presence and absence of a feature, respectively; ❍ indicates partial availability.

Figure 5: The pipeline of GATE, our proposed Geolocation Agentic-workflow with Tool Enhancement approach. The input image is first registered into stable img_id references. GATE then follows a ReAct-style loop: <Think> summarizes the latest evidence and plans the next step, <Action> invokes an image or knowledge tool, and the tool response is returned as <Obs> to update the agent state. Any new images in <Obs> are re-registered, and the loop repeats until the final answer.

Backbone	Level 1: Visual Cues				Level 2: BrowseComp-style
Backbone	City	State	Country	Avg.	City	State	Country	Avg.
Direct Inference
GPT-4o	5.8	16.4	35.7	23.1	10.5	15.8	11.1	11.9
GPT-4.1	5.8	14.5	30.4	19.6	5.3	5.3	15.9	11.9
GPT-5	11.5	21.8	38.0	27.6	15.8	10.8	20.6	17.8
GPT-5.2	7.7	9.1	30.4	18.6	21.1	21.1	20.6	20.8

Table 3: Main results on GeoBrowse. All accuracy scores are reported as percentages. Each entry is averaged over 3 runs. Avg. is the macro-average over city/state/country.

Figure 6: Tool-use distribution on GeoBrowse. Statistics are aggregated over all tool calls produced by GATE with the Gemini-3-Pro backbone, across Level 1 (geolocation tasks) and Level 2 (multi-step reasoning tasks requiring external knowledge).

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Subset	N	Agree(%)	$\kappa_{\text{country}}$	$\kappa_{\text{state}}$	$\kappa_{\text{city}}$	$\kappa^{w}_{\text{trace}}$
Level 1	199	82.0	0.93	0.86	0.72	0.62
Level 2	101	76.0	0.91	0.82	0.66	0.55
Macro avg. (L1/L2)	300	79.0	0.92	0.84	0.69	0.59

Appendix

Table 8: Inter-annotator agreement (IAA) on a blinded double-annotated subset. Agree(%) is strict city-level exact match.

\kappa^{w}

denotes weighted Cohen’s kappa (quadratic weights) for ordinal 1–4 trace-quality ratings (verifiability/completeness/coherence averaged).

Judge	Agreement (%)	95% CI	Cohen’s $\kappa$
GPT-4o	95.0	[91.9, 97.9]	0.90
Claude-3.5-Sonnet	91.1	[83.9, 95.2]	0.82
Gemini-2.5-Pro	94.8	[90.9, 96.5]	0.89
Qwen2.5-72B-Instruct	93.1	[86.4, 96.6]	0.86

Appendix

Table 9: Inter-rater agreement between LLM judges and human experts on GeoBrowse (binary 0–1 labels,

N{=}300

). 95% Wilson confidence intervals are reported for raw agreement.

Geospatial Reasoning Benchmarks

See all 31 papers

Learning to Wander: Improving the Global Image Geolocation Ability of LMMs via Actionable Reasoning
Mar 11, 2026Yushuo Zheng, Huiyu Duan, Zicheng Zhang +2Multimodal Large Language Models VLM Reasoning
Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory
Perceive to Hypothesize, Verify to Ground: An Agentic Reasoning Framework for Open-World Geo-Localization
Aug 30, 2026Yutian Jiang, Ruijie Li, Sisuo Lyu +4Multimodal Grounding Geospatial Reasoning
The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology
Google Earth AI Geospatial Reasoning Agent: Capabilities and a Systematic Evaluation
Oct 11, 2026Noam Slonim, Aviv Slobodkin, Gil Fidel +34Geospatial QA Geospatial Reasoning
Google Research · Google Geo · Spatial Bytes

Mar 11, 2026J/K move · Enter open · S save

Learning to Wander: Improving the Global Image Geolocation Ability of LMMs via Actionable Reasoning

Yushuo Zheng, Huiyu Duan, Zicheng Zhang, Xiaohong Liu+1

Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory

Multimodal Large Language Models VLM Reasoning Cross-View Geo-Localization Active Visual Perception

Open paper AskPDF arXiv

Geolocation, the task of identifying the geographic location of an image, requires abundant world knowledge and complex reasoning abilities. Though advanced large multimodal models (LMMs) have shown superior aforementioned capabilities, their performance on the geolocation task remains unexplored. To this end, we introduce WanderBench, the first open access global geolocation benchmark designed for actionable geolocation reasoning in embodied scenarios. WanderBench contains over 32K panoramas across six continents, organized as navigable graphs that enable physical actions such as rotation and movement, transforming geolocation from static recognition into interactive exploration. Building on this foundation, we propose GeoAoT (Action of Thought), a Geolocation framework with Action of Though, which couples reasoning with embodied actions. Instead of generating textual reasoning chains, GeoAoT produces actionable plans such as, approaching landmarks or adjusting viewpoints, to actively reduce uncertainty. We further establish an evaluation protocol that jointly measures geolocation accuracy and difficulty-aware geolocation questioning ability. Experiments on 19 large multimodal models show that GeoAoT achieves superior fine-grained localization and stronger generalization in dynamic environments. WanderBench and GeoAoT define a new paradigm for actionable, reasoning driven geolocation in embodied visual understanding.

Abstract

Geospatial Reasoning Benchmarks

Explore similar work

Learning to Wander: Improving the Global Image Geolocation Ability of LMMs via Actionable Reasoning

Perceive to Hypothesize, Verify to Ground: An Agentic Reasoning Framework for Open-World Geo-Localization

Google Earth AI Geospatial Reasoning Agent: Capabilities and a Systematic Evaluation