Vision-language model (VLM) agents that act in the open world need two abilities: world knowledge to interpret what they see, and exploration to find what they cannot yet see. We study how agents combine the two through interactive visual geolocation. GeoGuesser Bench contains 282 tasks in an offline Street View environment where agents rotate, zoom, and move between real panoramas. Its three task types shift the balance from knowledge to exploration: agents (1) report the coordinates of their starting location, (2) name the enclosing street, city, state, or country, or (3) find, read, and count nearby objects. The coordinate tasks in (1) also include 42 real GeoGuessr World Championship rounds, whose expert guesses provide a human reference. We evaluate 23 VLMs: the best accuracy is 62.5% on place naming and 35.0% on object finding, and the best agent averages 4352.4 GeoGuessr points on the championship rounds, against 4770.3 for the better expert in each round. Beyond these headline numbers, world knowledge and exploration are only weakly correlated across models: agents that localize well do not necessarily explore well. To understand where agents fail, we categorize their errors into vision, navigation, and tool-use failures. Further ablations show that the strongest agents, such as Claude Opus 4.8, still perform well with limited views, while weaker agents degrade; customizing the environment thus adjusts task difficulty. The environment's verifiable rewards and restorable state also support training, and our Planner-Worker-Compressor harness cuts planner tokens by 57-61% by compacting the interaction history. We hope GeoGuesser Bench and its environment serve as a high-quality foundation for simulating real-world exploration at scale.
Figures & tables
Figure 1: Example GeoGuesser Bench episodes. Read left to right; calls below each new view specify the transition. Boxes mark natural (green) and human-made (red) clues. From top: Precise Localization, Place Naming, and Navigation Challenge. Prompts are abbreviated or illustrative.
Figure 2: Starting locations of each split. Competition and Synthetic are the two Precise Localization splits; Place Naming and Navigation Challenge favor urban and suburban areas (Section 5.2 ). Empty regions largely reflect scarce or absent Street View imagery, as in mainland China and much of Russia and Africa.
Precise Localization (Score ∣ Dist)
Place Naming
Navigation Challenge
Model
Competition
Synthetic
(%)
(%)
Human experts
4770.3 ∣ 70.2
–
–
–
Claude Opus 4.8
4352.4 ± 128 ∣ 206.9
4549.5 ± 83 ∣ 140.8
60.00 ± 3.6
26.25 ± 3.3
Claude Opus 4.7
4306.4 ± 146 ∣ 222.8
4501.4 ± 96 ∣ 156.7
56.25 ± 3.6
31.25 ± 3.4
GPT-5.5
3815.1 ± 182 ∣ 403.5
3467.8 ± 161 ∣ 545.8
62.50 ± 3.5
32.50 ± 3.4
Claude Fable 5
2821.1 ± 248 ∣ 853.7
3164.2 ± 175 ∣ 682.5
62.50 ± 3.5
35.00 ± 3.5
Table 1: Results on GeoGuesser Bench. For the Competition and Synthetic splits of Precise Localization (Section 5.1 ), cells show the mean GeoGuessr score over all entries, with invalid responses scored 0, and the score-equivalent distance in km (Eq. 1 ). The human row averages the better of the two players’ guesses in each Competition round (Section 6 ). Models are sorted by average rank, the mean of a model’s ranks in the four result columns (1 is best; ties share the mean rank). Bold marks the best model in each column.
Figure 3: Exploration and region granularity. (a) Mean move_* calls per completed episode for the four strongest models by mean Precise Localization score: all four move little on Precise Localization, and three move far more on Navigation Challenge. (b) Accuracy of the five strongest Place Naming models by the granularity the question asks for. Claude Fable 5’s dip at the state level comes from the step budget: all eight state questions it missed ended at the step limit. Tables 6 and 7 list the values.
Figure 5
Model
80 Steps
No Movement
Top Half
Bottom Half
Web Search
C
S
C
S
C
S
C
S
C
S
Claude Opus 4.8
4594
4805
4646
4556
4368
4386
4369
4313
4551
4380
GLM-5V-Turbo
3520
4089
4249
4356
3061
3960
3093
3480
3544
4022
Qwen3.7-Plus
3884
3930
3686
3998
2925
3171
3271
3097
3080
3339
Claude Opus 4.6
3708
3741
3749
3018
2527
2583
1928
2879
2688
3069
Claude Sonnet 4.6
2791
3193
3626
4203
2337
2922
2446
2707
3206
2767
Table 3: Ablation results across all conditions. Mean GeoGuessr score (0–5000) per model on the two Precise Localization splits, Competition (C, n=42 ) and Synthetic (S, n=80 ), with invalid responses assigned zero. 80 Steps : step limit raised from 40 to 80, navigation enabled. No Movement : move_* tools removed, look-and-zoom only, 40 steps. Top/Bottom Half : only the named half of each 1920×1080 capture retained, 40 steps, navigation enabled. Web Search : search_engine_query and fetch_url_content added, full images, 40 steps. Models are sorted by average rank over the ten columns; best value per column in bold. GPT-5.5’s Web Search cell on Synthetic averages 79 entries because one question was not attempted (1809 if it is counted as zero). Claude Opus 4.6’s Top Half, Bottom Half, and Web Search runs used OpenRouter; its other runs used the provider’s API.
Figure 5: Visual coverage within one panorama. Bright regions fall within the reconstructed camera views of an example no-movement episode; dark regions were never viewed. The mask shows which directions were viewed, not what the agent attended to.
Figure 6: Planner–Worker–Compressor framework. The Compressor condenses recent actions and key-frame notes into a brief and focus_hint ; selected images remain available to the Planner. The Planner either returns an answer or instructs the Worker to act, producing new observations for Memory. Dashed arrows denote image inputs to the models.
Precise Localization
Place Naming
Navigation Challenge
Planner tokens, no Compressor
11.06M
6.20M
11.18M
Planner tokens, Compressor
4.36M
2.44M
4.80M
Token reduction
60.6%
60.6%
57.1%
Result, no Compressor
4686
30.00%
24.39%
Result, Compressor
4737
26.25%
24.39%
Table 4: Planner tokens and task results with and without Compressor. M denotes millions of tokens; reductions are computed from the displayed counts. Precise Localization reports the GeoGuessr score (the mean distance error falls from 166.89 km to 89.56 km with Compressor); Place Naming and Navigation Challenge report accuracy.
Figure 7: Haversine Distance Visualization. Haversine distance from New York City to San Francisco (~4129 km; score=313) and New Delhi (~11755 km; score=2).
Figure 8: Camera field of view on the unit sphere. A perspective camera at the origin projects rays through a rectangular image plane (gold) onto the unit sphere. Normalizing each ray yields the spherical viewport (teal), the footprint Ft of a single view in Eq. 3 , whose curved boundary is set by the camera field of view rather than by a latitude–longitude rectangle. The default h=90∘ horizontal field of view and 16:9 aspect ratio give v≈58.72∘ vertically. The cutaway removes the forward hemisphere to expose the observer and viewing pyramid; the dashed silhouette marks the full sphere.
Figure 9: Panorama coverage versus performance–No Movement, Precise Localization. Observations from ten models on Competition and Synthetic are pooled in one plot of panorama coverage (Eq. 3 ) against GeoGuessr score; color denotes maximum absolute pitch. Invalid responses remain at zero score, and zero coverage denotes an empty reconstructed trace. The dotted line at approximately 0.49 indicates the coverage limit of a full horizon sweep at the default field of view; pitch changes or wider fields of view can exceed it.
Precise Localization
Place Naming
Navigation Challenge
Model
Competition Steps
Synthetic Steps
Steps
Steps
Claude Fable 5
29.1 ± 12.5
23.9 ± 15.3
20.0 ± 16.0
31.1 ± 11.1
Claude Opus 4.8
12.8 ± 8.9
9.7 ± 7.9
11.6 ± 11.3
27.0 ± 13.2
Claude Opus 4.7
11.8 ± 8.0
9.8 ± 9.0
14.7 ± 13.4
25.1 ± 12.3
Claude Opus 4.6
24.4 ± 12.5
24.5 ± 14.5
24.8 ± 15.3
33.5 ± 10.7
Claude Sonnet 4.6
29.4 ± 13.1
23.2 ± 13.9
24.1 ± 15.3
34.5 ± 10.3
Table 5: Average steps per episode (mean ± standard deviation). Number of agent steps (tool-call rounds plus the final answer) the model takes before committing a response, per task split. Statistics are computed over completed episodes only; episodes that ended in an API error, such as requests exceeding the provider’s size limit (HTTP 413) on long, image-heavy episodes of Claude Fable 5 and Claude Opus 4.6, are excluded. Claude Opus rows use the direct Anthropic endpoint. The step budget is 40.
Model
Comp.
Synth.
Place Naming
Navigation Challenge
Claude Opus 4.8
3.07
2.33
3.15
8.68
Claude Opus 4.7
2.43
2.34
7.44
9.28
GPT-5.4
2.83
2.48
1.74
3.30
Qwen3.7-Plus
2.90
2.58
2.17
8.17
Table 6: Average movement calls per episode. Mean number of move_* tool calls per episode for the four strongest models by mean Precise Localization score (Competition and Synthetic), by task type; plotted in Figure 3 a. Counts are over completed episodes only.
Model
Country ( n=27 )
State ( n=11 )
City ( n=31 )
Street ( n=11 )
Claude Fable 5
88.9
27.3
58.1
45.5
Claude Opus 4.8
88.9
72.7
41.9
27.3
Claude Opus 4.7
81.5
54.5
45.2
27.3
GPT-5.5
85.2
72.7
48.4
36.4
GPT-5.4 Pro
81.5
54.5
51.6
36.4
Table 7: Place Naming accuracy by question granularity. Accuracy (%) of the five strongest models on Place Naming, by the geographic granularity the question asks for; plotted in Figure 3 b. Bold : best per granularity.
Competition
Synthetic
Step limit
Model
Median
Zero
Step
Abst.
Median
Zero
Step
Abst.
Place Naming
Navigation Challenge
Claude Opus 4.8
4923
2
2
0
4984
2
2
0
4
27
Claude Opus 4.7
4949
4
1
3
4988
4
2
2
9
17
GPT-5.5
4763
7
7
0
4968
18
18
0
7
10
Claude Fable 5
4872
18
15
0
4994
29
28
0
22
28
Qwen3.7-Plus
4411
2
0
1
4608
1
0
1
3
17
Table 8: Precise Localization score distribution and failure causes. For each Precise Localization split: the median GeoGuessr score, the number of zero-score episodes (Competition: of 42; Synthetic: of 80), and how many of them ended at the 40-step limit, with no answer in the final turn (Step; Section 6 ), or with an abstention before the limit (Abst.); the remaining zeros are API errors or episodes without a final reply. The last two columns count the Place Naming and Navigation Challenge episodes (of 80) that ended at the step limit in the same sense. Rows follow Table 1 .
Model
Competition Score ∣ Dist
Synthetic Score ∣ Dist
Zero-score rate %
Claude Opus 4.8
4570.0 ∣ 134.1
4666.2 ∣ 103.1
3.28
Claude Opus 4.7
4759.7 ∣ 73.5
4738.3 ∣ 80.2
6.56
GPT-5.5
4578.1 ∣ 131.5
4474.6 ∣ 165.6
20.49
Claude Fable 5
4937.0 ∣ 18.9
4963.4 ∣ 11.0
38.52
Qwen3.7-Plus
4162.6 ∣ 273.4
4046.5 ∣ 315.6
2.46
GPT-5.4 Pro
4780.0 ∣ 67.1
4742.0 ∣ 79.0
45.08
Table 9: Score on answered episodes. Mean GeoGuessr score ∣ distance in km (Eq. 1 ) over episodes with a non-zero score on the two Precise Localization splits, Competition ( n=42 ) and Synthetic ( n=80 ), and the percentage of episodes that score zero, pooled over both ( n=122 ). Zero scores come from abstentions, episodes that end at the step limit, and errors (Table 8 ). Bold marks the highest score and the lowest zero-score rate.
ID
Failure Mode
Description
Affected Stage
FM 1
Vision Failure
Errors in interpreting visual input or applying pretrained knowledge, including hallucinated landmark recognition, target object misclassification, and faulty perception of scene content.
Vision
FM 2
Navigational Reasoning Failure
Errors in spatial and navigational reasoning, including poor route planning, imprecise movement execution, and lost spatial awareness across observations.
Planning / Execution
FM 3
Tool-Use Failure
Errors in tool interface use, including malformed or invalid calls, hallucinated tool capabilities, and refusal to engage due to misunderstood tool semantics.
Tool Use
Table 10: Taxonomy of observed failure modes. Each mode represents a distinct category of error exhibited by visual agents during geospatial reasoning tasks.
Figure 10: Failure-mode examples. (a) FM2, Place Naming (“What street are we on?”), intermediate view: the agent overlooks blue signage and crosswalks that suggest a nearby street sign, and answers “I do not know.” (b) FM1, the same question at another location: from its final view, the agent answers “Las Vegas Boulevard South” based on the skyline and roadside businesses instead of exploring toward the intended view. (c) FM1, Navigation Challenge (“How many distinct, third floor patios does this building have?”): the model fixates on a house with a gabled roof rather than the intended building. Figures 12 – 14 give the full prompts and responses.
Figure 11: FM2 (navigational reasoning failure) example: lack of object permanence. Final view of Qwen3.5-122B-A10B for a Navigation Challenge prompt. The model miscounts the lanterns hanging just above and ignores the lanterns it passed on the way to this viewpoint.
Figure 12: FM2 (navigational reasoning failure) example: poor route planning. Intermediate view of Qwen3.5-397B-A17B for a Place Naming prompt. The agent overlooks the blue signage and crosswalks, which suggest that a street sign is nearby.
Figure 13: FM1 (vision failure) example: hallucinated landmark recognition. Final view of GPT-5.2 and the intended final view for a Place Naming prompt. Instead of exploring further and grounding its answer in concrete features of the scene, the agent infers its location from sparse evidence and pretrained knowledge.
Figure 14: FM1 (vision failure) example: fixation on a distractor. Intended target and the target chosen by Claude Opus 4.6 for a Navigation Challenge prompt. The model fixates on the house with a gabled roof.
Figure 15: FM3 (tool-use failure) example. Response of Qwen3.5-122B-A10B to a Precise Localization prompt before taking any action. The agent misunderstands what capture_view does (Section 6 ).
Modern Vision-Language Models (VLMs) perform well above the human baseline in image geolocalization, a task critically important in disaster response, OSINT verification, and location privacy. However, most efforts to study AI behavior on the task remain limited to static image-based retrieval, classification, and predictions. We argue that faithful recreation of the task should involve embodied navigation, where a multimodal agent autonomously explores its surroundings to gather observations before submitting a prediction. To this end, we introduce GeoAgent, an agentic environment-based benchmark that requires agents to navigate Street View environments to refine their geolocalization through sequential reasoning. Our analysis shows that modern VLMs struggle to discern regional patterns while succeeding at country- and continent-level predictions. When compared to static image-based baselines, agentic navigation significantly improves accuracy across established metrics. We also note severe bias in a developed/developing region context across frontier model architectures and poor self-improvement capabilities given incorrect priors. Overall, our work establishes the challenges of embodied navigation and geospatial reasoning. We publicly release our code and the GeoAgent environment: https://geoagent-benchmark.github.io
Deep research agents integrate fragmented evidence through multi-step tool use. BrowseComp offers a text-only testbed for such agents, but existing multimodal benchmarks rarely require both weak visual cues composition and BrowseComp-style multi-hop verification. Geolocation is a natural testbed because answers depend on combining multiple ambiguous visual cues and validating them with open-web evidence. Thus, we introduce GeoBrowse, a geolocation benchmark that combines visual reasoning with knowledge-intensive multi-hop queries. Level 1 tests extracting and composing fragmented visual cues, and Level 2 increases query difficulty by injecting long-tail knowledge and obfuscating key entities. To support evaluation, we provide an agentic workflow GATE with five think-with-image tools and four knowledge-intensive tools, and release expert-annotated stepwise traces grounded in verifiable evidence for trajectory-level analysis. Experiments show that GATE outperforms direct inference and open-source agents, indicating that no-tool, search-only or image-only setups are insufficient. Gains come from coherent, level-specific tool-use plans rather than more tool calls, as they more reliably reach annotated key evidence steps and make fewer errors when integrating into the final decision. The GeoBrowse bernchmark and codes are provided in https://github.com/ornamentt/GeoBrowse
Geolocation, the task of identifying the geographic location of an image, requires abundant world knowledge and complex reasoning abilities. Though advanced large multimodal models (LMMs) have shown superior aforementioned capabilities, their performance on the geolocation task remains unexplored. To this end, we introduce WanderBench, the first open access global geolocation benchmark designed for actionable geolocation reasoning in embodied scenarios. WanderBench contains over 32K panoramas across six continents, organized as navigable graphs that enable physical actions such as rotation and movement, transforming geolocation from static recognition into interactive exploration. Building on this foundation, we propose GeoAoT (Action of Thought), a Geolocation framework with Action of Though, which couples reasoning with embodied actions. Instead of generating textual reasoning chains, GeoAoT produces actionable plans such as, approaching landmarks or adjusting viewpoints, to actively reduce uncertainty. We further establish an evaluation protocol that jointly measures geolocation accuracy and difficulty-aware geolocation questioning ability. Experiments on 19 large multimodal models show that GeoAoT achieves superior fine-grained localization and stronger generalization in dynamic environments. WanderBench and GeoAoT define a new paradigm for actionable, reasoning driven geolocation in embodied visual understanding.
Yushuo Zheng, Huiyu Duan, Zicheng Zhang +2
Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory