GeoAgent: Evaluating VLM Geolocalization Through Embodied Navigation

Arka Mukherjee; Soham Roy; Kartikeya Trivedi; Shreya Ghosh

cs.CVAug 30, 2026

GeoAgent: Evaluating VLM Geolocalization Through Embodied Navigation

Authors: Arka Mukherjee, Soham Roy, Kartikeya Trivedi, Shreya Ghosh

Organizations: KIIT Bhubaneswar · IIT Bhubaneswar

Abstract

Modern Vision-Language Models (VLMs) perform well above the human baseline in image geolocalization, a task critically important in disaster response, OSINT verification, and location privacy. However, most efforts to study AI behavior on the task remain limited to static image-based retrieval, classification, and predictions. We argue that faithful recreation of the task should involve embodied navigation, where a multimodal agent autonomously explores its surroundings to gather observations before submitting a prediction. To this end, we introduce GeoAgent, an agentic environment-based benchmark that requires agents to navigate Street View environments to refine their geolocalization through sequential reasoning. Our analysis shows that modern VLMs struggle to discern regional patterns while succeeding at country- and continent-level predictions. When compared to static image-based baselines, agentic navigation significantly improves accuracy across established metrics. We also note severe bias in a developed/developing region context across frontier model architectures and poor self-improvement capabilities given incorrect priors. Overall, our work establishes the challenges of embodied navigation and geospatial reasoning. We publicly release our code and the GeoAgent environment: https://geoagent-benchmark.github.io

Figures & tables

Figure 1: An overview of the GeoAgent framework, with an example VLM (GPT-5-Mini) interacting. Each agent is provided with a toolbox containing nine actions to explore the environment and submit a guess through iteratively refining their geolocalization prediction.

Figure 2: A screenshot of the navigable GeoAgent interface with the collected data.

Tier	# Cities	Samples	Share (%)	Example Cities	Developed	Developing
Tier 1	5	34	2.85	New York, Tokyo	5	0
Tier 2	10	86	7.14	Rome, Mumbai	5	5
Tier 3	25	257	21.42	Bern, Bangalore	10	15
Tier 4	25	343	28.56	Oulu, Mombasa	12	13
Tier 5	35	480	40.00	Inuvik, Siem Reap	14	21
Total	100	1,200	100.00	–	46	54

Table 1: Tier-wise composition of the final dataset (1,200 samples) with example cities and development status.

Action	Description
ROTATE:<degrees>	Rotate to a specific heading (0–360°)
ROTATE_LEFT	Rotate 90° left from current view
ROTATE_RIGHT	Rotate 90° right from current view
ROTATE_BEHIND	Rotate 180° to look behind
LOOK_UP	Tilt the camera up by 20°
LOOK_DOWN	Tilt the camera down by 20°

Table 2: Available tool calls for agents to explore the GeoAgent environment.

Condition	Model	Mean Dist.	Median Dist.	Reas. Len.	Cont. Acc (%)	Country Acc (%)	City Acc (%)
Human Baseline		2753.6 $\pm$ 202.2	995.9 $\pm$ 253.4	–	57.6 $\pm$ 2.5	36.6 $\pm$ 2.5	7.1 $\pm$ 1.3
	Claude Haiku 4.5	2951.7 $\pm$ 48.9	1114.6 $\pm$ 27.4	329.9 $\pm$ 1.6	64.4 $\pm$ 0.2	43.5 $\pm$ 0.1	9.4 $\pm$ 0.2
	GPT-5 Mini	776.7 $\pm$ 10.9	232.5 $\pm$ 0.1	205.9 $\pm$ 0.1	90.3 $\pm$ 0.7	80.4 $\pm$ 0.6	30.3 $\pm$ 0.4
	Gemini 3.0 Flash	1446.8 $\pm$ 74.8	118.5 $\pm$ 11.7	232.3 $\pm$ 1.9	70.0 $\pm$ 0.5	65.6 $\pm$ 0.6	39.5 $\pm$ 1.0
Static 4-View	Gemma 3 27B	2669.2 $\pm$ 18.0	754.7 $\pm$ 0.5	198.0 $\pm$ 0.2	49.5 $\pm$ 0.3	37.1 $\pm$ 0.2	12.7 $\pm$ 0.2
	Llama 4 Scout	1586.1 $\pm$ 61.2	458.0 $\pm$ 3.7	200.4 $\pm$ 0.3	72.7 $\pm$ 0.3	59.9 $\pm$ 0.7	20.8 $\pm$ 0.3

Table 3: Performance comparison across conditions and models. Bold = best, underline = second-best per condition. Accuracies reported in %. Lower distance and reasoning length are better; higher accuracy is better.

Figure 3: Aggregated performance improvement per action (out of 8) per model, stratified by final correct and incorrect guesses. We notice model performance generally improves with each extra action, but calibrates when the outcome is incorrect. Gemini 3.0 Flash is an outlier with a collapse on incorrect city guesses beyond 5 actions.

Figure 4: Developed–developing delta across all models and navigation modes. Bordered cells exclude zero in their confidence intervals. Each cell shows the difference (developed –developing); negative values indicate stronger performance on developed regions. All models exhibit a developed-region bias with 50–103% lower mean distance, and most secure higher accuracies for developed regions. Gemma 3 27B is an exception, with slightly higher city accuracy in developing regions across all three conditions. Gemini 3.0 Flash shows the largest country-level deltas (>20%) in all modes, suggesting its strong performance is disproportionately driven by developed-region familiarity.

Model	Exploration behavior				Mean Haversine distance (km)
Model	Actions	Moves	Rotations	Invalid (%)	Agentic	Random Walk	$\Delta$
GPT-5 Mini	4.94 $\pm$ 0.07	2.42 $\pm$ 0.12	1.51 $\pm$ 0.06	0.00	900 $\pm$ 74	1581 $\pm$ 50	$+681$
Gemini 3.0 Flash	4.96 $\pm$ 0.25	3.57 $\pm$ 0.25	0.39 $\pm$ 0.02	0.00	1099 $\pm$ 100	1565 $\pm$ 484	$+466$
Claude Haiku 4.5	5.61 $\pm$ 0.49	2.79 $\pm$ 0.36	1.59 $\pm$ 0.14	0.04	2181 $\pm$ 87	3260 $\pm$ 30	$\mathbf{+1079}$
Gemma 3 27B	5.96 $\pm$ 0.25	4.20 $\pm$ 0.34	0.76 $\pm$ 0.49	0.00	1689 $\pm$ 148	2205 $\pm$ 37	$+516$
Llama 4 Scout	7.55 $\pm$ 0.04	0.41 $\pm$ 0.07	6.14 $\pm$ 0.07	0.00	1779 $\pm$ 36	2783 $\pm$ 55	$+1004$

Table 4: Exploration behavior and the causal contribution of goal-directed navigation under the 8-action budget. Actions is the mean number of turns used before the agent commits; Moves and Rotations are the mean counts of each per round; Invalid is the share of emitted actions the environment could not parse.

\Delta

is random-walk minus agentic mean distance, so positive values mean model-chosen navigation beats random movement. Every model with functioning tool use gains 466–1079 km; Geo-R1 7B, which emits an unparseable action in 87.3% of turns and never navigates, is the sole model that does worse than random walk (Appendix I ).

Figure 5: Agent behavior and localization accuracy across geographic tiers. Models struggle significantly with less identifiable locations, with city guessing accuracies dropping below 26% on Tier 5. Interestingly, Llama 4 Scout and GPT-5 Mini maintain their country guessing accuracies across tiers while Gemma3 and Haiku 4.5 show collapse down the tier list. Other than Llama 4 Scout, mean Haversine distance generally increases by tier.

Model	Agentic $\leftrightarrow$ 4-View	Agentic $\leftrightarrow$ 8-View
Claude Haiku 4.5	41.7%	39.0%
Gemini 3.0 Flash	52.7%	51.9%
Gemma 3 27B	37.2%	36.6%
Llama 4 Scout	39.7%	43.2%
GPT-5 mini	50.0%	48.5%

Table 5: Agentic vocabulary overlap (top 50 TF-IDF words) with static multiview settings. Reported values are the mean across all geographic categories.

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Aggregated mean distance improvement over actions per model.

Figure 9: Examples of 4-view and 8-view composite images used in the multiview baseline. The top strip shows four spawn-point screenshots at cardinal headings (4-view). The bottom two rows show the full 8-view composite, with spawn-point views (blue labels) in the first row and post- MOVE_FORWARD views (green labels) in the second row, each captured at identical heading offsets.

Feature	Outcome	Pearson $r$	Pearson $p$	Spearman $\rho$	Rejects Null Hypothesis ( $H_{0}$ )?
Reasoning Length	Continent Correct	$0.129$	$0.0000$	$0.140$	YES
	Country Correct	$0.119$	$0.0000$	$0.137$	YES
	City Correct	$0.070$	$0.0000$	$0.090$	YES
	Distance Km	$-0.130$	$0.0000$	$-0.142$	YES
	Points	$0.181$	$0.0000$	$0.241$	YES
Screenshots Taken	Continent Correct	$0.097$	$0.0000$	$0.128$	YES

Appendix

Table 6: Correlation analysis between agent behavior features and task outcomes. Pearson and Spearman correlation coefficients are reported along with hypothesis test results.

Model	Dev. Level	# Actions	# Moves	# Rotations	Mean Reas. Len.	Accuracy (%)			Mean Dist. (km)
	Dev. Level	# Actions	# Moves	# Rotations	Mean Reas. Len.	Continent	Country	City	Mean Dist. (km)
Random Baseline	Developed	-	-	-	-	$0.00\pm 0.0$	$0.00\pm 0.0$	$0.00\pm 0.0$	$9982.18\pm 306.7$
Random Baseline	Developing	-	-	-	-	$0.00\pm 0.0$	$0.00\pm 0.0$	$0.00\pm 0.0$	$10007.06\pm 255.4$
Claude Haiku 4.5	Developed	$6.29\pm 0.1$	$3.15\pm 0.1$	$2.04\pm 0.1$	$434.7\pm 7.3$	$68.23\pm 6.9$	$53.22\pm 4.4$	$17.93\pm 1.7$	$1782.64\pm 263.2$
Claude Haiku 4.5	Developing	$5.40\pm 0.0$	$2.68\pm 0.1$	$1.64\pm 0.0$	$415.4\pm 8.9$	$61.53\pm 1.4$	$43.92\pm 4.9$	$14.08\pm 1.4$	$3712.72\pm 410.9$
Gemma 3 27B	Developed	$6.92\pm 0.1$	$2.70\pm 0.2$	$3.22\pm 0.2$	$220.0\pm 2.0$	$45.94\pm 2.7$	$36.35\pm 1.1$	$22.81\pm 0.5$	$1307.74\pm 315.4$

Appendix

Table 7: Agent behavior and accuracy breakdown across developed and developing regions. Models generally make similar exploration actions across both region types, but the final outcomes vary widely. Most models secure higher continent, country, and city guessing accuracies, while Gemma 3 27B reverses this trend. Importantly, all tested VLMs show an acute developed region bias in mean Haversine distance, resulting in 50-103% better numbers than the equivalent developing region result (lower is better).

Metric	Gemma-3-27B	Gemma-3-27B	Llama-4 Scout	Llama-4 Scout	GPT-5-mini	GPT-5-mini
	(Unlimited Agent)	(Limited Agent)	(Unlimited Agent)	(Limited Agent)	(Unlimited Agent)	(Limited Agent)
Avg Distance (km)	1577.65 $\pm$ 323.61	1689.33 $\pm$ 147.50	3136.53 $\pm$ 228.14	1779.45 $\pm$ 35.50	1218.41 $\pm$ 311.27	899.99 $\pm$ 73.51
Avg Actions / Round	5.36 $\pm$ 0.08	5.96 $\pm$ 0.25	22.99 $\pm$ 1.68	7.55 $\pm$ 0.04	10.13 $\pm$ 0.47	4.94 $\pm$ 0.07
Avg Reasoning Length	586.40 $\pm$ 12.12	268.52 $\pm$ 100.38	523.63 $\pm$ 8.27	200.35 $\pm$ 0.20	566.30 $\pm$ 12.73	182.77 $\pm$ 0.72
Continent Accuracy	79.0 $\pm$ 7.0	63.9 $\pm$ 2.0	69.0 $\pm$ 3.0	70.1 $\pm$ 1.0	88.0 $\pm$ 5.0	88.6 $\pm$ 0.0
Country Accuracy	61.0 $\pm$ 8.0	54.0 $\pm$ 3.0	59.0 $\pm$ 5.0	58.4 $\pm$ 1.0	79.0 $\pm$ 7.0	81.2 $\pm$ 1.0

Appendix

Table 8: GeoAgent performance of selected models in unlimited vs. limited action step exploration.

Figure 10: Model improvement behaviour under a good start (bottom-1% of

d_{1}

, pooled GPT-5 Mini and Gemini 3.0 Flash;

n=54

, median

d_{1}=0.28

km). Drift fraction (

\max_{k}(d_{k})>2d_{1}

) is 20.4% in this slice versus 15.0% over the full dataset.

Model	Total Tokens	Total Cost ($)	Cost / Round ($)
Claude Haiku 4.5	42,416,725	187.60	0.0174
GPT-5-mini	54,747,219	39.43	0.00355
Llama 4 Scout	89,050,944	10.90	0.00098
Gemini 3 Flash	44,421,928	8.50	0.00079
Gemma 3 27B	23,296,767	2.33	0.00021

Appendix

Table 9: Summary of total tokens consumed, total cost in USD, and the cost per round for each model.

Model	Actions/round	Limited	Limited	Actions/round	Unlimited	Unlimited
	for Limited	$/round	Total $	for Unlimited	$/round	Total $
Gemma 3 27B	5.96 $\pm$ 0.25	$0.00004525	$0.1629	5.36 $\pm$ 0.08	$0.000431	$0.1292
GPT-5 Mini	4.94 $\pm$ 0.07	$0.00060825	$2.1897	10.13 $\pm$ 0.47	$0.018215	$5.4644
Llama 4 Scout	7.55 $\pm$ 0.04	$0.00018642	$0.6713	22.99 $\pm$ 1.68	$0.010409	$3.1226

Appendix

Table 10: Cost analysis of limited and unlimited agentic exploration.

Metric	Developed	Developing	$\boldsymbol{\Delta}$	$p$
Image age (years)	3.09	2.51	$-0.58$ (dev’g newer)	$0.034$
Quality score	0.979	0.982	$+0.003$ (dev’g higher)	$0.030$
Sharpness (Var. Laplacian)	1,499	1,540	$+42$ (n.s.)	$0.70$

Appendix

Table 11: Per-panorama image quality medians across developed and developing samples (Mann–Whitney

U

\Delta

is reported as developing minus developed.

Figure 11: Per-tier and per-development-level distributions of image age, quality score, and sharpness (

n=1{,}200

). No monotonic degradation is observed from Tier 1 to Tier 5.

Model	Invalid actions (%)	Rounds w/ no nav. (%)
Claude Haiku 4.5	0.04	–
GPT-5 Mini	0.00	–
Gemini 3.0 Flash	0.00	–
Gemma 3 27B	0.00	–
Llama 4 Scout	0.00	–
Geo-R1 7B	87.1	99.8

Appendix

Table 12: Tool-call validity under agentic navigation, over all emitted actions in three runs per model. Every general-purpose VLM produces well-formed tool calls essentially always, while the specialized Geo-R1 7B model does not.

Model	$\Delta$ mean dist (km)	$\Delta$ city acc	$\Delta$ country acc	Dev. coef. log-dist OLS	Dev. coef. city logit
	[95% CI]	[95% CI]	[95% CI]	(p)	(p)
GPT-5 Mini	$-792\ [-1261,-392]$	$-0.022\ [-.155,+.106]$	$+0.071\ [-.009,+.157]$	$-0.24\ (.46)$	$+0.41\ (.14)$
Gemini 3.0 Flash	$-627\ [-1134,-201]$	$-0.053\ [-.168,+.071]$	$+0.025\ [-.027,+.084]$	$-0.37\ (.14)$	$+0.32\ (.13)$
Claude Haiku 4.5	$-1057\ [-1891,-252]$	$+0.046\ [-.036,+.134]$	$+0.070\ [-.056,+.196]$	$+0.30\ (.25)$	$-0.16\ (.59)$
Gemma 3 27B	$-1045\ [-1649,-485]$	$-0.007\ [-.108,+.089]$	$+0.041\ [-.069,+.151]$	$-0.00\ (1.00)$	$+0.19\ (.44)$
Llama 4 Scout	$-864\ [-1699,-126]$	$+0.017\ [-.057,+.091]$	$+0.048\ [-.055,+.158]$	$-0.06\ (.82)$	$+0.33\ (.22)$

Appendix

Table 13: Cluster bootstrap results by developed minus developing cities on the agentic condition over 10 iterations. Negative distance delta = better in developed cities.

UN M49 Subregion	$n$	% ref.	% parent	% sample
Northern America	294	22.8	24.0	24.5
Northern Europe	139	11.2	11.3	11.6
South America	136	3.5	11.0	11.3
Southern Asia	124	3.8	10.6	10.3
South-Eastern Asia	101	5.8	8.0	8.4
Eastern Europe	77	6.0	5.7	6.4

Appendix

Table 14: Composition and retention by UN M49 subregion. % ref. is an independently sampled 10,000-row Street View reference collection built with the same city-targeting heuristic; % parent is the 1,994-row candidate pool retrieved for this study; % sample is the final evaluation set. All parent

\to

sample shifts are

\leq 0.9

pp.

Sub-region (M49)	GPT-5 Mini	Gemini Flash	Claude Haiku	Gemma 3 27B	Llama4 Scout	Geo-R1 7B
Western Asia	215	343	1,245	419	1,558	728
Eastern Europe	350	698	834	540	1,387	300
Western Europe	572	560	407	1,117	918	404
Northern America	656	783	2,061	1,143	1,370	872
Southern Asia	1,173	1,202	1,507	1,263	1,279	1,159
South America	1,326	1,151	3,810	2,841	4,077	3,483

Appendix

Table 15: Mean Haversine distance (km) in the GeoAgent task by a few exemplar UN M49 sub-regions under the limited agentic condition.

Figure 12: A screenshot of the portal for collecting human baselines. It’s roughly the same as LLM agents, with the added field for adding observation chains in plain text and speech-to-text.

Figure 13: The human annotation pipeline allows us to collect detailed reasoning chains as we ask our participants to think out loud.

Annotator	Distance Error (km)	Continent Acc.	Country Acc.	City Acc.
Annotator A	$2346.8\pm 1128.0$ [956, 4692]	$0.200\pm 0.133$	$0.000\pm 0.000$	$0.000\pm 0.000$
Annotator B	$3050.9\pm 1199.4$ [1062, 5545]	$0.200\pm 0.133$	$0.100\pm 0.100$	$0.000\pm 0.000$
Annotator C	$1720.2\pm 270.1$ [1237, 2273]	$0.709\pm 0.038$	$0.468\pm 0.042$	$0.043\pm 0.017$
Annotator D	$3009.5\pm 629.1$ [1829, 4319]	$0.333\pm 0.065$	$0.241\pm 0.059$	$0.111\pm 0.043$
Annotator E	$3751.4\pm 414.7$ [2966, 4598]	$0.565\pm 0.046$	$0.348\pm 0.045$	$0.087\pm 0.026$
Annotator F	$3118.4\pm 468.2$ [2257, 4033]	$0.640\pm 0.069$	$0.380\pm 0.069$	$0.100\pm 0.043$

Appendix

Table 16: Per-annotator geolocation performance metrics reported as mean

\pm

standard error, along with 95% bootstrap confidence intervals.

Country	Claude Haiku 4.5	Gemini 3.0 Flash	Gemma 3 27B	Llama 4 Scout	GPT-5 mini
India	indian, india, auto, tropical, market	india, delhi, indian, rickshaws, auto	india, hindi, indian, hindi signage, auto rickshaws	india, tuk, english, urban, script	indian, india, tropical, south asian, south
Japan	japanese, japanese residential, ginkgo, japan, tokyo	japanese, japan, tokyo, utility, nakano	japanese, autumn, japanese writing, retaining, concrete retaining	japanese, plaque, japanese characters, presence japanese, japan	ginkgo, ginkgo leaves, japanese, leaves, tokyo
Norway	nordic, norway, norwegian, road, scandinavian	norway, norwegian, oslo, sweden, birch	norway, scandinavian, road, sweden, norwegian	picket, european, white picket, picket fence, white	norway, norwegian, scandinavian, picket, birch
Switzerland	swiss, tunnel, alpine, european, roofs ivy	switzerland, bern, swiss, zurich, shutters	german, tram tracks, tram tracks, park	shutters, chinese, european, architecture shutters, chinese characters	swiss, park, switzerland, european, chinese
Taiwan	japanese, chinese, urban, taiwan, taiwanese	taipei, taiwan, traditional chinese, chinese, xinyi	chinese characters, chinese, characters, taiwan, scooters	chinese, chinese characters, characters, presence chinese, asian	chinese, taiwan, taipei, traditional chinese, traditional
Thailand	river, bridge, cable, mekong, stayed	bangkok, soi, thai, thailand, bridge	river, chao phraya, chao, phraya, phraya river	river, bridge, water, stayed bridge, cable	river, phraya, chao, chao phraya, bangkok

Appendix

Table 17: Qualitative geographic descriptors produced by different vision-language models across countries. Models generally reason with nationality-specific terms.

Country	Top Words
Argentina	spanish, andean, colombian, colonial, white, argentine, argentina, altitude, high altitude, urban
Australia	australian, modern, australia, melbourne, red, brick, white, tropical, trees, vegetation
Bangladesh	bengali, tropical, india, indian, vegetation, bengali script, bangladesh, tropical vegetation, kerala, script
Brazil	brazilian, brazil, tropical, portuguese, vegetation, tropical vegetation, modern, urban, portuguese signage, portuguese language
Cambodia	cambodia, rural, tropical, vegetation, red, soil, tropical vegetation, laterite, red laterite, laos
Canada	canadian, north american, american, prairie, canada, rural, midwest, trees, flat, road

Appendix

Table 18: TF-IDF qualitative geographic descriptors for Claude Haiku 4.5 under agentic navigation.

Country	Top Words
Argentina	spanish, spanish speaking, presence spanish, spanish text, latin american, latin, south, american, speaking country, speaking
Australia	australian, english, palm, palm trees, australia, modern, city, skyline, city skyline, cars
Bangladesh	bengali, bangladesh, script, bengali script, tuk, tropical, rickshaws, presence bengali, south asian, palm
Brazil	portuguese, portuguese speaking, brazil, presence portuguese, cars, portuguese text, latin american, speaking, speaking country, spanish
Cambodia	dirt, khmer, tropical, cambodia, dirt road, khmer script, southeast, shows dirt, wall, dirt path
Canada	north, north american, typical north, american, suburban, canada, suggest north, houses, vehicles, cars

Appendix

Table 19: TF-IDF qualitative geographic descriptors for Llama 4 Scout under agentic navigation.

Country	Top Words
Argentina	spanish, argentina, argentine, buenos aires, buenos, aires, ushuaia, spanish language, uruguay, american
Australia	australian, melbourne, australia, sydney, laneway, australian style, english, darwin, tropical, inner
Bangladesh	bengali, bangladesh, bengali script, tropical, dhaka, rickshaws, rickshaw, script, cycle, india
Brazil	portuguese, brazilian, brazil, tropical, portuguese language, brazilian style, indicate brazil, portuguese signage, curitiba, brazilian city
Canada	prairie, canadian, north american, american, north, canada, saskatchewan, alberta, canadian prairie, flat prairie
France	paris, french, central paris, france, parisian, haussmann, haussmann style, lib, french language, le

Appendix

Table 20: TF-IDF qualitative geographic descriptors for GPT-5 Mini under agentic navigation.

Field	Type	Description
image_id	string	MD5 hash of the panorama image bytes — unique instance identifier
pano_id	string	Google Street View panorama ID
image_filename	string	Filename within the collection archive
image_path	string	Relative path to the stored image file
ground_truth_lat / ground_truth_lng	float	Coordinates of the Street View camera
actual_lat / actual_lng	float	Coordinates of the Google-reported actual location

Appendix

Table 21: Dataset metadata fields and descriptions.

Geospatial Reasoning Benchmarks

See all 31 papers

May 29, 2026

ERGeoBench:A Comprehensive Benchmark for Embodied Reasoning and Geo-localization in Multimodal Large Language Models

Multimodal large language models (MLLMs) have shown strong potential as embodied agents, yet embodied geo-localization remains underexplored due to the lack of fine-grained evaluation. We introduce ERGeoBench, a diagnostic benchmark for vision-driven embodied geo-localization. ERGeoBench evaluates models under three progressive settings -- single-view, panorama-view, and embodied-view -- where agents may actively acquire observations through sequential changes in yaw, pitch, and zoom. The benchmark contains 2,207 globally distributed street-view panoramas and measures four complementary capabilities: foundational perception, spatial awareness, common sense reasoning, and geo-localization reasoning. Evaluations of leading proprietary and open-source MLLMs show that current models can infer high-level geographic semantics, but still struggle with fine-grained perceptual operations, metric localization, and spatial consistency across views. We further observe that geo-localization is strongly correlated with the other capability dimensions, suggesting that accurate localization depends on integrated perception, spatial reasoning, and commonsense inference rather than isolated visual recognition. Overall, ERGeoBench provides a unified framework for diagnosing and advancing human-like embodied geo-localization. Project Page: https://kaixuewen.github.io/ERGeoBench/

Kaiwen Xue, Tao Wei, Guoxin Zhang +5

Beijing University of Posts and Telecommunications, Beijing, China · State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China · School of Materials Science and Engineering, Shanghai Jiao Tong University, Shanghai, China +2

Multimodal Large Language Models Visual Spatial Reasoning Multimodal Model Evaluation Spatial Reasoning Benchmarks Geospatial Reasoning Visually Grounded Reasoning+1

Ask PDF

Mar 11, 2026

Learning to Wander: Improving the Global Image Geolocation Ability of LMMs via Actionable Reasoning

Geolocation, the task of identifying the geographic location of an image, requires abundant world knowledge and complex reasoning abilities. Though advanced large multimodal models (LMMs) have shown superior aforementioned capabilities, their performance on the geolocation task remains unexplored. To this end, we introduce WanderBench, the first open access global geolocation benchmark designed for actionable geolocation reasoning in embodied scenarios. WanderBench contains over 32K panoramas across six continents, organized as navigable graphs that enable physical actions such as rotation and movement, transforming geolocation from static recognition into interactive exploration. Building on this foundation, we propose GeoAoT (Action of Thought), a Geolocation framework with Action of Though, which couples reasoning with embodied actions. Instead of generating textual reasoning chains, GeoAoT produces actionable plans such as, approaching landmarks or adjusting viewpoints, to actively reduce uncertainty. We further establish an evaluation protocol that jointly measures geolocation accuracy and difficulty-aware geolocation questioning ability. Experiments on 19 large multimodal models show that GeoAoT achieves superior fine-grained localization and stronger generalization in dynamic environments. WanderBench and GeoAoT define a new paradigm for actionable, reasoning driven geolocation in embodied visual understanding.

Yushuo Zheng, Huiyu Duan, Zicheng Zhang +2

Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory

Multimodal Large Language Models VLM Reasoning Cross-View Geo-Localization Active Visual Perception

Ask PDF

Sep 30, 2026

KilometerVision: A New Frontier for Large-Scale Spatial Intelligence in VLMs

We push the frontier of large-scale spatial intelligence in Vision-Language Models (VLMs) and introduce the first benchmark that probes geographical layout understanding from real-world videos, spanning up to 1km distances. Inspired by the cognitive science literature, we evaluate models against the hierarchical stages of human spatial awareness: anchoring via landmarks, connecting them through routes, and integrating these into global mental maps. Extensive experiments reveal a fundamental divergence in how current AI models process spatial information. Instead of utilising true path integration or forming geometric survey knowledge, we find that VLMs rely almost entirely on 2D visual recognition and text-matching to bypass complex spatial reasoning. The benchmark is publicly available at https://perception-test-challenge.github.io/kilometervision.html.

Aravindh Mahendran, Michael King, Matthew Koichi Grimes +12

Google DeepMind, Berlin, Germany · Google DeepMind, London, UK · Princeton University, Princeton, USA +3

Spatial Reasoning Benchmarks Visual Spatial Reasoning Geospatial Reasoning VLM Reasoning Spatial Reasoning in Language Models Geospatial Reasoning Benchmarks

Ask PDF

Abstract

Geospatial Reasoning Benchmarks

Explore similar work

ERGeoBench:A Comprehensive Benchmark for Embodied Reasoning and Geo-localization in Multimodal Large Language Models

Learning to Wander: Improving the Global Image Geolocation Ability of LMMs via Actionable Reasoning

KilometerVision: A New Frontier for Large-Scale Spatial Intelligence in VLMs