Geospatial Reasoning Benchmarks

Follow this topic Ask about this topic

Latest papers

31 papers8 papers in the last four weeks, up 100% on the four weeks before. 0.0% of all new papers.

Google Earth AI Geospatial Reasoning Agent: Capabilities and a Systematic Evaluation
Oct 11, 2026Noam Slonim, Aviv Slobodkin, Gil Fidel +34Geospatial QA Geospatial Reasoning
Google Research · Google Geo · Spatial Bytes
PlacePulse: Learning Transferable Place Representations from Structural Geographic Data
Oct 9, 2026Lucía García-Duarte, Miguel Álvarez, Diego Vicente +2Geospatial Reasoning Benchmarks Geospatial Representation Learning
(CARTO)
Mind the Spatial Gap: A Benchmark for Geospatial Reasoning and Constrained Spatial Editing
Oct 9, 2026Zheng Ren, Yanning Wang, Ziheng Huang +4Geospatial Reasoning Benchmarks Geospatial Reasoning
Urban Governance and Design (UGOD) Thrust The Hong Kong University of Science and Technology (Guangzhou) Guangzhou, China
GeoFitBench-LLM: A Benchmark for Assessing Large Language Models' Capabilities in Geospatial Data Requirements Elicitation for Geo-Analytic Tasks
Oct 8, 2026Amirhossein Vahdat, Thierry Badard, Luc LamontagneLLM Evaluation Geospatial Reasoning Benchmarks
Département des Sciences Géomatiques, Université Laval, QC, Canada · Département d’informatique et de génie logiciel, Université Laval, QC, Canada
GeoNatureAgent (GNA): A Framework and Benchmark for Pre-Production Evaluation of Tool-Using Agents on Geospatial and Environmental Tasks
Oct 6, 2026Gabriel Diaz-Ireland, Diego Prieto-Herráez, Mario García Peces +3Tool-Use Evaluation LLM Agent Evaluation
Universidad Católica de Ávila (UCAV) Ávila, Spain · Independent Researcher Madrid, Spain · Department of Earth and Planetary Sciences, Johns Hopkins University Baltimore, MD, USA +1
KilometerVision: A New Frontier for Large-Scale Spatial Intelligence in VLMs
Sep 30, 2026Aravindh Mahendran, Michael King, Matthew Koichi Grimes +12Spatial Reasoning Benchmarks Visual Spatial Reasoning
Google DeepMind, Berlin, Germany · Google DeepMind, London, UK · Princeton University, Princeton, USA +3
Network-based Spatial Context Retrieval for Open-weight LLMs: A Faithfulness Benchmark for Grounded Geographic Reasoning
Sep 30, 2026Joan PerezSpatial Reasoning Benchmarks LLM Grounding
Urban Geo Analytics
MIND the Gap: A Geographic Implicit Neural Representation with Adjustable Spatial Scale
Sep 21, 2026Isaac Corley, Arjun Rao, Esther Rolf +7Geospatial Reasoning Benchmarks Geospatial Representation Learning
Taylor Geospatial · University of British Columbia · Vector Institute +7
GeoContext: One Context Ladder, Two Failure Modes in Vision-Language Geolocation: Flat Reliance on User-Provided Location Context and False Confirmation of Location Claims
Sep 4, 2026Yifan Zhang, Kai WangGeospatial Reasoning Benchmarks VLM Evaluation
GeoAgent: Evaluating VLM Geolocalization Through Embodied Navigation
Aug 30, 2026Arka Mukherjee, Soham Roy, Kartikeya Trivedi +1Geospatial Reasoning Benchmarks VLM Reasoning
KIIT Bhubaneswar · IIT Bhubaneswar
LifePlanner: Evaluating LLM Agents for Geo-spatial Planning with Social Media Data
Aug 25, 2026Zhen Dong, Yuning Peng, Yutao Shi +4Geospatial Reasoning Benchmarks LLM Agent Evaluation
Wuhan University · Hong Kong University of Science and Technology
GeoRefer-Bench: A Benchmark from Referring Pixels to Verifiable Geospatial Reasoning
Aug 25, 2026Shuaishuai Cao, Min Huang, Meng Tang +3Spatial Reasoning Benchmarks Geospatial Reasoning
Central South University · Jiangxi Normal University · Hohai University +2
GeoBenchLLM: A Comprehensive Benchmark for Evaluating LLMs on Geo-Related Tasks
Aug 7, 2026Rodrigo Ferreira Rodrigues, Karim Radouane, Jose G Moreno +1Spatial Reasoning Benchmarks LLM Evaluation
University of Toulouse Toulouse, France
MultiGlobeQA: A Multilingual and Globally Diverse Benchmark for Geospatial Reasoning
Aug 4, 2026Martin Böckling, Elizaveta Nosova, Heiko Paulheim +1Spatial Reasoning Multilingual QA
Data and Web Science Group, University of Mannheim, Germany
Geo-Embed: Towards Unified Multimodal Embeddings for Urban Understanding
Aug 4, 2026Jiapeng Li, Yong Li, Junjie Zhou +2Geospatial Representation Learning Cross-Modal Retrieval
1Peking University · 2Beijing University of Posts and Telecommunications
GISAgentBench: A Practitioner-Sourced Benchmark for Evaluating LLM Agents on GIS Tasks
Aug 3, 2026Abhinav Pothuri, Zhe Jiang, Zelin Xu +1LLM Agent Evaluation AI Agent Benchmarks
Department of Computer & Information Science & Engineering, University of Florida · Department of Geography, University of Florida
Do Maps Still Matter for Machines: Revisiting the Role of Choropleth Maps in Foundation Model Spatial Understanding
Jul 20, 2026Zhiwei Wei, Yonghe Sun, Zhenjia Liu +5Data Visualization Spatial Reasoning Benchmarks
aSchool of Geographic Sciences, Hunan Normal University, Changsha, China · bHunan Key Laboratory of Geospatial Big Data Mining and Application, Changsha, China · cSchool of Information and Communication Engineering, Beijing University of Posts and Telecommunications, Beijing, China +3
TerraLogic: A Benchmark for Hierarchical Geospatial Reasoning in Earth Observation
Jul 14, 2026Yuhang Yan, Linchao Mou, Bokang Yang +1Spatial Reasoning Hierarchical Planning
The Chinese University of Hong Kong (Shenzhen) China · Technical University of Munich Germany
MapReason-OSM: Can Vision-Language Models Make Graph-Verifiable Mobility Decisions from Street Maps ?
Jun 21, 2026Srinivas Venkatanarayanan, Clement Pakkam IsaacGeospatial Reasoning Spatial Reasoning Benchmarks
NVIDIA · University of Central Florida · University of South Florida
GeoNatureAgent Benchmark: Benchmarking LLM Agents for Environmental Geospatial Analysis Across Frontier and Open-Weight Foundation Models
Jun 11, 2026Gabriel Diaz-Ireland, Diego Prieto-Herráez, Mario García Peces +2LLM Agent Evaluation Geospatial Reasoning
Universidad Católica de Ávila (UCAV) Ávila, Spain · Johns Hopkins University Baltimore, MD, USA · Independent Researcher Madrid, Spain +1
Plan2Map: A Multimodal Benchmark for Document-Grounded Geospatial Boundary Reconstruction from Planning Records
Jun 1, 2026Fabian Degen, Oishi Deb, Jindong Gu +4Geospatial Reasoning Geospatial Reasoning Benchmarks
University of Oxford · 2Google DeepMind
ERGeoBench:A Comprehensive Benchmark for Embodied Reasoning and Geo-localization in Multimodal Large Language Models
May 29, 2026Kaiwen Xue, Tao Wei, Guoxin Zhang +5Multimodal Large Language Models Visual Spatial Reasoning
Beijing University of Posts and Telecommunications, Beijing, China · State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China · School of Materials Science and Engineering, Shanghai Jiao Tong University, Shanghai, China +2
GeoX: Mastering Geospatial Reasoning Through Self-Play and Verifiable Rewards
May 19, 2026Kyeongjin Ahn, Seungeon Lee, Krishna P. Gummadi +1Spatial Reasoning Self-Play RL
KAIST · MPI-SWS · MPI-SP
GeoR-Bench: Evaluating Geoscience Visual Reasoning
May 12, 2026Yushuo Zheng, Zicheng Zhang, Huiyu Duan +7Remote Sensing Image Understanding Geospatial Reasoning
Shanghai Jiao Tong University · Beijing University of Technology
EVGeoQA: Benchmarking LLMs on Dynamic, Multi-Objective Geo-Spatial Exploration
Apr 8, 2026Jianfei Wu, Zhichun Wang, Zhensheng Wang +1Geospatial QA Geospatial Reasoning Benchmarks
School of Artificial Intelligence, Beijing Normal University, Beijing 100875, China · Beijing Key Laboratory of Artificial Intelligence for Education, Beijing 100875, China · Engineering Research Center of Intelligent Technology and Educational Application, Ministry of Education, Beijing 100875, China +1
GeoBrowse: A Geolocation Benchmark for Agentic Tool Use with Expert-Annotated Reasoning Traces
Apr 5, 2026Xinyu Geng, Yanjing Xiao, Yuyang Zhang +5Geospatial Reasoning Benchmarks Tool-Using Vision-Language Agents
Hong Kong University of Science and Technology
GISclaw: A Comprehensive Open-Source LLM Agent System for Realistic Multi-Step Geospatial Analysis
Mar 27, 2026Jinzhen Han, JinByeong Lee, Yuri Shim +2Geospatial Reasoning Benchmarks Geospatial Reasoning
Department of Civil, Architectural & Environment Engineering, Sungkyunkwan University, Suwon, South Korea · Ministry of the Interior and Safety, South Korea · School of Geography, University of Leeds, United Kingdom +1
TimeSpot: Benchmarking Geo-Temporal Understanding in Vision-Language Models in Real-World Settings
Mar 4, 2026Azmine Toushik Wasi, Shahriyar Zaman Ridoy, Koushik Ahamed Tonmoy +5Spatiotemporal Reasoning Geospatial Reasoning Benchmarks
Computational Intelligence and Operations Laboratory (CIOL), Bangladesh · Shahjalal University of Science and Technology (SUST), Sylhet, Bangladesh · North South University (NSU), Dhaka, Bangladesh +1
OpenEarthAgent: A Unified Framework for Tool-Augmented Geospatial Agents
Feb 19, 2026Akashah Shabbir, Muhammad Umer Sheikh, Muhammad Akhtar Munir +8Geospatial Reasoning Benchmarks Remote Sensing
Mohamed bin Zayed University of AI, United Arab Emirates · IBM Research, Europe · Linköping University, Sweden +2
GPSBench: Do Large Language Models Understand GPS Coordinates?
Feb 18, 2026Thinh Hung Truong, Jey Han Lau, Jianzhong QiGeospatial Reasoning Geospatial Reasoning Benchmarks
University of Melbourne.

Oct 11, 2026J/K move · Enter open · S save

Google Earth AI Geospatial Reasoning Agent: Capabilities and a Systematic Evaluation

Noam Slonim, Aviv Slobodkin, Gil Fidel, Gal Moshitch+33

Google Research · Google Geo · Spatial Bytes

Geospatial QA Geospatial Reasoning AI Agent Benchmarks Geospatial Reasoning Benchmarks Geospatial Foundation Models

Open paper AskPDF arXiv

Google Earth AI is a family of geospatial AI models, together with an agent that orchestrates them. We present this agent, Google Earth AI's Geospatial Reasoning agent (GRA), a Gemini-powered research system that combines geospatial data sources, foundation models, and tools to answer questions about our planet. GRA organizes these into modular domain skills and pairs them with upfront geospatial planning to decompose complex queries into multi-stage workflows. Evaluating such systems is hard: real-world problems admit multiple valid workflows, require data discovery across large catalogs, and rely on evolving ground truth. We introduce two benchmarks and release their development splits. GeoExpertQueries poses complex analytical problems grounded in practitioner use cases and refined by geospatial experts; each is paired with a consensus golden workflow from three independent experts and scored by a human-validated, tool-agnostic autorater. GeoKTools consists of templated queries with deterministic reference solvers that systematically vary the number of required tool types. We also use two external benchmarks, CBGB and UnivEARTH; the latter we find to be saturated. To isolate the contribution of GRA's tools and its agentic orchestration, we compare four configurations, each with Gemini 3.1 Pro and 3.8 Flash: a standalone LLM, the LLM with basic tools, the LLM with GRA's skills and tools but without its upfront planning and execution guidance, and GRA itself. With at most basic tools, an advanced model recovers much of the answer on templated queries but satisfies only a small fraction of the expert workflows on open-ended problems; GRA performs well on both. The largest step comes from GRA's skills and tools; on open-ended problems, its planning and execution guidance add a further gain with Gemini 3.8 Flash, letting Flash match or exceed Pro at lower cost.