Enabling Intrinsic Reasoning over Dense Geospatial Embeddings with DFR-Gemma

Xuechen Zhang; Aviv Slobodkin; Joydeep Paul; Mandar Sharma; Samet Oymak; Shravya Shetty; Gautam Prasad

cs.CLApr 8, 2026

Enabling Intrinsic Reasoning over Dense Geospatial Embeddings with DFR-Gemma

Authors: Xuechen Zhang, Aviv Slobodkin, Joydeep Paul, Mandar Sharma, Samet Oymak, Shravya Shetty, Gautam Prasad

Organizations: Google Research Research · University of Michigan

Abstract

Representation learning for geospatial and spatio-temporal data plays a critical role in enabling general-purpose geospatial intelligence. Recent geospatial foundation models, such as the Population Dynamics Foundation Model (PDFM), encode complex population and mobility dynamics into compact embeddings. However, their integration with Large Language Models (LLMs) remains limited. Existing approaches to LLM integration treat these embeddings as retrieval indices or convert them into textual descriptions for reasoning, introducing redundancy, token inefficiency, and numerical inaccuracies. We propose Direct Feature Reasoning-Gemma (DFR-Gemma), a novel framework that enables LLMs to reason directly over dense geospatial embeddings. DFR aligns high-dimensional embeddings with the latent space of an LLM via a lightweight projector, allowing embeddings to be injected as semantic tokens alongside natural language instructions. This design eliminates the need for intermediate textual representations and enables intrinsic reasoning over spatial features. To evaluate this paradigm, we introduce a multi-task geospatial benchmark that pairs embeddings with diverse question-answer tasks, including feature querying, comparison, and semantic description. Experimental results show that DFR allows LLMs to decode latent spatial patterns and perform accurate zero-shot reasoning across tasks, while significantly improving efficiency compared to text-based baselines. Our results demonstrate that treating embeddings as primary data inputs, provides a more direct, efficient, and scalable approach to multimodal geospatial intelligence.

Figures & tables

Figure 1: Comparison of geospatial reasoning pipelines. DFR (top) enables direct reasoning over embeddings, while prior methods rely on fragmented pipelines or noisy text/raw inputs, resulting in inefficiency and errors.

Figure 2: The architecture (left) and corresponding pseudocode (right) illustrate the end-to-end integration of geospatial features into a frozen LLM. Detaied explaination are in Section 3 and Section 5 . In DFR-Gemma, the Projector maps dense each PDFM embedding directly into $N$ Aligned Soft Tokens. We shown $N=2$ as an example in the figure. To retain pre-trained linguistic intelligence, the LLM Backbone remains frozen while only the Cross-Modal Projector is optimized to align the geospatial manifold with the LLM’s latent space. During inference, the Positional Re-indexer recomputes sequence IDs for the interleaved text and soft tokens. This ensures the attention mechanism correctly interprets spatial-textual relationships, enabling the model to decode latent patterns and perform intrinsic, zero-shot reasoning validated against ground truth.

Type	Method	Single-Embed Queries		Feature Description		Multi-Embed Queries
		Gemma	Qwen	Gemma	Qwen	Gemma	Qwen
No Training	Zero Context	0.67	0.54	173.59	254.30	0.21	0.24
	Unprocessed Raw Input	0.63	0.61	284.81	298.03	0.18	0.30
	Raw Data Description	0.70	0.76	152.45	351.87	0.46	0.53
No LLM	MLP	0.76		/		0.37
	LightGBM	0.81		/		0.39

Table 1: Performance Comparison on Geospatial Reasoning Tasks. Performance of description is measured by perplexity(ppl). Others are measured by Accuracy (%). DFR-Gemma consistently outperforms the baselines, particularly in complex multi-modal tasks, demonstrating the effectiveness of reasoning directly over latent geospatial embeddings compared to textual serialization.

Panel of Figure 3 — Figure 3: Effect of soft token count $N$ under Separate and Mix training strategies.

Figure 4: Token length distribution (min–max range with average marker).

Figure 5: Absolute Value Estimation using 4-Option Multiple-Choice Evaluation.

Method	Single-Embed Queries	Feature Description	Multi-Embed Queries	HellaSwag	GPQA Diamond
DFR-Gemma	0.75	21.89	0.65	0.77 $\star$	0.15 $\star$
Projector+First layer	0.78	20.71	0.63	0.66	0.09
Projector+Full LLM	0.76	15.64	0.67	0.53	0.04

Table 2: Compare DFR-Gemma (only the projector is trainable) against two increasingly intensive training baselines.

\star

are results from Gemma (2025) , which is the official report. HellaSwag Zellers et al. (2019) and GPQA Diamond Rein et al. (2024) are benchmarks commonly used to evaluate reasoning capabilities in language models.

Type	Method	Single-Embed	Multiple-Embed
No Training	Zero Context	0.61 (-0.06)	0.14 (-0.07)
	Unprocessed Raw Input	0.57 (-0.06)	0.13 (-0.05)
	Raw Data Description	0.66 (-0.04)	0.42 (-0.04)
Ours	Separate, N=1	0.74 (-0.01)	0.62 (-0.03)
Ours	Mix, N=4	0.79 (+0)	0.73 (+0.01)

Table 3: Evaluation of Model Stability across Linguistic Styles. Accuracy scores are shown for out-of-domain (OOD) writing styles, with the change from the original baseline performance noted in parentheses. DFR-Gemma maintains near-perfect consistency (marginal to zero delta), whereas text-based baselines experience significant drops due to sensitivity to syntactic and stylistic shifts.

Panel of Figure 6 — Figure 6: Model Robustness to Geographic Distribution Shifts. We evaluate the transferability of the DFR framework from postal-code-level training to county-level testing.

Figure 7: Multi-hop Reasoning.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Accuracy
$\mathcal{L}_{\text{CE}}$	0.79
$\mathcal{L}_{\text{total}}$	0.70

Appendix

Table 4: Hybrid Contrastive Learning Performance.

Geospatial Reasoning

Geospatial reasoning studies how AI interprets locations, maps, spatial relationships, and Earth-observation data to answer questions and make decisions.

See all 77 papers

Aug 6, 2025

GeoGR^2:Zero-Shot Geospatial Inference via Geostatistically-Guided Iterative Refinement with LLMs

Standard large language model prompting treats geospatial inference as independent, instance-wise prediction, ignoring the fundamental spatial dependencies that govern geographic reality. Consequently, even advanced models struggle with spatial consistency and exhibit severe biases toward populous regions. To bridge this gap, we propose GeoGR^2 (Geospatial Graph Refine Reasoning), a framework that formalizes zero-shot geospatial prediction as an iterative message-passing process on a dynamically constructed graph. Unlike static retrieval methods, GeoGR^2 instantiates three dynamic operators via collaborating operators: (1) a Topology Operator that constructs graph topology to enforce the Spatial Markov property; (2) a Feature Operator that enriches nodes with task-relevant semantic covariates; and (3) an Update Operator that performs natural language message passing to iteratively minimize spatial discrepancy. Theoretically, we frame this refinement as a contraction mapping that approximates the fixed point of a global consistency equation. Empirically, we validate GeoGR^2 on diverse physical and socioeconomic tasks. Results demonstrate that by explicitly embedding geostatistical inductive biases, GeoGR^2 significantly outperforms standard prompting baselines, while effectively mitigating systematic geographic bias. Our framework leverages large language models' intrinsic capacity for understanding spatial correlations through explicit topological scaffolding, without resorting to general graph reasoning paradigms. The code of GeoGR^2 is available at https://github.com/JinfanTang/GeoGRR.

Jinfan Tang, Kunming Wu, Xieruifeng Gong +4

College of Computer Science, Sichuan University · National Key Laboratory of Fundamental Science on Synthetic Vision, Sichuan University · College of Life Science, Sichuan University

Zero-Shot Learning Geospatial Reasoning Spatial Reasoning Spatial Context Modeling

Ask PDF

Jun 3, 2026

From Symbolic to Geometric: Enabling Spatial Reasoning in Large Language Models

Recent large language models (LLMs) often appear to exhibit spatial reasoning ability; however, this capability is largely symbolic, arising from pattern matching over spatial language rather than true geometric reasoning over space. Because LLMs operate on discrete tokens, they lack native support for continuous spatial representations, explicit geometric computation, and structured spatial operators. To address this limitation, we introduce the Spatial Language Model (SLM), the first multimodal LLM that treats location information as a first-class modality and enables geometric spatial reasoning within the model's inference process. SLM directly operates on learned spatial representations rather than textual descriptions of spatial relations. To support effective training, we construct a Spatial Instruction Dataset that aligns spatial representations, atomic geometric operations, and natural language instructions. We further propose a new benchmark named SpatialEval, which is designed to evaluate spatial reasoning across attributes, distance, topology, and relative-position tasks. Extensive experiments show that SLM significantly outperforms existing LLM-based approaches that rely on symbolic reasoning via prompt engineering or textual abstraction, demonstrating the benefits of integrating geometric spatial representations for robust spatial reasoning. Our instruction dataset, evaluation benchmark, model training codes, and models' checkpoints can be found at: \hyperlink{https://github.com/chuchen2017/SLM}{https://github.com/chuchen2017/SLM}.

Chen Chu, Bita Azarijoo, Li Xiong +2

University of Southern California Los Angeles, CA, USA · Emory University Atlanta, GA, USA · Novateur Research Solutions Ashburn, Virginia, USA

Spatial Reasoning Benchmarks Multimodal Large Language Models Geometric Representation Learning Spatial Reasoning Spatial Reasoning in Language Models

Ask PDF

Oct 12, 2026

Geospatial Vector Geometry as a Native Spatial Modality for Vision-Language Models

Geospatial reasoning links VLMs to real-world action and decision making. Reliable reasoning requires accurate spatial information and effective spatial representations. However, current VLMs obtain this information mainly from language and vision. At larger geographic scales, limited visual observations make it difficult to establish accurate spatial references and recover precise spatial relations. This leads to a bottleneck for VLM spatial perception and reasoning. This paper proposes that geospatial vector geometry can become a native spatial modality within a VLM and introduces GeoVec-VLM. First, we develop a structured geometry representation that preserves the spatial structure and arrangement of vector objects, building an explicit spatial basis. Second, we introduce a geometry interaction strategy that brings this basis into multimodal computation, where it interacts with visual content and language references according to the queried spatial relations. Together, these designs allow perception and reasoning to operate on shared geometric references. Across five spatial reasoning tasks, GeoVec-VLM improves mean accuracy by about 11% relative to the strongest baseline, maintains its gains under geographic transfer, and shows greater geometric consistency. Our results show that geospatial vector geometry offers a promising internal spatial basis for VLM spatial reasoning. https://github.com/TXFR/GeoVector-VLM

Huang Fei, Boyi Chen, Hao Xue +3

The Hong Kong University of Science and Technology (Guangzhou), China · The Hong Kong University of Science and Technology, Hong Kong SAR, China

Vision-Language Models Geospatial Reasoning Geospatial Representation Learning Vision-Language Grounding

Ask PDF

Abstract

Geospatial Reasoning

Explore similar work

GeoGR^2:Zero-Shot Geospatial Inference via Geostatistically-Guided Iterative Refinement with LLMs

From Symbolic to Geometric: Enabling Spatial Reasoning in Large Language Models

Geospatial Vector Geometry as a Native Spatial Modality for Vision-Language Models