Do LLMs Build Spatial World Models? Evidence from Grid-World Maze Tasks

Weijiang Li; Yilin Zhu; Rajarshi Das; Parijat Dube

cs.AIApr 12, 2026

Do LLMs Build Spatial World Models? Evidence from Grid-World Maze Tasks

Authors: Weijiang Li, Yilin Zhu, Rajarshi Das, Parijat Dube

Organizations: University of Notre Dame · Columbia University · MQube Cognition

Abstract

Foundation models have shown remarkable performance across diverse tasks, yet their ability to construct internal spatial world models for reasoning and planning remains unclear. We systematically evaluate the spatial understanding of large language models through maze tasks, a controlled testing context requiring multi-step planning and spatial abstraction. Across comprehensive experiments with Gemini-2.5-Flash, GPT-5-mini, Claude-Haiku-4.5, and DeepSeek-Chat, we uncover significant discrepancies in spatial reasoning that challenge assumptions about LLM planning capabilities. Using chain-of-thought prompting, Gemini achieves 80-86% accuracy on smaller mazes (5x5 to 7x7 grids) with tokenized adjacency representations, but performance collapses to 16-34% with visual grid formats, which is a 2-5x difference, suggesting representation-dependent rather than format-invariant spatial reasoning. We further probe spatial understanding through sequential proximity questions and compositional distance comparisons. Despite achieving 96-99% semantic coverage in reasoning traces, models fail to leverage this understanding for consistent spatial computations, indicating that they treat each question independently rather than building cumulative spatial knowledge. Our findings based on the maze-solving tasks suggest that LLMs do not develop robust spatial world models, but rather exhibit representation-specific and prompting-dependent reasoning that succeeds only under narrow conditions. These results have critical implications for deploying foundation models in applications requiring spatial abstraction.

Figures & tables

Figure 1: Overview of the main maze navigation task. This illustrates our systematic framework for evaluating whether LLMs construct spatial abstractions or rely on surface-level pattern matching. Models are asked to navigate mazes presented in two distinct formats: adjacency lists and visual character grids. Here, we present a comparison between direct querying LLMs, where models tend to fail the task by getting lost in the maze, and successfully solving the task through reasoning-based prompting. The latter can elicit more structured and valid navigation paths.

Figure 2: An overview of model performance across all tasks and maze sizes, with each spoke representing a maze-solving task and grid size, and colored lines representing the average performance of the LLM over all mazes generated for that task.

	5×5			6×6			7×7			8×8			9×9
Model	SC	R-L	LLM	SC	R-L	LLM	SC	R-L	LLM	SC	R-L	LLM	SC	R-L	LLM
Gemini-2.5-Flash	0.96	0.85	96.67	0.94	0.77	93.23	0.95	0.75	94.17	0.92	0.67	94.13	0.91	0.57	90.43
GPT-5-mini	0.98	0.89	93.47	0.98	0.87	92.19	0.97	0.74	90.23	0.93	0.52	85.00	0.94	0.58	82.48
Claude-Haiku-4.5	0.86	0.40	85.90	0.85	0.39	87.50	0.85	0.37	86.53	0.86	0.39	85.71	0.87	0.36	85.0
DeepSeek-Chat	0.86	0.27	87.40	0.86	0.26	87.04	0.86	0.23	86.84	0.81	0.10	88.33	0.82	0.08	87.62

Table 1: Reasoning quality results for sequential questions with point reuse, measured by semantic coverage (SC), ROUGE-L (R-L), and LLM-as-a-judge (LLM)

	5×5			6×6			7×7			8×8			9×9
Model	SC	R-L	LLM	SC	R-L	LLM	SC	R-L	LLM	SC	R-L	LLM	SC	R-L	LLM
Gemini-2.5-Flash	0.93	0.42	9.49	0.92	0.39	16.25	0.95	0.52	17.86	0.95	0.51	14.32	0.95	0.56	12.36
GPT-5-mini	0.95	0.59	23.4	0.95	0.60	17.1	0.96	0.82	16.1	0.96	0.94	0.0	0.98	0.99	0.0
Claude-Haiku-4.5	0.94	0.55	8.16	0.96	0.62	7.0	0.94	0.54	10.0	0.95	0.55	4.6	0.95	0.59	13.0
DeepSeek-Chat	0.96	0.59	3.0	0.96	0.60	0.0	0.94	0.56	5.0	0.95	0.55	5.6	0.93	0.50	10.9

Table 2: Reasoning quality results for compositional distance comparison, measured by semantic coverage (SC), ROUGE-L (R-L), and LLM-as-a-judge (LLM)

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Train Grid Size	Test Grid Size
Train Grid Size	5×5	6×6	7×7	8×8	9×9
5×5	0.806	0.407	0.251	0.152	0.093
6×6	0.856	0.769	0.449	0.258	0.168
7×7	0.820	0.746	0.691	0.400	0.248
8×8	0.799	0.718	0.673	0.614	0.373
9×9	0.778	0.694	0.623	0.567	0.537

Appendix

Table 3: Fine-tuned model performance on different grid sizes (Path Accuracy). Each model is trained with 8000 examples and tested with 2000 examples over 10 epochs.

Model	Question	5×5	6×6	7×7	8×8	9×9
Gemini-2.5-Flash	Q0	0.94	0.88	0.78	0.76	0.70
	Q1	1.00	0.90	0.80	0.78	0.62
	Q2	0.96	0.92	0.90	0.78	0.68
	Q3	0.98	0.98	0.92	0.84	0.74
GPT-5-mini	Q0	1.00	0.94	0.92	0.82	0.70
	Q1	1.00	0.98	0.88	0.68	0.54

Appendix

Table 7: Results for sequential questions with point reuse

Model	Question	5×5	6×6	7×7	8×8	9×9
Gemini-2.5-Flash	Q0	1.00	0.92	0.82	0.68	0.52
	Q1	0.98	0.88	0.84	0.68	0.50
	Q2	0.94	0.94	0.74	0.68	0.74
GPT-5-mini	Q0	1.00	0.96	0.44	0.16	0.04
	Q1	1.00	0.94	0.46	0.18	0.04
	Q2	0.96	0.92	0.42	0.16	0.04

Appendix

Table 8: Results for compositional distance comparison accuracy by maze size and question index

Model	Q 0	Q 1	Q 2	Overall	$\Delta$
Gemini-2.5-Flash	78.8	77.6	80.8	79.1	+2.6
GPT-5-mini	52.0	52.4	50.0	51.5	-2.2
Claude-Haiku-4.5	65.2	66.8	67.6	66.5	+1.6
DeepSeek-Chat	77.6	76.0	74.4	76.0	-2.4

Appendix

Table 9: Analysis of composition distance experiment results. Q 0 tests the distance from corner A (top-left) to center M vs corner B. Q 1 tests the distance from corner D (bottom-right) to center M vs corner C. Q 2 tests the distance from corner B to corner C vs center M, which is designed to benefit from composing Q 0 and Q 1 .

\Delta

indicates Q 2 performance relative to avg(Q 0 , Q 1 ).

Explore similar work

Mar 27, 2026

From Human Cognition to Neural Activations: Probing the Computational Primitives of Spatial Reasoning in LLMs

As spatial intelligence becomes an increasingly important capability for foundation models, it remains unclear whether large language models' (LLMs) performance on spatial reasoning benchmarks reflects structured internal spatial representations or reliance on linguistic heuristics. We address this question from a mechanistic perspective by examining how spatial information is internally represented and used. Drawing on computational theories of human spatial cognition, we decompose spatial reasoning into three primitives, relational composition, representational transformation, and stateful spatial updating, and design controlled task families for each. We evaluate multilingual LLMs in English, Chinese, and Arabic under single pass inference, and analyze internal representations using linear probing, sparse autoencoder based feature analysis, and causal interventions. We find that task relevant spatial information is encoded in intermediate layers and can causally influence behavior, but these representations are transient, fragmented across task families, and weakly integrated into final predictions. Cross linguistic analysis further reveals mechanistic degeneracy, where similar behavioral performance arises from distinct internal pathways. Overall, our results suggest that current LLMs exhibit limited and context dependent spatial representations rather than robust, general purpose spatial reasoning, highlighting the need for mechanistic evaluation beyond benchmark accuracy.

Jiyuan An, Liner Yang, Mengyan Wang +3

National Language Resources Monitoring and Research Center for Print Media, Beijing Language and Culture University, Beijing, China · School of Information Science, Beijing Language and Culture University, Beijing, China · School of artificial intelligence, Beijing normal university, Beijing, China

Spatial Reasoning LLM Interpretability

Ask PDF

May 27, 2026

Do LLMs Build World Models From Text? A Multilingual Diagnostic of Spatial Reasoning

Whether large language models (LLMs) construct internal spatial world models from pure-text descriptions remains contested, and whether such capabilities transfer across languages has not been systematically studied. We introduce MentalMap, a multilingual diagnostic benchmark with a six-level capability hierarchy (L0-L5) spanning atomic spatial facts to generative world-graph construction, together with four diagnostic axes probing frame of reference, reading-direction bias, reasoning-effort allocation, and hallucination. MentalMap is built from 100 ProcTHOR household scenes, covers eight typologically diverse languages plus a structured-text control, and contains 39 task families across 1,950 evaluation cells. Evaluating thirteen LLMs across scales and model families, we identify a universal L3 reasoning cliff: no model retains even half of its L0 performance on viewpoint reasoning once baseline atomic accuracy exceeds 40%. The cliff persists across languages, scales, and prompting strategies, while structured-output failures and reasoning patterns vary substantially across models. Human evaluation under the identical pure-text protocol reproduces the same failure pattern, suggesting that the bottleneck arises from text-only working memory constraints rather than being specific to current LLM architectures. Our findings reframe pure-text spatial reasoning as a multi-axis world-modeling problem and motivate multimodal and scratchpad-augmented reasoning as future directions.

Zhikai Pan, Chih-Ting Liao, Chunrui Liu +5

University of New South Wales · Essential Energy · University of Alabama at Birmingham +3

LLM World Models Multilingual Language Model Evaluation Spatial Reasoning Benchmarks World Model Learning Cross-Lingual Reasoning in Language Models Spatial Reasoning

Ask PDF

Mar 10, 2026

World2Mind: Cognition Toolkit for Allocentric Spatial Reasoning in Foundation Models

Achieving robust spatial reasoning remains a fundamental challenge for current Multimodal Foundation Models (MFMs). Existing methods either overfit statistical shortcuts via 3D grounding data or remain confined to 2D visual perception, limiting both spatial reasoning accuracy and generalization in unseen scenarios. Inspired by the spatial cognitive mapping mechanisms of biological intelligence, we propose World2Mind, a training-free spatial intelligence toolkit. At its core, World2Mind leverages 3D reconstruction and instance segmentation models to construct structured spatial cognitive maps, empowering MFMs to proactively acquire targeted spatial knowledge regarding interested landmarks and routes of interest. To provide robust geometric-topological priors, World2Mind synthesizes an Allocentric-Spatial Tree (AST) that uses elliptical parameters to model the top-down layout of landmarks accurately. To mitigate the inherent inaccuracies of 3D reconstruction, we introduce a three-stage reasoning chain comprising tool invocation assessment, modality-decoupled cue collection, and geometry-semantics interwoven reasoning. Extensive experiments demonstrate that World2Mind boosts the performance of frontier models, such as GPT-5.2, by 5%~18%. Astonishingly, relying solely on the AST-structured text, purely text-only foundation models can perform complex 3D spatial reasoning, achieving performance approaching that of advanced multimodal models.

Shouwei Ruan, Bin Wang, Zhenyu Wu +4

Institute of Artificial Intelligence, Beihang University · Huawei Noah’s Ark Lab · Dept. of Comp. Sci. and Tech., Institute for AI, Tsinghua-Bosch Joint ML Center, THBI Lab, BNRist Center, Tsinghua University

Spatial Reasoning 3D Spatial Reasoning Multimodal Foundation Models

Ask PDF