Do LLMs Build Spatial World Models? Evidence from Grid-World Maze Tasks

Weijiang Li; Yilin Zhu; Rajarshi Das; Parijat Dube

cs.AIApr 12, 2026

Do LLMs Build Spatial World Models? Evidence from Grid-World Maze Tasks

Authors: Weijiang Li, Yilin Zhu, Rajarshi Das, Parijat Dube

Organizations: University of Notre Dame · Columbia University · MQube Cognition

Abstract

Foundation models have shown remarkable performance across diverse tasks, yet their ability to construct internal spatial world models for reasoning and planning remains unclear. We systematically evaluate the spatial understanding of large language models through maze tasks, a controlled testing context requiring multi-step planning and spatial abstraction. Across comprehensive experiments with Gemini-2.5-Flash, GPT-5-mini, Claude-Haiku-4.5, and DeepSeek-Chat, we uncover significant discrepancies in spatial reasoning that challenge assumptions about LLM planning capabilities. Using chain-of-thought prompting, Gemini achieves 80-86% accuracy on smaller mazes (5x5 to 7x7 grids) with tokenized adjacency representations, but performance collapses to 16-34% with visual grid formats, which is a 2-5x difference, suggesting representation-dependent rather than format-invariant spatial reasoning. We further probe spatial understanding through sequential proximity questions and compositional distance comparisons. Despite achieving 96-99% semantic coverage in reasoning traces, models fail to leverage this understanding for consistent spatial computations, indicating that they treat each question independently rather than building cumulative spatial knowledge. Our findings based on the maze-solving tasks suggest that LLMs do not develop robust spatial world models, but rather exhibit representation-specific and prompting-dependent reasoning that succeeds only under narrow conditions. These results have critical implications for deploying foundation models in applications requiring spatial abstraction.

Figures & tables

Figure 1: Overview of the main maze navigation task. This illustrates our systematic framework for evaluating whether LLMs construct spatial abstractions or rely on surface-level pattern matching. Models are asked to navigate mazes presented in two distinct formats: adjacency lists and visual character grids. Here, we present a comparison between direct querying LLMs, where models tend to fail the task by getting lost in the maze, and successfully solving the task through reasoning-based prompting. The latter can elicit more structured and valid navigation paths.

Figure 2: An overview of model performance across all tasks and maze sizes, with each spoke representing a maze-solving task and grid size, and colored lines representing the average performance of the LLM over all mazes generated for that task.

	5×5			6×6			7×7			8×8			9×9
Model	SC	R-L	LLM	SC	R-L	LLM	SC	R-L	LLM	SC	R-L	LLM	SC	R-L	LLM
Gemini-2.5-Flash	0.96	0.85	96.67	0.94	0.77	93.23	0.95	0.75	94.17	0.92	0.67	94.13	0.91	0.57	90.43
GPT-5-mini	0.98	0.89	93.47	0.98	0.87	92.19	0.97	0.74	90.23	0.93	0.52	85.00	0.94	0.58	82.48
Claude-Haiku-4.5	0.86	0.40	85.90	0.85	0.39	87.50	0.85	0.37	86.53	0.86	0.39	85.71	0.87	0.36	85.0
DeepSeek-Chat	0.86	0.27	87.40	0.86	0.26	87.04	0.86	0.23	86.84	0.81	0.10	88.33	0.82	0.08	87.62

Table 1: Reasoning quality results for sequential questions with point reuse, measured by semantic coverage (SC), ROUGE-L (R-L), and LLM-as-a-judge (LLM)

	5×5			6×6			7×7			8×8			9×9
Model	SC	R-L	LLM	SC	R-L	LLM	SC	R-L	LLM	SC	R-L	LLM	SC	R-L	LLM
Gemini-2.5-Flash	0.93	0.42	9.49	0.92	0.39	16.25	0.95	0.52	17.86	0.95	0.51	14.32	0.95	0.56	12.36
GPT-5-mini	0.95	0.59	23.4	0.95	0.60	17.1	0.96	0.82	16.1	0.96	0.94	0.0	0.98	0.99	0.0
Claude-Haiku-4.5	0.94	0.55	8.16	0.96	0.62	7.0	0.94	0.54	10.0	0.95	0.55	4.6	0.95	0.59	13.0
DeepSeek-Chat	0.96	0.59	3.0	0.96	0.60	0.0	0.94	0.56	5.0	0.95	0.55	5.6	0.93	0.50	10.9

Table 2: Reasoning quality results for compositional distance comparison, measured by semantic coverage (SC), ROUGE-L (R-L), and LLM-as-a-judge (LLM)

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Train Grid Size	Test Grid Size
Train Grid Size	5×5	6×6	7×7	8×8	9×9
5×5	0.806	0.407	0.251	0.152	0.093
6×6	0.856	0.769	0.449	0.258	0.168
7×7	0.820	0.746	0.691	0.400	0.248
8×8	0.799	0.718	0.673	0.614	0.373
9×9	0.778	0.694	0.623	0.567	0.537

Appendix

Table 3: Fine-tuned model performance on different grid sizes (Path Accuracy). Each model is trained with 8000 examples and tested with 2000 examples over 10 epochs.

Model	Question	5×5	6×6	7×7	8×8	9×9
Gemini-2.5-Flash	Q0	0.94	0.88	0.78	0.76	0.70
	Q1	1.00	0.90	0.80	0.78	0.62
	Q2	0.96	0.92	0.90	0.78	0.68
	Q3	0.98	0.98	0.92	0.84	0.74
GPT-5-mini	Q0	1.00	0.94	0.92	0.82	0.70
	Q1	1.00	0.98	0.88	0.68	0.54

Appendix

Table 7: Results for sequential questions with point reuse

Model	Question	5×5	6×6	7×7	8×8	9×9
Gemini-2.5-Flash	Q0	1.00	0.92	0.82	0.68	0.52
	Q1	0.98	0.88	0.84	0.68	0.50
	Q2	0.94	0.94	0.74	0.68	0.74
GPT-5-mini	Q0	1.00	0.96	0.44	0.16	0.04
	Q1	1.00	0.94	0.46	0.18	0.04
	Q2	0.96	0.92	0.42	0.16	0.04

Appendix

Table 8: Results for compositional distance comparison accuracy by maze size and question index

Model	Q 0	Q 1	Q 2	Overall	$\Delta$
Gemini-2.5-Flash	78.8	77.6	80.8	79.1	+2.6
GPT-5-mini	52.0	52.4	50.0	51.5	-2.2
Claude-Haiku-4.5	65.2	66.8	67.6	66.5	+1.6
DeepSeek-Chat	77.6	76.0	74.4	76.0	-2.4

Appendix

Table 9: Analysis of composition distance experiment results. Q 0 tests the distance from corner A (top-left) to center M vs corner B. Q 1 tests the distance from corner D (bottom-right) to center M vs corner C. Q 2 tests the distance from corner B to corner C vs center M, which is designed to benefit from composing Q 0 and Q 1 .

\Delta

indicates Q 2 performance relative to avg(Q 0 , Q 1 ).

Explore similar work

From Human Cognition to Neural Activations: Probing the Computational Primitives of Spatial Reasoning in LLMs
Mar 27, 2026Jiyuan An, Liner Yang, Mengyan Wang +3Spatial Reasoning LLM Interpretability
National Language Resources Monitoring and Research Center for Print Media, Beijing Language and Culture University, Beijing, China · School of Information Science, Beijing Language and Culture University, Beijing, China · School of artificial intelligence, Beijing normal university, Beijing, China
Do LLMs Build World Models From Text? A Multilingual Diagnostic of Spatial Reasoning
May 27, 2026Zhikai Pan, Chih-Ting Liao, Chunrui Liu +5LLM World Models Multilingual Language Model Evaluation
University of New South Wales · Essential Energy · University of Alabama at Birmingham +3
World2Mind: Cognition Toolkit for Allocentric Spatial Reasoning in Foundation Models
Mar 10, 2026Shouwei Ruan, Bin Wang, Zhenyu Wu +4Spatial Reasoning 3D Spatial Reasoning
Institute of Artificial Intelligence, Beihang University · Huawei Noah’s Ark Lab · Dept. of Comp. Sci. and Tech., Institute for AI, Tsinghua-Bosch Joint ML Center, THBI Lab, BNRist Center, Tsinghua University

Mar 27, 2026J/K move · Enter open · S save

From Human Cognition to Neural Activations: Probing the Computational Primitives of Spatial Reasoning in LLMs

Jiyuan An, Liner Yang, Mengyan Wang, Luming Lu+2

National Language Resources Monitoring and Research Center for Print Media, Beijing Language and Culture University, Beijing, China · School of Information Science, Beijing Language and Culture University, Beijing, China · School of artificial intelligence, Beijing normal university, Beijing, China

Spatial Reasoning LLM Interpretability

Open paper AskPDF arXiv

As spatial intelligence becomes an increasingly important capability for foundation models, it remains unclear whether large language models' (LLMs) performance on spatial reasoning benchmarks reflects structured internal spatial representations or reliance on linguistic heuristics. We address this question from a mechanistic perspective by examining how spatial information is internally represented and used. Drawing on computational theories of human spatial cognition, we decompose spatial reasoning into three primitives, relational composition, representational transformation, and stateful spatial updating, and design controlled task families for each. We evaluate multilingual LLMs in English, Chinese, and Arabic under single pass inference, and analyze internal representations using linear probing, sparse autoencoder based feature analysis, and causal interventions. We find that task relevant spatial information is encoded in intermediate layers and can causally influence behavior, but these representations are transient, fragmented across task families, and weakly integrated into final predictions. Cross linguistic analysis further reveals mechanistic degeneracy, where similar behavioral performance arises from distinct internal pathways. Overall, our results suggest that current LLMs exhibit limited and context dependent spatial representations rather than robust, general purpose spatial reasoning, highlighting the need for mechanistic evaluation beyond benchmark accuracy.

Abstract

Explore similar work

From Human Cognition to Neural Activations: Probing the Computational Primitives of Spatial Reasoning in LLMs

Do LLMs Build World Models From Text? A Multilingual Diagnostic of Spatial Reasoning

World2Mind: Cognition Toolkit for Allocentric Spatial Reasoning in Foundation Models