From Pixels to BFS: High Maze Accuracy Does Not Imply Visual Planning

Alberto G. Rodriguez Salgado

cs.LGMar 27, 2026

From Pixels to BFS: High Maze Accuracy Does Not Imply Visual Planning

Authors: Alberto G. Rodriguez Salgado

Organizations: Independent Researcher

Abstract

How do multimodal models solve visual spatial tasks -- through genuine planning, or through brute-force search in token space? We introduce MazeBench, a benchmark of 110 procedurally generated maze images across nine controlled groups, and evaluate 16 model configurations from OpenAI, Anthropic, Google, and Alibaba. GPT-5.4 solves 91% and Gemini 3.1 Pro 79%, but these scores are misleading: models typically translate images into text grids and then enumerate paths step by step, consuming 1,710--22,818 tokens per solve for a task humans do quickly. Without added reasoning budgets, all configurations score only 2--12%; on 20 $\times$ 20 ultra-hard mazes, they hit token limits and fail. Qualitative traces reveal a common two-stage strategy: image-to-grid translation followed by token-level search, effectively BFS in prose. A text-grid ablation shows Claude Sonnet 4.6 rising from 6% on images to 80% when given the correct grid, isolating weak visual extraction from downstream search. When explicitly instructed not to construct a grid or perform graph search, models still revert to the same enumeration strategy. MazeBench therefore shows that high accuracy on visual planning tasks does not imply human-like spatial understanding.

Figures & tables

Model	Solved	Reach.%	Lat.(s)
GPT-5.5 (medium)	96/100	99	28.6
GPT-5.5 (low)	92/100	99	13.5
GPT-5.4 (medium)	91/100	95	38.1
GPT-5.4 (low)	85/100	92	23.8
Gemini 3.1 Pro	79/100	86	51.9
Gemini 3 Flash	53/100	82	32.3

Table 1: Main leaderboard on the 100-maze core set. Solved requires correct reachability, correct shortest path length, and exact path match. Reach.% is reachability accuracy. Lat. is average latency per maze.

		Solves per Group
Model	Reason.	A	B	C	D	E	F	G	H	Total
		/8	/15	/15	/12	/14	/10	/16	/10	/100
GPT-5.5	med.	8	13	13	12	14	10	16	10	96
GPT-5.5	low	8	15	11	10	14	10	14	10	92
GPT-5.5	none	8	2	0	0	2	0	3	0	15
GPT-5.4	med.	7	14	13	12	13	9	13	10	91

Table 2: Per-group solve counts for all 19 model configurations. Groups: A =Diagnostic, B =Grid Scale, C =Wall Density, D =Trap Ablation, E =Unreachable, F =Border Walls, G =Combined Hard, H =Palette Stress. GPT-5.5 medium is the first configuration to clear Group G (Combined Hard) at full 16/16. GPT-5.4’s none

\to

low transition (+73 solves) is the largest single improvement; GPT-5.5 shows a similar +77 jump (15

\to

92). All Claude models remain at 2–6 regardless of reasoning effort. Gemini models achieve strong results via hidden internal thinking.

Model	Solved	Tot. Tok.	Tok/Solve
GPT-5.5 (medium)	96	209K	2,172
GPT-5.5 (low)	92	101K	1,096
GPT-5.4 (medium)	91	265K	2,913
GPT-5.4 (low)	85	145K	1,710
Gemini 3.1 Pro	79	731K	9,250
Gemini 3 Flash	53	804K	15,171

Table 3: Thinking token efficiency. Tot. Tok. is total thinking + output tokens across all 100 mazes. Tok/Solve is the average total tokens consumed per correctly solved maze. Lower is more efficient.

Maze	GT	Pred.	Path	Solved	Lat.
101	✓	–	40 $\to$ –	$\triangle$	137s
102	✓	✓	30 $\to$ 30		75s
103	✓	✓	42 $\to$ 42		103s
104	$\times$	$\times$	–	✓	101s
105	✓	–	32 $\to$ –	$\triangle$	87s
106	✓	✓	37 $\to$ 37	✓	88s

Table 4: GPT-5.5 (medium) on the 10 ultra-hard 20

\times

20 mazes. GT = ground-truth reachability; Pred. = model prediction; Path = GT length

\to

predicted length. ✓= correct,

\times

= wrong,

\triangle

= correct reachability but incomplete output (hit token limit). The model solves one reachable maze (37 steps) plus all three unreachable cases; it falsely declares the 41-step reachable maze unreachable and runs out of tokens on the 40- and 32-step mazes.

Model	Input	Solved	Tok/Solve	Lat.
Sonnet 4.6	Image (none)	6/100	14,313	16.9s
Sonnet 4.6	Text grid (low)	80/100	3,222	34.8s
Opus 4.7	Image (low)	31/100	13,189	29.1s
Opus 4.7	Text grid (low)	90/100	2,285	18.7s

Table 5: Image vs. text grid on 100 mazes for two Claude generations. Bypassing vision yields

13\times

more solves for Sonnet 4.6 and

2.9\times

more for Opus 4.7. Opus 4.7 on text grids reaches 90%—matching GPT-5.4 medium on images (91%) at lower tokens per solve (2,285 vs. 2,913).

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 2: Group A: Diagnostic (8 mazes). Empty or near-empty grids with trivial straight-line paths.

Figure 3: Group B: Grid Scale (15 mazes). Constant 25% wall density, grid sizes from $5\times 5$ to $13\times 13$ .

Figure 4: Group C: Wall Density (15 mazes). Constant $9\times 9$ grid, density from 0% to 45%.

Figure 5: Group D: Trap Ablation (12 mazes). Six matched pairs—control (no traps) and treatment (with traps)—sharing the same random seed.

Figure 6: Group E: Unreachable (14 mazes). All mazes have no valid path from start to goal.

Figure 7: Group F: Border Walls (10 mazes). Five matched pairs with and without a wall border ring.

Figure 8: Group G: Combined Hard (16 mazes). Large grids with high wall density, traps, and borders.

Figure 9: Group H: Palette Stress (10 mazes). Same maze structure rendered across four visual palettes.

Figure 10: Group X: Ultra-Hard (10 mazes). $20\times 20$ grids with 8–25 traps and shortest paths of 28–42 steps.

VLM Reasoning

Vision-language model reasoning studies how multimodal systems use visual evidence and language to answer questions, plan, and explain.

See all 853 papers

Rule-Compliant Visual Spatial Planning for Multimodal Large Language Models
Aug 20, 2026Yu Chen, Ting Lei, Yaoyi Li +3Multimodal Large Language Models Spatial Reasoning Benchmarks
Wangxuan Institute of Computer Technology, Peking University · Yinwang Intelligent Technology Co., Ltd
Mind the Gap Between Spatial Reasoning and Acting! Step-by-Step Evaluation of Agents With Spatial-Gym
Apr 10, 2026Lars Benedikt Kaesberg, Tianyu Yang, Niklas Bauer +3Spatial Reasoning Benchmarks Spatial Reasoning
University of Göttingen, Germany
Video Models Reason Early: Exploiting Plan Commitment for Maze Solving
Mar 31, 2026Kaleb Newman, Tyler Zhu, Olga RussakovskyVideo Diffusion Models Video Reasoning
Princeton University

Aug 20, 2026J/K move · Enter open · S save

Rule-Compliant Visual Spatial Planning for Multimodal Large Language Models

Yu Chen, Ting Lei, Yaoyi Li, Jia Cai+2

Wangxuan Institute of Computer Technology, Peking University · Yinwang Intelligent Technology Co., Ltd

Multimodal Large Language Models Spatial Reasoning Benchmarks Visual Spatial Reasoning

Open paper AskPDF arXiv

Multimodal large language models (MLLMs) combine linguistic reasoning with visual perception, yet their ability to perform visual spatial planning under explicit or previously unseen rule constraints remains underexplored. This setting requires models to jointly understand spatial layouts, interpret natural-language rules, and plan valid actions accordingly. To address this gap, we introduce RuleMaze, a controllable benchmark in which MLLMs must navigate mazes while obeying natural-language rules of varying complexity. RuleMaze isolates rule-compliant spatial planning by requiring accurate perception, rule interpretation, and constrained action planning. To enable scalable and systematic rule construction, we propose Language-Logic-Function Hybridization, which automatically generates natural-language rules and translates them into logical representations and executable validators, eliminating manual rule engineering. To improve rule following and generalization, we introduce Disentangled Multimodal Planning (DMP), which separates perception, execution, and rule verification through interpretable reasoning primitives. By disentangling these components, DMP facilitates systematic generalization to more complex and previously unseen rules, while providing transparent intermediate planning traces. Experiments demonstrate that DMP substantially improves rule compliance and planning success compared to end-to-end textual planning baselines. Overall, RuleMaze establishes a principled benchmark for studying grounded and interpretable rule-based spatial planning in MLLMs. Code is available at https://github.com/oceanflowlab/RuleMaze.

Abstract

VLM Reasoning

Explore similar work

Rule-Compliant Visual Spatial Planning for Multimodal Large Language Models

Mind the Gap Between Spatial Reasoning and Acting! Step-by-Step Evaluation of Agents With Spatial-Gym

Video Models Reason Early: Exploiting Plan Commitment for Maze Solving