Exploration and goal-directed navigation in unknown layouts are central to inspection, logistics, and search-and-rescue. We ask whether large language models (LLMs) can function as text-only controllers under partial observability -- without code execution, tools, or program synthesis. We introduce a reproducible benchmark with oracle localisation in fixed ASCII gridworlds: each step reveals only a local 5×5 window around the agent and the model must select one of UP/RIGHT/DOWN/LEFT. Nine contemporary LLMs ranging from open/proprietary, dense / Mixture of Experts and instruction- vs. reasoning-tuned are evaluated on two tasks across three layouts of increasing difficulty: Exploration (maximising revealed cells) and Navigation (reach the goal on the shortest path). The experimental results are evaluated on quantitative metrics including success rate, efficiency such as normalised coverage and path length vs. oracle as well as qualitative analysis. Reasoning-tuned models reliably complete navigation across all layouts, yet remain less efficient than oracle paths. Few-shot demonstrations in the prompt chiefly help these Reasoning-tuned models by reducing invalid moves and shortening paths, while classic dense instruction models remain inconsistent. We observe characteristic action priors (UP/RIGHT) that can induce looping under partial observability. Overall, training regimen and test-time deliberation predict control ability better than raw parameter count. These findings suggest lightweight hybridisation with classical online planners as a practical route to deployable partial map systems.
Figures & tables
Figure 1: Decision loop of an LLM-driven gridworld agent (1) Environment: Observation + action evaluation (reward & success signals); Agent: prompt generation, stop condition monitoring (2) Agent: Structured prompt to LLM (system message, last action, observation) (3) LLM: Generated decision of movement direction (4) Agent: Prompt validation, action forwarding to environment
Figure 2: Full view of the used gridworlds with the oracle baseline for shortest navigation path with a length of (a) 26, (b) 25 and (c) 24. At the start of each attempt it is obscured with ? except for a 5×5 window centred on R and g
Exploration
Navigation
Model
0 - Shot
5 - Shot
0 - Shot
5 - Shot
Random Actions
53.71 %
8.00 % (282)
GPT-5 (Reasoning)
100.00 %
100.00 %
100.00 % (26)
100.00 % (26)
GPT-OSS-120B (Reasoning)
100.00 %
100.00 %
100.00 % (29)
100.00 % (28)
GPT-5-mini (Reasoning)
99.47 %
100.00 %
100.00 % (60)
100.00 % (28)
Gemma-3-27B-Instruct
39.97 %
61.23 %
100.00 % (106)
100.00 % (37)
Table 1: Easy: Per model exploration coverage for exploration and success rate (average steps) for navigation. The oracle path is 26 cells. n=13
Exploration
Navigation
Model
0 - Shot
5 - Shot
0 - Shot
5 - Shot
Random Actions
45.76 %
3.00 % (316)
GPT-5 (Reasoning)
100.00 %
100.00 %
100.00 % (48)
100.00 % (44)
GPT-OSS-120B (Reasoning)
99.50 %
99.94 %
100.00 % (52)
100.00 % (46)
GPT-5-mini (Reasoning)
99.00 %
100.00 %
0.00 % ( ∞ )
0.00 % ( ∞ )
Qwen3-QwQ-32B
22.70 %
68.22 %
0.00 % ( ∞ )
30.77 % (122)
Table 2: Medium: Per model exploration coverage for exploration and success rate (average steps) for navigation. The oracle path is 25 cells. n=13
Exploration
Navigation
Model
0 - Shot
5 - Shot
0 - Shot
5 - Shot
Random Actions
35.64 %
7.80 % (283)
GPT-5 (Reasoning)
89.22 %
100.00 %
100.00 % (32)
100.00 % (32)
GPT-5-mini (Reasoning)
62.91 %
83.21 %
100.00 % (56)
100.00 % (50)
GPT-OSS-120B (Reasoning)
58.56 %
45.21 %
100.00 % (43)
100.00 % (40)
DeepSeek-R1 (685B / 37B)
42.97 %
51.55 %
30.00 % (207)
53.85 % (123)
Table 3: Hard: Per model exploration coverage for exploration and success rate (average steps) for navigation. The oracle path is 24 cells. n=13
Large language model (LLM) based agents are often criticized for lacking spatial understanding and mainly exploiting statistical text patterns. We investigate their spatial comprehension through an architecture combining geometrical tools with a LLM serving as a high-level orchestrator in grid-world environments. The agent first collects geodesic trajectories, which are then vector-quantized to extract a representative subset. Offline, the LLM associates a natural language description of the underlying behavioral patterns to each selected trajectory, making it a tool. Online, the LLM chooses the appropriate tool conditioned on the current state and goal. Low-level control is handled by primitive actions that execute the trajectory associated with the tool. From an agentic AI perspective, this approach separates learning into two levels: tool discovery is handled through unsupervised quantization of trajectories, while reasoning and decision-making are handled by the LLM. We test the approach in a partially observable dynamic 2D grid environment with an open vision-language model (Qwen3.6-35B-A3B). Pairing the geometry-derived tool library with an agent-centered zoom tool and a collision detection tool lets a fast, non-reasoning configuration match the goal-reaching rate of a much more costly chain-of-thought version, while cutting the cost of a decision from minutes to seconds.
Gabriel Turinici
CEREMADE CNRS Université Paris Dauphine - PSL Paris 75116 FRANCE
Language Model (LM) agents are increasingly used in complex open-ended decision-making tasks, from AI coding to physical AI. A core requirement in these settings is the ability to both explore the problem space and exploit acquired knowledge effectively. However, systematically distinguishing and quantifying exploration and exploitation from observed actions without access to the agent's internal policy remains challenging. To address this, we design controllable environments inspired by practical embodied AI scenarios. Each environment consists of a partially observable 2D grid map and an unknown task Directed Acyclic Graph (DAG). The map generation can be programmatically adjusted to emphasize exploration or exploitation difficulty. To enable policy-agnostic evaluation, we design a metric to quantify exploration and exploitation errors from agent's actions. We evaluate a variety of frontier LM agents and find that even state-of-the-art models struggle on our task, with different models exhibiting distinct failure modes. We further observe that reasoning models solve the task more effectively and show both exploration and exploitation can be significantly improved through minimal harness engineering. We release our code here.
Jaden Park, Jungtaek Kim, Jongwon Jeong +3
University of Wisconsin–Madison · KRAFTON · Ludo Robotics
We introduce GSU, a text-only grid dataset to evaluate the spatial reasoning capabilities of LLMs over 3 core tasks: navigation, object localization, and structure composition. By forgoing visual inputs, isolating spatial reasoning from perception, we show that while most models grasp basic grid concepts, they struggle with frames of reference relative to an embodied agent and identifying 3D shapes from coordinate lists. We also find that exposure to a visual modality does not provide a generalizable understanding of 3D space that VLMs are able to utilize for these tasks. Finally, we show that while the very latest frontier models can solve the provided tasks (though harder variants may still stump them), fully fine-tuning a small LM or LORA fine-tuning a small LLM show potential to match frontier model performance, suggesting an avenue for specialized embodied agents.