Modern 3D game levels rely heavily on visual guidance, yet the navigability of level layouts remains difficult to quantify. Prior work either simulates play in simplified environments or analyzes static screenshots for visual affordances, but neither setting faithfully captures how players explore complex, real-world game levels. In this paper, we build on an existing open-source visual affordance detector and instantiate a screen-only exploration and navigation agent that operates purely from visual affordances. Our agent consumes live game frames, identifies salient interest points, and drives a simple finite-state controller over a minimal action space to explore Dark Souls-style linear levels and attempt to reach expected goal regions. Pilot experiments show that the agent can traverse most required segments and exhibits meaningful visual navigation behavior, but also highlight that limitations of the underlying visual model prevent truly comprehensive and reliable auto-navigation. We argue that this system provides a concrete, shared baseline and evaluation protocol for visual navigation in complex games, and we call for more attention to this necessary task. Our results suggest that purely vision-based sense-making models, with discrete single-modality inputs and without explicit reasoning, can effectively support navigation and environment understanding in idealized settings, but are unlikely to be a general solution on their own.
Figures & tables
Game
Route ID
Description
Runs
Milestones
DS1
DS1-UP
Undead Parish: from the dragon bridge up to the Gargoyle church
10
6
DS1
DS1-PW
Painted World of Ariamis: from the starting bridge to the staircase before the Undead Dragon
10
6
DS3
DS3-GA
Grand Archives: from the first bonfire to the second-floor landing
10
6
DS3
DS3-IR
Irithyll of the Boreal Valley: from the bonfire to the plaza before the second bonfire
10
6
ER
ER
Raya Lucaria Academy: from the Schoolhouse Classroom up to the rooftop exit
10
6
BMW
BMW
Chapter 3 precept corridor: from the lower shrine toward the upper courtyard
10
6
Table 1. Pilot study configuration. Game abbreviations: DS1 = Dark Souls I, DS3 = Dark Souls III, ER = Elden Ring, BMW = Black Myth: Wukong. Each route is decomposed into 6 milestones, yielding Kℓ navigation segments per run.
Route
Method
RS (%)
MS (%)
Seg. dur. (s)
Fwd time (s)
DS3-GA
Naive
20
83.3±29.3
84.9±56.8
22.4±11.6
FSM
50
88.3±16.8
70.6±66.6
17.1±13.1
Full
20
78.3±31.8
99.0±100.0
32.0±30.2
DS3-IR
Naive
20
73.3±27.5
117.9±59.9
87.6±44.6
FSM
30
71.7±24.1
137.4±85.7
50.5±32.3
Full
20
75.0±26.9
103.3±65.0
43.2±31.4
Table 2. Overall navigation performance per route and method. RS is the route success rate, the percentage of full runs that reach all six milestones; MS is the per-milestone segment success rate. Durations are reported in seconds.
Route
Method
M1
M2
M3
M4
M5
M6
DS3-GA
Naive
100%
100%
20%
80%
100%
100%
FSM
100%
100%
60%
70%
100%
100%
Full
100%
100%
50%
20%
100%
100%
DS3-IR
Naive
100%
80%
100%
60%
80%
20%
FSM
100%
70%
60%
70%
100%
30%
Full
100%
100%
60%
60%
100%
30%
Table 3. Per-milestone segment success rates for all routes. M1 - M6 denote successive milestone segments along each route. Game abbreviations as in Table 1 .
Figure 1. Top-down map of the Dark Souls III Grand Archives test route. The red polyline shows the designer-specified path from the bonfire (bottom) toward the second-floor landing (top). Red boxes mark the six visual milestones M1 - M6 used in our experiments; M2 and M3 correspond to the challenging mid-route transitions analyzed in Section 5.4 .
Figure 2. Irithyll M3 : the MSTP correctly highlights the upper landing, but the avatar gets stuck on local geometry.
Figure 3. Raya Lucaria Academy from M1 to M2 : the doorway is correctly detected as an MSTP, but the avatar gets stuck between a breakable table and an effigy.
Figure 4. Painted World from M2 to M3 : the stairs are correctly detected as an MSTP, but require a narrow, oblique approach angle that is easy to miss from the avatar’s starting pose.
Figure 5. Grand Archives milestone M2 : a near-symmetric scene where both the data and the route favor turning left.
Figure 6. Irithyll milestone M6 : a visually similar symmetry, but here the intended route is the darker right-hand doorway that the model often overlooks.
Figure 7. Black Myth: Wukong route. The red polyline shows the evaluation path and milestones M1 and M2 ; green arrows sketch the routes that players and our agents are visually drawn to first, toward left and upper side shrines, before noticing the narrow main-path gap near M2 .
Figure 8. Grand Archives milestone M3 : decorative shelves and lighting dominate the frame, drawing the MSTP away from the right-hand turn that actually continues the route.
Figure 9. Elden Ring Raya Lucaria Academy route (ER). The red polyline shows the evaluation path from the Classroom site of grace up through the interior loop and balconies to the rooftop exit; red boxes mark milestones M1 - M6 .
Figure 10. Dark Souls I Painted World of Ariamis (DS1-PW) route. The polyline and rectangles indicate the evaluation path and milestones M1 - M6 .
Figure 11. Dark Souls III Irithyll of the Boreal Valley (DS3-IR) route. The red polyline shows the evaluation path from the bonfire toward the cathedral approach; red boxes mark the six visual milestones M1 - M6 used in our experiments.
Figure 12. Dark Souls I Undead Parish (DS1-UP) route used in our experiments. The magenta polyline shows the evaluation path from the dragon bridge toward the Gargoyle church; magenta rectangles mark the six visual milestones M1 - M6 .
Fixed 3D Gaussian Splatting (3DGS) reconstructions provide realistic novel views but lack the traversability constraints, valid goals, and closed-loop protocols required for navigation evaluation. We introduce NavArena, an automated framework that transforms fixed 3DGS reconstructions into benchmarks for goal-oriented visual navigation. NavArena integrates a frozen 3DGS model for egocentric RGB-D rendering, an occupancy costmap derived from Gaussian density and height statistics for reachability and collision queries, and semantic goal candidates lifted from multi-view open-vocabulary masks. These components support the automatic generation and unified closed-loop evaluation of goal-oriented navigation episodes. Across more than 2{,}000 scenes, NavArena generates 22.2 million expert trajectories. Spatial and semantic evaluations assess the derived navigation representations, while policy rollouts demonstrate the diagnostic value of the unified evaluation protocol. NavArena enables scalable and reproducible navigation evaluation on large-scale 3DGS reconstructions, and all benchmark-generation tools, evaluation protocols, and derived assets will be released publicly.
Visual Language Navigation foundation models aim to unify deep reasoning for grounded spatial decisions with broad versatility for diverse embodied tasks. Current approaches typically achieve this integration via monolithic policies that map observations directly to actions, yet they often suffer from coordinate drift and poor handling of long-tail semantics. Furthermore, these black-box mappings lack interpretability, hindering the simultaneous achievement of generality, robustness, and transparency. We present ABot-N1, a step toward a general Visual Language Navigation foundation model, that addresses these challenges by decoupling cognition from control via a slow-fast architecture guided by dual visual-language signals. More specifically, a slow vision-language reasoner performs explicit Chain-of-Thought reasoning while producing a pixel goal. This compact set of image-space anchor points serves as a universal interface for diverse tasks, including point-goal, object-goal, poi-goal, instruction-following, and person-following. Subsequently, a fast action expert leverages both the textual cues and the pixel guidance to generate continuous waypoints at the native control frequency. By bridging high-level intents and low-level control through pixel-grounded anchors paired with explicit linguistic traces, our approach ensures robust, generalizable, and interpretable navigation across simulation and real-world benchmarks. ABot-N1 establishes new state-of-the-art records, delivering massive gains specifically in urban-scale navigation: boosting POI arrival by 35.0% (to 77.3%) and achieving 95.4%/92.9% SR in complex indoor and outdoor scenes. It also maintains superior robustness across object-reaching, person-following, and instruction-following tasks. New Point-Goal/POI-Goal benchmarks are released as open source to advance the field of urban-scale navigation.
Open-world navigation requires robots to make decisions in complex everyday environments while adapting to flexible task requirements. Conventional navigation approaches often rely on dense 3D reconstruction and hand-crafted goal metrics, which limits their generalization across tasks and environments. Recent advances in vision-language navigation (VLN) and vision-language-action (VLA) models enable end-to-end policies conditioned on natural language, but typically require interactive training, large-scale data collection, or task-specific fine-tuning with a mobile agent. We formulate navigation as a sparse subgoal identification and reaching problem and observe that providing visual anchoring targets for high-level semantic priors enables highly efficient goal-conditioned navigation. Based on this insight, we select visual frontiers as semantic anchors and propose OpenFrontier, a navigation framework that requires no task-specific training or fine-tuning and seamlessly integrates diverse vision-language prior models. OpenFrontier enables efficient navigation with a lightweight system design, without dense 3D semantic mapping, task-specific policy training, or model fine-tuning. We evaluate OpenFrontier across multiple navigation benchmarks and demonstrate strong zero-shot performance, as well as effective real-world deployment on a mobile robot.
Esteban Padilla-Cerdio, Boyang Sun, Marc Pollefeys +1
1ETH Zurich · 2Microsoft Spatial AI Lab · University of Bonn