R2F: Repurposing Ray Frontiers for LLM-free Object Navigation
Authors: Francesco Argenziano, John Mark Alexis Marcelo, Michele Brienza, Abdel Hakim Drid, Emanuele Musumeci, Daniele Nardi, Domenico D. Bloisi, Vincenzo Suriani
Organizations: Department of Computer, Automation and Management Engineering, Sapienza University of Rome, Rome, Italy · Department of Electronics and Automation, Mohamed Khider University of Biskra, Biskra, Algeria · International University of Rome UNINT, Rome, Italy
Zero-shot open-vocabulary object navigation has progressed rapidly with the emergence of large Vision-Language Models (VLMs) and Large Language Models (LLMs), now widely used as high-level decision-makers instead of end-to-end policies. Although effective, such systems often rely on iterative large-model queries at inference time, introducing latency and computational overhead that limit real-time deployment. To address this problem, we repurpose ray frontiers (R2F), a recently proposed frontier-based exploration paradigm, to develop an LLM-free framework for indoor open-vocabulary object navigation. While ray frontiers were originally used to bias exploration using semantic cues carried along rays, we reinterpret frontier regions as explicit, direction-conditioned semantic hypotheses that serve as navigation goals. Language-aligned features accumulated along out-of-range rays are stored sparsely at frontiers, where each region maintains multiple directional embeddings encoding plausible unseen content. In this way, navigation then reduces to embedding-based frontier scoring and goal tracking within a classical mapping and planning pipeline, eliminating iterative large-model reasoning. We further introduce R2F-VLN, a lightweight extension for free-form language instructions using syntactic parsing and relational verification without additional VLM or LLM components. Experiments in Habitat-sim and on a real robotic platform demonstrate competitive state-of-the-art zero-shot performance with real-time execution, achieving up to 6 times faster runtime than VLM-based alternatives.
Figures & tables
Fig. 2: R2F system overview and execution schedule. The agent receives RGB-D observations and a text query. RGB images are processed by RADIO with a Neighborhood-Aware attention modification (NA) to produce dense open-vocabulary features (NA-RADIO), while the query is encoded with SigLIP. From out-of-range depth pixels, semantic rays are sampled, binned by direction, and associated with frontier regions, forming Semantic Ray Frontiers that accumulate directional semantic evidence. In parallel, depth updates a volumetric occupancy map, from which frontier regions are periodically recomputed ( low frequency ), while semantic ray accumulation runs continuously ( high frequency ). Frontier regions are scored via cosine similarity with the query embedding, and the navigation policy ( R2F or R2F-VLN ) selects and tracks the highest-scoring frontier until the goal is detected or exploration continues.
Fig. 3: NA-RADIO’s feature maps in comparison with different text query. Both the text query embedding and the visual features lie in the SigLIP [ 37 ] embedding space thanks to RADIO’s adapter. In practice, we observe that informative similarity values typically lie in the range [−0.10,0.15] .
Task
Method
SR (%)
SPL (%)
t (s)
ObjectNav
VLN-Game [ 2 ]
76.7
28.0
122.0
3D-MEM [ 31 ]
26.7
15.9
61.2
VLFM [ 5 ]
40.0
7.71
84.2
OpenFrontier [ 8 ]
23.3
7.12
245.0
R2F (ours)
78.3
29.6
32.7
VLN
VLN-Game [ 2 ]
43.7
22.7
504.0
TABLE I: Comparison across ObjectNav and VLN tasks with the three baselines. We report SR , SPL , and Execution Time ( t ). Bold is best and underlined is second-best.
Fig. 4: Pipeline execution on a real robot for the “Find a sink” goal. The robot navigates between the boundaries based on their semantic value until it reaches the goal.
Open-world navigation requires robots to make decisions in complex everyday environments while adapting to flexible task requirements. Conventional navigation approaches often rely on dense 3D reconstruction and hand-crafted goal metrics, which limits their generalization across tasks and environments. Recent advances in vision-language navigation (VLN) and vision-language-action (VLA) models enable end-to-end policies conditioned on natural language, but typically require interactive training, large-scale data collection, or task-specific fine-tuning with a mobile agent. We formulate navigation as a sparse subgoal identification and reaching problem and observe that providing visual anchoring targets for high-level semantic priors enables highly efficient goal-conditioned navigation. Based on this insight, we select visual frontiers as semantic anchors and propose OpenFrontier, a navigation framework that requires no task-specific training or fine-tuning and seamlessly integrates diverse vision-language prior models. OpenFrontier enables efficient navigation with a lightweight system design, without dense 3D semantic mapping, task-specific policy training, or model fine-tuning. We evaluate OpenFrontier across multiple navigation benchmarks and demonstrate strong zero-shot performance, as well as effective real-world deployment on a mobile robot.
Esteban Padilla-Cerdio, Boyang Sun, Marc Pollefeys +1
1ETH Zurich · 2Microsoft Spatial AI Lab · University of Bonn
Zero-shot object-goal navigation aims to enable an intelligent agent to explore and navigate to objects of unknown categories in an unfamiliar environment without specific target training. In zero-shot navigation tasks, pre-trained large models are usually employed to leverage their prior knowledge for guiding the agent's navigation. However, existing zero-shot object-goal navigation methods based on large language models (LLMs) merely utilize LLMs as flat reasoning tools to directly associate objects or regions. They lack the hierarchical spatial cognition modeling of human-like room semantics to object localization, which leads to strong blindness in exploration, insufficient accuracy in semantic association, and failure to fully unleash the common-sense reasoning potential of LLMs. This paper proposes an LLM-driven hierarchical room-to-object (HRO) framework for zero-shot object-goal navigation, which guides the agent to explore and navigate to the target object in a coarse-to-fine manner. Experiments on Gibson and HM3D datasets verify that our HRO framework achieves superior success rate and generalization over existing LLM-based methods, underscoring LLMs' strong potential for zero-shot object-goal navigation.
Luyuan Jia, Yinfeng Yu
School of Computer Science and Technology, Xinjiang University, Urumqi 830017, China. · Joint Research Laboratory for Embodied Intelligence, Xinjiang University. · Joint International Research Laboratory of Silk Road Multilingual Cognitive Computing, Xinjiang University.
Recent advances in large vision-language models (VLMs) and large language models (LLMs) have enabled zero-shot approaches to visual language navigation (VLN), where an agent follows natural language instructions using only ego perception and reasoning. However, existing zero-shot methods typically construct a naive observation graph and perform per-step VLM-LLM inference on it, resulting in high latency and computation costs that limit real-time deployment. To address this, we present SFCo-Nav, an efficient zero-shot VLN framework inspired by the principle of slow-fast cognitive collaboration. SFCo-Nav integrates three key modules: 1) a slow LLM-based planner that produces a strategic chain of subgoals, each linked to an imagined object graph; 2) a fast reactive navigator for real-time object graph construction and subgoal execution; and 3) a lightweight asynchronous slow-fast bridge aligns advanced structured, attributed imagined and perceived graphs to estimate navigation confidence, triggering the slow LLM planner only when necessary. To the best of our knowledge, SFCo-Nav is the first slow-fast collaboration zero-shot VLN system supporting asynchronous LLM triggering according to the internal confidence. Evaluated on the public R2R and REVERIE benchmarks, SFCo-Nav matches or exceeds prior state-of-the-art zero-shot VLN success rates while cutting total token consumption per trajectory by over 50% and running more than 3.5 times faster. Finally, we demonstrate SFCo-Nav on a legged robot in a hotel suite, showcasing its efficiency and practicality in indoor environments.
Chaoran Xiong, Litao Wei, Xinhao Hu +5
The authors are with the Shanghai Key Laboratory of Navigation and Location Based Services, Shanghai Jiao Tong University (SJTU), Shanghai 200240, China. · Litao Wei is also with Zhiyuan College, SJTU. · Ling Pei is also with the State Key Laboratory of Submarine Geoscience, SJTU.