cs.ROAug 27, 2026

RTNav: Towards Real-Time Zero-Shot Object Navigation

Authors: Easop Lee, Lingyu Zhang, Boyuan Chen

Organizations: All authors are with Duke University.

Abstract

Navigation in unknown environments to find unforeseen objects has become increasingly feasible with capable vision and language foundation models. However, these models also introduce non-negligible inference latency, which becomes an important concern when agents must operate continuously in the real world. Most state-of-the-art methods are still developed in synchronous simulators, where the environment waits for the agent to act and inference time is effectively free. As a result, agents are often designed around the sequential execution of perception, reasoning, and action, with little regard for time constraints. Under real-time execution, where wall-clock time counts towards the task budget, the inefficiencies of these architectures become clear. We show that recent zero-shot object navigation methods suffer consistent performance degradation under such realistic timing conditions. Motivated by this observation, we propose RTNav, a simple but effective architecture that treats inference latency, asynchronous environment stepping, and bounded compute as explicit design considerations. Evaluated on real-time variants of HM3D-v1, HM3D-v2, and HM3D-OVON, RTNav improves the success rate by up to 11% and the Success weighted by Completion Time by up to 5.1 points over prior work.

Figures & tables

Explore similar work

Mar 25, 2026

ReMemNav: Memory-Based Decision Correction and Target Verification for Zero-Shot Object Navigation

Zero-shot object navigation requires agents to locate unseen targets in unfamiliar environments without prior maps or task-specific training. Despite the commonsense reasoning ability of vision-language models (VLMs), existing mapless navigators often suffer from repeated exploration and premature stopping caused by limited historical context and false-positive target predictions. We propose ReMemNav, a training-free framework that combines lightweight semantic grounding, memory-based decision correction, and target verification. RAM-derived semantic priors guide panoramic direction selection, while a geometry-triggered correction mechanism retrieves historical scene descriptions when the selected direction points toward previously visited regions. When a target is predicted to be visible, ReMemNav verifies the corresponding single-view observation before the final approach. Depth-based action sampling then converts the high-level decision into collision-free motion. Experiments on HM3D and MP3D show that ReMemNav achieves higher success rates and path efficiency than existing training-free zero-shot baselines. Specifically, ReMemNav achieves absolute SR/SPL gains of 1.7%/5.3% on HM3D v0.1, 18.2%/11.1% on HM3D v0.2, and 8.7%/6.9% on MP3D.
Mar 19, 2026

SR-Nav: Spatial Relationships Matter for Zero-shot Object Goal Navigation

Zero-shot object-goal navigation aims to find target objects in unseen environments using only egocentric observation. Recent methods leverage foundation models' comprehension and reasoning capabilities to enhance navigation performance. However, when faced with poor viewpoints or weak semantic cues, foundation models often fail to support reliable reasoning in both perception and planning, resulting in inefficient or failed navigation. We observe that inherent relationships among objects and regions encode structured scene priors, which help agents infer plausible target locations even under partial observations. Motivated by this insight, we propose Spatial Relation-aware Navigation (SR-Nav), a framework that models both observed and experience-based spatial relationships to enhance both perception and planning. Specifically, SR-Nav first constructs a Dynamic Spatial Relationship Graph (DSRG) that encodes the target-centered spatial relationships through the foundation models and updates dynamically with real-time observations. We then introduce a Relation-aware Matching Module. It utilizes relationship matching instead of naive detection, leveraging diverse relationships in the DSRG to verify and correct errors, enhancing visual perception robustness. Finally, we design a Dynamic Relationship Planning Module to reduce the planning search space by dynamically computing the optimal paths based on the DSRG from the current position, thereby guiding planning and reducing exploration redundancy. Experiments on HM3D show that our method achieves state-of-the-art performance in both success rate and navigation efficiency. The code will be publicly available at https://github.com/Mzyw-1314/SR-Nav
Apr 21, 2026

LiveVLN: Breaking the Stop-and-Go Loop in Vision-Language Navigation

Recent navigation systems achieve strong benchmark results, yet real-world deployment often remains visibly stop-and-go. This bottleneck arises because the sense-inference-execution loop is still blocking: after each new observation, the controller must wait for sensing, transmission, and inference before motion can continue. Reducing action-generation cost alone therefore does not remove redundant waiting. To address this issue, we present LiveVLN, a training-free framework for more continuous embodied navigation by augmenting pretrained VLM navigators with multi-step action continuation. Instead of pausing for each full sense-and-inference round, LiveVLN overlaps execution with the processing of newly arrived observations, allowing refreshed future actions to be handed off before the current executable prefix is exhausted. This design keeps actions continuously available during motion, reducing idle waiting and enabling smoother online execution. The framework operates at runtime and can be integrated with compatible pretrained VLM navigators. Across R2R and RxR, LiveVLN preserves benchmark performance while reducing waiting time and improving action availability. In real-world deployments, it cuts average episode waiting time by up to 77.7%77.7\% and shortens wall-clock episode time by 12.6%12.6\% on StreamVLN and 19.6%19.6\% on NaVIDA, yielding more coherent execution during deployment. Code is available at https://github.com/NIneeeeeem/LiveVLN.