Indoor navigation to task-relevant targets remains difficult when the target is small, occluded, or visually indistinct. Meanwhile, smart appliances, mobile devices, and IoT tags emit Wi-Fi signals during everyday communication, providing a pervasive but underused spatial cue. We study visual-Wi-Fi navigation, where a robot reaches a signal-emitting device in an unmapped environment using onboard visual observations and Wi-Fi packets overheard from the device, without connecting to the device or modifying infrastructure. Realizing this requires pose-dependent Wi-Fi observations during closed-loop policy learning and reliable spatial cues from bearing measurements corrupted by indoor multipath propagation on a moving robot. We present WiFiSpaces, which extends the Habitat embodied simulator with online ray-traced Wi-Fi sensing, rendering Wi-Fi observations for a robot-mounted antenna array at continuous poses during policy rollouts. On top of it, we build ViFiNav, a localization-aware navigation framework: an online source-localization algorithm exploits robot motion to disambiguate and robustly aggregate bearings from multiple viewpoints into a confidence-aware source estimate, and a learned policy fuses this evolving estimate with depth observations and learns to act on it as it is refined. Experiments show that exploiting robot motion is key to accurate source estimates, and that fine-tuning on ray-traced Wi-Fi observations in WiFiSpaces improves the success rate of ViFiNav. The policy transfers zero-shot to unseen scene datasets, and we deploy it on a real robot with a Wi-Fi sensor built from a commodity router, without any real-world training.
Figures & tables
Fig. 1 : Visual–Wi-Fi navigation in everyday indoor environments. (a) Using visual and Wi-Fi observations, an agent navigates toward signal-emitting devices in an unmapped indoor environment. Examples: 1. finding objects equipped with signal-emitting IoT tags (e.g., a package with an AirTag-style Wi-Fi tag), 2. finding the user with a carried mobile device, and 3. finding non-connected objects via associated signal-emitting devices (e.g., food in a smart fridge). (b) Our robot: a commodity Wi-Fi router mounted on a Hello Robot Stretch 3 serves as the Wi-Fi sensor.
Platform
Nav. tasks
Visual obs.
Wireless obs.
Learning interface
Embodied simulators [ 32 , 34 , 22 , 30 ]
✓
✓
–
✓
Wireless simulators [ 1 , 31 , 9 ]
–
–
✓
–
WiFiSpaces (ours)
✓
✓
✓
✓
TABLE I : Simulation platforms for visual–Wi-Fi navigation.
Fig. 2 : System overview. From CSI Ht and the robot pose, the online Wi-Fi source localization module of ViFiNav (disambiguation and multi-view fusion, Fig. 4 ) produces a confidence-tagged source estimate, which the policy fuses with depth and previous-action features to output the action at . They form a closed loop: the policy’s motion selects the localizer’s viewpoints, and the policy is trained on the evolving estimate rather than on a fixed goal. The policy is trained with reinforcement learning (RL) in WiFiSpaces , using a reward available only in simulation, and deployed zero-shot on a real robot.
Fig. 3 : WiFiSpaces renders the Wi-Fi observation on the fly. Given the scene mesh, the transmitting device and the robot pose, propagation paths, including reflections off scene surfaces, are ray-traced per antenna element; the per-path delays and gains form the channel taps, from which the Wi-Fi CSI Ht at the current pose is derived.
Fig. 4 : Wi-Fi source localization. (a) A ULA measurement yields a true and a mirror bearing; under in-place rotation only the true one stays fixed in the world frame, so it is kept. (b) Verified bearings from different viewpoints are fused by RANSAC, keeping the hypothesis with most inliers.
Fig. 5 : The router re-housed in a 3D-printed case as a four-antenna ULA.
Method
Goal Cue
SR ↑
SPL ↑
DTG ↓
Loc. ↓
No-goal ViFiNav
–
0.16
0.10
10.27
–
PointNav (oracle)
GT PointGoal
87.22
69.88
1.66
–
ViFiNav (oracle)
GT AoA
82.30
60.83
2.10
<10−3
PointNav
Est. Goal
49.76
34.57
4.99
3.23
Loc-then-PointNav
Est. Goal
17.78
8.91
6.50
2.85
ViFiNav (w/o FT)
Wi-Fi Obs.
53.02
35.82
4.58
2.56
TABLE II : Main results on HSSD test scenes. GT: ground truth; oracle: GT goal cue; Wi-Fi Obs.: CSI simulated by WiFiSpaces ; Est. Goal: estimate of our online localization module; w/o FT: no fine-tuning on Wi-Fi observations. SR, SPL in percent; DTG, Loc. in meters.
Fig. 6 : Qualitative comparison on three HSSD test episodes (columns). Rows: ViFiNav (top), PointNav (middle), Loc-then-PointNav (bottom). Stars mark successive source estimates, coloured by time. ViFiNav ’s estimates migrate toward the source as it moves, keeping its path short. PointNav follows each updated estimate and detours. Loc-then-PointNav freezes one early estimate and cannot recover when it is wrong.
Loc. Alg.
Amb. Resolver
Pretrain
SR ↑
SPL ↑
DTG ↓
Loc. Err. ↓
LS
–
–
1.75
1.30
7.84
48.74
LS
✓
–
43.33
28.14
4.91
4.34
RANSAC
–
–
47.62
27.95
4.70
2.03
RANSAC
✓
–
52.22
30.17
4.20
1.26
RANSAC
✓
✓
68.57
44.12
3.04
1.24
TABLE III : Effect of the fusion estimator, ambiguity resolution, and two-stage training. LS: least squares.
Fig. 7 : Localization module analysis on 10,000 HSSD training-split episodes, replayed along their shortest paths. (a) Cumulative distribution function (CDF) of direction error before (raw AoA) and after (resolved bearing) motion-based ambiguity resolution. (b) Mean and median source localization error per confidence bin.
Dataset
Scenes
Episodes
Geo. dist. (m)
SR ↑
SPL ↑
DTG ↓
Loc. ↓
HSSD (in-domain)
42
1,260
11.30
68.57
44.12
3.04
1.24
ReplicaCAD
21
1,050
7.44
75.05
47.14
1.29
0.72
ProcTHOR
1,000
5,000
7.19
85.22
50.83
1.29
0.20
TABLE IV : Zero-shot cross-dataset evaluation of the HSSD-trained ViFiNav , with test-set statistics (scenes, episodes, mean geodesic start–goal distance).
Category-level
Instance-level
Method
SR ↑
SPL ↑
DTG ↓
SR ↑
SPL ↑
DTG ↓
ObjectNav (official)
60.95
28.38
2.68
25.67
13.20
7.55
ObjectNav (matched)
61.45
27.26
2.84
25.07
12.86
7.69
ViFiNav (zero-shot)
–
–
–
56.29
30.40
4.59
TABLE V : ObjectNav policies vs. ViFiNav on HSSD ObjectNav episodes. Category-level metrics count any instance of the goal category, instance-level metrics only the designated instance, at which the Wi-Fi source is placed for ViFiNav ; success distance 0.5m to the instance viewpoints at both levels. “Official” follows the HSSD recipe [ 19 ] ; “matched” uses the embodiment of ViFiNav (Sec. VI ).
Fig. 8 : Representative real-world trials: (a) line-of-sight, (b) non-line-of-sight, and (c) an application scenario in which the Wi-Fi source (a phone) is inside a package. Green box: Wi-Fi source; red: robot trajectory.
Smart factory inspection requires robots to reach connected equipment without a prior map or known target coordinates. Radio frequency (RF) signals from the target can provide directional cues to complement visual observations when occlusion or poor lighting limits target detection. However, multipath propagation can distort these cues, making it difficult to infer the target's true direction from instantaneous RF measurements. To enable navigation research under these conditions, we first construct a Habitat Sionna RT benchmark that uses detailed scene geometry and assigned material properties to generate aligned visual and RF observations in response to robot actions. Building on this benchmark, we propose an uncertainty aware multimodal navigation framework that jointly estimates target direction and its uncertainty from a history of RF, visual, and pose observations. These estimates inform action selection alongside visual context. Experiments in unseen scenes show relative improvements of 18.2% in success rate (SR) and 11.5% in success weighted by path length (SPL) over the strongest evaluated baseline.
Wenlihan Lu, Tianshun Li, Liuqing Yang +1
Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China
Open-world navigation requires robots to make decisions in complex everyday environments while adapting to flexible task requirements. Conventional navigation approaches often rely on dense 3D reconstruction and hand-crafted goal metrics, which limits their generalization across tasks and environments. Recent advances in vision-language navigation (VLN) and vision-language-action (VLA) models enable end-to-end policies conditioned on natural language, but typically require interactive training, large-scale data collection, or task-specific fine-tuning with a mobile agent. We formulate navigation as a sparse subgoal identification and reaching problem and observe that providing visual anchoring targets for high-level semantic priors enables highly efficient goal-conditioned navigation. Based on this insight, we select visual frontiers as semantic anchors and propose OpenFrontier, a navigation framework that requires no task-specific training or fine-tuning and seamlessly integrates diverse vision-language prior models. OpenFrontier enables efficient navigation with a lightweight system design, without dense 3D semantic mapping, task-specific policy training, or model fine-tuning. We evaluate OpenFrontier across multiple navigation benchmarks and demonstrate strong zero-shot performance, as well as effective real-world deployment on a mobile robot.
Esteban Padilla-Cerdio, Boyang Sun, Marc Pollefeys +1
1ETH Zurich · 2Microsoft Spatial AI Lab · University of Bonn
Object navigation requires a robot to search for an unobserved target in an unknown environment by deciding where to explore next under partial observability. Effective search resembles human-like exploration: selectively probing visually promising frontiers while relying on spatial memory to avoid redundant revisits. We propose IntentNav, a spatial-visual imitation framework that learns human-like ObjectNav policies from human demonstrations. To infer high-level search intent from low-level human actions, we introduce Frontier-based Human-Intent Labeling, which looks ahead in human demonstrations and labels the frontier that best explains the demonstrator's future search direction. We construct a spatial-visual candidate space, where BEV memory tracks explored regions, unexplored frontiers, and trajectory history, while egocentric visual memory provides semantic cues for each candidate. A VLM policy is trained to select among these grounded candidates, using Intent-Aligned Objective to encourage consistent and human-like exploration. IntentNav achieves state-of-the-art performance on the MP3D, HM3D-v1 and HM3D-v2 ObjectNav benchmarks. The proposed candidate-level navigation interface transfers zero-shot to wheeled, quadruped, and humanoid robots without further VLM fine-tuning. Project page.
Yuxin Cai, Zongtai Li, Maonan Wang +9
Nanyang Technological University · Carnegie Mellon University · The Chinese University of Hong Kong +1