Indoor navigation to task-relevant targets remains difficult when the target is small, occluded, or visually indistinct. Meanwhile, smart appliances, mobile devices, and IoT tags emit Wi-Fi signals during everyday communication, providing a pervasive but underused spatial cue. We study visual-Wi-Fi navigation, where a robot reaches a signal-emitting device in an unmapped environment using onboard visual observations and Wi-Fi packets overheard from the device, without connecting to the device or modifying infrastructure. Realizing this requires pose-dependent Wi-Fi observations during closed-loop policy learning and reliable spatial cues from bearing measurements corrupted by indoor multipath propagation on a moving robot. We present WiFiSpaces, which extends the Habitat embodied simulator with online ray-traced Wi-Fi sensing, rendering Wi-Fi observations for a robot-mounted antenna array at continuous poses during policy rollouts. On top of it, we build ViFiNav, a localization-aware navigation framework: an online source-localization algorithm exploits robot motion to disambiguate and robustly aggregate bearings from multiple viewpoints into a confidence-aware source estimate, and a learned policy fuses this evolving estimate with depth observations and learns to act on it as it is refined. Experiments show that exploiting robot motion is key to accurate source estimates, and that fine-tuning on ray-traced Wi-Fi observations in WiFiSpaces improves the success rate of ViFiNav. The policy transfers zero-shot to unseen scene datasets, and we deploy it on a real robot with a Wi-Fi sensor built from a commodity router, without any real-world training.
Figures & tables
Fig. 1 : Visual–Wi-Fi navigation in everyday indoor environments. (a) Using visual and Wi-Fi observations, an agent navigates toward signal-emitting devices in an unmapped indoor environment. Examples: 1. finding objects equipped with signal-emitting IoT tags (e.g., a package with an AirTag-style Wi-Fi tag), 2. finding the user with a carried mobile device, and 3. finding non-connected objects via associated signal-emitting devices (e.g., food in a smart fridge). (b) Our robot: a commodity Wi-Fi router mounted on a Hello Robot Stretch 3 serves as the Wi-Fi sensor.
Platform
Nav. tasks
Visual obs.
Wireless obs.
Learning interface
Embodied simulators [ 32 , 34 , 22 , 30 ]
✓
✓
–
✓
Wireless simulators [ 1 , 31 , 9 ]
–
–
✓
–
WiFiSpaces (ours)
✓
✓
✓
✓
TABLE I : Simulation platforms for visual–Wi-Fi navigation.
Fig. 2 : System overview. From CSI Ht and the robot pose, the online Wi-Fi source localization module of ViFiNav (disambiguation and multi-view fusion, Fig. 4 ) produces a confidence-tagged source estimate, which the policy fuses with depth and previous-action features to output the action at . They form a closed loop: the policy’s motion selects the localizer’s viewpoints, and the policy is trained on the evolving estimate rather than on a fixed goal. The policy is trained with reinforcement learning (RL) in WiFiSpaces , using a reward available only in simulation, and deployed zero-shot on a real robot.
Fig. 3 : WiFiSpaces renders the Wi-Fi observation on the fly. Given the scene mesh, the transmitting device and the robot pose, propagation paths, including reflections off scene surfaces, are ray-traced per antenna element; the per-path delays and gains form the channel taps, from which the Wi-Fi CSI Ht at the current pose is derived.
Fig. 4 : Wi-Fi source localization. (a) A ULA measurement yields a true and a mirror bearing; under in-place rotation only the true one stays fixed in the world frame, so it is kept. (b) Verified bearings from different viewpoints are fused by RANSAC, keeping the hypothesis with most inliers.
Fig. 5 : The router re-housed in a 3D-printed case as a four-antenna ULA.
Method
Goal Cue
SR ↑
SPL ↑
DTG ↓
Loc. ↓
No-goal ViFiNav
–
0.16
0.10
10.27
–
PointNav (oracle)
GT PointGoal
87.22
69.88
1.66
–
ViFiNav (oracle)
GT AoA
82.30
60.83
2.10
<10−3
PointNav
Est. Goal
49.76
34.57
4.99
3.23
Loc-then-PointNav
Est. Goal
17.78
8.91
6.50
2.85
ViFiNav (w/o FT)
Wi-Fi Obs.
53.02
35.82
4.58
2.56
TABLE II : Main results on HSSD test scenes. GT: ground truth; oracle: GT goal cue; Wi-Fi Obs.: CSI simulated by WiFiSpaces ; Est. Goal: estimate of our online localization module; w/o FT: no fine-tuning on Wi-Fi observations. SR, SPL in percent; DTG, Loc. in meters.
Fig. 6 : Qualitative comparison on three HSSD test episodes (columns). Rows: ViFiNav (top), PointNav (middle), Loc-then-PointNav (bottom). Stars mark successive source estimates, coloured by time. ViFiNav ’s estimates migrate toward the source as it moves, keeping its path short. PointNav follows each updated estimate and detours. Loc-then-PointNav freezes one early estimate and cannot recover when it is wrong.
Loc. Alg.
Amb. Resolver
Pretrain
SR ↑
SPL ↑
DTG ↓
Loc. Err. ↓
LS
–
–
1.75
1.30
7.84
48.74
LS
✓
–
43.33
28.14
4.91
4.34
RANSAC
–
–
47.62
27.95
4.70
2.03
RANSAC
✓
–
52.22
30.17
4.20
1.26
RANSAC
✓
✓
68.57
44.12
3.04
1.24
TABLE III : Effect of the fusion estimator, ambiguity resolution, and two-stage training. LS: least squares.
Fig. 7 : Localization module analysis on 10,000 HSSD training-split episodes, replayed along their shortest paths. (a) Cumulative distribution function (CDF) of direction error before (raw AoA) and after (resolved bearing) motion-based ambiguity resolution. (b) Mean and median source localization error per confidence bin.
Dataset
Scenes
Episodes
Geo. dist. (m)
SR ↑
SPL ↑
DTG ↓
Loc. ↓
HSSD (in-domain)
42
1,260
11.30
68.57
44.12
3.04
1.24
ReplicaCAD
21
1,050
7.44
75.05
47.14
1.29
0.72
ProcTHOR
1,000
5,000
7.19
85.22
50.83
1.29
0.20
TABLE IV : Zero-shot cross-dataset evaluation of the HSSD-trained ViFiNav , with test-set statistics (scenes, episodes, mean geodesic start–goal distance).
Category-level
Instance-level
Method
SR ↑
SPL ↑
DTG ↓
SR ↑
SPL ↑
DTG ↓
ObjectNav (official)
60.95
28.38
2.68
25.67
13.20
7.55
ObjectNav (matched)
61.45
27.26
2.84
25.07
12.86
7.69
ViFiNav (zero-shot)
–
–
–
56.29
30.40
4.59
TABLE V : ObjectNav policies vs. ViFiNav on HSSD ObjectNav episodes. Category-level metrics count any instance of the goal category, instance-level metrics only the designated instance, at which the Wi-Fi source is placed for ViFiNav ; success distance 0.5m to the instance viewpoints at both levels. “Official” follows the HSSD recipe [ 19 ] ; “matched” uses the embodiment of ViFiNav (Sec. VI ).
Fig. 8 : Representative real-world trials: (a) line-of-sight, (b) non-line-of-sight, and (c) an application scenario in which the Wi-Fi source (a phone) is inside a package. Green box: Wi-Fi source; red: robot trajectory.