Organizations: Department of Mechanical and Mechatronics Engineering, University of Waterloo, Waterloo, ON, Canada · Department of Mechatronics Engineering, Ontario Tech University, Oshawa, ON, Canada · Department of Earth and Space Science Engineering, York University, Toronto, ON, Canada · University of Toronto Institute for Aerospace Studies and the University of Toronto Robotics Institute, University of Toronto, Toronto, ON, Canada · Department of Mechanical and Materials Engineering, Queen’s University, Kingston, ON, Canada
We address the Person Goal Navigation (PersonNav) problem, enabling robots to search for people under realistic constraints on information accessibility and language uncertainty. This tackles the current solutions revolving around rigid person finding systems requiring exact knowledge of the individual for item-delivery in indoor settings. While the focus in literature is on learned methods using unavailable public or sparse data due to human privacy. We propose a planning framework that combines distance and semantic information about the recipient (habits and intent) weighted by the trust of the user-provided information. A synthetic benchmark of scenarios is also developed including actors, items, and requests to evaluate performance before real-world deployment aiming to simulate natural language human-robot interactions. Results show our informed search outperforms classical distance-based graph baselines, while semantics alone lead to ungrounded, sporadic search. Our method achieves strong gains over baselines, with an LLM-based variant performing comparably, and its explicit belief representation naturally supports future Bayesian filtering. Hardware tests demonstrate our method supports real-world embodiment able to leverage between semantic and distance information in a real setting. This work moves toward more intelligent mobile service agents capable of human-like, informed search in realistic environments to be leveraged in day-to-day use. https://anonymous.4open.science/r/personnavsite-4ED2/index.html.
Figures & tables
Type
Certainty
Example
None
-
Hey can you give this screwdriver to Jane?
Recency
High
This hammer is for Deshawn, he was right over by the lounge just a brief moment ago.
Behavioural
Low
Give this folder to Martin, not too sure but they’re sometimes at the lab.
Contextual
Low
Can you give Ramirez the whistle. He may need it for running drills today, but can’t say for certain, so try the field.
Behav. & Context.
High
Could you bring this laptop to Sana? She’s normally in the studio editing, and she’ll definitely need it for that.
TABLE I: Hint Type Taxonomy and Examples
Fig. 1: CSIR Framework Belief Generator
Fig. 2: Synthetic Scenario (Actor, Item, and Request) Generator
Fig. 3: Annotated Office, Hospital, and University Map (yellow circle markers indicate agent start locations)
Office
Hospital
Engineering University
Average
Method
Travel Eff.
Top-3 Acc.
Visited
Plan Time
Travel Eff.
Top-3 Acc.
Visited
Plan Time
Travel Eff.
Top-3 Acc.
Visited
Plan Time
Travel Eff.
Top-3 Loc.
Plan Time.
Graph-Based Planners
Random
0.22
0.17
9.42
-
0.22
0.19
7.75
-
0.21
0.20
8.92
-
0.22
0.19
-
Greedy
0.36
0.13
10.53
-
0.48
0.20
7.22
-
0.43
0.18
10.10
-
0.42
0.17
-
Held-Karp TSP
0.41
0.25
9.80
0.11
0.44
0.16
7.56
0.01
0.44
0.20
9.81
0.05
0.43
0.20
0.06
Semantic Planners
TABLE II: Simulation Summary Performance Accross All Environments
Fig. 4: Sankey Diagram to Show Hint and Certainty Distribution to Hint Correct/Incorrect (Office Environment)
Model
Train Method
Val. Acc.
Scenario Acc.
DeBERTa-v3-base
Fine-Tuning
0.86
0.67
Qwen3-0.6B-
Embedding
Linear Head
0.80
0.82
Qwen3-0.6B-
Embedding
Non-Linear Head
0.87
0.94
TABLE III: Certainty classification accuracy on validation and scenario datasets
Fig. 5: Violin Plot of Travel Efficiency Results Across All Methods (All Environments)
Belief Gen.
Travel Eff.
Top-3 Acc.
KL Div.
Gen. Time
Bi-Encoder
0.38
0.37
1.55
0.03
Cross-Encoder
0.48
0.47
1.86
0.01
LLM
0.52
0.58
1.36
2.16
CSIR (Belief)
0.51
0.56
1.58
3.70*
TABLE IV: Belief Generator Ablation Results (*Note: Gen. time is computed using the hospital and university results since parallelization was not introduced for the office yet)
Hint Type
Travel Eff.
KL Div.
None
0.46
1.63
Recency
0.80
1.49
Behavioural
0.67
1.80
Contextual
0.69
1.58
Behav. & Context.
0.77
1.42
TABLE V: Hint Type Ablation Results
Fig. 6: Hardware Experiment Recency and Behavioural Hint Requests