Text-goal instance navigation (TGIN) asks an agent to resolve a single, free-form description into actions that reach the correct object instance among same-category distractors. We present Context-Nav, which elevates long, contextual captions from a local matching cue to a global exploration prior and verifies candidates through 3D spatial reasoning. First, we compute dense text-image alignments for a value map that ranks frontiers -- guiding exploration toward regions consistent with the entire description rather than early detections. Second, upon observing a candidate, we perform a viewpoint-aware relation check: the agent samples plausible observer poses, aligns local frames, and accepts a target only if the spatial relations can be satisfied from at least one viewpoint. The pipeline requires no task-specific training or fine-tuning; we attain state-of-the-art performance on InstanceNav and CoIN-Bench. Ablations show that (i) encoding full captions into the value map avoids wasted motion and (ii) explicit, viewpoint-aware 3D verification prevents semantically plausible but incorrect stops. This suggests that geometry-grounded spatial reasoning is a scalable alternative to heavy policy training or human-in-the-loop interaction for fine-grained instance disambiguation in cluttered 3D scenes.
Figures & tables
Figure 2: Overall pipeline of Context-Nav. Given RGB-D observations, odometry, and a free-form text goal, the perception and mapping modules use GOAL-CLIP, open-vocabulary detection, and 3D projection to build an occupancy map, a context-conditioned value map, and an instance-level map. Whenever a target object candidate is detected, the verification module checks intrinsic attributes with a VLM and extrinsic attributes through 3D spatial reasoning to decide whether to terminate or continue exploring.
Method
Model Condition
InstanceNav
CoIN-Bench
Input
Training-free
SR ↑
SPL ↑
Val Seen
Val Seen Synonyms
Val Unseen
SR ↑
SPL ↑
SR ↑
SPL ↑
SR ↑
SPL ↑
GOAT [ 19 ]
d
✗
17.0
8.8
6.6
3.1
13.1
6.5
0.2
0.1
PSL † [ 34 ]
d
✗
26.0
10.2
8.8
3.3
8.9
2.8
4.6
1.4
VLFM [ 48 ]
c
✓
14.9
9.3
0.4
0.3
0.0
0.0
0.0
0.0
AIUTA [ 35 ]
c
✓
-
-
7.4
2.9
14.4
8.0
6.7
2.3
Table 1: Benchmark results on InstanceNav and CoIN-Bench . Comparison of RL-trained policies, training-free modular baselines, and the proposed Context-Nav on InstanceNav and the three CoIN-Bench splits ( Val Seen , Val Seen Synonyms , Val Unseen ). Input type c denotes a category-level goal specification, while d denotes a language description of the target.
Figure 3: Stage-wise qualitative example of context-driven navigation. An episode where the agent must find a dresser described as “located next to the bed” and “a white dresser with a mirror on top”. Early dresser candidate is not selected because context objects are absent; after the bed is detected, the map concentrates around the corresponding room, frontier selection focuses on that area, and a dresser that satisfies both intrinsic attributes and 3D spatial relations with the bed and mirror is finally verified as the goal.
Backbone
Prompt
CoIN-Bench
SR ↑
SPL ↑
BLIP-2 [ 23 ]
Category
15.9
7.3
Intrinsic Attributes
17.8
8.2
Full text
16.4
9.5
GOAL-CLIP [ 13 ]
Category
13.7
7.6
Intrinsic Attributes
16.7
9.7
Table 2: Ablation of similarity backbone and prompt on CoIN-Bench Val Seen Synonyms. We compare BLIP-2 and GOAL-CLIP under different prompt designs (category only, category with intrinsic attributes, and full contextual text).
Method
CoIN-Bench
SR ↑
SPL ↑
Nearest frontier exploration
10.6
4.6
Remove VLM category verification
11.1
7.1
Remove attribute verification
12.5
7.7
Remove context verification
12.0
8.4
Full Approach
20.3
10.9
Table 3: Ablation of pipeline components on CoIN-Bench Val Seen Synonyms. Replacing value-map–guided frontier ranking with a nearest-frontier heuristic or removing VLM category, attribute, or context verification each degrades SR and SPL.
Figure 4: Qualitative results across diverse categories and context descriptions. Successful episodes on CoIN-Bench for nine different target categories, showing top-down trajectories and corresponding goal views. The instructions span a wide range of natural language, from captions that only specify extrinsic context to descriptions that combine intrinsic and extrinsic attributes, and from short hints to detailed multi-sentence goals.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1 : Viewpoint-aware 3D verification of extrinsic attributes. Starting from the extrinsic part of the goal description, Context-Nav extracts context objects and spatial-relation triples, builds instance-level 3D point clouds, and samples candidate viewpoints around the reference–target pairs. For each candidate viewpoint, a local frame is aligned and the corresponding spatial predicates are evaluated; the figure visualizes an example viewpoint from which all extrinsic relations are satisfied simultaneously, together with the resulting relation checks for the confirmed target instance.
Method
Model Condition
InstanceNav
Input
Training-free
SR ↑
SPL ↑
PSL [ 34 ]
d
✗
18.0
7.2
VLFM [ 48 ]
c
✓
9.5
5.7
UniGoal [ 47 ]
d
✓
13.5
6.7
Ours
d
✓
21.2
7.9
Appendix
Table S1: Benchmark results on InstanceNav. Comparison of RL-trained policies, training-free modular baselines, and the proposed Context-Nav on InstanceNav [ 34 ] under the stricter CoIN-Bench [ 35 ] success criteria. Input type c denotes a category-level goal specification, while d denotes a language description of the target.
Component
Det/Seg Modules
Value-map Update
3D Association
VLM Inference
Viewpoint-aware Verification
Context-Nav
0.12
0.11
0.29
0.49
0.0004
Appendix
Table S2 : Per-call latency (in seconds) of each module.
Figure S2 : Qualitative comparison on CoIN-Bench. Context-Nav trajectories are compared with those of the RL-trained PSL [ 34 ] policy and the training-free AIUTA [ 35 ] agent on CoIN-Bench episodes featuring multiple same-category distractors. For each text goal, top-down trajectories are overlaid on the floor map: Context-Nav is shown in orange and the baselines in light gray. Insets show the final egocentric view and outcome label for each method: Target indicates that the correct instance is reached within the step budget, Time-out denotes failure to reach any candidate in time, and Distractor indicates that the agent stops at a different instance of the same category. In these examples, Context-Nav consistently reaches the correct instance, whereas the baselines either time out or stop at distractors.
Figure S3 : Qualitative comparison on InstanceNav. Representative episodes from InstanceNav [ 34 ] compare Context-Nav with the RL-trained PSL [ 34 ] policy and the training-free UniGoal [ 47 ] pipeline. As in Fig. S2 , top-down trajectories and final goal views are visualized, drawing Context-Nav in orange and the baselines in light gray. The terminal state is annotated as Target when the correct instance is reached, Distractor when the agent stops at a different instance of the same category, and Off-target when the agent stops at an object from a different category than the described target. These examples highlight that Context-Nav successfully reaches the correct goal instances, whereas PSL and UniGoal often time out, stop at same-category distractors, or end up off-target at objects of incorrect categories.
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing · University of Chinese Academy of Sciences, Beijing · Institute of Computing Technology, Chinese Academy of Sciences, Beijing