BEACON: Language-Conditioned Navigation Affordance Prediction under Occlusion

Xinyu Gao; Gang Chen; Javier Alonso-Mora

cs.ROMar 10, 2026

BEACON: Language-Conditioned Navigation Affordance Prediction under Occlusion

Authors: Xinyu Gao, Gang Chen, Javier Alonso-Mora

Organizations: Department of Cognitive Robotics (CoR), Delft University of Technology

Abstract

Language-conditioned local navigation requires a robot to infer a nearby traversable target location from its current observation and an open-vocabulary, relational instruction. Existing vision-language spatial grounding methods usually rely on vision-language models (VLMs) to reason in image space, producing 2D predictions tied to visible pixels. As a result, they struggle to infer target locations in occluded regions, typically caused by furniture or moving humans. To address this issue, we propose BEACON, which predicts an ego-centric Bird's-Eye View (BEV) affordance heatmap over a bounded local region including occluded areas. Given an instruction and surround-view RGB-D observations from four directions around the robot, BEACON predicts the BEV heatmap by injecting spatial cues into a VLM and fusing the VLM's output with depth-derived BEV features. Using an occlusion-aware dataset built in the Habitat simulator, we conduct detailed experimental analysis to validate both our BEV space formulation and the design choices of each module. Our method improves the accuracy averaged across geodesic thresholds by 22.74 percentage points over the state-of-the-art image-space baseline on the validation subset with occluded target locations. Our project page is: https://xin-yu-gao.github.io/beacon.

Figures & tables

Fig. 2 : Examples of language-conditioned local navigation under occlusion. The blue boxes mark the robot, the red boxes highlight humans and objects that cause occlusions, and the green boxes indicate target regions.

Fig. 3 : BEACON overview. Stage 1 performs auto-derived ego-centric instruction tuning with ego-centric 3D position encoding to train the Ego-Aligned VLM. Stage 2 initializes the Ego-Aligned VLM weights from Stage 1, combines the resulting instruction-conditioned output with Geometry-Aware BEV features, and predicts an ego-centric BEV navigation affordance heatmap via a Post-Fusion Affordance Decoder. The two stages use different supervision signals, and inference selects the navigation target by taking the argmax.

Method	Input	Output	Full Validation Set (%)				Occluded-Target Subset (%)
Method	Input	Output	${\overline{\mathrm{GeoAcc}}}$ $\uparrow$	${\overline{\mathrm{EucAcc}}}$ $\uparrow$	SIR $\downarrow$	${\overline{\mathrm{GeoAcc}}}_{\mathrm{snap}}^{\dagger}\uparrow$	${\overline{\mathrm{GeoAcc}}}$ $\uparrow$	${\overline{\mathrm{EucAcc}}}$ $\uparrow$	SIR $\downarrow$	${\overline{\mathrm{GeoAcc}}}_{\mathrm{snap}}^{\dagger}\uparrow$
General-purpose VLM baselines
ChatGPT-4o [ 38 ]	RGB	image point	9.69	20.65	57.25	18.94	5.69	11.55	54.03	10.39
ChatGPT-4o [ 38 ] (oracle-view)	RGB	image point	15.97	30.79	48.20	28.28	9.52	17.11	41.68	15.31
Spatial-grounding VLM baselines with oracle-view selection
RoboPoint-13B [ 1 ]	RGB	image point	20.86	32.59	39.34	30.96	15.43	23.46	35.18	21.88

TABLE I : Overall quantitative results on local navigation target prediction, comparing image-space baselines, straightforward trained alternative, and BEACON on the full validation set and occluded-target subset. Best results are shown in bold .

Stage 1 Tuning	3D Pos. Enc.	BEV Encoder	BEV Output	Val. (%)	Occluded-Target Subset (%)
Stage 1 Tuning	3D Pos. Enc.	BEV Encoder	BEV Output	${\overline{\mathrm{GeoAcc}}}$ $\uparrow$	${\overline{\mathrm{GeoAcc}}}$ $\uparrow$	${\overline{\mathrm{EucAcc}}}$ $\uparrow$	SIR $\downarrow$
Ego-Aligned VLM design ablations
		✓	✓	54.76	40.06	42.49	2.37
✓		✓	✓	54.36	40.22	42.64	2.62
	✓	✓	✓	53.59	37.93	40.31	2.50
BEV-space design ablations

TABLE II : Ablation study of key Ego-Aligned VLM and BEV-space design choices. Best results are shown in bold .

$F_{\mathrm{BEV}}^{\mathrm{Img}}$	$F_{\mathrm{BEV}}^{\mathrm{Geom}}$	$G$	Full Val. (%)	Occ. Subset (%)
$F_{\mathrm{BEV}}^{\mathrm{Img}}$	$F_{\mathrm{BEV}}^{\mathrm{Geom}}$	$G$	${\overline{\mathrm{GeoAcc}}}$ $\uparrow$ / SIR $\downarrow$	${\overline{\mathrm{GeoAcc}}}$ $\uparrow$ / SIR $\downarrow$
$\checkmark$			55.99 / 3.22	41.52 / 3.77
	$\checkmark$		51.67 / 9.54	36.93 / 12.51
$\checkmark$	$\checkmark$		56.96 / 2.12	42.34 / 2.62
$\checkmark$	$\checkmark$	$\checkmark$	57.72 / 2.13	42.83 / 2.60

TABLE III: Ablation study on

F_{\mathrm{BEV}}

components.

Panel of Figure 6 — Fig. 4 : Qualitative examples of language-conditioned navigation affordance prediction, comparing BEACON’s BEV affordance predictions with the image-space baselines RoboPoint [ 1 ] and RoboRefer [ 2 ] . Target regions are shown in green .

Robot Navigation

Robot navigation studies how robots perceive, localize, plan, and move safely toward goals in static or changing environments.

See all 554 papers

Offline Vision-Language Navigation with Geometric Goal Localization for Outdoor Environments
Jul 24, 2026Ali Salmasi, Xianjia Yu, Tomi WesterlundSmall Language Models Robotics
Turku Intelligent Embedded and Robotic Systems (TIERS) Lab, University of Turku, Turku, Finland.
Contextual Observer Grounding: Evaluating Situated Spatial Reasoning in Vision-Language Models
Sep 7, 2026Mimo Shirasaka, Haochen Zhang, Yonatan BiskVision-Language Grounding Visually Grounded Reasoning
The University of Tokyo Tokyo, Japan · Carnegie Mellon University Pittsburgh, PA, USA
CapNav: Benchmarking Vision Language Models on Capability-conditioned Indoor Navigation
Feb 20, 2026Xia Su, Ruiqi Chen, Benlin Liu +4Embodied Spatial Reasoning VLN Benchmarks
University of Washington Seattle, WA, USA · University of California, Santa Cruz Santa Cruz, CA, USA

Jul 24, 2026J/K move · Enter open · S save

Offline Vision-Language Navigation with Geometric Goal Localization for Outdoor Environments

Ali Salmasi, Xianjia Yu, Tomi Westerlund

Turku Intelligent Embedded and Robotic Systems (TIERS) Lab, University of Turku, Turku, Finland.

Small Language Models Robotics Vision-Language Navigation

Open paper AskPDF arXiv

Foundation-model-based vision-language navigation (VLN) has advanced autonomous robot navigation by enabling robots to interpret natural-language instructions, identify semantic goals, and follow user-specified behavioral rules. However, existing VLN systems rely heavily on cloud-hosted foundation models for language understanding and semantic grounding, limiting their applicability where network connectivity is unavailable and reliable metric goal localization is required. Although recent small language models (SLMs) enable fully onboard inference, their suitability for navigation instruction decomposition has not been systematically evaluated. This paper makes three contributions toward fully onboard VLN for outdoor environments. First, we present the first systematic benchmark of 17 edge-deployable SLMs against 4 online APIs for robotic navigation instruction decomposition, evaluating accuracy and latency on human-annotated instructions across three computing platforms and providing practical guidance for selecting onboard language models. Second, we propose a lightweight hybrid semantic-geometric goal localization framework that combines open-vocabulary object detection, prompted segmentation, and LiDAR geometry to estimate metric goals, while maintaining visual bearing guidance when reliable geometric observations are unavailable. Third, we integrate these advances into Edge-BehAV, a fully onboard extension of the BehAV architecture that enables cloud-independent behavior-guided navigation. Experimental results show that the best offline SLM matches the instruction decomposition performance of the strongest cloud API while running approximately 9x faster and without network connectivity. The proposed goal localization framework reduces mean goal-distance error from 2.05 m to 0.20 m at lower computational cost, and the complete system succeeds in 31 of 32 closed-loop outdoor trials.

Abstract

Robot Navigation

Explore similar work

Offline Vision-Language Navigation with Geometric Goal Localization for Outdoor Environments

Contextual Observer Grounding: Evaluating Situated Spatial Reasoning in Vision-Language Models

CapNav: Benchmarking Vision Language Models on Capability-conditioned Indoor Navigation