FloorPlan-VLN: A New Paradigm for Floor Plan Guided Vision-Language Navigation

Kehan Chen; Yan Huang; Dong An; Jiawei He; Yifei Su; Jing Liu; Nianfeng Liu; Liang Wang

cs.ROMar 18, 2026

FloorPlan-VLN: A New Paradigm for Floor Plan Guided Vision-Language Navigation

Authors: Kehan Chen, Yan Huang, Dong An, Jiawei He, Yifei Su, Jing Liu, Nianfeng Liu, Liang Wang

Organizations: New Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences and School of Artificial Intelligence, University of Chinese Academy of Sciences, China · AMap, Alibaba Group · BAAI · Xiaomi Robotics Lab ·

Abstract

Existing Vision-Language Navigation (VLN) task requires agents to follow verbose instructions, ignoring some potentially useful global spatial priors, limiting their capability to reason about spatial structures. Although human-readable spatial schematics (e.g., floor plans) are ubiquitous in real-world buildings, current agents lack the cognitive ability to comprehend and utilize them. To bridge this gap, we introduce FloorPlan-VLN, a new paradigm that leverages structured semantic floor plans as global spatial priors to enable navigation with only concise instructions. We first construct the FloorPlan-VLN dataset, which comprises over 10k episodes across 72 scenes. It pairs more than 100 semantically annotated floor plans with Matterport3D-based navigation trajectories and concise instructions that omit step-by-step guidance. Then, we propose a simple yet effective method FP-Nav that uses a dual-view, spatio-temporally aligned video sequence, and auxiliary reasoning tasks to align observations, floor plans, and instructions. When evaluated under this new benchmark, our method significantly outperforms adapted state-of-the-art VLN baselines, achieving more than a 60% relative improvement in navigation success rate. Furthermore, comprehensive noise modeling and real-world deployments demonstrate the feasibility and robustness of FP-Nav to actuation drift and floor plan distortions. These results validate the effectiveness of floor plan guided navigation and highlight FloorPlan-VLN as a promising step toward more spatially intelligent navigation.

Figures & tables

Fig. 1: Illustration of the FloorPlan-VLN paradigm. Floor plans are derived from Matterport3D scenes, and concise region-based instructions are constructed to train agents to navigate by leveraging floor plans as global spatial priors.

Task / Paradigm	Instruction Type	Spatial Prior Type	Semantic Info.	Goal Specification	Primary Challenge
Traditional VLN [ 1 ]	Verbose (step-by-step)	✗(Unknown)	✗	Linguistic Landmark	Trajectory Grounding
Hand-Drawn Navigation [ 9 , 10 ]	✗(None)	Informal Sketch	✗	Trajectory Annotation	Informal Cue Interpretation
FloorPlan Point Navigation [ 11 , 12 ]	✗(None)	Geometric Boundary	✗	Point Goal Coordinates	Localization & Planning
FloorPlan-VLN (Ours)	Concise (Region-based)	Vectorized Layout	✓(RTA/RIA)	Linguistic Landmark	Spatio-Semantic Alignment

TABLE I: Conceptual comparison between FloorPlan-VLN and related navigation paradigms. Our task introduces a unique combination of concise language and semantic-rich floor plans , shifting the focus to spatio-semantic reasoning.

Split	FloorPlan-R2R Dataset							FloorPlan-RxR Dataset
	Scenes	Trajs	Eps	FPs	Regions	RTs	MTL	Scenes	Trajs	Eps	FPs	Regions	RTs	MTL
Train	61	2,998	9,002	111	945	30	9.27	59	5328	15978	115	942	30	15.04
Val-Seen	51	226	679	80	885	30	9.57	56	593	1778	94	864	29	14.90
Val-Unseen	11	487	1,461	18	161	22	9.04	11	925	2775	21	157	22	13.97

TABLE II: The FloorPlan-VLN Dataset derives trajectories from two major VLN datasets, R2R and RxR, both built on Matterport3D scenes. RxR includes longer trajectories and poses greater navigation challenges than R2R. Abbreviations: Val = Validation, Trajs = Trajectories, Eps = Episodes, FPs = Floor Plans, RTs = Region Types, MTL = Mean Trajectory Length (meters).

Fig. 3: Hand drawn maps can be converted to standardized representation.

Fig. 4: Dataset analysis of the FloorPlan-R2R training split.

Fig. 5: Overview of the FP-Nav framework. The model takes a spatio-temporally aligned dual-view video stream as input, jointly encoding floor plans and egocentric observations. Three auxiliary tasks are designed to strengthen cross-modal alignment, and the model autoregressively predicts navigation actions.

Fig. 6: Comparison of four input strategies.

Fig. 7: Three auxiliary tasks designed to strengthen the alignment among instructions, egocentric observations, and floor plans.

Method	FloorPlan-R2R								FloorPlan-RxR
	Val Seen				Val Unseen				Val Seen				Val Unseen
	NE $\downarrow$	OSR $\uparrow$	SR $\uparrow$	SPL $\uparrow$	NE $\downarrow$	OSR $\uparrow$	SR $\uparrow$	SPL $\uparrow$	NE $\downarrow$	OSR $\uparrow$	SR $\uparrow$	SPL $\uparrow$	NE $\downarrow$	OSR $\uparrow$	SR $\uparrow$	SPL $\uparrow$
Qwen-zs	9.6	3.1	2.2	1.9	9.3	3.4	2.4	2.0	12.0	8.6	5.6	4.4	11.1	11.2	7.2	5.7
NaVILA-zs	9.9	19.0	8.8	0.6	10.1	17.6	5.5	3.5	12.9	20.1	7.1	4.5	11.2	20.0	6.9	4.4
StreamVLN-zs	9.1	24.3	11.5	6.5	8.8	30.3	14.6	7.0	11.6	27.9	15.0	9.2	10.8	26.9	14.4	7.7
InternVLA-N1-zs	7.3	26.5	10.6	9.2	8.2	19.4	7.5	7.0	10.4	22.4	12.3	9.7	10.0	22.4	10.0	8.0

TABLE III: Comparison with baselines across FloorPlan-R2R and FloorPlan-RxR.

Fig. 8: Action frequency after merging consecutive steps in the FloorPlan-R2R finetuning dataset.

Fig. 9: Dataset comparison between FloorPlan-R2R Val-Seen and FloorPlan-RxR on Val-Seen split.

Fig. 11: Visualization of actuation noises and scale noises.

Experiment 1: Robustness to actuation noises
#	Noise	NE $\downarrow$	OSR $\uparrow$	SR $\uparrow$	SPL $\uparrow$
1	$\sigma_{\text{move}}=0.00,\sigma_{\text{rot}}=0.00$	9.8	35.2	20.9	15.3
2	$\sigma_{\text{move}}=0.10,\sigma_{\text{rot}}=0.05$	8.4	42.4	22.0	14.5
3	$\sigma_{\text{move}}=0.30,\sigma_{\text{rot}}=0.10$	8.9	39.5	21.4	11.5
4	$\sigma_{\text{move}}=0.50,\sigma_{\text{rot}}=0.30$	8.60	37.1	17.5	6.7
Experiment 2: Robustness to floor plan scale noises

TABLE IV: Robustness to actuation, floor plan scale, and floor plan distortion noises.

#	Method	NE $\downarrow$	OSR $\uparrow$	SR $\uparrow$	SPL $\uparrow$
1	static separate input	9.8	3.9	2.5	2.3
2	dual-stream temporal fusion	9.8	9.8	4.2	2.4
3	interleaved frame fusion	9.6	4.4	2.6	2.4
4	spatio-temporally aligned	9.8	35.2	20.9	15.3

TABLE V: Ablation study on different input strategies.

Fig. 12: An example of a successful episode using the spatio-temporally aligned input strategy. At each step, the agent’s pose is plotted on the floor plan and concatenated with the corresponding egocentric observation to form a synchronized dual-view input sequence.

#	Method	NE $\downarrow$	OSR $\uparrow$	SR $\uparrow$	SPL $\uparrow$
1	full floorplan	9.8	35.2	20.9	15.3
2	mask 25%	10.1	28.7	17.7	13.4
3	mask 50%	10.0	29.8	16.8	13.0
4	mask 75%	10.2	30.7	14.4	10.5
5	mask 100%	10.8	32.0	12.3	8.0
6	random floorplan	10.7	34.4	9.1	5.5

TABLE VI: Investigation of whether the model truly learns to use floor plans during navigation.

#	Method	NE $\downarrow$	OSR $\uparrow$	SR $\uparrow$	SPL $\uparrow$
1	w/o aux tasks	10.0	30.5	17.8	13.0
2	RL	9.8	32.0	19.1	12.8
3	RL + TR	9.7	32.0	20.1	14.5
4	RL + TR + IS	9.8	35.2	20.9	15.3

TABLE VII: Effect of auxiliary tasks on navigation performance.

#	Method	NE $\downarrow$	OSR $\uparrow$	SR $\uparrow$	SPL $\uparrow$
1	MLP	10.4	3.2	2.3	1.9
2	MLP + LLM	9.8	35.2	20.9	15.3
3	MLP + LLM + Vision	9.6	37.4	25.9	21.7

TABLE VIII: Effect of training different model components on navigation performance.

Fig. 13: Real-world experimental scenes and the corresponding floor plan.

#	Method	NE $\downarrow$	SR $\uparrow$
1	Navid-ft	9.3	8.0
2	FP-Nav-v-rxr	6.4	24.0

TABLE IX: The results of real-world experiments.

Vision-Language Navigation

Vision-Language Navigation studies how embodied agents use natural-language instructions and visual observations to plan and execute routes.

See all 290 papers

FloVerse: Floor Plan-Guided Multi-Modal Navigation
Jun 12, 2026Weiqi Huang, Shuangyi Dong, Jiaxin Li +3Embodied Navigation Robot Imitation Learning
School of Computer Science & Technology, Beijing Institute of Technology
VLN-NF: Feasibility-Aware Vision-and-Language Navigation with False-Premise Instructions
Apr 12, 2026Hung-Ting Su, Ting-Jun Wang, Jia-Fong Yeh +2VLN Benchmarks Vision-Language Navigation
National Taiwan University · DRIC · National Tsing Hua University
AgentVLN: Towards Agentic Vision-and-Language Navigation
Mar 18, 2026Zihao Xin, Wentong Li, Yixuan Jiang +6Vision-Language Navigation Robot Navigation
Nanjing University of Aeronautics and Astronautics · Shandong University · Zhejiang University

Jun 12, 2026J/K move · Enter open · S save

FloVerse: Floor Plan-Guided Multi-Modal Navigation

Weiqi Huang, Shuangyi Dong, Jiaxin Li, Yifei Guo+2

School of Computer Science & Technology, Beijing Institute of Technology

Embodied Navigation Robot Imitation Learning

Open paper AskPDF arXiv

Floor plans encapsulate compact spatial priors, enabling agents to navigate unseen scenes more efficiently. While prior work has explored floor plan-guided navigation, it has focused mainly on PointNav and a limited set of environments. To bridge this gap, we introduce FloVerse, a new task for floor plan-guided embodied navigation that unifies PointNav, ObjectNav, and ImageNav. To support FloVerse, we assemble FloVerse-1.6K, a large-scale dataset of 1.6K scenes from HM3D and Gibson 4+, paired with corresponding floor plans, comprising 240K expert trajectories and 12M RGBD frames. We further propose ThreeDiff, a two-stage imitation learning policy comprising a planner, a diffusion-based multimodal goal-reasoning module trained via masked-modality modeling, and a refiner, a depth-based trajectory-refinement module for safe execution. Extensive experiments demonstrate that (1) floor-plan priors improve navigation performance across all goal modalities, and (2) ThreeDiff implicitly captures spatial information from floor plans. These results underscore the effectiveness of spatial priors and validate our proposed unified approach for floor plan-guided embodied navigation.

Abstract

Vision-Language Navigation

Explore similar work

FloVerse: Floor Plan-Guided Multi-Modal Navigation

VLN-NF: Feasibility-Aware Vision-and-Language Navigation with False-Premise Instructions

AgentVLN: Towards Agentic Vision-and-Language Navigation