Interactive Navigation Beyond Reachability: Benchmark and Agentic Framework

Delun Li; Xueyao Wei; Rui Chen; Yifei Wang; Junzhi Yu; Wenjie Song; Xingyu Chen

cs.ROOct 12, 2026

Interactive Navigation Beyond Reachability: Benchmark and Agentic Framework

Authors: Delun Li, Xueyao Wei, Rui Chen, Yifei Wang, Junzhi Yu, Wenjie Song, Xingyu Chen

Organizations: Beijing Institute of Technology · Zhongguancun Academy · Peking University

Abstract

Existing embodied navigation tasks commonly assume that traversability and object states remain fixed during an episode. Yet reaching a goal may require opening a door to access a room and then opening a container to reveal a hidden target. We formulate interactive navigation beyond reachability, unifying channel-induced changes in spatial connectivity, container-induced changes in target visibility, and their sequential dependencies. We introduce MultINav-Bench, a 3K-episode ObjectGoal benchmark in MolmoSpaces spanning channel, container, and mixed tasks, with interaction-necessity annotations covering required and unnecessary episodes. Its data generation pipeline automates task construction across multiple task types. The evaluation protocol separately measures navigation success, interaction effectiveness, and cost. We further propose MultINav-Agent, a three-stage agentic framework built on a persistent, state-conditioned interaction graph. Semantic interactive perception grounds entities and interaction states; a vision-language model selects task-relevant actions based on expected changes in connectivity and target observability; and post-interaction verification updates the graph to guide subsequent navigation, exploration, or interaction. Experiments on MultINav-Bench show that MultINav-Agent consistently outperforms adapted baselines across channel, container, and mixed tasks.

Figures & tables

Benchmark	Endpoint	Chan.	Cont.	Obj.	Evaluated state effect	Reported scale / envs.
Non-interactive benchmarks (fixed environment state)
Habitat ObjectNav [ 9 ]	ObjectGoal				Static	2.8M eps. / 90 envs.
MolmoSpaces Navigate-to [ 8 ]	ObjectGoal				Static	2K traj. / 679 envs.
R2R [ 2 ]	InstructionGoal				Static	21.6K instr. / 90 envs.
VLN-CE [ 13 ]	InstructionGoal				Static	R2R annot. / 90 envs.
Interactive benchmarks (environment-changing actions)

Table 1: Comparison of interaction scope and evaluated state effects. Chan., Cont., and Obj. denote channels, containers, and manipulable objects. Reported scale follows the respective publications.

Figure 2: Construction and composition of MultINav-Bench. Top: channel interactions alter reachability and, in a route-shortening subset, route efficiency, whereas container interactions change target visibility and accessibility. Bottom: benchmark construction and data collection. Right: required-interaction counts and feasible-route lengths for the 3,000 episodes.

Figure 3: MultINav-Agent framework . Perception builds a persistent, state-conditioned interaction graph; candidate plans are formed; interaction proposals are filtered for reachability, observability, and freshness; a VLM scores admissible plans by task relevance; and post-interaction verification updates the graph to guide subsequent action selection.

Method	Tasks	SR (%) $\uparrow$	Domain
MultINav-Agent (ours)	2,000	53.0	ProcTHOR
RING †	2,000	48.0	ProcTHOR
DualVLN †	2,000	21.0	Habitat
SPOC	2,000	20.15	ProcTHOR
VLFM	2,000	11.90	Habitat

Table 2: MolmoSpaces NavToObj success ( † leaderboard).

Method / Split	SR (%) $\uparrow$	SPL $\uparrow$	ISR (%) $\uparrow$	IP (%) $\uparrow$	Cost $\downarrow$
	Navigation & Task		Interaction		Overall
MoMa-LLM	10.53	0.0848	23.69	27.59	0.9309
Channel	19.60	0.1758	54.28	40.61	0.8651
Container	10.30	0.0685	2.00	2.37	0.9324
Mixed	1.70	0.0101	25.37	39.81	0.9951
MultINav-Agent (ours)	41.63	0.2903	49.38	46.79	0.7743

Table 3: Interactive navigation results on MultINav-Bench , grouped by method. Each method is evaluated on 3,000 episodes, with 1,000 per split. Each method’s first row reports Overall, followed by task splits. Bold indicates the best value for each metric within the same split.

Method	SR (%) $\uparrow$	SPL $\uparrow$	ISR (%) $\uparrow$	IP (%) $\uparrow$	Cost $\downarrow$
	Navigation & Task		Interaction		Overall
Full	23.0	0.107	41.8	40.4	0.961
Flat Object Memory	21.0	0.100	43.8	26.7	0.963
Greedy Selection	22.0	0.134	40.9	36.9	0.933
Perception-only Update	5.5	0.033	26.6	36.6	0.988

Table 4: Ablation results on the same fixed 200-episode Mixed subset . Bold indicates the best value in each column.

Panel of Figure 4 — Figure 4: Sequential interactions and graph updates. The robot opens two doors to reach a refrigerator, then opens it to reveal the target. Insets show evolving semantic maps and interaction graphs; right: final graph and trajectory (blue: navigation/exploration; orange: interactions).

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Split / recipe	$N$	Notes
Channel total	1000	253 houses
single-path door closed	366	required
distractor doors closed	346	unnecessary
critical+distractor	288	required
Container total	1000	569 houses
Mixed total	1000	369 houses

Appendix

Table 5: Domain recipes and necessity labels in MultINav-Bench (light).

Domain	$N$	Path med	Path mean	Int. mean
Channel	1000	8.12	8.71	0.65
Container	1000	5.93	7.99	1.10
Mixed	1000	11.50	13.58	2.01
All	3000	8.69	10.09	1.25

Appendix

Table 6: GT path length (m) and mean required-interaction count from the light manifest.

Task form	Stored goal specification
ObjectGoal	Target category/instance, distance and head-camera visibility success criteria; primary MultINav-Bench split.
PointGoal	Grounded planar goal (optional yaw) with interaction-aware route feasibility and path-delta annotations.
InstructionGoal	Text grounded to oracle steps and entity IDs, with hidden/partial/explicit interaction disclosure.

Appendix

Table 7: Three task specifications supported by the shared collector.

Panel of Figure 6 — Figure 6: Illustrative InstructionGoal disclosure levels generated from one grounded interaction plan. These are task-specification examples, not results from a frozen InstructionGoal evaluation split.

Embodied Navigation

Embodied navigation studies how robots perceive, map, and move through environments to reach goals and follow instructions.

See all 138 papers

NavHarness: Adaptive Goals for Agentic Vision-Language Navigation
Sep 30, 2026Haoxiang Shi, Zaijing Li, Muhe Ding +3Vision-Language Navigation Multimodal Agents
Harbin Institute of Technology (Shenzhen) · Pengcheng Laboratory
Ask When It Pays: Cost-Aware Open-Ended Interaction for Instance Goal Navigation
Jun 2, 2026Xunyi Zhao, Sihao Lin, Gengze Zhou +5Embodied Navigation AI Agent Benchmarks
1Adelaide University · 2Responsible AI Research Centre, Australian Institute for Machine Learning · Institute for Infocomm Research (I2R), A*STAR +2
RAGNav: A Retrieval-Augmented Topological Reasoning Framework for Multi-Goal Visual-Language Navigation
Mar 4, 2026Ling Luo, Qiangian BaiVision-Language Navigation Visual Spatial Reasoning
Southwestern University of Finance and Economics, Chengdu, 611130, China

Sep 30, 2026J/K move · Enter open · S save

NavHarness: Adaptive Goals for Agentic Vision-Language Navigation

Haoxiang Shi, Zaijing Li, Muhe Ding, Xiang Deng+2

Harbin Institute of Technology (Shenzhen) · Pengcheng Laboratory

Vision-Language Navigation Multimodal Agents

Open paper AskPDF arXiv

Vision-Language Navigation (VLN) requires embodied agents to generate actions based on instructions and observations. General-purpose multimodal agents offer a promising basis for this task, but selecting plausible local actions does not ensure that execution remains consistent with the intended route, particularly in long-horizon tasks. Moreover, the accumulated interaction history increases the input required for subsequent decisions, resulting in a significant inference overhead. To this end, we introduce \method, an Agentic VLN framework that includes a Goal Agent that sets adaptive goals for local actions, a Verify Agent that dynamically verifies whether a goal has been completed, a Memory Agent for multimodal context compression, and a Visuomotor Agent to execute adaptive goals. Specifically, the Goal Agent formulates adaptive goals based on the instruction, current observation, and execution history. Then the Visuomotor Agent executes navigation actions to achieve each goal, while the Verify Agent uses a goal-specific verification question to dynamically assess whether the observed outcomes satisfy the intended completion condition. Verified goal completion then marks a boundary for the Memory Agent to compress the corresponding multimodal interaction history while preserving information needed for subsequent navigation. We evaluate navigation on R2R-CE and RxR-CE, examine framework variants across three model backbones, and study context evolution during execution. For Real-World evaluation, \method achieves 83.3% success and 1.51,m navigation error across eight challenging routes evaluated three times each.

Abstract

Embodied Navigation

Explore similar work

NavHarness: Adaptive Goals for Agentic Vision-Language Navigation

Ask When It Pays: Cost-Aware Open-Ended Interaction for Instance Goal Navigation

RAGNav: A Retrieval-Augmented Topological Reasoning Framework for Multi-Goal Visual-Language Navigation