LangMap: A Human-Verified Benchmark for Hierarchical Open-Vocabulary Goal Navigation

Bo Miao; Weijia Liu; Jun Luo; Lachlan Shinnick; Jian Liu; Thomas Hamilton-Smith; Yuhe Yang; Zijie Wu

cs.CVFeb 2, 2026

LangMap: A Human-Verified Benchmark for Hierarchical Open-Vocabulary Goal Navigation

Authors: Bo Miao, Weijia Liu, Jun Luo, Lachlan Shinnick, Jian Liu, Thomas Hamilton-Smith, Yuhe Yang, Zijie Wu, +3 more

Organizations: AIML, Adelaide University · East China Normal University

Abstract

Language-conditioned goal navigation (LGN) requires embodied agents to locate user-specified targets without step-by-step guidance. However, existing benchmarks largely focus on category-level goals or rely on instance descriptions generated by vision-language models, which often contain ambiguities and semantic errors, limiting systematic and reliable evaluation. We introduce HieraNav, an open-vocabulary LGN task with goals at four hierarchical semantic levels: scene, room, region, and instance. We present Language as a Map (LangMap), the first LGN benchmark to enrich real-world indoor 3D scans with human-verified semantic annotations supporting tasks across all four goal levels. Built on HM3D using a contrastive annotation protocol that compares same-scene regions and instances, LangMap provides region labels and discriminative region and instance descriptions covering 414 object categories and contains over 18K tasks. Each target has concise and detailed descriptions, enabling evaluation across instruction styles. Automated and human evaluations validate our annotation quality: our descriptions improve text-to-view matching accuracy over GOAT-Bench's by 23 points on all shared annotated instances, and an independent human audit yields 92.5% unique-and-correct matches. We also propose PlaNaVid, an RGB-only baseline that combines Bounded Diverse Memory with high-level planning to prime a reactive policy for multi-goal navigation, achieving top-tier success rates without depth, 3D scene representations, or object masks. Further analyses reveal that exploration and hierarchical disambiguation failures become more prominent at finer goal levels, while long-tail categories, small objects, distant targets, timely stopping, and multi-goal completion remain challenging. Benchmark and code: https://bo-miao.github.io/LangMap

Figures & tables

Figure 3 : HieraNav requires agents to navigate to language-specified targets at four levels: scene, room, region, and instance. LangMap enables evaluation with mixed-level multi-goal episodes and single-goal tasks. Descriptions cover intrinsic attributes, spatial context, and open-world knowledge.

Eval Benchmark

Goal Granularity

Region Annotation

Object Annotation

Small Obj

Tasks

Scene

Room

Region

Instance

Cat

Desc

Words

Cat

Desc

Words

RoboTHOR [ 65 ]

✓

\times

\times

\times

\times

\times

ObjNav-MP3D [ 21 ]

✓

\times

\times

\times

\times

\times

ObjNav-HM3D [ 2 ]

✓

\times

\times

\times

\times

\times

HM3D-OVON [ 1 ]

✓

\times

\times

\times

178

\times

\times

4.2%

9000

LHPR-VLN [ 6 ]

✓ †

\times

\times

\times

\times

960

Table 1: Statistics of LGN evaluation benchmarks. Cat/Desc/Words: number of categories, number of discriminative descriptions, and average word count (concise/detailed). Small Obj: percentage of objects with mean frame coverage below 3.3% across look-down, forward, and look-up views. Blanks and

\times

indicate unsupported settings. † : VLM-generated descriptions (GOAT-Bench) and inferred room types (LHPR-VLN). LangMap provides human-verified multi-granularity annotations.

Figure 4 : LangMap distributions. (a) Region label frequency. (b) Instances per object category, with sampled labels. (c) Ground-truth geodesic distances of navigation tasks across four semantic levels.

Figure 5 : Overview of our data processing pipeline.

Figure 6 : Examples of the contrastive annotation process. (a) Region annotation: annotators compare same-category regions within a scene. (b) Instance annotation: annotators compare same-category or semantically related instances using object views and verified region context. All annotations are cross-checked for correctness and discriminability.

Figure 7 : Overview of PlaNaVid, a decoupled RGB-only framework using RGB and language without depth, 3D representations, or object masks. Bounded Diverse Memory maintains a bounded history with near-uniform temporal coverage and provides diverse context for a VLM planner, which selects an initial waypoint and heading per goal to prime reactive policy for multi-goal navigation.

Method	Observation	Mask	3D Map	Local Nav.	Multi-Goal			Single-Goal		Latency (s/goal)
Method	Observation	Mask	3D Map	Local Nav.	SR $\uparrow$	SeqSR@2 $\uparrow$	SPL $\uparrow$	SR $\uparrow$	SPL $\uparrow$	Latency (s/goal)
3D-Mem-3B [ 68 ]	RGB-D (pano.)	✓	✓	Shortest	13.5	1.1	6.4	8.5	1.9	147.2
3D-Mem-7B [ 68 ]	RGB-D (pano.)	✓	✓	Shortest	30.1	5.1	17.3	13.7	6.2	95.6
3D-Mem-3B † [ 68 ]	RGB-D (pano.)	✓	✓	Shortest	20.4	2.2	11.3	15.3	2.8	147.2
3D-Mem-7B † [ 68 ]	RGB-D (pano.)	✓	✓	Shortest	36.8	10.0	21.2	21.2	8.7	95.6
MTU3D [ 69 ]	RGB-D (pano.)	✓	✓	Shortest	41.2	11.0	24.3	29.9	15.3	52.1

Table 2: Evaluation on LangMap with concise descriptions . -3B/-7B denote planner sizes. † Reference only: 3D-Mem’s original success criterion (within 1m of a target viewpoint, allowing up to 2m from the goal). Local Nav.: simulator pathfinder (Shortest) or learned policy (Learned).

Config	Method	Scene		Room		Region		Region-D		Instance		Instance-D
		SR $\uparrow$	SPL $\uparrow$	SR $\uparrow$	SPL $\uparrow$	SR $\uparrow$	SPL $\uparrow$	SR $\uparrow$	SPL $\uparrow$	SR $\uparrow$	SPL $\uparrow$	SR $\uparrow$	SPL $\uparrow$
RGB-D,Mask	3D-Mem-3B	11.2	2.5	11.8	2.8	8.2	1.8	6.8	1.4	4.7	1.1	5.1	1.1
	3D-Mem-7B	16.4	7.2	12.6	6.0	14.0	6.1	15.1	5.6	13.0	5.8	16.3	7.0
	MTU3D	31.4	15.7	32.5	16.2	33.6	16.8	33.7	16.2	23.8	12.9	28.7	15.2
RGB	PSL	6.0	1.4	6.6	1.9	7.3	2.1	9.0	2.6	6.4	1.9	8.5	2.4
	SenseAct-M	10.2	5.6	8.3	4.4	8.5	4.3	9.5	4.8	7.7	4.0	7.4	4.1

Table 3: Single-goal results on LangMap. Descriptions are concise unless marked ‘-D’ (detailed).

Table 10

Method	Category Frequency				Path Length						Object Size
Method	Head		Long-tail		Short		Medium		Long		Non-small		Small
	SR $\uparrow$	SPL $\uparrow$	SR $\uparrow$	SPL $\uparrow$	SR $\uparrow$	SPL $\uparrow$	SR $\uparrow$	SPL $\uparrow$	SR $\uparrow$	SPL $\uparrow$	SR $\uparrow$	SPL $\uparrow$	SR $\uparrow$	SPL $\uparrow$
3D-Mem-7B	14.4	6.6	11.1	4.6	27.3	12.8	11.6	4.9	4.4	2.0	14.7	6.7	11.0	4.6
MTU3D	32.1	16.3	22.0	11.5	46.3	21.7	28.0	14.8	17.4	9.7	32.8	16.7	20.5	10.0
Uni-NaVid	31.3	15.9	26.1	13.2	42.2	21.5	29.7	14.7	19.4	10.3	32.6	16.5	23.0	10.9
PlaNaVid-3B	32.4	15.8	27.4	13.0	43.8	21.7	30.5	14.4	20.7	10.4	34.0	16.4	22.4	10.5

Table 6: Single-goal analysis by category frequency, optimal path length, and object size.

Components			Results
Mem	GUU	SDR	SR	SeqSR	SPL	#Frames $\downarrow$
			34.4	10.4	15.0	-
✓			40.4	13.2	16.9	656
✓	✓		40.9	13.2	17.3	50
✓		✓	42.0	14.2	17.8	629
✓	✓	✓	42.8	14.6	18.1	50

Table 7 : Ablation of PlaNaVid.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8 : Representative object-view and region-view capture.

Figure 9 : Comparison of instance-level descriptions in GOAT-Bench and LangMap. Blue marks semantic errors and green marks ambiguous descriptions that match multiple objects. Our human-verified descriptions provide correct, fine-grained semantics and uniquely refer to the target instance in the scene.

Figure 10 : Our annotation comparison viewer for qualitative analysis. It displays the target and same-category distractors to assess whether each description uniquely identifies the target.

Figure 11 : Failure cases of PlaNaVid. Green boxes mark ground-truth targets and red boxes mark distractors at the stop location.

Method	SeqSR@1	SeqSR@2	SeqSR@3	SeqSR@4	SeqSR@5
PSL	0.0	0.0	0.0	0.0	0.0
SenseAct-M	6.0	1.0	0.1	0.1	0.1
3D-Mem-3B	6.9	1.1	0.3	0.0	0.0
3D-Mem-7B	12.4	5.1	1.9	0.8	0.1
MTU3D	25.0	11.0	5.4	2.5	1.1
Uni-NaVid	27.1	10.4	3.5	1.4	0.6

Appendix

Table 8: SeqSR@1–5 Results on LangMap.

	$N_{\max}$	$K$	SR	SeqSR@2	SPL
Latest Memory	50	10	36.4	11.4	14.9
Full Memory	656	10	40.4	13.2	16.9
Ours	25	10	41.5	13.8	17.3
	50	5	40.0	12.2	17.0
	50	10	42.8	14.6	18.1
	75	10	42.3	13.8	17.7

Appendix

Table 9: Ablation of

N_{\max}

and

K

. Ours uses Bounded Diverse Memory.

Config	Method	SR $\uparrow$	SeqSR@2 $\uparrow$	SPL $\uparrow$
RGB-D,Mask	3D-Mem-3B	13.5 [11.8, 15.2]	1.1 [0.4, 1.9]	6.4 [4.9, 8.1]
	3D-Mem-7B	30.1 [26.6, 33.7]	5.1 [3.1, 7.4]	17.3 [15.0, 19.8]
	MTU3D	41.2 [37.8, 44.7]	11.0 [8.1, 14.0]	24.3 [21.9, 26.7]
	MTU3D ∗	38.0 [34.5, 41.4]	7.9 [5.4, 10.6]	21.0 [18.8, 23.1]
RGB	Uni-NaVid	34.4 [31.0, 37.6]	10.4 [7.9, 12.9]	15.0 [13.3, 16.7]
	PlaNaVid-3B	41.5 [37.9, 44.9]	14.4 [11.1, 17.9]	17.5 [15.5, 19.5]

Appendix

Table 10: Multi-goal results (%) with 95% scene-cluster bootstrap CIs. ∗ : MTU3D with a horizontal field of view matched to PlaNaVid.

Figure 12 : Example of sequential multi-goal navigation. The colored paths denote different goals.

Explore similar work

LH-AVLN: A Benchmark for Long-Horizon Audio-Visual-Language Navigation
Jul 4, 2026Rufeng Chen, Yue Chang, Zili Shao +4Vision-Language Navigation
The Hong Kong University of Science and Technology (Guangzhou) · Jilin University
Instance-Enriched Semantic Maps for Visual Language Navigation
Jul 14, 2026Jiho Hong, Eunae Kang, Sanghyun Kim +1Vision-Language Navigation Semantic Mapping
Department of Mechanical Engineering, Kyung Hee University, Yongin-si, 17104, Republic of Korea · Advanced Institutes of Convergence Technology (AICT), Suwon, 16229, Republic of Korea · School of Mechanical Engineering, Kyungpook National University, Daegu, 41566, Republic of Korea
AnchorVLN: Geometry-Anchored Vision-Language Grounding Reasoning for Open-Vocabulary Navigation
Sep 14, 2026Long Giang Vu, Chengkai Yao, Yuxin Liu +2Vision-Language Navigation Navigation
*These authors contributed equally.

Jul 4, 2026, cs.ROJ/K move · Enter open · S save

LH-AVLN: A Benchmark for Long-Horizon Audio-Visual-Language Navigation

Rufeng Chen, Yue Chang, Zili Shao, Zhaofan Zhang+3

The Hong Kong University of Science and Technology (Guangzhou) · Jilin University

Vision-Language Navigation

Open paper AskPDF arXiv

Embodied navigation is moving toward long-horizon missions, yet existing long-horizon benchmarks are largely acoustically silent, and audio-visual navigation tasks typically focus on a single goal. We introduce LH-AVLN, a benchmark for Long-Horizon Audio-Visual-Language Navigation that combines multi-goal mission execution, heterogeneous goal specifications, and persistent spatialized acoustic cues. In LH-AVLN, an agent receives a global mission of two to four goals specified by category, language description, or reference image, and navigates with RGB-D observations, pose, and binaural audio in indoor 3D environments. The benchmark supports both ordered and unordered missions, where alternating goal-associated sounds can guide non-line-of-sight search but may also become distractors as mission progress changes. We further develop PAG-Nav, a training-free reference agent that maintains a temporal uniform semantic map and performs progressive goal-state planning, using sound for search while reserving completion for visual-semantic verification. Experiments show that existing vision-language, memory-based, and audio-visual agents struggle to complete full LH-AVLN missions, and that PAG-Nav provides a stronger diagnostic baseline while leaving substantial room for future progress.

Abstract

Explore similar work

LH-AVLN: A Benchmark for Long-Horizon Audio-Visual-Language Navigation

Instance-Enriched Semantic Maps for Visual Language Navigation

AnchorVLN: Geometry-Anchored Vision-Language Grounding Reasoning for Open-Vocabulary Navigation