Lifelong Embodied Navigation Learning

Xudong Wang; Jiahua Dong; Baichen Liu; Qi Lyu; Lianqing Liu; Zhi Han

cs.ROMar 6, 2026

Lifelong Embodied Navigation Learning

Authors: Xudong Wang, Jiahua Dong, Baichen Liu, Qi Lyu, Lianqing Liu, Zhi Han

Organizations: State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Mohamed bin Zayed University of Artificial Intelligence

Abstract

Embodied navigation agents powered by large language models have shown strong performance on individual tasks but struggle to continually acquire new navigation skills, which suffer from catastrophic forgetting. We formalize this challenge as lifelong embodied navigation learning (LENL), where an agent is required to adapt to a sequence of navigation tasks spanning multiple scenes and diverse user instruction styles, while retaining previously learned knowledge. To tackle this problem, we propose Uni-Walker, a lifelong embodied navigation framework that decouples navigation knowledge into task-shared and task-specific components with Decoder Extension LoRA (DE-LoRA). To learn the shared knowledge, we design a knowledge inheritance strategy and an experts co-activation strategy to facilitate shared knowledge transfer and refinement across multiple navigation tasks. To learn the specific knowledge, we propose an expert subspace orthogonality constraint together and a navigation-specific chain-of-thought reasoning mechanism to capture specific knowledge and enhance instruction-style understanding. Extensive experiments demonstrate the superiority of Uni-Walker for building universal navigation agents with lifelong learning.

Figures & tables

Figure 1: Illustration of the proposed lifelong embodied navigation learning (LENL) task. The proposed Uni-Walker is able to evolve and continually learn multiple new navigation tasks based on learned navigation knowledge, for developing universal embodied navigation. In lifelong learning, the sequential navigation tasks include multi-scenes and multi-instruction styles (VLN, OLN, DUN).

Figure 2: Illustration of the LENL performance. (a) The catastrophic forgetting phenomenon under the LENL settings. (b) Our proposed Uni-Walker has a better anti-forgetting performance.

Figure 3: Illustration of the proposed Uni-Walker pipeline. It includes (a) a Decoder Extension LoRA Adaptation to achieve progressive knowledge decoupled learning, which decouples navigation knowledge into shared and specific parts, thereby facilitating new tasks learning using shared knowledge while avoiding forgetting. (b) a Task-Aware Knowledge Aggregation to automatically aggregate the learned knowledge according to a specific navigation task for task specific inference.

Figure 4: Illustration of the Navigation Chain-of-Thought to design various specific LLM chains of thought for specific instruction style tasks to facilitate the embodied navigation performance.

Figure 5: Illustration of the lifelong navigation benchmark. We establish a total of 18 navigation tasks for lifelong learning, including 18 navigation scenes and 3 types of user instruction styles (VLN is marked in blue color, OLN is green, and DUN is purple). We use the first 15 tasks for lifelong learning, and the last 3 tasks are used to evaluate unseen scene generalization performance.

Comparison Methods	S1	S2	S3	S4	S5	S6	S7	S8	S9	S10	S11	S12	S13	S14	S15	S16	S17	S18	Avg.
Seq-FT (sequential fine-tuning)	0	0	0	5	4	0	5	18	8	9	8	21	20	25	86	0	0	0	12
LwF-LoRA ( Li and Hoiem (2017) )	0	5	0	5	6	0	5	22	8	12	10	30	45	36	85	5	0	0	15
EWC-LoRA ( Xiang et al. (2023) )	0	5	0	10	4	0	5	20	8	9	8	31	49	40	86	5	5	0	16
Dense MoLE ( Chen et al. (2024) )	10	10	4	10	8	3	10	18	16	19	17	30	56	51	85	12	10	8	21
Sparse MoLE ( Dou et al. (2024) )	15	12	0	17	6	6	16	18	18	24	15	32	58	53	84	16	12	10	23
MoLA ( Gao et al. (2024) )	15	12	0	12	6	10	20	18	13	26	19	34	61	59	86	20	12	11	24

Table 1: Test results (task-wise SR

\uparrow

, %) of comparison experiment with our LENL settings.

Comparison Methods	S1	S2	S3	S4	S5	S6	S7	S8	S9	S10	S11	S12	S13	S14	S16	S17	S18	Avg.
Seq-FT (sequential fine-tuning)	100	100	100	94	93	100	93	78	90	88	88	61	77	62	100	100	100	85
LwF-LoRA ( Li and Hoiem (2017) )	100	93	100	94	90	100	93	74	90	84	84	44	49	45	94	100	100	80
EWC-LoRA ( Xiang et al. (2023) )	100	93	100	87	93	100	93	76	90	88	88	43	44	38	94	92	100	79
Dense MoLE ( Chen et al. (2024) )	86	85	95	87	87	95	86	79	81	74	73	44	36	22	86	84	85	71
Sparse MoLE ( Dou et al. (2024) )	78	82	100	79	90	89	77	79	79	68	77	41	34	18	81	81	82	69
MoLA ( Gao et al. (2024) )	78	82	100	85	90	82	71	79	85	65	70	37	31	9	76	82	80	67

Table 2: Test results (task-wise SR-F

\downarrow

, %) of comparison experiment with our LENL settings.

Comparisons	S16	S17	S18	Avg
HydraLoRA	18	14	16	16.0
BranchLoRA	28	20	15	21.0
O-LoRA + TAKA	65	53	36	51.3
SD-LoRA + TAKA	68	55	48	57.0
Uni-Walker (ours)	74	61	51	62.0

Table 3: Results (SR%) for Generalization.

Methods	KIS	ECAS	SSC	SR $\uparrow$	SR-F $\downarrow$	SPL $\uparrow$	SPL-F $\downarrow$	OSR $\uparrow$	OSR-F $\downarrow$
Baseline	✗	✗	✗	55.7	21.1	37.0	45.0	76.7	8.7
Uni-Walker w/o KIS	✗	✓	✓	60.3	14.2	50.2	23.9	77.6	7.7
Uni-Walker w/o SSC	✓	✓	✗	59.7	15.1	44.7	30.6	77.9	7.3
Uni-Walker w/o ECAS	✓	✗	✓	58.1	17.4	44.7	32.3	78.3	6.9
Our Uni-Walker	✓	✓	✓	67.3	4.3	62.3	5.7	81.3	3.5

Table 4: Ablation study results (%) for the methods with Task Shared Knowledge Exploration .

Methods	ESOC	NSCoT	SR $\uparrow$	SR-F $\downarrow$	SPL $\uparrow$	SPL-F $\downarrow$	OSR $\uparrow$	OSR-F $\downarrow$
Baseline	✗	✗	49.0	29.2	33.9	45.0	72.3	14.0
Uni-Walker w/o ESOC	✗	✓	63.5	9.8	60.6	8.2	79.7	5.3
Uni-Walker w/o NSCoT	✓	✗	51.1	27.3	35.5	46.3	75.3	10.5
Our Uni-Walker	✓	✓	67.3	4.3	62.3	5.7	81.3	3.5

Table 5: Ablation study results (%) for the methods with Task Specific Knowledge Exploration .

Methods	IM	OM	MM	SR $\uparrow$	SR-F $\downarrow$	SPL $\uparrow$	SPL-F $\downarrow$	OSR $\uparrow$	OSR-F $\downarrow$
Uni-Walker w IM	✓	✗	✗	35.0	50.1	23.2	65.0	46.6	49.5
Uni-Walker w OM	✗	✓	✗	65.1	9.6	62.7	7.5	80.1	5.5
Our Uni-Walker	✗	✗	✓	67.3	4.3	62.3	5.7	81.3	3.5

Table 6: Ablation study results (%) for the Task-Aware Knowledge Aggregation methods.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Task ID	Type	Scene ID	INST Style	Train INST Number	Test INST Number
S1	Continual Learning	1LXtFkjw3qL	VLN	801	99
S2	Continual Learning	1pXnuDYAj8r	VLN	813	90
S3	Continual Learning	pRbA3pwrgk9	OLN	336	195
S4	Continual Learning	5q7pvUzZiYa	VLN	738	63
S5	Continual Learning	759xd9YjKW5	OLN	693	147
S6	Continual Learning	VVfe2KiqLaN	DUN	255	24

Appendix

Table 7: Statistics of the 18 sequential tasks of LENL benchmark. Each task corresponds to a unique scene under one instruction style.

Hyper-parameter	Value
TOP-K activated expert	$K=2$
LoRA rank	$r=16$
balance hyperparameter of $\mathcal{L}_{ssc,t}$	$\lambda_{ssc}=0.1$
balance hyperparameter of $\mathcal{L}_{esoc,t}$	$\lambda_{esoc}=0.1$
perturbation for avoiding the computational avalanches	$\epsilon=0.01$
Fisher smoothing	$\omega=0.9$

Appendix

Table 8: Summary of primary hyper-parameters used in experiments.

Figure 7: Illustration of three visualization examples of VLN, OLN, DUN. We visualize each step of the Uni-Walker’s selection process and annotate it with arrows to clearly illustrate the navigation sequence.

Comparison Methods	S1	S2	S3	S4	S5	S6	S7	S8	S9	S10	S11	S12	S13	S14	S15	S16	S17	S18	Avg.
Seq-FT (sequential fine-tuning)	0	0	0	2	2	0	2	10	6	6	8	15	10	12	79	0	0	0	8
LwF-LoRA ( Li and Hoiem (2017) )	0	0	0	0	2	0	2	11	5	4	6	21	36	24	80	0	0	0	11
EWC-LoRA ( Xiang et al. (2023) )	0	2	0	4	0	0	0	10	6	2	4	25	32	30	80	0	0	0	11
Dense MoLE ( Chen et al. (2024) )	2	2	2	4	4	0	2	10	8	12	10	20	36	40	78	2	0	2	13
Sparse MoLE ( Dou et al. (2024) )	2	2	0	10	2	0	10	13	12	15	10	24	34	42	78	0	2	0	14
MoLA ( Gao et al. (2024) )	4	6	0	6	2	5	12	12	13	11	15	23	36	44	80	6	6	0	16

Appendix

Table 9: Test results (task-wise SPL

\uparrow

, %) of comparison experiment with our LENL settings.

Comparison Methods	S1	S2	S3	S4	S5	S6	S7	S8	S9	S10	S11	S12	S13	S14	S16	S17	S18	Avg.
Seq-FT (sequential fine-tuning)	100	100	100	97	96	100	97	88	88	92	86	72	88	79	100	100	100	88
LwF-LoRA ( Li and Hoiem (2017) )	100	100	100	100	96	100	97	87	90	95	90	61	57	58	100	100	100	85
EWC-LoRA ( Xiang et al. (2023) )	100	97	100	94	100	100	100	88	88	97	93	54	61	47	100	100	100	84
Dense MoLE ( Chen et al. (2024) )	97	97	97	94	92	100	97	88	85	84	83	63	57	30	98	100	96	81
Sparse MoLE ( Dou et al. (2024) )	97	97	100	85	96	100	85	85	77	79	83	56	60	26	100	97	100	79
MoLA ( Gao et al. (2024) )	94	91	100	91	96	90	82	86	75	85	75	57	57	24	93	91	100	77

Appendix

Table 10: Test results (task-wise SPL-F

\downarrow

, %) of comparison experiment with our LENL settings.

Comparison Methods	S1	S2	S3	S4	S5	S6	S7	S8	S9	S10	S11	S12	S13	S14	S15	S16	S17	S18	Avg.
Seq-FT (sequential fine-tuning)	12	12	14	16	18	16	17	19	16	18	15	35	36	36	100	16	15	15	24
LwF-LoRA ( Li and Hoiem (2017) )	12	15	16	15	13	15	16	29	16	32	32	45	55	51	100	13	15	16	28
EWC-LoRA ( Xiang et al. (2023) )	13	15	16	15	15	16	15	29	15	34	36	46	56	52	100	16	18	17	29
Dense MoLE ( Chen et al. (2024) )	21	22	18	23	16	18	19	31	16	33	41	48	69	68	100	22	20	17	33
Sparse MoLE ( Dou et al. (2024) )	23	23	19	25	19	21	23	36	19	26	40	47	69	72	100	25	26	18	35
MoLA ( Gao et al. (2024) )	26	24	16	24	19	23	25	37	21	29	45	48	69	73	100	28	22	26	36

Appendix

Table 11: Test results (task-wise OSR

\uparrow

, %) of comparison experiment with our LENL settings.

Comparison Methods	S1	S2	S3	S4	S5	S6	S7	S8	S9	S10	S11	S12	S13	S14	S16	S17	S18	Avg.
Seq-FT (sequential fine-tuning)	86	87	84	77	80	83	77	78	75	76	77	59	64	58	83	83	83	73
LwF-LoRA ( Li and Hoiem (2017) )	86	83	82	79	86	84	80	66	75	57	54	48	45	40	86	83	81	67
EWC-LoRA ( Xiang et al. (2023) )	85	83	82	79	84	83	81	66	77	55	48	47	44	39	83	79	80	66
Dense MoLE ( Chen et al. (2024) )	75	75	80	67	83	80	76	64	75	56	41	44	31	20	77	77	80	61
Sparse MoLE ( Dou et al. (2024) )	73	74	79	64	79	77	71	58	71	65	42	45	31	15	74	70	79	59
MoLA ( Gao et al. (2024) )	69	73	82	66	79	75	68	56	68	61	35	44	31	14	71	74	70	58

Appendix

Table 12: Test results (task-wise OSR-F

\downarrow

, %) of comparison experiment with our LENL settings.

Methods	S1	S2	S3	S4	S5	S6	S7	S8	S9	S10	S11	S12	S13	S14	S15	Avg.
Baseline	50	56	46	51	39	38	51	71	46	60	51	48	82	60	86	56
Uni-Walker w/o KIS	61	60	51	55	43	46	56	74	48	68	59	49	86	63	86	60
Uni-Walker w/o SSC	60	59	49	56	42	45	56	75	48	69	58	47	84	62	86	60
Uni-Walker w/o ECAS	59	58	48	53	42	45	55	72	46	62	55	46	82	62	86	58
Uni-Walker (ours)	67	67	75	67	57	50	67	83	50	73	62	53	88	65	86	67

Appendix

Table 13: Ablation study results ( SR

\uparrow

, %) for the methods with task-shared navigation knowledge exploration.

Methods	S1	S2	S3	S4	S5	S6	S7	S8	S9	S10	S11	S12	S13	S14	Avg.
Baseline	29	20	42	36	35	27	27	16	12	20	22	14	9	8	21
Uni-Walker w/o KIS	13	14	35	31	28	12	20	13	8	9	9	13	4	3	14
Uni-Walker w/o SSC	14	16	38	30	30	13	20	12	8	8	11	16	7	5	15
Uni-Walker w/o ECAS	16	17	39	34	30	13	21	15	12	17	15	18	9	5	17
Uni-Walker (ours)	4	4	5	16	5	4	4	2	4	3	5	5	2	0	4

Appendix

Table 14: Ablation study results ( F-SR

\downarrow

, %) for the methods with task-shared navigation knowledge exploration.

Methods	S1	S2	S3	S4	S5	S6	S7	S8	S9	S10	S11	S12	S13	S14	S15	Avg.
Baseline	29	30	30	42	25	23	32	35	24	33	34	25	65	48	80	37
Uni-Walker w/o KIS	36	37	38	49	33	35	48	75	51	46	46	45	78	56	80	50
Uni-Walker w/o SSC	36	36	37	48	33	35	46	43	36	43	45	42	76	51	80	46
Uni-Walker w/o ECAS	35	36	35	49	32	32	45	43	35	42	42	40	75	49	80	45
Uni-Walker (ours)	66	63	59	61	49	50	65	74	50	72	56	51	82	56	80	62

Appendix

Table 15: Ablation study results ( SPL

\uparrow

, %) for the methods with task-shared navigation knowledge exploration.

Methods	S1	S2	S3	S4	S5	S6	S7	S8	S9	S10	S11	S12	S13	S14	Avg.
Baseline	57	54	52	35	52	56	54	59	55	56	42	56	24	17	45
Uni-Walker w/o KIS	47	43	39	25	37	33	30	12	4	39	22	21	9	3	24
Uni-Walker w/o SSC	47	45	40	26	37	33	33	49	32	43	24	26	12	12	31
Uni-Walker w/o ECAS	49	45	44	25	38	38	35	49	34	44	29	30	13	16	32
Uni-Walker (ours)	3	3	5	6	6	4	6	13	6	4	5	11	5	3	6

Appendix

Table 16: Ablation study results ( F-SPL

\downarrow

, %) for the methods with task-shared navigation knowledge exploration.

Methods	S1	S2	S3	S4	S5	S6	S7	S8	S9	S10	S11	S12	S13	S14	S15	Avg.
Baseline	75	80	79	63	81	81	70	82	62	70	61	80	91	75	100	77
Uni-Walker w/o KIS	77	82	78	65	82	88	71	82	62	70	61	80	91	75	100	78
Uni-Walker w/o SSC	78	81	79	66	82	89	71	82	61	71	62	80	91	75	100	78
Uni-Walker w/o ECAS	77	81	79	66	81	88	72	82	62	71	61	81	95	78	100	78
Uni-Walker (ours)	82	87	88	67	90	90	72	83	63	73	62	84	100	78	100	81

Appendix

Table 17: Ablation study results ( OSR

\uparrow

, %) for the methods with task-shared navigation knowledge exploration.

Methods	S1	S2	S3	S4	S5	S6	S7	S8	S9	S10	S11	S12	S13	S14	Avg.
Baseline	12	10	12	10	12	12	10	4	9	7	6	7	9	12	9
Uni-Walker w/o KIS	9	8	13	7	11	4	9	4	9	7	6	7	9	12	8
Uni-Walker w/o SSC	8	9	12	6	11	3	9	4	10	5	5	7	9	12	7
Uni-Walker w/o ECAS	9	9	12	6	12	4	8	4	9	5	6	6	5	8	7
Uni-Walker (ours)	4	2	2	4	2	2	8	2	7	3	5	2	0	8	3

Appendix

Table 18: Ablation study results ( OSR-F

\downarrow

, %) for the methods with task-shared navigation knowledge exploration.

Methods	S1	S2	S3	S4	S5	S6	S7	S8	S9	S10	S11	S12	S13	S14	S15	Avg.
Baseline	45	50	42	46	42	32	42	60	41	52	48	41	76	59	71	50
Uni-Walker w/o ESOC	65	62	70	61	54	45	59	78	48	69	59	50	85	63	85	64
Uni-Walker w/o NSCoT	46	51	43	46	43	33	45	61	42	55	49	44	77	60	71	51
Uni-Walker (ours)	67	67	75	67	57	50	67	83	50	73	62	53	88	65	86	67

Appendix

Table 19: Ablation study results ( SR

\uparrow

, %) for the methods with task-specific navigation knowledge exploration.

Methods	S1	S2	S3	S4	S5	S6	S7	S8	S9	S10	S11	S12	S13	S14	S15	Avg.
Baseline	36	29	47	43	30	38	40	29	21	31	26	27	16	9	17	29
Uni-Walker w/o ESOC	7	11	11	24	10	13	16	8	8	8	9	11	6	3	1	10
Uni-Walker w/o NSCoT	34	27	46	43	28	37	36	28	19	27	25	21	14	8	17	27
Uni-Walker (ours)	4	4	5	16	5	4	4	2	4	3	5	5	2	0	0	4

Appendix

Table 20: Ablation study results ( F-SR

\downarrow

, %) for the methods with task-specific navigation knowledge exploration.

Methods	S1	S2	S3	S4	S5	S6	S7	S8	S9	S10	S11	S12	S13	S14	S15	Avg.
Baseline	27	28	29	41	22	20	30	31	22	31	30	22	60	40	75	34
Uni-Walker w/o ESOC	65	61	55	60	45	49	64	73	48	70	55	50	81	55	78	61
Uni-Walker w/o NSCoT	29	31	31	42	23	21	32	33	25	33	31	23	61	41	76	35
Uni-Walker (ours)	66	63	59	61	49	50	65	74	50	72	56	51	82	56	80	62

Appendix

Table 21: Ablation study results ( SPL

\uparrow

, %) for the methods with task-specific navigation knowledge exploration.

Methods	S1	S2	S3	S4	S5	S6	S7	S8	S9	S10	S11	S12	S13	S14	S15	Avg.
Baseline	60	57	53	37	58	62	57	64	58	59	49	61	30	31	6	45
Uni-Walker w/o ESOC	4	6	11	8	13	6	7	14	9	7	7	12	6	5	3	8
Uni-Walker w/o NSCoT	57	52	50	35	56	60	54	61	53	56	47	60	29	29	5	46
Uni-Walker (ours)	3	3	5	6	6	4	6	13	6	4	5	11	5	3	0	6

Appendix

Table 22: Ablation study results ( F-SPL

\downarrow

, %) for the methods with task-specific navigation knowledge exploration.

Methods	S1	S2	S3	S4	S5	S6	S7	S8	S9	S10	S11	S12	S13	S14	S15	Avg.
Baseline	65	71	70	53	77	80	66	81	60	70	59	77	85	71	100	72
Uni-Walker w/o ESOC	80	86	79	66	89	89	71	82	63	70	62	84	100	75	100	80
Uni-Walker w/o NSCoT	67	73	72	55	80	82	67	82	62	70	61	81	100	78	100	75
Uni-Walker (ours)	82	87	88	67	90	90	72	83	63	73	62	84	100	78	100	81

Appendix

Table 23: Ablation study results ( OSR

\uparrow

, %) for the methods with task-specific navigation knowledge exploration.

Methods	S1	S2	S3	S4	S5	S6	S7	S8	S9	S10	S11	S12	S13	S14	Avg.
Baseline	24	20	22	24	16	13	15	5	12	7	9	10	15	16	14
Uni-Walker w/o ESOC	6	3	12	6	3	3	9	4	7	7	5	2	0	12	5
Uni-Walker w/o NSCoT	21	18	20	21	13	11	14	4	9	7	6	6	0	8	11
Uni-Walker (ours)	4	2	2	4	2	2	8	2	7	3	5	2	0	8	3

Appendix

Table 24: Ablation study results ( F-OSR

\downarrow

, %) for the methods with task-specific navigation knowledge exploration.

Comparison Methods	Avg SR $\uparrow$ (%)	Avg SPL $\uparrow$ (%)	Avg OSR $\uparrow$ (%)
NaviLLM Zheng et al. (2024)	50	39	58
ScaleVLN Wang et al. (2023b)	53	42	61
SAME Zhou et al. (2024)	55	45	62
Uni-Walker	66	61	81

Appendix

Table 25: Comparison of the other state-of-the-art universal navigation agents: averaged metrics on the 18-task with our LENL settings.

Embodied Navigation

Embodied navigation studies how robots perceive, map, and move through environments to reach goals and follow instructions.

See all 131 papers

Sep 28, 2026

NavHarness: Towards Lifelong Embodied Navigation

Frontier models can now perform well on individual embodied navigation tasks through multi-round multimodal reasoning with simple tools. Across successive tasks, however, an agent must also rely on an evolving map and earlier search records, both of which may be incomplete or conflict with new observations. We present NavHarness, a training-free embodied harness towards lifelong navigation that makes memory processing part of the navigation loop. During navigation, its multi-round agentic session draws on maps, task records, and house knowledge, checking them against observations and recording corrections to guide its actions. NavHarness preserves this experience across fresh conversations for new tasks or recovery attempts, while outcome verification and run-end summaries support its later reuse. On GOAT-Bench, NavHarness improves s-SR over context-only independent sessions by 18.6 points with Astra and 22.6 with Opus 5. Using SLAM-estimated poses, NavHarness with GPT-6 Astra achieves state-of-the-art task success of 83.7 s-SR with 36.9 e-SR on GOAT-Bench and 85.9 s-SR on IR2R-CE. To understand these gains, we examine how experience is carried between sessions and find that structured recovery handovers outperform length-matched summaries. In extended deployments across houses, consolidation improves navigation beyond retaining maps and task records, with case studies showing how agents use earlier experience to interpret new goals, investigate unresolved questions, and resume failed searches. We suggest that progress towards lifelong navigation depends on how successive reasoning sessions build on prior experience, alongside improvements in single-task capability.

Xunyi Zhao, Jian Zhou, Sihao Lin +6

Adelaide University · Responsible AI Research Centre, AIML · CSIRO Data61

Embodied Navigation Memory-Augmented Agents LLM Agent Harnesses

Ask PDF

Sep 14, 2026

HarnessVLN: Unifying Training-Free Embodied Navigation through an Agent Harness

Embodied navigation requires agents to ground instructions or object goals in spatial observations and translate plans into successful execution. As multimodal large language models (MLLMs) become increasingly capable, they offer stronger support for navigation without task-specific training; however, improved semantic reasoning alone does not ensure that proposed actions remain consistent with spatial evidence, task progress, and execution outcomes. We introduce HarnessVLN, a zero-shot, training-free framework that unifies instruction-following and object-goal navigation through a shared Agent Harness. The Harness coordinates perception, memory, and execution tools through a unified interface, validating planner proposals for evidential support, geometric feasibility, and subgoal consistency before dispatch. It jointly manages hierarchical event memory and a persistent Spatiotemporal Graph to track task progress, preserve spatial evidence, and contextualize failures. Structured execution feedback updates this shared state, guiding subsequent planning, recovery, and termination. Across R2R, RxR, HM3D-v2, and HM3D-OVON, HarnessVLN achieves success rates of 59.6%, 51.4%, 76.0%, and 59.3%, respectively, outperforming prior training-free methods. Humanoid robot deployment further demonstrates its applicability to both navigation tasks in real-world environments. The project page is available at [https://agibot-harnessvln.netlify.app/].

Yang Chen, Lirong Che, Zhenyu Huang +7

Nanjing University · AGIBOT · Tsinghua University

Embodied Navigation Large Language Model-Based Robot Planning Vision-Language Navigation Object-Goal Navigation

Ask PDF

Apr 21, 2026

LiveVLN: Breaking the Stop-and-Go Loop in Vision-Language Navigation

Recent navigation systems achieve strong benchmark results, yet real-world deployment often remains visibly stop-and-go. This bottleneck arises because the sense-inference-execution loop is still blocking: after each new observation, the controller must wait for sensing, transmission, and inference before motion can continue. Reducing action-generation cost alone therefore does not remove redundant waiting. To address this issue, we present LiveVLN, a training-free framework for more continuous embodied navigation by augmenting pretrained VLM navigators with multi-step action continuation. Instead of pausing for each full sense-and-inference round, LiveVLN overlaps execution with the processing of newly arrived observations, allowing refreshed future actions to be handed off before the current executable prefix is exhausted. This design keeps actions continuously available during motion, reducing idle waiting and enabling smoother online execution. The framework operates at runtime and can be integrated with compatible pretrained VLM navigators. Across R2R and RxR, LiveVLN preserves benchmark performance while reducing waiting time and improving action availability. In real-world deployments, it cuts average episode waiting time by up to

77.7\%

and shortens wall-clock episode time by

12.6\%

on StreamVLN and

19.6\%

on NaVIDA, yielding more coherent execution during deployment. Code is available at https://github.com/NIneeeeeem/LiveVLN.

Xiangchen Wang, Weiye Zhu, Teng Wang +5

Southern University of Science and Technology · Harbin Institute of Technology, Shenzhen · Spatialtemporal AI

Vision-Language Navigation Action Chunking VLMs for Robotics

Ask PDF

Abstract

Embodied Navigation

Explore similar work

NavHarness: Towards Lifelong Embodied Navigation

HarnessVLN: Unifying Training-Free Embodied Navigation through an Agent Harness

LiveVLN: Breaking the Stop-and-Go Loop in Vision-Language Navigation