Large vision-language models (LVLMs) have shown strong potential for embodied vision-language navigation (VLN), yet their performance degrades on long-horizon trajectories due to context-length limits, redundant historical observations, and unstable spatial alignment across time. These issues make it difficult to both compress long navigation histories and reliably reuse past spatial experience. To address these limitations, we propose MemoryVLN, a unified memory-driven navigation system built around memory construction and management. During memory construction, Long Sequence Compression employs forgetting-curve-inspired temporal sampling to preserve dense recent evidence and sparse yet representative distant observations under a limited token budget. Ego-Aware Memory Representation further integrates 2D semantic tokens, 3D geometric tokens, and 6D agent poses, allowing each memory to encode both what was observed and where it was observed. During memory management, Hierarchical Memory Bank maintains in-episode and context-level memories through cross-context warm-start and confidence-controlled write-back. Memory Enhancement then retrieves the Top-K relevant memories according to the current navigation state and fuses them with the current visual tokens. Together, these mechanisms transform raw navigation history into compact, spatially grounded, selectively retrievable, and reusable navigation experience. Extensive experiments on R2R-CE and RxR-CE demonstrate consistent improvements over the state-of-the-art baselines by a substantial margin (i.e., 2% and 6.5% in successful rate).
Figures & tables
Figure 2: Detailed pipeline of MemoryVLN. (a) Long Sequence Compression performs inter-chunk budget allocation and intra-chunk key-frame selection. (b) Ego-Aware Visuospatial Representation integrates ViT-based 2D tokens, VGGT-based 3D tokens, and 6D agent poses. (c) Memory Bank maintains context-level and in-episode memories through warm start, write-back, and short-/long-term memory updates. (d) Memory Enhancement retrieves Top- K relevant memories and residually fuse them with the current-frame tokens for action prediction.
Method
Observation
R2R Val-Unseen
RxR Val-Unseen
Pano.
Odo.
Depth
S.RGB
NE ↓
OS ↑
SR ↑
SPL ↑
NE ↓
SR ↑
SPL ↑
nDTW ↑
HPN+DN ∗ [ 24 ]
✓
✓
✓
–
6.31
40.0
36.0
34.0
–
–
–
–
CMA ∗ [ 27 ]
✓
✓
✓
–
6.20
52.0
41.0
36.0
8.76
26.5
22.1
47.0
Sim2Sim ∗ [ 26 ]
✓
✓
✓
–
6.07
52.0
43.0
36.0
–
–
–
–
VLN-BERT ∗ [ 13 ]
✓
✓
✓
–
5.74
53.0
44.0
39.0
8.98
27.0
22.6
46.7
Ego 2 -Map ∗ [ 41 ]
✓
✓
✓
–
5.54
56.0
47.0
41.0
–
–
–
–
Table 1: Comparison with SOTA methods on R2R-CE and RxR-CE val-unseen split. ∗ denotes methods using a waypoint predictor, while † denotes models trained only with VLN-CE expert trajectories without additional navigation data. Pano, Odo, Depth, and S.RGB respectively represent panoramic view, odometry, depth, and single RGB.
R2R-CE
RxR-CE
Variant
NE ↓
SR ↑
SPL ↑
NE ↓
SR ↑
SPL ↑
w/o Long-Seq. Comp.
OOM
OOM
w/o Pos-Encoding
3.66
52.75
49.25
3.35
57.24
48.74
w/o In-Episode
3.57
53.34
49.06
3.30
56.69
48.13
w/o Context
3.55
54.16
49.65
3.29
57.37
49.09
Full MemoryVLN
3.52
54.76
50.26
3.33
57.84
50.42
Table 2: Component ablations on the R2R-CE and RxR-CE val-unseen splits.
Figure 3: Qualitative evaluation in Habitat simulation and a real indoor environment. Top: JanusVLN gradually deviates from the instructed route and fails, whereas MemoryVLN preserves historical and spatial context and reaches the target. Bottom: MemoryVLN successfully executes two real-world instructions with different routes and target locations.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Component
Value
Role
Action backbone
Qwen2.5-VL
Language-conditioned action prediction.
Geometry backbone
frozen VGGT
Produces geometric tokens; no VGGT parameters are updated.
Training / evaluation seed
42
Training and evaluation use this seed; evaluation seeds Python, NumPy, PyTorch CPU, and all CUDA devices.
Reported training runs
1
All reported results come from one training run with seed 42; no across-run variation is reported.
8 ml.hpcpni2.28xlarge instances; each has 112 vCPUs, 1960 GiB host memory, and 2.5.1-cu12.4.1-py3.9-ubuntu22.04 .
Appendix
Table 3: Implementation values used for the reported results. The submitted config.json records the model and memory settings; an evaluation command may explicitly override these values.
Figure 4: Selected R2R val-unseen comparisons for SR and SPL as functions of training data. The dashed line marks 46.4% training data, the last data-matched comparison point; the reported-final point to its right is marked with an asterisk because it is not training-data matched. Annotations report MemoryVLN minus JanusVLN in percentage points.
Dataset
Metric
N
JanusVLN
MemoryVLN
Δ (pp)
95% CI
Test
p
R2R
SR
1,839
52.8
54.8
+2.0
[-0.7, 4.6]
exact McNemar
0.158
R2R
SPL
1,839
49.0
50.3
+1.2
[-1.3, 3.7]
Wilcoxon signed-rank
0.327
RxR
SR
3,669
51.0
57.2
+6.2
[4.5, 8.0]
exact McNemar
2.71×10−12
RxR
SPL
3,570
43.5
48.7
+5.2
[3.8, 6.7]
Wilcoxon signed-rank
1.27×10−12
Appendix
Table 4: Paired significance tests on the same evaluation episodes. Differences are MemoryVLN minus JanusVLN in percentage points; bootstrap intervals quantify episode-level uncertainty and do not represent across-run variability. For RxR SPL, 99 pairs with non-finite values are excluded, leaving 3,570 valid episode pairs.
Figure 5: Representative MemoryVLN failure trajectories. From top to bottom, the rows illustrate room-level branch confusion, ordinal-sensitive final-doorway grounding, a landmark-turn error not corrected by context memory, and long-instruction progress loss. Each row pairs egocentric observations with a top-down map showing the goal, shortest path, and executed agent trajectory.
Dataset
Scene / Episode
Steps
NE
Oracle NE
Coll.
Failure type
R2R
QUCTc6BB5sX / 577
78
28.61
14.13
0
Semantic branch confusion
R2R
x8F5xyUWy9e / 231
60
10.76
6.08
3
Final-doorway grounding
R2R
EU6Fwq7SyZv / 1206
56
10.28
5.82
0
Failed-context reuse
RxR
2azQ1b91cZZ / 5691
44
16.40
12.49
3
Progress loss / early stop
Appendix
Table 5: Four representative failure cases in the same top-to-bottom order as Fig. 5 . All four trajectories have zero Success, SPL, and Oracle Success.
Figure 6: Qualitative comparison between JanusVLN and MemoryVLN on four Habitat episodes under identical instructions. Each example combines representative egocentric observations with top-down trajectories, where the yellow star marks the goal, the green curve marks the shortest path, and the blue curve marks the executed agent trajectory. JanusVLN fails in these examples, whereas MemoryVLN reaches the target region and stops successfully.
Figure 7: Qualitative real-world navigation results for five natural-language instructions using a Unitree GO2 quadruped in an indoor office environment. Navigation-policy inference is performed on a single NVIDIA GeForce RTX 4090 GPU. The egocentric sequences show the robot following corridor and doorway transitions, passing the specified landmarks, and stopping at the requested destination. The targets include a printer, a coffee machine, a designated room, a water dispenser, and a conference-room podium.
Institute of Automation, Chinese Academy of Sciences, Beijing, China · Southern University of Science and Technology, Shenzhen, China · Pengcheng Laboratory, Shenzhen, China +3