Organizations: The University of Tokyo Tokyo, Japan · Xi’an Jiaotong University Xi’an, China · Shenzhen University of Advanced Technology Shenzhen, China
Existing feedback-based methods for online vision-and-language navigation (VLN) primarily learn from current episode outcomes, leaving previously acquired feedback underutilized. Inspired by human episodic memory, we propose R2A, an online VLN adaptation framework that continually reuses successful and failed navigation experiences for policy learning. Specifically, Success-Failure Memory (SFM) retains successful and failed navigation episodes with acquired feedback and retrieves task-relevant experiences based on incoming instructions, enabling historical feedback to guide subsequent navigation. Prior-Anchored Adaptation (PAA) learns from these experiences by reinforcing successful trajectories and suppressing failed ones relative to a frozen pretrained policy, while regularizing behavioral drift. Together, these modules enable continual feedback reuse and stable policy adaptation without source-domain data or expert supervision. Extensive experiments across three benchmarks, REVERIE, R2R, and R2R-CE, and five navigation backbones demonstrate consistent performance gains. On the REVERIE validation unseen split, R2A improves DUET's SR from 46.98% to 69.84% and SPL from 33.73% to 50.98%.
Figures & tables
Figure 1: Learning from past successes and failures for navigation. (a) People recall successful experiences and mistakes to guide navigation. (b) Similar visual cues may cause an agent to confuse a kitchen with a laundry room; replaying a past failure helps correct this mistake. Inspired by episodic memory, R 2 A reuses successful and failed experiences with outcome feedback for adaptation.
Figure 2: Overview of R 2 A. (a) The agent navigates using the adapted policy and obtains queried feedback or a self-predicted outcome. (b) Success–Failure Memory: Before navigation, the incoming instruction triggers the retrieval of the most similar experience from the entire memory based on instruction similarity, together with a randomly sampled experience. (c) Prior-Anchored Adaptation : The preceding episode and recalled experiences jointly update the policy through reference-relative outcome learning and pretrained-policy regularization.
Method
FB
Val Seen
Val Unseen
Test Unseen
OSR ↑
SR ↑
SPL ↑
RGSPL ↑
OSR ↑
SR ↑
SPL ↑
RGSPL ↑
OSR ↑
SR ↑
SPL ↑
RGSPL ↑
HAMT [ 5 ]
–
47.65
43.29
40.19
25.18
36.84
32.95
30.20
17.28
33.41
30.40
26.67
13.08
w/ Tent [ 37 ]
✗
46.03
43.43
40.78
25.81
32.60
30.56
28.23
14.48
25.06
23.73
21.78
10.82
w/ FSTTA [ 12 ]
✗
48.21
42.87
39.56
24.58
36.78
32.89
30.51
17.20
33.39
30.39
26.65
13.61
w/ IDEA [ 20 ]
✗
50.67
47.33
42.13
26.82
39.87
34.92
31.52
17.76
38.14
32.81
28.52
14.45
w/ FeedTTA [ 24 ]
✓
62.97
55.80
49.70
31.80
40.73
35.05
31.60
17.83
38.62
34.14
29.07
14.36
Table 1: Comparison with test-time adaptation methods on REVERIE. Bold indicates the best TTA result within each.
Method
Val Seen
Val Unseen
TL ↓
NE ↓
SR ↑
SPL ↑
TL ↓
NE ↓
SR ↑
SPL ↑
ETPNav [ 2 ]
11.78
3.95
66
59
11.99
4.71
57
49
w/ Tent [ 37 ]
11.36
3.94
66
59
11.57
4.74
57
49
w/ FSTTA [ 12 ]
11.35
3.93
66
59
11.57
4.77
57
49
w/ IDEA [ 20 ]
10.87
3.82
68
62
11.17
4.47
59
51
w/ FeedTTA [ 24 ]
10.88
3.85
67
61
11.99
4.47
58
50
Table 2: Comparison with VLN-TTA methods on R2R-CE.
Method
Val Seen
Val Unseen
TL ↓
NE ↓
SR ↑
SPL ↑
TL ↓
NE ↓
SR ↑
SPL ↑
DUET [ 6 ]
12.33
2.28
79
73
13.94
3.31
72
60
w/ Tent [ 37 ]
12.17
2.38
78
72
13.78
3.42
72
60
w/ FSTTA [ 12 ]
13.39
2.25
79
73
14.64
3.03
75
62
w/ IDEA [ 20 ]
11.23
2.03
81
76
12.47
2.91
76
67
w/ FeedTTA [ 24 ]
11.49
2.09
80
75
13.52
2.95
75
65
Table 3: Comparison with VLN-TTA methods on R2R.
Figure 3: Feedback efficiency and online adaptation on REVERIE val-unseen. (a) Performance under different feedback budgets. (b) Comparison of feedback query strategies. (c) Evolution of navigation SR throughout online adaptation.
Figure 4: Analysis of PAA components on REVERIE val-unseen. (a) Navigation performance under different outcome weighting and reference choices. (b) Paired SR differences relative to full PAA.
Figure 5: Qualitative examples of task-cued recall on REVERIE val-unseen. (a) Failure memory: recalling a previously failed navigation experience helps the agent avoid stopping at an incorrect sink. (b) Success memory: recalling a successful experience guides the agent toward the target phone. In both cases, Task-Cued Recall reaches the intended destination, while uniform replay fails.
Variant
REVERIE
R2R
SR ↑
SPL ↑
SR ↑
SPL ↑
DUET (no adaptation)
46.98
33.73
72
60
R2A (full)
69.84±0.70
50.98
77
69
w/o memory
64.49±0.86
48.55
74
66
w/o anchor
56.74±7.34
39.45
73
63
Success-only memory
66.19±0.27
46.51
74
65
Table 4: Effects of feedback replay and prior anchoring. Results use DUET on REVERIE and R2R val-unseen. Success-only memory retains failure feedback in current updates; w/o failure feedback excludes it from both current updates and replay.
Method
REVERIE
R2R
SR ↑
SPL ↑
SR ↑
SPL ↑
Uniform Replay
66.54
48.12
74
65
Global Shuffled-Cue
66.82
48.34
74
65
Scene-Based Recall
67.87
49.91
75
67
TCR w/o Same-Goal
69.26
50.40
76
68
TCR (Ours)
69.84
50.98
77
69
Table 5: Comparison of replay strategies on val unseen.