Lifelong Embodied Navigation Learning
Organizations: State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Mohamed bin Zayed University of Artificial Intelligence
Abstract
Embodied navigation agents powered by large language models have shown strong performance on individual tasks but struggle to continually acquire new navigation skills, which suffer from catastrophic forgetting. We formalize this challenge as lifelong embodied navigation learning (LENL), where an agent is required to adapt to a sequence of navigation tasks spanning multiple scenes and diverse user instruction styles, while retaining previously learned knowledge. To tackle this problem, we propose Uni-Walker, a lifelong embodied navigation framework that decouples navigation knowledge into task-shared and task-specific components with Decoder Extension LoRA (DE-LoRA). To learn the shared knowledge, we design a knowledge inheritance strategy and an experts co-activation strategy to facilitate shared knowledge transfer and refinement across multiple navigation tasks. To learn the specific knowledge, we propose an expert subspace orthogonality constraint together and a navigation-specific chain-of-thought reasoning mechanism to capture specific knowledge and enhance instruction-style understanding. Extensive experiments demonstrate the superiority of Uni-Walker for building universal navigation agents with lifelong learning.
Figures & tables
| Comparison Methods | S1 | S2 | S3 | S4 | S5 | S6 | S7 | S8 | S9 | S10 | S11 | S12 | S13 | S14 | S15 | S16 | S17 | S18 | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Seq-FT (sequential fine-tuning) | 0 | 0 | 0 | 5 | 4 | 0 | 5 | 18 | 8 | 9 | 8 | 21 | 20 | 25 | 86 | 0 | 0 | 0 | 12 |
| LwF-LoRA ( Li and Hoiem (2017) ) | 0 | 5 | 0 | 5 | 6 | 0 | 5 | 22 | 8 | 12 | 10 | 30 | 45 | 36 | 85 | 5 | 0 | 0 | 15 |
| EWC-LoRA ( Xiang et al. (2023) ) | 0 | 5 | 0 | 10 | 4 | 0 | 5 | 20 | 8 | 9 | 8 | 31 | 49 | 40 | 86 | 5 | 5 | 0 | 16 |
| Dense MoLE ( Chen et al. (2024) ) | 10 | 10 | 4 | 10 | 8 | 3 | 10 | 18 | 16 | 19 | 17 | 30 | 56 | 51 | 85 | 12 | 10 | 8 | 21 |
| Sparse MoLE ( Dou et al. (2024) ) | 15 | 12 | 0 | 17 | 6 | 6 | 16 | 18 | 18 | 24 | 15 | 32 | 58 | 53 | 84 | 16 | 12 | 10 | 23 |
| MoLA ( Gao et al. (2024) ) | 15 | 12 | 0 | 12 | 6 | 10 | 20 | 18 | 13 | 26 | 19 | 34 | 61 | 59 | 86 | 20 | 12 | 11 | 24 |
| Comparison Methods | S1 | S2 | S3 | S4 | S5 | S6 | S7 | S8 | S9 | S10 | S11 | S12 | S13 | S14 | S15 | S16 | S17 | S18 | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Seq-FT (sequential fine-tuning) | 100 | 100 | 100 | 94 | 93 | 100 | 93 | 78 | 90 | 88 | 88 | 61 | 77 | 62 | 0 | 100 | 100 | 100 | 85 |
| LwF-LoRA ( Li and Hoiem (2017) ) | 100 | 93 | 100 | 94 | 90 | 100 | 93 | 74 | 90 | 84 | 84 | 44 | 49 | 45 | 0 | 94 | 100 | 100 | 80 |
| EWC-LoRA ( Xiang et al. (2023) ) | 100 | 93 | 100 | 87 | 93 | 100 | 93 | 76 | 90 | 88 | 88 | 43 | 44 | 38 | 0 | 94 | 92 | 100 | 79 |
| Dense MoLE ( Chen et al. (2024) ) | 86 | 85 | 95 | 87 | 87 | 95 | 86 | 79 | 81 | 74 | 73 | 44 | 36 | 22 | 0 | 86 | 84 | 85 | 71 |
| Sparse MoLE ( Dou et al. (2024) ) | 78 | 82 | 100 | 79 | 90 | 89 | 77 | 79 | 79 | 68 | 77 | 41 | 34 | 18 | 0 | 81 | 81 | 82 | 69 |
| MoLA ( Gao et al. (2024) ) | 78 | 82 | 100 | 85 | 90 | 82 | 71 | 79 | 85 | 65 | 70 | 37 | 31 | 9 | 0 | 76 | 82 | 80 | 67 |
| Comparisons | S16 | S17 | S18 | Avg |
|---|---|---|---|---|
| HydraLoRA | 18 | 14 | 16 | 16.0 |
| BranchLoRA | 28 | 20 | 15 | 21.0 |
| O-LoRA + TAKA | 65 | 53 | 36 | 51.3 |
| SD-LoRA + TAKA | 68 | 55 | 48 | 57.0 |
| Uni-Walker (ours) | 74 | 61 | 51 | 62.0 |
| Methods | KIS | ECAS | SSC | SR | SR-F | SPL | SPL-F | OSR | OSR-F |
|---|---|---|---|---|---|---|---|---|---|
| Baseline | ✗ | ✗ | ✗ | 55.7 | 21.1 | 37.0 | 45.0 | 76.7 | 8.7 |
| Uni-Walker w/o KIS | ✗ | ✓ | ✓ | 60.3 | 14.2 | 50.2 | 23.9 | 77.6 | 7.7 |
| Uni-Walker w/o SSC | ✓ | ✓ | ✗ | 59.7 | 15.1 | 44.7 | 30.6 | 77.9 | 7.3 |
| Uni-Walker w/o ECAS | ✓ | ✗ | ✓ | 58.1 | 17.4 | 44.7 | 32.3 | 78.3 | 6.9 |
| Our Uni-Walker | ✓ | ✓ | ✓ | 67.3 | 4.3 | 62.3 | 5.7 | 81.3 | 3.5 |
| Methods | ESOC | NSCoT | SR | SR-F | SPL | SPL-F | OSR | OSR-F |
|---|---|---|---|---|---|---|---|---|
| Baseline | ✗ | ✗ | 49.0 | 29.2 | 33.9 | 45.0 | 72.3 | 14.0 |
| Uni-Walker w/o ESOC | ✗ | ✓ | 63.5 | 9.8 | 60.6 | 8.2 | 79.7 | 5.3 |
| Uni-Walker w/o NSCoT | ✓ | ✗ | 51.1 | 27.3 | 35.5 | 46.3 | 75.3 | 10.5 |
| Our Uni-Walker | ✓ | ✓ | 67.3 | 4.3 | 62.3 | 5.7 | 81.3 | 3.5 |
| Methods | IM | OM | MM | SR | SR-F | SPL | SPL-F | OSR | OSR-F |
|---|---|---|---|---|---|---|---|---|---|
| Uni-Walker w IM | ✓ | ✗ | ✗ | 35.0 | 50.1 | 23.2 | 65.0 | 46.6 | 49.5 |
| Uni-Walker w OM | ✗ | ✓ | ✗ | 65.1 | 9.6 | 62.7 | 7.5 | 80.1 | 5.5 |
| Our Uni-Walker | ✗ | ✗ | ✓ | 67.3 | 4.3 | 62.3 | 5.7 | 81.3 | 3.5 |
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
| Task ID | Type | Scene ID | INST Style | Train INST Number | Test INST Number |
|---|---|---|---|---|---|
| S1 | Continual Learning | 1LXtFkjw3qL | VLN | 801 | 99 |
| S2 | Continual Learning | 1pXnuDYAj8r | VLN | 813 | 90 |
| S3 | Continual Learning | pRbA3pwrgk9 | OLN | 336 | 195 |
| S4 | Continual Learning | 5q7pvUzZiYa | VLN | 738 | 63 |
| S5 | Continual Learning | 759xd9YjKW5 | OLN | 693 | 147 |
| S6 | Continual Learning | VVfe2KiqLaN | DUN | 255 | 24 |
| Hyper-parameter | Value |
|---|---|
| TOP-K activated expert | |
| LoRA rank | |
| balance hyperparameter of | |
| balance hyperparameter of | |
| perturbation for avoiding the computational avalanches | |
| Fisher smoothing |
| Comparison Methods | S1 | S2 | S3 | S4 | S5 | S6 | S7 | S8 | S9 | S10 | S11 | S12 | S13 | S14 | S15 | S16 | S17 | S18 | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Seq-FT (sequential fine-tuning) | 0 | 0 | 0 | 2 | 2 | 0 | 2 | 10 | 6 | 6 | 8 | 15 | 10 | 12 | 79 | 0 | 0 | 0 | 8 |
| LwF-LoRA ( Li and Hoiem (2017) ) | 0 | 0 | 0 | 0 | 2 | 0 | 2 | 11 | 5 | 4 | 6 | 21 | 36 | 24 | 80 | 0 | 0 | 0 | 11 |
| EWC-LoRA ( Xiang et al. (2023) ) | 0 | 2 | 0 | 4 | 0 | 0 | 0 | 10 | 6 | 2 | 4 | 25 | 32 | 30 | 80 | 0 | 0 | 0 | 11 |
| Dense MoLE ( Chen et al. (2024) ) | 2 | 2 | 2 | 4 | 4 | 0 | 2 | 10 | 8 | 12 | 10 | 20 | 36 | 40 | 78 | 2 | 0 | 2 | 13 |
| Sparse MoLE ( Dou et al. (2024) ) | 2 | 2 | 0 | 10 | 2 | 0 | 10 | 13 | 12 | 15 | 10 | 24 | 34 | 42 | 78 | 0 | 2 | 0 | 14 |
| MoLA ( Gao et al. (2024) ) | 4 | 6 | 0 | 6 | 2 | 5 | 12 | 12 | 13 | 11 | 15 | 23 | 36 | 44 | 80 | 6 | 6 | 0 | 16 |
| Comparison Methods | S1 | S2 | S3 | S4 | S5 | S6 | S7 | S8 | S9 | S10 | S11 | S12 | S13 | S14 | S15 | S16 | S17 | S18 | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Seq-FT (sequential fine-tuning) | 100 | 100 | 100 | 97 | 96 | 100 | 97 | 88 | 88 | 92 | 86 | 72 | 88 | 79 | 0 | 100 | 100 | 100 | 88 |
| LwF-LoRA ( Li and Hoiem (2017) ) | 100 | 100 | 100 | 100 | 96 | 100 | 97 | 87 | 90 | 95 | 90 | 61 | 57 | 58 | 0 | 100 | 100 | 100 | 85 |
| EWC-LoRA ( Xiang et al. (2023) ) | 100 | 97 | 100 | 94 | 100 | 100 | 100 | 88 | 88 | 97 | 93 | 54 | 61 | 47 | 0 | 100 | 100 | 100 | 84 |
| Dense MoLE ( Chen et al. (2024) ) | 97 | 97 | 97 | 94 | 92 | 100 | 97 | 88 | 85 | 84 | 83 | 63 | 57 | 30 | 0 | 98 | 100 | 96 | 81 |
| Sparse MoLE ( Dou et al. (2024) ) | 97 | 97 | 100 | 85 | 96 | 100 | 85 | 85 | 77 | 79 | 83 | 56 | 60 | 26 | 0 | 100 | 97 | 100 | 79 |
| MoLA ( Gao et al. (2024) ) | 94 | 91 | 100 | 91 | 96 | 90 | 82 | 86 | 75 | 85 | 75 | 57 | 57 | 24 | 0 | 93 | 91 | 100 | 77 |
| Comparison Methods | S1 | S2 | S3 | S4 | S5 | S6 | S7 | S8 | S9 | S10 | S11 | S12 | S13 | S14 | S15 | S16 | S17 | S18 | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Seq-FT (sequential fine-tuning) | 12 | 12 | 14 | 16 | 18 | 16 | 17 | 19 | 16 | 18 | 15 | 35 | 36 | 36 | 100 | 16 | 15 | 15 | 24 |
| LwF-LoRA ( Li and Hoiem (2017) ) | 12 | 15 | 16 | 15 | 13 | 15 | 16 | 29 | 16 | 32 | 32 | 45 | 55 | 51 | 100 | 13 | 15 | 16 | 28 |
| EWC-LoRA ( Xiang et al. (2023) ) | 13 | 15 | 16 | 15 | 15 | 16 | 15 | 29 | 15 | 34 | 36 | 46 | 56 | 52 | 100 | 16 | 18 | 17 | 29 |
| Dense MoLE ( Chen et al. (2024) ) | 21 | 22 | 18 | 23 | 16 | 18 | 19 | 31 | 16 | 33 | 41 | 48 | 69 | 68 | 100 | 22 | 20 | 17 | 33 |
| Sparse MoLE ( Dou et al. (2024) ) | 23 | 23 | 19 | 25 | 19 | 21 | 23 | 36 | 19 | 26 | 40 | 47 | 69 | 72 | 100 | 25 | 26 | 18 | 35 |
| MoLA ( Gao et al. (2024) ) | 26 | 24 | 16 | 24 | 19 | 23 | 25 | 37 | 21 | 29 | 45 | 48 | 69 | 73 | 100 | 28 | 22 | 26 | 36 |
| Comparison Methods | S1 | S2 | S3 | S4 | S5 | S6 | S7 | S8 | S9 | S10 | S11 | S12 | S13 | S14 | S15 | S16 | S17 | S18 | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Seq-FT (sequential fine-tuning) | 86 | 87 | 84 | 77 | 80 | 83 | 77 | 78 | 75 | 76 | 77 | 59 | 64 | 58 | 0 | 83 | 83 | 83 | 73 |
| LwF-LoRA ( Li and Hoiem (2017) ) | 86 | 83 | 82 | 79 | 86 | 84 | 80 | 66 | 75 | 57 | 54 | 48 | 45 | 40 | 0 | 86 | 83 | 81 | 67 |
| EWC-LoRA ( Xiang et al. (2023) ) | 85 | 83 | 82 | 79 | 84 | 83 | 81 | 66 | 77 | 55 | 48 | 47 | 44 | 39 | 0 | 83 | 79 | 80 | 66 |
| Dense MoLE ( Chen et al. (2024) ) | 75 | 75 | 80 | 67 | 83 | 80 | 76 | 64 | 75 | 56 | 41 | 44 | 31 | 20 | 0 | 77 | 77 | 80 | 61 |
| Sparse MoLE ( Dou et al. (2024) ) | 73 | 74 | 79 | 64 | 79 | 77 | 71 | 58 | 71 | 65 | 42 | 45 | 31 | 15 | 0 | 74 | 70 | 79 | 59 |
| MoLA ( Gao et al. (2024) ) | 69 | 73 | 82 | 66 | 79 | 75 | 68 | 56 | 68 | 61 | 35 | 44 | 31 | 14 | 0 | 71 | 74 | 70 | 58 |
| Methods | S1 | S2 | S3 | S4 | S5 | S6 | S7 | S8 | S9 | S10 | S11 | S12 | S13 | S14 | S15 | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Baseline | 50 | 56 | 46 | 51 | 39 | 38 | 51 | 71 | 46 | 60 | 51 | 48 | 82 | 60 | 86 | 56 |
| Uni-Walker w/o KIS | 61 | 60 | 51 | 55 | 43 | 46 | 56 | 74 | 48 | 68 | 59 | 49 | 86 | 63 | 86 | 60 |
| Uni-Walker w/o SSC | 60 | 59 | 49 | 56 | 42 | 45 | 56 | 75 | 48 | 69 | 58 | 47 | 84 | 62 | 86 | 60 |
| Uni-Walker w/o ECAS | 59 | 58 | 48 | 53 | 42 | 45 | 55 | 72 | 46 | 62 | 55 | 46 | 82 | 62 | 86 | 58 |
| Uni-Walker (ours) | 67 | 67 | 75 | 67 | 57 | 50 | 67 | 83 | 50 | 73 | 62 | 53 | 88 | 65 | 86 | 67 |
| Methods | S1 | S2 | S3 | S4 | S5 | S6 | S7 | S8 | S9 | S10 | S11 | S12 | S13 | S14 | S15 | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Baseline | 29 | 20 | 42 | 36 | 35 | 27 | 27 | 16 | 12 | 20 | 22 | 14 | 9 | 8 | 0 | 21 |
| Uni-Walker w/o KIS | 13 | 14 | 35 | 31 | 28 | 12 | 20 | 13 | 8 | 9 | 9 | 13 | 4 | 3 | 0 | 14 |
| Uni-Walker w/o SSC | 14 | 16 | 38 | 30 | 30 | 13 | 20 | 12 | 8 | 8 | 11 | 16 | 7 | 5 | 0 | 15 |
| Uni-Walker w/o ECAS | 16 | 17 | 39 | 34 | 30 | 13 | 21 | 15 | 12 | 17 | 15 | 18 | 9 | 5 | 0 | 17 |
| Uni-Walker (ours) | 4 | 4 | 5 | 16 | 5 | 4 | 4 | 2 | 4 | 3 | 5 | 5 | 2 | 0 | 0 | 4 |
| Methods | S1 | S2 | S3 | S4 | S5 | S6 | S7 | S8 | S9 | S10 | S11 | S12 | S13 | S14 | S15 | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Baseline | 29 | 30 | 30 | 42 | 25 | 23 | 32 | 35 | 24 | 33 | 34 | 25 | 65 | 48 | 80 | 37 |
| Uni-Walker w/o KIS | 36 | 37 | 38 | 49 | 33 | 35 | 48 | 75 | 51 | 46 | 46 | 45 | 78 | 56 | 80 | 50 |
| Uni-Walker w/o SSC | 36 | 36 | 37 | 48 | 33 | 35 | 46 | 43 | 36 | 43 | 45 | 42 | 76 | 51 | 80 | 46 |
| Uni-Walker w/o ECAS | 35 | 36 | 35 | 49 | 32 | 32 | 45 | 43 | 35 | 42 | 42 | 40 | 75 | 49 | 80 | 45 |
| Uni-Walker (ours) | 66 | 63 | 59 | 61 | 49 | 50 | 65 | 74 | 50 | 72 | 56 | 51 | 82 | 56 | 80 | 62 |
| Methods | S1 | S2 | S3 | S4 | S5 | S6 | S7 | S8 | S9 | S10 | S11 | S12 | S13 | S14 | S15 | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Baseline | 57 | 54 | 52 | 35 | 52 | 56 | 54 | 59 | 55 | 56 | 42 | 56 | 24 | 17 | 0 | 45 |
| Uni-Walker w/o KIS | 47 | 43 | 39 | 25 | 37 | 33 | 30 | 12 | 4 | 39 | 22 | 21 | 9 | 3 | 0 | 24 |
| Uni-Walker w/o SSC | 47 | 45 | 40 | 26 | 37 | 33 | 33 | 49 | 32 | 43 | 24 | 26 | 12 | 12 | 0 | 31 |
| Uni-Walker w/o ECAS | 49 | 45 | 44 | 25 | 38 | 38 | 35 | 49 | 34 | 44 | 29 | 30 | 13 | 16 | 0 | 32 |
| Uni-Walker (ours) | 3 | 3 | 5 | 6 | 6 | 4 | 6 | 13 | 6 | 4 | 5 | 11 | 5 | 3 | 0 | 6 |
| Methods | S1 | S2 | S3 | S4 | S5 | S6 | S7 | S8 | S9 | S10 | S11 | S12 | S13 | S14 | S15 | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Baseline | 75 | 80 | 79 | 63 | 81 | 81 | 70 | 82 | 62 | 70 | 61 | 80 | 91 | 75 | 100 | 77 |
| Uni-Walker w/o KIS | 77 | 82 | 78 | 65 | 82 | 88 | 71 | 82 | 62 | 70 | 61 | 80 | 91 | 75 | 100 | 78 |
| Uni-Walker w/o SSC | 78 | 81 | 79 | 66 | 82 | 89 | 71 | 82 | 61 | 71 | 62 | 80 | 91 | 75 | 100 | 78 |
| Uni-Walker w/o ECAS | 77 | 81 | 79 | 66 | 81 | 88 | 72 | 82 | 62 | 71 | 61 | 81 | 95 | 78 | 100 | 78 |
| Uni-Walker (ours) | 82 | 87 | 88 | 67 | 90 | 90 | 72 | 83 | 63 | 73 | 62 | 84 | 100 | 78 | 100 | 81 |
| Methods | S1 | S2 | S3 | S4 | S5 | S6 | S7 | S8 | S9 | S10 | S11 | S12 | S13 | S14 | S15 | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Baseline | 12 | 10 | 12 | 10 | 12 | 12 | 10 | 4 | 9 | 7 | 6 | 7 | 9 | 12 | 0 | 9 |
| Uni-Walker w/o KIS | 9 | 8 | 13 | 7 | 11 | 4 | 9 | 4 | 9 | 7 | 6 | 7 | 9 | 12 | 0 | 8 |
| Uni-Walker w/o SSC | 8 | 9 | 12 | 6 | 11 | 3 | 9 | 4 | 10 | 5 | 5 | 7 | 9 | 12 | 0 | 7 |
| Uni-Walker w/o ECAS | 9 | 9 | 12 | 6 | 12 | 4 | 8 | 4 | 9 | 5 | 6 | 6 | 5 | 8 | 0 | 7 |
| Uni-Walker (ours) | 4 | 2 | 2 | 4 | 2 | 2 | 8 | 2 | 7 | 3 | 5 | 2 | 0 | 8 | 0 | 3 |
| Methods | S1 | S2 | S3 | S4 | S5 | S6 | S7 | S8 | S9 | S10 | S11 | S12 | S13 | S14 | S15 | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Baseline | 45 | 50 | 42 | 46 | 42 | 32 | 42 | 60 | 41 | 52 | 48 | 41 | 76 | 59 | 71 | 50 |
| Uni-Walker w/o ESOC | 65 | 62 | 70 | 61 | 54 | 45 | 59 | 78 | 48 | 69 | 59 | 50 | 85 | 63 | 85 | 64 |
| Uni-Walker w/o NSCoT | 46 | 51 | 43 | 46 | 43 | 33 | 45 | 61 | 42 | 55 | 49 | 44 | 77 | 60 | 71 | 51 |
| Uni-Walker (ours) | 67 | 67 | 75 | 67 | 57 | 50 | 67 | 83 | 50 | 73 | 62 | 53 | 88 | 65 | 86 | 67 |
| Methods | S1 | S2 | S3 | S4 | S5 | S6 | S7 | S8 | S9 | S10 | S11 | S12 | S13 | S14 | S15 | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Baseline | 36 | 29 | 47 | 43 | 30 | 38 | 40 | 29 | 21 | 31 | 26 | 27 | 16 | 9 | 17 | 29 |
| Uni-Walker w/o ESOC | 7 | 11 | 11 | 24 | 10 | 13 | 16 | 8 | 8 | 8 | 9 | 11 | 6 | 3 | 1 | 10 |
| Uni-Walker w/o NSCoT | 34 | 27 | 46 | 43 | 28 | 37 | 36 | 28 | 19 | 27 | 25 | 21 | 14 | 8 | 17 | 27 |
| Uni-Walker (ours) | 4 | 4 | 5 | 16 | 5 | 4 | 4 | 2 | 4 | 3 | 5 | 5 | 2 | 0 | 0 | 4 |
| Methods | S1 | S2 | S3 | S4 | S5 | S6 | S7 | S8 | S9 | S10 | S11 | S12 | S13 | S14 | S15 | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Baseline | 27 | 28 | 29 | 41 | 22 | 20 | 30 | 31 | 22 | 31 | 30 | 22 | 60 | 40 | 75 | 34 |
| Uni-Walker w/o ESOC | 65 | 61 | 55 | 60 | 45 | 49 | 64 | 73 | 48 | 70 | 55 | 50 | 81 | 55 | 78 | 61 |
| Uni-Walker w/o NSCoT | 29 | 31 | 31 | 42 | 23 | 21 | 32 | 33 | 25 | 33 | 31 | 23 | 61 | 41 | 76 | 35 |
| Uni-Walker (ours) | 66 | 63 | 59 | 61 | 49 | 50 | 65 | 74 | 50 | 72 | 56 | 51 | 82 | 56 | 80 | 62 |
| Methods | S1 | S2 | S3 | S4 | S5 | S6 | S7 | S8 | S9 | S10 | S11 | S12 | S13 | S14 | S15 | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Baseline | 60 | 57 | 53 | 37 | 58 | 62 | 57 | 64 | 58 | 59 | 49 | 61 | 30 | 31 | 6 | 45 |
| Uni-Walker w/o ESOC | 4 | 6 | 11 | 8 | 13 | 6 | 7 | 14 | 9 | 7 | 7 | 12 | 6 | 5 | 3 | 8 |
| Uni-Walker w/o NSCoT | 57 | 52 | 50 | 35 | 56 | 60 | 54 | 61 | 53 | 56 | 47 | 60 | 29 | 29 | 5 | 46 |
| Uni-Walker (ours) | 3 | 3 | 5 | 6 | 6 | 4 | 6 | 13 | 6 | 4 | 5 | 11 | 5 | 3 | 0 | 6 |
| Methods | S1 | S2 | S3 | S4 | S5 | S6 | S7 | S8 | S9 | S10 | S11 | S12 | S13 | S14 | S15 | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Baseline | 65 | 71 | 70 | 53 | 77 | 80 | 66 | 81 | 60 | 70 | 59 | 77 | 85 | 71 | 100 | 72 |
| Uni-Walker w/o ESOC | 80 | 86 | 79 | 66 | 89 | 89 | 71 | 82 | 63 | 70 | 62 | 84 | 100 | 75 | 100 | 80 |
| Uni-Walker w/o NSCoT | 67 | 73 | 72 | 55 | 80 | 82 | 67 | 82 | 62 | 70 | 61 | 81 | 100 | 78 | 100 | 75 |
| Uni-Walker (ours) | 82 | 87 | 88 | 67 | 90 | 90 | 72 | 83 | 63 | 73 | 62 | 84 | 100 | 78 | 100 | 81 |
| Methods | S1 | S2 | S3 | S4 | S5 | S6 | S7 | S8 | S9 | S10 | S11 | S12 | S13 | S14 | S15 | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Baseline | 24 | 20 | 22 | 24 | 16 | 13 | 15 | 5 | 12 | 7 | 9 | 10 | 15 | 16 | 0 | 14 |
| Uni-Walker w/o ESOC | 6 | 3 | 12 | 6 | 3 | 3 | 9 | 4 | 7 | 7 | 5 | 2 | 0 | 12 | 0 | 5 |
| Uni-Walker w/o NSCoT | 21 | 18 | 20 | 21 | 13 | 11 | 14 | 4 | 9 | 7 | 6 | 6 | 0 | 8 | 0 | 11 |
| Uni-Walker (ours) | 4 | 2 | 2 | 4 | 2 | 2 | 8 | 2 | 7 | 3 | 5 | 2 | 0 | 8 | 0 | 3 |
| Comparison Methods | Avg SR (%) | Avg SPL (%) | Avg OSR (%) |
|---|---|---|---|
| NaviLLM Zheng et al. (2024) | 50 | 39 | 58 |
| ScaleVLN Wang et al. (2023b) | 53 | 42 | 61 |
| SAME Zhou et al. (2024) | 55 | 45 | 62 |
| Uni-Walker | 66 | 61 | 81 |
Embodied Navigation
Embodied navigation studies how robots perceive, map, and move through environments to reach goals and follow instructions.
See all 131 papers