TagaVLM: Topology-Aware Global Action Reasoning for Vision-Language Navigation
Organizations: School of Information Science and Technology, Beijing University of Technology, China. · Now at Imperial College London, U.K.
Abstract
Vision-Language Navigation (VLN) presents a unique challenge for Large Vision-Language Models (VLMs) due to their inherent architectural mismatch: VLMs are primarily pretrained on static, disembodied vision-language tasks, which fundamentally clash with the dynamic, embodied, and spatially-structured nature of navigation. Existing large-model-based methods often resort to converting rich visual and spatial information into text, forcing models to implicitly infer complex visual-topological relationships or limiting their global action capabilities. To bridge this gap, we propose TagaVLM (Topology-Aware Global Action reasoning), an end-to-end framework that explicitly injects topological structures into the VLM backbone. To introduce topological edge information, Spatial Topology Aware Residual Attention (STAR-Att) directly integrates it into the VLM's self-attention mechanism, enabling intrinsic spatial reasoning while preserving pretrained knowledge. To enhance topological node information, an Interleaved Navigation Prompt strengthens node-level visual-text alignment. Finally, with the embedded topological graph, the model is capable of global action reasoning, allowing for robust path correction. On the R2R benchmark, TagaVLM achieves state-of-the-art performance among large-model-based methods, with a Success Rate (SR) of 51.09% and SPL of 47.18 in unseen environments, outperforming prior work by 3.39% in SR and 9.08 in SPL. This demonstrates that, for embodied spatial reasoning, targeted enhancements on smaller open-source VLMs can be more effective than brute-force model scaling. The code can be found on our project page: https://apex-bjut.github.io/Taga-VLM
Figures & tables
| Method | Backbone | Val Seen | Val Unseen | ||||||||
| TL | NE | OSR | SR | SPL | TL | NE | OSR | SR | SPL | ||
| Seq2Seq [ 1 ] | LSTM | 11.33 | 6.01 | 53 | 39 | - | 8.39 | 7.81 | 28 | 21 | - |
| Speaker Follower [ 38 ] | LSTM | - | 3.36 | 74 | 66 | - | - | 6.62 | 45 | 36 | - |
| HAMT [ 26 ] | Cross-Modal Transformer | 11.15 | 2.52 | - | 76 | 72 | 11.46 | 2.29 | - | 66 | 61 |
| DUET [ 4 ] | Cross-Modal Transformer | 12.32 | 2.28 | 86 | 79 | 73 | 13.94 | 3.31 | 81 | 72 | 60 |
| BEVBert [ 5 ] | Cross-Modal Transformer | 13.56 | 1.67 | 88 | 81 | 74 | 14.55 | 2.81 | 84 | 75 | 64 |
| STAR-Att | INP | GA | AD | TL | NE | OSR | SR | SPL | |
| a | ✘ | ✘ | ✘ | ✘ | 13.85 | 9.05 | 27.37 | 17.28 | 13.01 |
| b | ✔ | ✘ | ✘ | ✘ | 12.76 | 7.74 | 35.67 | 26.14 | 20.81 |
| c | ✔ | ✔ | ✘ | ✘ | 9.08 | 6.49 | 47.47 | 38.40 | 35.61 |
| d | ✔ | ✘ | ✔ | ✘ | 10.85 | 7.50 | 42.40 | 31.97 | 27.63 |
| e | ✔ | ✔ | ✔ | ✘ | 10.12 | 6.06 | 52.41 | 42.06 | 37.73 |
| f | ✔ | ✔ | ✔ | ✔ | 9.8 | 5.57 | 55.09 | 45.72 | 41.91 |
| STAR-Att | Text-Based | TL | NE | OSR | SR | SPL | |
| a | ✘ | ✘ | 10.10 | 6.27 | 49.38 | 39.76 | 35.67 |
| b | ✘ | ✔ | 9.93 | 6.34 | 51.60 | 40.70 | 36.92 |
| c | ✔ | ✘ | 10.12 | 6.06 | 52.41 | 42.06 | 37.73 |
Vision-Language Navigation
Vision-Language Navigation studies how embodied agents use natural-language instructions and visual observations to plan and execute routes.
See all 290 papers