Aerial vision-language navigation (VLN) asks a drone to find an object described in natural language. The description usually locates the object relative to named landmarks, such as streets and buildings. We observe that current navigators often bring the drone close to the target and then stop at the wrong place: they reach the target without finding it. This gap is shared by navigators of very different designs, from imitation learners to a fine-tuned 7B vision-language model. A simple test reveals the cause. When we replace the target object in the instruction with another one next to the same landmarks, no navigator goes to it, and most stop where they did before. Navigators learn to reach the landmarks, not to identify the described object. We therefore propose MAP-VP, a plug-and-play module for the last step. After any navigator stops, MAP-VP draws the landmarks named in the instruction onto the drone's current view, using the landmark outlines that the benchmark provides on a map. A fine-tuned 4B vision-language model then locates the target relative to these landmarks. The training data are generated automatically from the benchmark annotations. Neither drawing the landmarks nor fine-tuning helps much alone, but together they are highly effective. Without retraining any navigator, MAP-VP significantly improves all six navigators we test on CityNav. It roughly doubles the success rate of the two strongest trained navigators, raising HETT from 25.9% to 54.6% and FlightGPT from 24.2% to 48.9%, whereas CityNav's official detector-based last step adds less than two points to HETT.
Figures & tables
Figure 2: Reaching is not finding (test-unseen). (a) Failed episodes of each navigator by whether the target is inside the view from 50 m at the stop, and the share that MAP-VP turns into successes. (b) Median shift of the end point when the target in the instruction is replaced by another object next to the same landmarks (386 episodes; the new object lies a median of 40 m away). MAP-VP is applied at HETT’s end points. Exact numbers are in the supplementary material ( Tabs. 4 and 5 ).
Figure 3: Overview of MAP-VP. (a) After any navigator stops, MAP-VP draws the instruction’s map landmarks onto the view (1–2); a fine-tuned VLM returns a box or null (3); the drone flies to the box center (4), or climbs to 100 m and looks again. (b) Training views from the train split, at HETT’s decision points (V1) and at random points near the target (V2), labeled with the target box or null .
Navigator
+ MAP-VP
+ MAP-VP + LA
Val SR
Navigator
Type
SR
SPL
SR@10
SR
SPL
SR@10
SR
SPL
SR@10
nav → ours+LA
Seq2Seq [ 1 ] †‡
imitation, end-to-end
1.19
0.72
0.33
3.87
2.28
2.85
6.98
3.65
4.67
0.84 → 6.16
CMA [ 11 ] †‡
imitation, end-to-end
1.04
0.49
0.29
4.61
2.18
3.09
9.19
3.85
5.84
0.84 → 7.84
MGP †§ [ 12 ]
map-based goal prediction
8.46
8.01
2.06
41.56
34.72
28.40
48.31
38.87
32.61
7.91 → 44.69
FlightGPT † [ 3 ]
7B VLM, SFT + GRPO
24.24
21.95
7.48
45.05
36.51
31.47
48.85
38.18
33.91
16.52 → 43.19
HETT † [ 5 ]
two-stage, IL + RL
25.94
21.17
10.09
51.56
38.31
36.38
54.55
39.96
37.99
23.40 → 51.65
Table 1: MAP-VP on six navigators (CityNav test-unseen, 5,281 episodes with the revised annotations of HETT [ 5 ] ). † Reproduced by us with the released code and weights. ‡ 5,117 episodes kept by the official VLN-CE loader; Seq2Seq and CMA do not use the landmark polygons that MAP-VP draws ( Sec. F ). § With CityNav’s official goal-selection step; MAP-VP replaces this step and starts where the drone stops ( Sec. E ). LA: look again from 100 m when the view from 50 m gives “not visible”. MAP-VP stays within each navigator’s step limit ( Sec. G ), and its flight counts in SPL. Every gain is significant: the paired bootstrap 95% confidence interval of the SR gain excludes zero in every row, e.g . HETT +25.6 [+24.2, +27.0]. Val SR: held-out validation set (4,867 episodes).
Method
Revised
Original
MGP [ 12 ]
0 9.70
0 6.38
FlightGPT [ 3 ]
24.47
21.20
HTNav [ 6 ]
25.49
22.23
HETT [ 5 ]
28.90
22.97
GeoNav [ 28 ]
–
25.9
CLOSER-VLN [ 15 ]
–
32.01
Table 2: Comparison with reported results (test-unseen SR, %). Revised: annotations of HETT [ 5 ] . Numbers of other methods are as reported; the revised numbers of MGP and FlightGPT are taken from [ 6 ] . Our original results use all 5,311 episodes. † Reproduced by us with the released code and weights. HETT’s weights are trained on the revised annotations only.
Figure 4: Success of MAP-VP vs . the navigator’s end-point distance (test-unseen, view from 50 m only). Points are distance bins; the last one pools all end points beyond 75 m. All six navigators follow one curve, and so does a stress test that places the end point at a set distance from the target in a random direction (dashed, 1,000 episodes per distance). Shaded: the target is always inside the view.
Figure 5: Anchors × recognition supervision. SR (%) of the last step at HETT’s end points (test-unseen). Drawing the landmarks helps a zero-shot VLM little (+2.2) but helps a lot after fine-tuning on our data (+7.8). Validation results are in Tab. 6 (supplementary material).
Landmark positions
Extra unnamed box
Model
correct
shifted 40 m
on target
on distractor
Zero-shot
39.06
–
46.7 → 65.6
38.2 → 31.2
MAP-VP
51.30
34.24
61.2 → 61.4
50.8 → 50.1
Table 3: Does the model use the drawn landmarks, and how? Left: SR when all drawn landmarks are shifted by 40 m; for reference, the model fine-tuned without drawn landmarks reaches 43.53. Right: SR without → with one extra unnamed box, on the subsets where a target box (4,404 episodes) or a distractor box (5,081 episodes) can be drawn.
Figure 6: Qualitative examples (test-unseen). Dashed green: the target. (a) Distances from the chosen box to the target. (b) Only the drawn outline is shifted by 40 m (image, instruction, and visible target unchanged); the box of MAP-VP moves with it.
Among failures
Reached
Recovered
Navigator
SR
≤ 50 m
in view
but failed
(in view)
Seq2Seq
1.2
4.0
5.3
–
53.7
CMA
1.0
5.5
7.1
–
50.4
MGP ∗
13.7
48.0
55.9
–
61.7
FlightGPT
24.2
51.1
56.2
17.3
57.6
HETT
25.9
59.7
66.5
21.5
59.5
Table 4: Where navigators fail (test-unseen, %). “ ≤ 50 m” and “in view”: share of failed episodes that end within 50 m of the target, or with the target inside the drone’s final view from 50 m. “Reached but failed”: share of all episodes that pass within 20 m of the target but stop outside; this needs per-step trajectories, which we have for FlightGPT and HETT. “Recovered”: share of the in-view failures that our last-step module turns into successes. ∗ Where the drone stops, before CityNav’s goal-selection step ( Sec. E ).
Median shift (m)
Gain in new- target SR (386)
Gain (1,099)
Gain, > 40 m (688)
Seq2Seq ‡
0.0
+0.0 [+0.0, +0.0]
+0.0
+0.0
CMA ‡
0.0
+0.0 [+0.0, +0.0]
+0.1
+0.0
MGP
0.0
− 1.3 [ − 2.8, +0.0]
− 0.5
− 0.3
HETT
3.9
+0.0 [ − 2.3, +2.3]
− 0.3
− 1.6
FlightGPT
17.7
− 0.5 [ − 4.4, +3.4]
− 0.4
+0.4
MAP-VP
22.1
+20.5 [+15.8, +25.1]
+13.1
+13.1
Table 5: Target swap (test-unseen). Main columns: the 386 episodes with exactly one candidate new target ( Sec. B ). Shift: distance between the end points with the original and the swapped instruction. With the original instruction, a stop can be near the new object by chance. Gain: SR for the new target with the swapped minus with the original instruction (10.1 → 30.6 for MAP-VP, applied at HETT’s end points), with paired bootstrap 95% confidence intervals. Last two columns: all 1,099 episodes with a designated new target, and those whose new and original targets are more than 40 m apart, so that the two success areas do not overlap. ‡ 371, 1,058, and 658 episodes kept by the official VLN-CE loader.
Test
Val
Last step at HETT’s end point
SR
on LM
SR
Stop (HETT)
25.94
–
23.40
GroundingDINO, target phrase
27.84
–
23.14
Zero-shot VLM, no landmarks drawn
36.91
17.4
33.45
Zero-shot VLM, landmarks drawn
39.06
23.1
35.05
Fine-tuned, no landmarks drawn
43.53
14.9
41.89
Table 6: Anchors × recognition supervision (SR, %). “on LM”: share of predicted boxes whose center falls inside a drawn landmark outline. Interaction on test: (51.30−43.53)−(39.06−36.91)=+5.6 [+4.0, +7.2]; on val: (47.65−41.89)−(35.05−33.45)=+4.2 [+2.5, +5.7]. All fine-tuned models use the same 60k samples, and all models the same prompt.
Selector at MGP’s predicted goal
Revised
Original
None (stop at the goal)
21.95
17.17
GroundingDINO, as released [ 12 , 18 ]
0 8.46
0 6.57
GroundingDINO, coordinates fixed
22.44
18.19
MAP-VP
49.02
38.13
MAP-VP + LA
52.77
42.14
Table 7: Last-step selectors at the same viewpoints (test-unseen SR, %). Revised: 5,281 episodes; original: all 5,311 episodes. The two GroundingDINO rows differ only in the conversion from the box to world coordinates ( Sec. E ). The comparison is slightly unfavorable to MAP-VP, because we charge the flight from the drone’s actual position to the predicted goal, which the official step does not.
Figure 7: Failures of MAP-VP when the target is in view (test-unseen, HETT end points, view from 50 m). Dashed green: the target; red: the choice of MAP-VP; yellow: the drawn landmarks. (a) A typical failure: the chosen car is on the named street but is not the target. (b) A description that requires counting. (c) Causes of all such failures, and SR with and without counting or ordinal words in the description.
Setting (original annotations)
NE (m)
SR
OSR
Reported [ 12 ]
93.8
6.38
26.04
Released code as is
87.5
5.91
11.11
Issue 1 fixed ( our setting )
94.1
6.57
26.00
Issues 1 and 2 fixed
85.4
18.19
32.48
Table 8: Reproducing MGP with the released checkpoint (all 5,311 episodes). The final position is the output of CityNav’s goal-selection step, as in the official evaluation. OSR counts an episode if the trajectory or the final position comes within 20 m of the target.
Method
Text
Images (every step)
Maps and coordinates (every step)
Form of the landmark input
Landmark polygons (train / test)
Output
Seq2Seq, CMA
instruction (word tokens)
current RGB and depth
–
–
– / –
actions
MGP
parsed landmark, target, and surrounding phrases (no raw instruction)
current RGB and depth
whole-block maps: view, explored area, landmarks, detections of the target and surrounding phrases
polygons filled into a map channel
✓/ ✓
goal, then actions
HETT
instruction (BERT)
current RGB
whole-block maps: view, explored area, landmarks; landmark centroids; position history
polygons filled into a map channel; centroid coordinates as numbers
✓/ ✓
goal and actions
FlightGPT
instruction and current pixel position
whole-block orthophoto with landmarks painted; current view
–
polygons filled in red and names written on the orthophoto
✓/ ✓
goal
LM-centroid
–
–
landmark polygons
mean of the polygon vertices, used as the goal
– / ✓
goal
MAP-VP
instruction and parsed phrases
view below the final position, with landmarks drawn
–
outlines and names drawn on the current view
✓/ ✓
box, then one move
Table 9: Inputs of each method , read from the released code; training and test inputs are the same. All views are rendered from the same geo-referenced orthophoto at the drone’s pose, and all methods that use landmarks read the same lists of landmark names and polygons provided by CityNav. No method receives the polygon coordinates as text. When MAP-VP looks again, it also receives the view from 100 m.
Figure 8: Every input of each method for the test episode of (“A dark red vehicle parked along Grafton Street across from a dark blue car”). All inputs are produced by each method’s released code at the same position, HETT’s end point, 50 m above the ground. Maps cover the whole block (410 m). Every navigator sees the same ground as MAP-VP; MAP-VP additionally draws the landmark that the navigators already receive as a map, a centroid, or a painted mask.
Earlier start (%)
+ MAP-VP
+ MAP-VP + LA
Navigator
Step limit (released code)
Taken over by MAP-VP
w/o LA
w/ LA
Tab. 1
Not counted
Tab. 1
Not counted
Seq2Seq
200 actions
climb + 20 actions
52.0
61.1
3.87 / 2.28
3.89 / 2.22
6.98 / 3.65
6.98 / 3.43
CMA
200 actions
climb + 20 actions
41.1
48.9
4.61 / 2.18
4.71 / 2.20
9.19 / 3.85
10.03 / 3.99
MGP
20 steps × 5 actions
last 4 steps (5 with LA)
34.2
34.9
41.56 / 34.72
42.06 / 34.85
48.31 / 38.87
49.06 / 39.07
FlightGPT
2 steps × 75 actions
actions left only
0
0
45.05 / 36.51
44.84 / 35.85
48.85 / 38.18
48.70 / 37.52
HETT
20 steps × 10 actions
last 2 steps (3 with LA)
98.7
98.8
51.56 / 38.31
51.30 / 37.51
54.55 / 39.96
53.95 / 38.59
Table 10: Step budget (test-unseen, SR / SPL). Tab. 1 : our results, in which the drone flies with the navigator’s own actions, at most 20 actions plus one step to look again, and the whole episode stays within the step limit of the navigator’s released code. Not counted: the drone flies straight to the predicted point, and this flight is added to the path length but not counted in steps. Earlier start: share of episodes in which MAP-VP takes over before the navigator ends.
Validation Seen
Validation Unseen
Test Unseen
Method
NE ↓
SR ↑
OSR ↑
SPL ↑
NE ↓
SR ↑
OSR ↑
SPL ↑
NE ↓
SR ↑
OSR ↑
SPL ↑
Seq2Seq
363.7
0.89
9.66
0.63
392.0
0.80
10.15
0.58
380.2
1.19
11.59
0.72
Seq2Seq + MAP-VP + LA
224.7
6.42
14.40
3.80
235.8
5.93
13.93
3.60
223.3
6.98
16.47
3.65
CMA
197.1
0.89
8.86
0.60
207.3
0.80
7.36
0.47
183.2
1.04
8.13
0.49
CMA + MAP-VP + LA
168.6
8.91
16.39
4.45
182.8
6.88
13.17
3.25
157.1
9.19
16.08
3.85
MGP
54.7
9.61
35.86
8.82
64.8
6.36
24.22
5.91
60.5
8.46
31.21
8.01
Table 11: Results on all CityNav splits in the format of prior work (revised annotations). Validation: held-out episodes. Seq2Seq and CMA: episodes kept by the official VLN-CE loader (2,257 / 2,513 / 5,117).
Easy
Medium
Hard
Method
NE ↓
SR ↑
OSR ↑
SPL ↑
NE ↓
SR ↑
OSR ↑
SPL ↑
NE ↓
SR ↑
OSR ↑
SPL ↑
Seq2Seq
430.3
2.30
17.21
1.24
379.0
0.49
7.71
0.39
311.4
0.41
8.04
0.35
Seq2Seq + MAP-VP + LA
261.0
9.41
23.19
3.93
218.8
4.81
11.72
2.81
175.5
5.98
12.36
4.17
CMA
165.2
1.37
14.31
0.25
177.1
0.99
6.11
0.76
215.1
0.62
1.72
0.52
CMA + MAP-VP + LA
134.0
10.98
24.22
2.14
152.0
7.83
12.28
4.44
195.2
8.17
8.93
5.58
MGP
62.4
7.26
48.68
6.29
60.2
9.12
25.74
8.94
58.3
9.40
13.34
9.33
Table 12: Results on the three difficulty levels of test-unseen defined by CityNav (2,081 / 1,678 / 1,522 episodes with the revised annotations).
Aerial Vision-and-Language Navigation (VLN) requires UAVs to locate language-specified goals in large-scale urban environments, where objects often appear small and visually ambiguous from high-altitude viewpoints. Existing methods largely follow a reactive paradigm common in indoor VLN, repeatedly updating navigation states from local observations while underutilizing the geospatial structure that remains stable during navigation. As a result, visually similar distractors can cause a reactive goal search to repeatedly reinforce locally plausible goal hypotheses based on passively accumulated evidence, leading to unreliable localization. To address this, we propose GeoReasoner, a robust framework that shifts the paradigm of Aerial VLN from reactive trial-and-error to geospatial-guided active reasoning. Specifically, GeoReasoner first performs pre-search geospatial reasoning to derive region-level priors and constrain the goal hypothesis space. Building on these, it further conducts hypothesis-driven exploration to actively acquire discriminative evidence and refine competing goal hypotheses. Finally, consistency-aware goal verification adversarially assesses complementary evidence sources before stopping, hence reducing premature or erroneous goal confirmations. Experiments on the real-world CityNav benchmark demonstrate that GeoReasoner sets a new SOTA among zero-shot Aerial VLN methods while competing favorably with supervised approaches.
Jiawei Ge, Wenkang Gong, Jiuxin Cao +8
Southeast University · Institute of AI for Industries, Chinese Academy of Sciences · The Hong Kong Polytechnic University +3
Aerial Vision-and-Language Navigation (Aerial VLN) enables unmanned aerial vehicles (UAVs) to follow natural language instructions and navigate complex urban environments. While recent advances have achieved progress through large-scale memory graphs and lookahead path planning, they remain limited by shallow instruction understanding and high computational cost. In particular, existing methods rely primarily on landmark descriptions, overlooking directional cues "a key source of spatial context in human navigation". In this work, we propose LookasideVLN, a new paradigm that exploits directional cues in natural language to achieve both more accurate spatial reasoning and greater computational efficiency. LookasideVLN comprises three core components: (1) an Egocentric Lookaside Graph (ELG) that dynamically encodes instruction-relevant landmarks and their directional relationships, (2) a Spatial Landmark Knowledge Base (SLKB) that provides lightweight memory retrieval from prior navigation experiences, and (3) a Lookaside MLLM Navigation Agent that aligns multimodal information from user instructions, visual observations, and landmark-direction information from ELG for path planning. Extensive experiments show that LookasideVLN significantly outperforms the state-of-the-art CityNavAgent, even with a single-level lookahead, demonstrating that leveraging directional cues is a powerful yet efficient strategy for Aerial VLN.
Yuwei Ning, Ganlong Zhao, Yipeng Qin +4
Sun Yat-sen University · Peng Cheng Laboratory · The Chinese University of Hong Kong +4
Aerial vision-language navigation (AVLN) requires a UAV to reach a language-described target from partial egocentric observations while reasoning over large outdoor scenes. Existing approaches typically fuse view-centered RGB observations and globally anchored history maps as generic visual embeddings, leaving cross-modal spatial alignment to be learned from sparse navigation supervision. We present SP-Guide, a transformer- based goal-prediction framework that addresses this weakness of alignment through a Spatial Prior Cross-Modal Encoder, an Instruction Enhanced Decoder, and a Goal Convergence Filter for terminal target refinement. Experiments implemented on the CityNav benchmark show that SP-Guide improves success rate over the representative baselines, with the largest gains on unseen environments. These results indicate that physically grounded spatial priors provide an effective inductive bias for robust AVLN.
Haoxiang Tang, Dong Zhou, Shen Tan +2
School of Astronautics, Harbin Institute of Technology, Harbin, 150001, China · The 28th Research Institute, China Electronics Technology Group Corporation, Nanjing, 210001, China