Organizations: Nanjing University of Aeronautics and Astronautics, China Key Laboratory of Brain-Machine Intelligence Technology, Ministry of Education, China
Inspired by the general Vision-and-Language Navigation (VLN) task, aerial VLN has attracted widespread attention, owing to its significant practical value in applications such as logistics delivery and urban inspection. However, existing methods face several challenges in complex urban environments, including insufficient generalization to unseen scenes, suboptimal performance in long-range path planning, and inadequate understanding of spatial continuity. To address these challenges, we propose HTNav, a new collaborative navigation framework that integrates Imitation Learning (IL) and Reinforcement Learning (RL) within a hybrid IL-RL framework. This framework adopts a staged training mechanism to ensure the stability of the basic navigation strategy while enhancing its environmental exploration capability. By integrating a tiered decision-making mechanism, it achieves collaborative interaction between macro-level path planning and fine-grained action control. Furthermore, a map representation learning module is introduced to deepen its understanding of spatial continuity in open domains. On the CityNav benchmark, our method achieves state-of-the-art performance across all scene levels and task difficulties. Experimental results demonstrate that this framework significantly improves navigation precision and robustness in complex urban environments.
Figures & tables
Figure 1: Significant challenges include: (a–b) show decreased success rates and increased navigation errors when generalizing from seen to unseen scenes; (c) illustrates UAVs losing track of the goal in long-range navigation; (d) reveals directional errors caused by failing to interpret complex spatial information.
Figure 2: Architecture of HTNav. The model processes visual observations and generates a multi-layered navigation map from state and target information. Features extracted from RGB, depth, and map inputs by their respective encoders are fed into a three-head prediction module: a Value Prediction Head for expected cumulative reward, a Progress Prediction Head for navigation status, and a Goal Prediction Head for the final destination. In parallel, a decision module uses this information to generate waypoints and execute actions.
Figure 3: A schematic of the Tiered Decision Mechanism. The plot shows the UAV’s ground-truth and predicted paths, annotated with critical waypoints and action decisions (e.g. ”Turn left”).
Figure 4: Schematic of the map representation learning module that integrates multi-source map inputs with a residual encoder and spatial-channel compression for goal coordinate regression.
Method
Validation Seen
Validation Unseen
Test Unseen
NE ↓
SR ↑
OSR ↑
SPL ↑
NE ↓
SR ↑
OSR ↑
SPL ↑
NE ↓
SR ↑
OSR ↑
SPL ↑
Random
222.3
0.00
1.15
0.00
223.0
0.00
0.90
0.00
208.8
0.00
1.44
0.00
Seq2Seq+GSM
58.5
8.43
17.31
7.28
78.6
5.13
10.90
4.65
98.1
3.81
13.82
2.79
CMA+GSM
68.0
6.25
13.28
5.40
75.9
4.38
9.29
3.90
94.6
4.68
12.01
4.05
MGP
59.7
8.69
35.51
8.28
75.1
5.84
22.19
5.56
93.8
6.38
26.04
6.08
MGP*
54.1
10.96
40.82
10.42
65.6
8.33
26.75
7.64
82.6
9.70
31.46
8.22
Table 1: Performance comparison of methods on CityNav dataset (* indicates results obtained using the revised dataset).
Method
Easy
Medium
Hard
NE ↓
SR ↑
OSR ↑
SPL ↑
NE ↓
SR ↑
OSR ↑
SPL ↑
NE ↓
SR ↑
OSR ↑
SPL ↑
Random
127.5
0.00
3.60
0.00
212.0
0.00
0.00
0.00
319.8
0.00
0.00
0.00
Seq2Seq
238.8
3.07
14.70
2.64
246.5
0.43
3.87
0.38
253.1
0.48
4.38
0.44
CMA
260.7
0.49
16.69
0.44
241.2
1.10
7.67
1.09
253.8
0.96
1.64
0.95
MGP
98.9
6.15
39.89
5.48
90.9
6.29
21.47
6.21
90.0
6.80
12.10
6.78
HTNav
68.9
19.99
45.18
14.94
68.2
20.40
39.44
18.69
68.1
21.91
34.14
21.11
Table 2: Results on Test-Unseen Set across different difficulty levels.
A
B
C
Validation Seen
Validation Unseen
Test Unseen
NE ↓
SR ↑
OSR ↑
SPL ↑
NE ↓
SR ↑
OSR ↑
SPL ↑
NE ↓
SR ↑
OSR ↑
SPL ↑
MGP(baseline)
54.1
10.96
40.82
10.42
65.6
8.33
26.75
7.64
82.6
9.70
31.46
8.22
MGP+IL-RL
47.1
23.56
45.55
20.03
58.3
12.83
34.07
10.18
47.8
18.95
43.08
14.82
✓
-
-
40.7
26.15
49.03
24.81
52.1
15.09
33.15
14.19
44.0
24.03
44.99
20.26
✓
✓
-
39.8
28.07
49.45
24.72
51.2
16.05
37.19
14.09
41.0
24.18
46.53
21.02
✓
-
✓
40.5
27.77
49.35
24.68
51.4
15.72
35.52
14.16
41.8
24.16
47.45
21.03
Table 3: Ablation study: A = Tiered structure, B = Residual map encoder, C = SCConv module(results are on the revised dataset).
Figure 5: Visualization of navigation trajectories. The left panel illustrates the long-path navigation capability, the middle panel demonstrates the small-target localization ability, and the right panel exhibits the efficiency of the navigation path.
λRL
Test-Unseen
NE ↓
SR ↑
OSR ↑
SPL ↑
0(w/o RL)
53.0
18.90
40.28
17.53
0.10
44.0
21.89
44.16
19.88
0.15
43.6
24.16
48.02
21.03
0.20
40.3
25.49
48.82
21.56
0.25
41.9
24.29
47.42
20.89
Table 4: Performance metrics under different RL weights.
Method
SR ↑
MGP [ 19 ]
5.84
w/o target & surroundings maps
5.81
w/o landmark map
0.47
HTNav
28.30
w/ target & surroundings maps
28.62
w/o landmark map
1.86
Table 5: Effectiveness evaluation of Target and Surroundings Map.
Tianmushan Laboratory, Beihang University, Hangzhou 311115, China · Hangzhou International Innovation Institute, Beihang University, Hangzhou 311115, China · Foshan Graduate School of Innovation, Northeastern University, Foshan, China +3
School of Software Engineering, Huazhong University of Science and Technology · Nanyang Technological University, Singapore · Interdisciplinary Graduate Programme University College London