RACO: Reliability-Aware Coarse-Goal Optimization for Inspection-Oriented UAV Vision-Language Navigation

Sen Wang; Yiming Sun; Jiaxuan He; Pengfei Zhu

cs.ROAug 24, 2026

RACO: Reliability-Aware Coarse-Goal Optimization for Inspection-Oriented UAV Vision-Language Navigation

Authors: Sen Wang, Yiming Sun, Jiaxuan He, Pengfei Zhu

Organizations: School of Automation, Southeast University, Nanjing, China · School of Science and the School of Engineering, The Hong Kong University of Science and Technology, Hong Kong

Abstract

UAV vision-language navigation (UAV-VLN) is commonly evaluated as goal reaching, but inspection-oriented deployment requires the agent to stop within a valid inspection region and avoid falsely confirming visually or semantically similar distractors. This requirement exposes a key weakness in existing coarse-to-fine UAV-VLN policies: the coarse goal predicted before local refinement is often treated as reliable, although it may drift toward plausible but incorrect object regions and limit the ability of the local stage to recover. To systematically evaluate this problem, we introduce LG-UVI, an object-centric inspection evaluation setting derived from CityNav/CityRefer. LG-UVI extends standard UAV-VLN episodes with target objects, hard distractors, type-aware inspection regions, and diagnostics for inspection-region arrival and object-level confirmation. To address this inspection-oriented setting, we further propose RACO, a reliability-aware adaptive coarse-to-fine navigation framework. Instead of treating the predicted coarse goal as a fixed waypoint, RACO views it as a runtime hypothesis and uses object-level candidate anchors to check and correct coarse localization before Stage 1 and at the Stage 1-to-Stage 2 boundary. RACO also applies scale-adaptive terminal refinement to handle terminal near-miss cases using runtime-observable geometric and anchor-based evidence. Under a unified online evaluation protocol, RACO improves SR over the reproduced HETT baseline by 9.53 and 7.98 percentage points on validation-unseen and test-unseen, respectively. It also improves inspection-region arrival and reduces false verification risk, showing that coarse-goal reliability optimization is an effective complement to existing coarse-to-fine UAV-VLN policies.

Figures & tables

Figure 1: Failure diagnosis for inspection-oriented UAV-VLN. (a) Typical HETT failures: invalid inspection states, coarse-goal drift to same-category distractors, and terminal near-misses. (b) Non-exclusive failure symptoms among failed episodes. (c) Final-to-target residual distances, with the 20–35 m near-miss band highlighted. Ground-truth distances are used only for offline diagnosis.

Split	Episodes	Building	Car	Ground	Parking	Avg. Cand.	Avg. Hard Distr.	Zone Policy
Train Seen	21,878	8,268	11,641	1,226	743	30.84	14.73	center / contour
Val Seen	2,470	1,057	1,226	131	56	30.88	14.84	center / contour
Val Unseen	2,697	1,141	1,026	392	138	29.71	14.58	center / contour
Test Unseen	5,281	2,750	1,693	631	207	30.55	14.74	center / contour

Table 1: Statistics of LG-UVI. Each split preserves the original CityNav/CityRefer navigation episodes and adds object-centric inspection annotations. Candidate pools contain the target object and same-category alternatives, while hard distractors are the nearest same-category objects used for false-verification diagnosis. Cars use center-radius inspection regions, whereas buildings, ground regions, and parking areas use contour-buffer inspection regions.

Figure 2: Overview of the proposed RACO framework. RACO augments a two-stage UAV-VLN backbone with stage-aware coarse-goal reliability correction and scale-adaptive terminal inspection refinement, and the lower panel visualizes the resulting correction process in one representative episode.

Model	Validation Seen				Validation Unseen				Test Unseen
	NE $\downarrow$	SR $\uparrow$	OSR $\uparrow$	SPL $\uparrow$	NE $\downarrow$	SR $\uparrow$	OSR $\uparrow$	SPL $\uparrow$	NE $\downarrow$	SR $\uparrow$	OSR $\uparrow$	SPL $\uparrow$
Random	222.3	0.00	1.15	0.00	223.0	0.00	0.90	0.00	208.8	0.00	1.44	0.00
Human	9.1	89.31	96.40	60.17	9.4	88.39	95.54	62.66	9.8	87.86	95.29	57.04
Seq2Seq	257.1	1.81	7.89	1.58	317.4	0.79	8.82	0.61	245.3	1.50	8.34	1.30
CMA	240.8	0.95	9.42	0.92	268.8	0.65	7.86	0.63	252.6	0.82	9.70	0.79
AerialVLN	65.6	9.77	23.77	8.64	81.8	6.79	17.91	5.73	64.1	8.09	19.13	5.91

Table 2: Main navigation results on the LG-UVI episodes. The results of Random, Human, Seq2Seq, CMA, AerialVLN, and MGP are reported from prior work for reference. HETT is the reproduced end-to-end reference, whereas RACO-Base shares the backbone, scene prior, and evaluation protocol with RACO but disables all runtime correction modules. Best automatic results are shown in Bold .

Split	Model	Zone-SR $\uparrow$	OSA $\uparrow$	ISR $\uparrow$	FVR $\downarrow$
Val Seen	HETT	40.65	10.85	8.91	82.27
	RACO	45.02	11.17	7.85	58.14
Val Unseen	HETT	27.88	15.87	11.98	76.68
	RACO	33.96	16.09	12.72	57.17
Test Unseen	HETT	38.38	16.78	13.69	77.18
	RACO	44.52	15.87	12.63	62.09

Table 3: Inspection-oriented diagnostics on LG-UVI under the same online protocol.

Variant	Val Seen						Val Unseen
Variant	SR $\uparrow$	OSR $\uparrow$	SPL $\uparrow$	NE $\downarrow$	Zone-SR $\uparrow$	FVR $\downarrow$	SR $\uparrow$	OSR $\uparrow$	SPL $\uparrow$	NE $\downarrow$	Zone-SR $\uparrow$	FVR $\downarrow$
RACO-Base	30.32	48.10	25.78	37.90	41.42	77.29	19.39	34.63	16.18	53.16	28.18	70.27
Pre-stage only	31.34	49.43	26.75	37.44	42.02	60.24	21.73	37.12	18.40	50.97	30.29	59.49
Boundary only	31.66	49.31	26.68	37.55	42.67	59.85	21.99	37.63	17.87	52.28	30.44	59.51
Pre-stage + boundary	32.23	49.96	27.34	37.28	42.71	62.71	23.43	39.12	19.46	50.45	31.70	60.33
Full w/o type filtering	32.55	49.03	27.26	37.27	42.39	57.13	24.14	38.01	20.17	51.21	31.48	53.99

Table 4: Ablation results on LG-UVI under the same online protocol. “Pre-stage + boundary” excludes terminal refinement, whereas “Full w/o type filtering” enables all three correction stages without type-aware anchor filtering. Best results for each metric are shown in Bold .

Figure 3: Qualitative correction cases on the LG-UVI development split. Each panel compares the original HETT trajectory with RACO under the same instruction. RACO corrects failures at the pre-stage, stage-boundary, and terminal refinement stages.

Module	Dim.	Main runtime feature groups
Pre-stage gate	24	Candidate distance statistics, type counts, type cue
Pre-stage ranker	47	Goal/agent geometry, type match, local density, group statistics
Boundary gate	118	Stage-1 trajectory, endpoint geometry, candidate-score distribution
Terminal scorer	46	Terminal geometry, proposed action, recent progress, gate diagnostics

Table S1: Runtime feature groups used by the learned decision modules. The dimension excludes ground-truth-only quantities used for offline labeling.

Split	Variant	SR $\uparrow$	SPL $\uparrow$	NE $\downarrow$	Zone-SR $\uparrow$	ISR $\uparrow$
Val Seen	No terminal	32.23	27.34	37.28	42.71	7.53
	Action len. 21	32.02	26.98	37.39	42.87	7.57
	Fixed 5 m	32.91	27.89	37.23	42.91	7.57
	Scale-adaptive	35.55	29.61	36.85	45.02	7.85
Val Unseen	No terminal	23.43	19.46	50.45	31.70	12.38
	Fixed 5 m	24.43	20.33	50.36	32.18	12.46

Table S2: Terminal-refinement ablation on LG-UVI. All variants use the same pre-stage and stage-boundary correction modules. “Action len. 21” increases the execution budget without applying terminal correction, whereas “Fixed 5 m” applies a constant terminal displacement. Best results within each split are shown in bold.

Method	Test Unseen SR $\uparrow$
Nearest-type heuristic	27.70
Nearest-type + geometry terminal	30.35
Full RACO	33.71

Table S3: Test Unseen SR for Full RACO and non-learning controls using the same annotation-derived object map.

Map condition	SR $\uparrow$	$\Delta$ SR
Clean map	27.59	0.00
Delete 10% objects	27.29	-0.30
Delete 20% objects	26.88	-0.71
Delete 30% objects	26.59	-1.00
Position noise (5 m)	24.10	-3.49
Position noise (10 m)	23.58	-4.01

Table S4: Validation Unseen SR under synthetic object-map perturbations. The reported change is measured relative to the clean-map condition.

Runtime module	Test trigger rate
Pre-stage correction	8.9%
Boundary correction	10.5%
Terminal refinement	29.9%

Table S5: Activation rates of the three RACO correction stages on Test Unseen.

UAV Navigation

Unmanned aerial vehicle (UAV) navigation develops methods for locating and guiding drones safely through known, unknown, and changing environments.

See all 272 papers

See-and-Reach: Precise Vision-Language Navigation for UAVs within the Field of View
Jun 18, 2026Fanfu Xue, En Yu, Yantian Shen +5UAV Navigation Vision-Language Navigation
School of Information Science and Engineering, Shandong University, Qingdao 266237, China · Faculty of Engineering and Information Technology, University of Technology Sydney, Sydney, NSW, Australia · School of Computer Science and Technology, Shandong University, Qingdao 266237, China +3
From Semantic Grounding to Decision Optimization: A Unified Framework for Long-Horizon UAV Vision-Language Navigation
Aug 10, 2026Zeyuan Ma, Jiaxin Chen, Di HuangUAV Navigation Vision-Language Navigation
State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China · School of Computer Science and Engineering, Beihang University, Beijing, China
Last-Meter Precision Navigation for UAVs: A Diffusion-Refined Aerial Visual Servoing Approach
Jul 5, 2026Yaxuan Li, Jiarui Zeng, Shaofei Huang +1Diffusion-Based Planning UAV Navigation
FST and ICI, University of Macau · Macau SAR, China

Jun 18, 2026J/K move · Enter open · S save

See-and-Reach: Precise Vision-Language Navigation for UAVs within the Field of View

Fanfu Xue, En Yu, Yantian Shen, Zhikun Hu+4

School of Information Science and Engineering, Shandong University, Qingdao 266237, China · Faculty of Engineering and Information Technology, University of Technology Sydney, Sydney, NSW, Australia · School of Computer Science and Technology, Shandong University, Qingdao 266237, China+3

UAV Navigation Vision-Language Navigation Aerial VLN 3D Visual Grounding

Open paper AskPDF arXiv

UAV Vision-Language Navigation (UAV-VLN) is typically formulated as a holistic search-and-reach problem, where long-range target discovery and final target approach are optimized and evaluated jointly. This formulation makes it difficult to assess a critical capability of aerial embodied agents, namely whether a UAV can accurately ground a visible target and translate vision-language evidence into precise 3D motion once the target enters its field of view. To address this limitation, we introduce UAV-VLN-FOV, a target-visible navigation task that isolates the see-and-reach stage and enables a more diagnostic evaluation of terminal reaching ability. We further propose 3DG-VLN, a vision-language waypoint prediction framework guided by dynamic 3D direction cues to enhance fine-grained visual grounding and spatial direction alignment for precise target reaching. Specifically, 3DG-VLN adaptively processes high-resolution front-view and downward-view observations to preserve fine-grained visual and geometric details for target grounding. It also updates the target-relative direction online during closed-loop navigation, allowing the agent to maintain spatial alignment with the target and reduce accumulated direction drift. To support this task, we construct a dedicated high-resolution benchmark which contains 2,717 trajectories with target-oriented high-level instructions, high-resolution front-view and downward-view egocentric observations, and continuous 3D waypoint annotations. Experiments show that 3DG-VLN outperforms competitive UAV-VLN baselines, achieving a 13.82% improvement in success rate. Real-world trials further demonstrate the potential of 3DG-VLN for practical see-and-reach navigation. The source code and benchmark are available at https://github.com/xuefanfu/3DG-VLN.

Abstract

UAV Navigation

Explore similar work

See-and-Reach: Precise Vision-Language Navigation for UAVs within the Field of View

From Semantic Grounding to Decision Optimization: A Unified Framework for Long-Horizon UAV Vision-Language Navigation

Last-Meter Precision Navigation for UAVs: A Diffusion-Refined Aerial Visual Servoing Approach