WareFly-VLA: A Vision-Language-Action Framework for UAV Navigation and Human Tracking in Smart Warehouses

Thinh D. Le; Son T. Nguyen; Duong Q. Nguyen; Dung D. Le; Ngo Anh Vien; H. Nguyen-Xuan

cs.ROOct 6, 2026

WareFly-VLA: A Vision-Language-Action Framework for UAV Navigation and Human Tracking in Smart Warehouses

Authors: Thinh D. Le, Son T. Nguyen, Duong Q. Nguyen, Dung D. Le, Ngo Anh Vien, H. Nguyen-Xuan

Organizations: Center for AI Research, VinUniversity, Ho Chi Minh City, Vietnam · College of Engineering and Computer Science, VinUniversity, Hanoi, Vietnam · VinRobotics, Hanoi, Vietnam

Abstract

Vision-Language-Action (VLA) models have achieved impressive results in robotic manipulation and ground-mobile navigation, yet language-conditioned control of unmanned aerial vehicles (UAVs) in smart warehouses remains largely unexplored, hindered by the lack of benchmarks that jointly provide continuous low-level flight actions, fine-grained natural-language target descriptions, and realistic industrial environments. This paper introduces WareFly-VLA, a photorealistic UAV VLA framework and dataset for language-guided human search, localization, and tracking in warehouse environments. It contains 507 human-teleoperated flight episodes and 8,504 high-resolution RGB transitions collected in NVIDIA Isaac Sim, each paired with a human-written appearance description of the target worker and a synchronized four-degree-of-freedom control command. Two aerial tasks are covered: target approach and person following, under occlusion, long-range search, altitude variation, and clutter. A unified benchmark of four open-source VLA architectures (SmolVLA, GR00T N1.7, pi_0 and OpenVLA) is established under a leakage-free episode-level protocol at two control rates. The results show that language-conditioned aerial control in warehouses is far from solved: performance drops substantially under strict generalization settings, continuous action modeling consistently outperforms discrete action tokenization, only the forward channel is reliably learnable from a single frame, and current foundation-model interfaces transfer poorly from ground and humanoid embodiments to aerial platforms. The synchronized video, language, action, pose, and difficulty annotations further support world-model research. The dataset, baselines, and evaluation protocol are released to support language-grounded aerial autonomy in smart warehouses.

Figures & tables

Figure 1: State-transition diagram of the human teleoperation policy used during data collection. Search states ( Look Around , Scan Shelves and Move to Open ) are shown in cyan. The check_open decision node is shown in amber, and the goal-directed Approach Target state is shown in green. From any search state, detecting the target in the camera frame triggers an immediate transition to Approach Target . If Scan Shelves ends with Search Exhausted , or if Move to Open ends with Arrived at open area , the policy returns to Look Around . The episode terminates when the drone reaches the target.

Property	Value
Total episodes	507
Non-terminal transitions	8,504
train / val	7,276 / 1,228
Train / val episodes	431 / 76
Frame rate	1 FPS
Image resolution	$1920\times 1440$ px

Table 1: WareFly-VLA dataset statistics.

Figure 2: Dataset overview. (a) Distribution of episode lengths; the dashed red line marks the mean. (b) Per-episode frame count sorted in ascending order. (c) Task-verb distribution across $15$ phrasings; approach and follow dominate, with thirteen additional verb families ensuring paraphrase diversity.

Figure 3: Drone flight trajectories in world coordinates. Left: four representative single-episode trajectories in the world-frame $xy$ -plane, color-coded from early (blue/purple) to late (yellow) via the plasma colormap, with the language instruction shown above each panel; green circles indicate episode start positions and red stars indicate end positions. Right: top-down composite of many episode trajectories overlaid on a bird’s-eye view of the photorealistic Isaac Sim warehouse. Solid line segments denote frames in which the drone has the target in view, while dashed line segments denote frames in which the target is out of view (search / re-acquisition). The composite illustrates the spatial diversity of pursuit paths across shelving rows, aisles and open areas and confirms that the dataset captures a wide range of pursuit behaviors rather than a single stereotyped maneuver.

Component	Mean	Std	Min	Max
$\Delta x_{\ell}$ (m)	$+0.864$	$0.688$	$-0.558$	$+5.615$
$\Delta y_{\ell}$ (m)	$-0.012$	$0.421$	$-5.124$	$+4.012$
$\Delta z$ (m)	$-0.000$	$0.013$	$-0.363$	$+0.567$
$\Delta\psi$ (rad)	$+0.096$	$0.355$	$-1.254$	$+2.197$

Table 2: Action component statistics over the

N{=}8{,}504

non-terminal frames.

Figure 5: Altitude $z$ (solid blue, left axis) and yaw angle $\psi$ (dashed red, right axis) over time for six episodes. Altitude remains nearly constant in most episodes, while heading changes dynamically as the drone turns to track the moving target.

Figure 6: Word cloud of appearance tokens in language instructions (task verbs and stop words removed). Term size reflects relative frequency. PPE vocabulary ( hard hat , safety vest , headphones ) dominates, alongside rich color and clothing-type diversity.

Figure 7: Sample frames from three episodes at four evenly-spaced time steps (left to right: $t{=}0,t_{1},t_{2},t_{\text{end}}$ ). Row labels show the corresponding language instruction (truncated). The warehouse background, diverse worker appearances and varying drone–target distances illustrate the visual complexity of the dataset.

Dataset	Plat- form	Lang. instr.	Cont. acts.	Src.	Episodes
UAV123 [ 33 ]	UAV	$\times$	✓	Real	$\sim$ 150
AerialVLN [ 30 ]	UAV	✓	$\times$	Sim	8,446
AirNav [ 12 ]	UAV	✓	$\times$	Real	35,000
UAV-VLA [ 41 ]	UAV	✓	$\times$	Real	100k
RaceVLA [ 42 ]	UAV	✓	✓	Real	$\sim$ 200
CognitiveDrone [ 34 ]	UAV	✓	✓	Sim	8,000

Table 3: Comparison with related datasets. ✓ = yes,

\times

= no, – = not applicable. “Cont. actions” indicates whether continuous low-level action labels (not high-level discrete commands or waypoints) are provided; “Scr.” indicates real-world capture vs. simulation.

Model	Parameters	Action Representation	Vision–Language Backbone	Adaptation
OpenVLA-7B	7.6B total 49.3M trainable	Autoregressive policy (256 bins per dimension)	Prismatic SigLIP + DINOv2 Llama-2 7B	LoRA
GR00T N1.7	3.1B total 1.62B trainable	Diffusion policy Horizon = 4	Cosmos-Reason2-2B	Fine-tuning
$\pi_{0}$ /OpenPI	3.3B total 145M trainable	Flow matching Horizon = 4	PaliGemma-3B	LoRA
SmolVLA	450M total 450M trainable	Flow matching Horizon = 4	SmolVLM-2 + Action Expert	Full fine-tuning

Table 4: Representative Vision–Language–Action architectures benchmarked on WareFly-VLA.

Model	$\Delta x_{\ell}$ (m)	$\Delta y_{\ell}$ (m)	$\Delta z$ (m)	$\Delta\psi$ (rad)
MAE $\downarrow$
Naive (constant-mean)	0.590	0.242	0.003	0.248
OpenVLA-7B [ 27 ]	0.620	0.368	0.005	0.409
GR00T N1.7 [ 38 ] †	5.728	3.867	0.637	0.555
$\pi_{0}$ /OpenPI [ 5 ]	0.413	0.244	0.003	0.220
SmolVLA [ 43 ]	0.468	0.264	0.003	0.243

Table 5: Per-step prediction performance on the WareFly-VLA held-out validation split. Results are reported as Mean Absolute Error (MAE) and Pearson correlation (

r

). All models are evaluated on the same 76 held-out episodes (

N=1,228

frames) under the canonical leakage-free split.

Figure 8: Cross-model comparison on the WareFly-VLA held-out validation split (76 identical episodes for all models). Left: Per-dimension MAE (lower is better). Right: Pearson correlation (higher is better). $\pi_{0}$ achieves the best overall performance, followed by SmolVLA, while OpenVLA shows limited generalization and GR00T fails to produce meaningful predictions. No model surpasses the naive baseline on $\Delta y_{\ell}$ or $\Delta z$ , highlighting the difficulty of lateral and vertical action prediction.

Model	$\Delta x_{\ell}$ (m)	$\Delta y_{\ell}$ (m)	$\Delta z$ (m)	$\Delta\psi$ (rad)
MAE $\downarrow$ ( $\times 10^{-2}$ , except $\Delta z$ )
Naive	$6.04{\pm}0.09$	$2.74{\pm}0.07$	$0.028{\pm}0.001$	$2.58{\pm}0.05$
$\pi_{0}$	4.80 ${\pm}$ 0.09	2.70 ${\pm}$ 0.06	0.037	2.62 ${\pm}$ 0.06
SmolVLA	5.33 ${\pm}$ 0.09	2.95 ${\pm}$ 0.06	0.038	2.68 ${\pm}$ 0.06
Pearson $r\uparrow$
$\pi_{0}$	$+0.51$	$+0.49$	$+0.12$	$+0.37$

Table 6: Per-step prediction accuracy at

10

FPS on the validation split (

76

episodes,

N{=}13{,}371

transitions). Values report MAE and Pearson

r

with

95\%

bootstrap confidence-interval half-widths.

Figure 9: Cross-model comparison at the $10$ FPS deployment-rate setting. Left: per-dimension MAE learned bars sit nearly flush with the naive baseline on $\Delta y_{\ell}$ / $\Delta z$ / $\Delta\psi$ , the consequence of $\sim$ $10\times$ -smaller per-frame actions. Right: Pearson $r$ preserves the $1$ FPS structure ( $\pi_{0}\!>\!$ SmolVLA; forward most learnable, vertical least). The contrast motivates reporting correlation alongside MAE at high control rates.

Figure 10: Per-dimension prediction-vs.-ground-truth scatter for $\pi_{0}$ (the strongest model) on the held-out validation split. The dashed diagonal denotes the identity line; $\Delta x_{\ell}$ shows a positive but range-compressed trend, while $\Delta y_{\ell}$ and $\Delta z$ collapse toward the prior mean the per-sample signature of the generalization gap.

Figure 11: Qualitative SmolVLA predictions on twenty randomly-sampled held-out frames. Solid blue arrows depict the ground-truth in-plane displacement; dashed purple arrows depict the predicted displacement. Predictions track the forward direction but under-reach on large displacements and miss lateral corrections, illustrating the generalization gap on truly unseen episodes.

Figure 12: Open-loop trajectory rollouts of SmolVLA predictions vs. expert trajectories on six held-out validation episodes. Solid blue is the expert path; dashed purple is the integrated prediction starting from the true initial pose; green squares mark start positions. Rollouts follow the gross forward progression but accumulate drift on curved segments where lateral/heading corrections are mi-predicted.

Figure 13: Cost–accuracy frontier. $x$ -axis: model size (log scale); $y$ -axis: mean validation MAE (symmetric log); marker area $\propto$ fine-tuning wall-clock time. SmolVLA and $\pi_{0}$ jointly define the Pareto frontier (cheapest vs. most accurate); OpenVLA is dominated and GR00T fails.

Rate	Model	Params	Lat. (ms)	p95	Hz
1 FPS	SmolVLA	$0.45$ B	$\mathbf{36.5}$	$124$	$\mathbf{27.4}$
	$\pi_{0}$ /OpenPI	$3.3$ B	$67.2$	$70$	$14.9$
	GR00T N1.7	$3.1$ B	$77.9$	$101$	$12.8$
	OpenVLA-7B	$7.6$ B	$365.1$	$376$	$2.7$
10 FPS	SmolVLA	$0.45$ B	$31.7^{\ast}$	$122$	$\mathbf{31.6}$
10 FPS	$\pi_{0}$ /OpenPI	$3.3$ B	$62.8$	$65$	$15.9$

Table 7: Single-frame inference latency on one H100 (

10

warm-up,

50

timed). “Rate” is sustained throughput; the last block is the

10

FPS frontier.

Figure 14: Runtime–accuracy frontier. $x$ -axis: single-frame inference latency (log ms, faster $\leftarrow$ ); $y$ -axis: held-out mean MAE (lower is better); bubble area $\propto$ parameter count. Vertical dashed lines mark the per-action time budgets for $1$ , $10$ and $30$ Hz control loops. SmolVLA and $\pi_{0}$ sit in the accurate-and-fast quadrant and clear the $10$ Hz budget; OpenVLA-7B violates even the $10$ Hz budget and GR00T is fast but inaccurate.

Rate	Model	ADE	FDE
1 FPS	$\pi_{0}$ /OpenPI	$\mathbf{2.55}$	$\mathbf{4.73}$
	SmolVLA	$3.06$	$5.95$
	OpenVLA-7B	$3.56$	$6.30$
	GR00T N1.7	$24.92$	$43.84$
10 FPS	$\pi_{0}$ /OpenPI	$\mathbf{3.12}$	$\mathbf{5.64}$
10 FPS	SmolVLA	$3.74$	$7.00$

Table 8: Open-loop rollout error (m) integrated from the true initial pose. ADE/FDE: average/final displacement error. The model ranking holds at both rates.

Figure 15: Open-loop rollout error at $1$ FPS: average (ADE, solid) vs. final (FDE, hatched) displacement error from the true initial pose. The flow-matching models drift least; OpenVLA trails; GR00T’s normalization failure produces order-of-magnitude larger drift. FDE exceeds ADE for every model, the signature of compounding error.

Figure 16: $\pi_{0}$ open-loop rollouts (orange) vs. expert paths (blue) on six $10$ FPS held-out episodes, integrated from the true initial pose (green square). Rollouts capture the gross forward progression but accumulate drift on curved segments where the unpredicted lateral and heading corrections compound the trajectory-level manifestation of the off-axis channels’ low correlation.

Panel of Figure 17 — Figure 17: World-model research roadmap enabled by WareFly-VLA. The dataset core is drawn as a quadrotor and aligns five synchronized supervision streams. Three predictive modules consume these streams: JEPA-style representation learning (1), action-conditioned latent dynamics (2) and language-conditioned prediction and reasoning (3). Their outputs fuse into model-based planning (4), which reaches hardware through Physical-AI and sim-to-real transfer (5). Solid colored arrows ( $E1$ – $E7$ ) denote forward modeling and control dependencies; the dashed arrow ( $E8$ ) returns deployment failures to data collection and closes the loop. The bottom band lists evaluation hooks already provided by the benchmark release.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

	OpenVLA	GR00T	$\pi_{0}$	SmolVLA
Adaptation	LoRA	proj.+DiT	LoRA	full
LoRA $r/\alpha$	$16/32$		expert head
Optimiser	AdamW	AdamW	AdamW	AdamW
Learning rate	$2{\times}10^{-4}$	$1{\times}10^{-4}$	$1{\times}10^{-4}$	$5{\times}10^{-5}$
Steps / epochs	$20$ ep	$6{,}000$ st	$15{,}000$ st	$15{,}000$ st
Batch size	$4$	$16$	$16$	$32$

Appendix

Table 1: Per-model fine-tuning hyperparameters used in the benchmark. All runs use a single H100

80

GB GPU. Total parameters / trainable parameters appear in Table 4 .

Figure 1: Training loss curves for the three models with step-level logs. OpenVLA optimizes cross-entropy over discrete action tokens; GR00T optimizes denoising score-matching; $\pi_{0}$ optimizes the flow-matching velocity objective. Annotations report the final loss value.

Figure 2: Per-dimension prediction-vs.-ground-truth scatter for SmolVLA on the held-out validation split. The dashed diagonal is the identity line; $\Delta x_{\ell}$ is positive but range-compressed while $\Delta y_{\ell}$ and $\Delta z$ collapse toward the prior mean.

Figure 3: Per-dimension prediction-vs.-ground-truth scatter for OpenVLA-7B on the held-out split. Note the horizontal banding caused by $256$ -bin discretization and the near-vertical collapse on $\Delta z$ where the discretization grid resolution exceeds the empirical action range.

Figure 4: Cumulative distribution of per-dimension absolute error across all four models on the held-out validation split. The $x$ -axis uses a symmetric log scale to preserve both small and large error magnitudes. $\pi_{0}$ (orange) and SmolVLA (purple) nearly coincide, with $\pi_{0}$ holding the right-tail edge; OpenVLA (red) is shifted right on $\Delta x_{\ell}$ ; GR00T (green) lies far above the empirical scale. On $\Delta y_{\ell}$ / $\Delta z$ all working models track the naive baseline.

Model	$\Delta x_{\ell}$	$\Delta y_{\ell}$	$\Delta z$	$\Delta\psi$
openvla (train)	0.472	0.297	0.004	0.358
openvla (val)	0.620	0.368	0.005	0.409
gap	+0.149	+0.070	+0.001	+0.052
smolvla (train)	0.083	0.050	0.002	0.063
smolvla (val)	0.468	0.264	0.003	0.243
gap	+0.385	+0.214	+0.002	+0.180

Appendix

Table 2: Train vs. validation MAE per dimension (the gap quantifies overfitting). Models with no train predictions available are omitted.

Verb	OpenVLA	GR00T	$\pi_{0}$	SmolVLA
accompany	0.300 (56)	2.594 (53)	0.221 (56)	0.197 (56)
approach	0.258 (229)	2.595 (289)	0.180 (229)	0.199 (229)
fly after	0.665 (19)	3.317 (14)	0.255 (19)	0.245 (19)
follow	0.310 (108)	3.349 (133)	0.202 (108)	0.233 (108)
go after	0.554 (46)	2.719 (62)	0.264 (46)	0.297 (46)
move closer	0.382 (52)	2.383 (45)	0.187 (52)	0.190 (52)

Appendix

Table 3: Mean validation error broken down by task-verb family (averaged across the four action dimensions). We omit verb families with fewer than

5

validation samples.

Figure 5: Per-dimension MAE on the train (lighter bars) and validation (darker bars) splits, for each model with train predictions available. Every model shows a large train–val gap; SmolVLA’s is the largest despite its lowest training error, while OpenVLA underfits even the training split.

Figure 6: Mean error (line) vs. ground-truth magnitude, with interquartile range shaded, binned into six quantile bins. SmolVLA’s error grows sub-linearly with $|GT|$ on the in-plane axes; OpenVLA’s error is roughly magnitude-independent (constant error floor); GR00T’s predictions are essentially decorrelated from $|GT|$ .

Figure 7: Per-episode MAE heatmap, log-color scaled, with rows corresponding to episodes (shortened UUID labels on the leftmost panel) and columns to models. Each panel shows one action dimension. Darker is lower error; the structural pattern is consistent across dimensions: flow-matching models ( smolvla , pi0 ) are uniformly low-error while the discrete and diffusion baselines are uniformly high.

Figure 8: Per-verb-family mean validation error (averaged across the four action dimensions) on a symmetric-log scale. Within each model, performance is stable across verb families; the model ranking is preserved across all linguistic conditions.

Figure 9: Mean error vs. normalized frame position in episode (deciles). All models are essentially flat, indicating no systematic within-episode error drift; this is the expected behavior for a well-curated dataset.

Figure 10: Same six held-out frames, with predictions overlaid from all four VLA models. Solid blue arrows are GT; dashed arrows are per-model predictions in the model’s reference color ( red : OpenVLA; green : GR00T; orange : $\pi_{0}$ ; purple : SmolVLA). The flow-matching predictions match GT direction but under-reach in magnitude; the discrete-token prediction undershoots further; the diffusion prediction points off-frame.

Figure 11: Per-dimension prediction–ground-truth Pearson correlation for all four baselines at $1$ FPS. Greener is higher. The forward channel is the most learnable for every model; $\Delta z$ is at chance throughout; $\pi_{0}$ dominates column-wise.

Figure 12: Pairwise per-episode win rate $P(\text{row beats column})$ in mean held-out MAE at $1$ FPS. The ordering $\pi_{0}\!>\!$ SmolVLA $\!>\!$ OpenVLA $\!>\!$ GR00T holds at the per-episode level, not merely in aggregate.

Panel of Figure 13 — Figure 13: Horizontal-speed calibration at $10$ FPS: ground-truth (blue) vs. predicted (orange) step-magnitude distributions for $\pi_{0}$ (left) and SmolVLA (right). Both models under-predict the spread (dashed means), the magnitude-compression signature.

Panel of Figure 14 — Figure 14: Yaw-turn confusion matrices at $10$ FPS (rows: ground-truth turn; columns: predicted) for $\pi_{0}$ (left) and SmolVLA (right). Straight motion is recovered well; left/right turns collapse toward “straight”, explaining the modest $\Delta\psi$ correlation.

Figure 15: Per-dimension prediction-vs.-ground-truth scatter for $\pi_{0}$ on the $10$ FPS held-out split ( $N{=}13{,}371$ ). The dashed diagonal is the identity line. The qualitative signature is identical to $1$ FPS (Fig. 10 ) at one-tenth the magnitude scale: a positive but range-compressed $\Delta x_{\ell}$ trend, partial $\Delta y_{\ell}$ structure and $\Delta z$ collapsed to the prior band.

Figure 16: SmolVLA per-dimension prediction-vs.-ground-truth scatter at $10$ FPS. The pattern matches $\pi_{0}$ (Fig. 15 ) with steeper $\Delta x_{\ell}$ range compression, consistent with SmolVLA’s larger generalization gap.

Figure 17: SmolVLA open-loop rollouts (orange) vs. expert paths (blue) on six $10$ FPS held-out episodes. Drift on curved segments is larger than $\pi_{0}$ ’s (Fig. 16 ), consistent with the ADE/FDE ranking of Table 8 .

Figure 18: Per-dimension absolute-error CDF at $10$ FPS (symmetric-log $x$ ). $\pi_{0}$ holds the right-tail edge over SmolVLA on $\Delta x_{\ell}$ , while both nearly coincide with the naive baseline on $\Delta y_{\ell}$ / $\Delta z$ / $\Delta\psi$ the cumulative view of the per-step MAE saturation at high control rate.

Explore similar work

Jul 7, 2026cs.RO

Vision Language Action (VLA) Models for Unmanned Aerial Robotics and Bimanual Manipulation: A Review

Vision Language Action (VLA) models unify visual perception, natural-language understanding, and action generation within a single foundation model, allowing a robot to follow instructions such as fold the towel or fly to the red building directly from camera images. Because VLAs inherit world knowledge from internet-scale pre-training, they have become the dominant framework for learning-based manipulation, with bimanual coordination serving as the most demanding testbed: two arms with 7 degrees of freedom each must move in concert to fold, assemble, and reorient objects. Unmanned aerial robotics faces a structurally similar challenge: a drone must coordinate thrust, attitude, and increasingly gripper commands from visual observations under strict latency and payload constraints. This review covers 183 contributions spanning 2017-2026 and organized along seven dimensions: VLA architectures, training recipes, action representations, bimanual coordination (2022-2026), unmanned aerial vehicle (UAV) navigation and control (2017-2026), language grounding, and cross-cutting concerns including memory and world models. We show that the coordination strategies, training recipes, and action representations developed for bimanual VLAs transfer to unmanned aerial systems and identify fourteen research directions across both domains.

Inkyu Sa, Chanoh Park, Hea-Min Lee +2

Chef Robotics, San Francisco, CA 94103, USA · RovifyLab, Gyeonggi 13840, Republic of Korea · IT Application Research Center, Jeonbuk Regional Branch, Korea Electronics Technology Institute (KETI), Jeonju 54853, Republic of Korea +1

Bimanual Manipulation Gripper

Ask PDF

Jul 16, 2026cs.RO

CosFly-VLA: A Spatially Aware Vision-Language-Action Model for UAV Tracking

Dynamic target tracking is essential for Unmanned Aerial Vehicles (UAVs) operating in complex urban environments, where both the target and the camera viewpoint change continuously. Existing Vision-Language-Action (VLA) policies can track visible targets effectively, but their performance often degrades when buildings, vegetation, or roadside objects block the line of sight. During sustained occlusion, a policy may lose the target state, execute actions toward an incorrect region, and amplify this error through subsequent observations until re-acquisition becomes impossible. To this end, we present CosFly-VLA, a spatially aware VLA model that jointly grounds the target, estimates its visibility, and generates continuous flight actions through a structured prediction interface. To train this policy, we use a large-scale recipe over diverse data sources. Spatially Grounded Continued Pretraining (CPT) on a 500k mixed pool injects UAV-view depth, distance, and 3-D spatial reasoning. A three-stage Curriculum-based Supervised Fine-Tuning (SFT) process then specializes the tracker through multi-head warm-up followed by two-stage curriculum learning over natural and hard / long-occlusion data. Chain-of-Thought (CoT) training subsequently teaches recovery-oriented reasoning traces before structured answers. Finally, a closed-loop Reinforcement Learning (RL) stage optimizes tracking behavior with a multi-component reward covering stand-off tracking, grounding quality, collision avoidance, and task success. Relative to OpenVLA, CosFly-VLA-0.8B reduces open-loop Average Displacement Error (ADE) by 34.1% on seen-test and 35.3% on unseen-test. Closed-loop optimization improves Success Rate (SR) by 29.8% and 2.5%, respectively. These results demonstrate progress from visible-frame imitation toward spatially grounded action-closed-loop control, evaluated under a shared oracle state history.

Ruilong Ren, Songsheng Cheng, Yunpeng Zhou +9

1Autel Robotics · 2Northeast Normal University · 3Southern University of Science and Technology +2

Aerial Vision-Language Navigation Unmanned Aerial Vehicle Detection Unmanned Aerial Vehicles

Ask PDF

Jul 3, 2026cs.RO

Exp2VLA: Enabling Vision-Language-Action for Drone Navigation from Expert Demonstrations

Vision-language-action (VLA) models open a new path toward intuitive robot control by directly linking perception, language, and action in a single end-to-end framework. Yet for UAVs, practical adoption remains difficult because existing solutions are either computationally heavy or insufficiently capable in complex environments. In this work, we propose a practical expert-distillation pipeline (Exp2VLA) for language-conditioned drone navigation. The core idea is to distill expert behavior, obtained from reinforcement learning, teleoperation, or other controllers, into training data that can be used to fine-tune compact VLA models. This allows existing control strategies to be transferred into a unified language-guided navigation model, reducing manual system integration and lowering the barrier for deploying new robot behaviors. Experiments in both sim-to-sim and simulation-in-the-loop settings across multi-object scenes show that the fine-tuned models can handle varied semantic commands and generalize to unseen target compositions. The proposed framework demonstrates how expert-policy distillation can help mechatronic systems move from specialized control modules toward more flexible and reusable robot intelligence.

Van Huyen Dang, Kabilesh Rajendran, Erdi Sayar +1

Automatic Control Group Paderborn University Paderborn, Germany

Autonomous Drones Unmanned Aerial Vehicles

Ask PDF