AeroManip-VLA: Scalable Vision-Language-Action Learning for Aerial Manipulation with RL-Generated Demonstrations

Rui Huang; Yanlin Mu; Lidong Li; Yucong Wang; Zichen Yan; Lin Zhao

cs.ROSep 29, 2026

AeroManip-VLA: Scalable Vision-Language-Action Learning for Aerial Manipulation with RL-Generated Demonstrations

Authors: Rui Huang, Yanlin Mu, Lidong Li, Yucong Wang, Zichen Yan, Lin Zhao

Organizations: National University of Singapore · Beijing Institute of Technology

Abstract

Aerial manipulators extend robotic manipulation into 3D workspaces that are difficult for ground-based robots to access, creating new opportunities for general-purpose manipulation. However, extending Vision-Language-Action (VLA) models to aerial robots introduces distinct challenges due to the tight coupling between manipulation and flight, continuously changing observations, and safety-critical physical interactions. These challenges demand diverse training data and systematic policy evaluation, yet collecting demonstrations and evaluating policies directly on physical aerial platforms are costly, difficult to scale, and hard to repeat under controlled conditions. We present AeroManip-VLA, a scalable benchmark for aerial VLA data generation and policy evaluation. AeroManip-VLA provides a GPU-accelerated simulation framework with low-level payload-aware flight and manipulation control in massively parallel environments. Building on this framework, we combine reusable reinforcement learning policies with expert task rules to automatically generate demonstrations without human teleoperation across diverse objects, environments, and randomized initial conditions. The generated data include basic skills such as grasping and placing, as well as long-horizon tasks that require both navigation and manipulation. We further introduce automated event labeling and trajectory categorization to filter demonstrations. These mechanisms enable fine-grained analysis of task progress, behavioral outcomes, and safety-related failures. Finally, we evaluate a range of imitation learning and VLA baselines across different task settings, revealing their performance characteristics and failure modes. Together, AeroManip-VLA enables scalable aerial manipulation data generation, structured trajectory analysis, and systematic VLA evaluation in simulation prior to real-world deployment.

Figures & tables

Figure 1: Overview of the AeroManip-VLA framework and dataset for aerial manipulation. Project page: https://ruihuangnus.github.io/AeroManip-VLA-page/

Figure 2: BC is substantially more interaction-efficient than PPO, but relies on expert demonstrations. BC markers report held-out success after training on successful expert trajectories from 1, 2, 4, and 8 collection rounds (256 rollouts each). PPO curves show training success from scratch (400-episode moving average), while dotted lines denote expert held-out success. The broken log-scale

x

-axis reports demonstration transitions for BC and environment interactions for PPO. One training seed per method.

Figure 3: PPO produces substantially more diverse successful approach trajectories than the expert and BC. Top: successful approach segments from drone spawn to within 5 cm of the planned grasp pose, shown in a start-normalised frame with the grasp pose at the origin and all trajectories starting at

(1,1)

. Expert and BC follow a similar L-shaped approach, whereas PPO spreads across the

X

–

Z

plane. Bottom: deviation from each policy’s own mean path, spatial coverage measured by occupied

0.05\times 0.05

cells, and velocity-direction entropy over 36 bins. Coverage and entropy are estimated from 200 random subsets of 80 episodes per policy; bars show the mean and 95% interval.

Figure 4: PPO discovers diverse yet substantially faster successful trajectories. Left: mean time to success decomposed into approach, alignment, grasp, lift, and hold phases. PPO reduces total execution time from 15.8 s to 3.8 s on TidyHouse and from approximately 14.7 s to 4.0 s on PrepareGroceries, primarily through a shorter approach phase. Right: distribution of successful episode lengths for each policy.

	Skill	Master Chef Can	Sugar Box	Tomato Soup Can	Tuna Fish Can	Pudding Box	Gelatin Box	Potted Meat Can	Bowl	Mean S.R. (Pick + Place)
	Pick	$2.0\pm 2.0$	$0.0\pm 0.0$	$0.0\pm 0.0$	$0.0\pm 0.0$	$0.0\pm 0.0$	$0.0\pm 0.0$	$10.0\pm 4.2$	$18.0\pm 5.4$
ACT	Place	$24.0\pm 6.0$	$98.0\pm 2.0$	$28.0\pm 6.3$	$82.0\pm 5.4$	$46.0\pm 7.0$	$48.0\pm 7.1$	$56.0\pm 7.0$	$60.0\pm 6.9$	29.5
	Pick	$4.0\pm 2.8$	$2.0\pm 2.0$	$6.0\pm 3.4$	$4.0\pm 2.8$	$2.0\pm 2.0$	$2.0\pm 2.0$	$2.0\pm 2.0$	$6.0\pm 3.4$
DP	Place	$50.0\pm 7.1$	$90.0\pm 4.2$	$50.0\pm 7.1$	$78.0\pm 5.9$	$82.0\pm 5.4$	$80.0\pm 5.7$	$32.0\pm 6.6$	$52.0\pm 7.1$	33.9
	Pick	$6.7\pm 4.6$	$6.7\pm 4.6$	$10.0\pm 5.5$	$20.0\pm 7.3$	$13.3\pm 6.2$	$13.3\pm 6.2$	$13.3\pm 6.2$	$6.7\pm 4.6$
PI0	Place	$80.0\pm 5.7$	$82.0\pm 5.4$	$52.0\pm 7.1$	$88.0\pm 4.6$	$52.0\pm 7.1$	$68.0\pm 6.6$	$86.0\pm 4.9$	$24.0\pm 6.0$	38.9

Table 1: Simulation results on AeroManip-VLA PrepareGroceries. For each skill–object pair, the table reports the success rate (%) over evaluation episodes together with its binomial standard error. Mean S.R. is the unweighted average over all 16 Pick/Place–object combinations.

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Representative qualitative rollouts in AeroManip-VLA: (a) short-horizon Pick , (b) short-horizon Place , (c) long-horizon Pick–Navigate–Place , and (d) long-horizon Open–Pick–Navigate–Place . Each sequence is paired with a language instruction and progresses from left to right in time. For each task, the top row shows third-person views, while the middle and bottom rows show onboard RGB and depth observations, respectively. Third-person views are used only for visualization and are not provided to the evaluated policies. All demonstration videos are available in the supplementary materials.

Figure 6: Observation configuration of AeroManip-VLA. The aerial platform is equipped with one forward-facing FPV camera and two gripper-mounted cameras, providing complementary global and local views for navigation and manipulation. The central third-person view illustrates the camera placement and is not provided to the evaluated policy.

Figure 7: GPU-parallel data collection and inference in AeroManip-VLA. Reinforcement learning policies automatically generate diverse aerial manipulation demonstrations across parallel simulation environments without human teleoperation, while batched VLA inference enables efficient, large-scale evaluation.

Figure 8: Qualitative comparison of AeroManip-VLA with ray tracing disabled (left) and enabled (right). Both views are rendered directly by the simulator, illustrating differences in lighting, shadows, and surface shading.

Figure 9: Interact benchmark comparing simulation-and-rendering throughput and GPU memory usage on an RTX 5090. Point labels indicate the number of parallel environments; per control step, each AeroManip-VLA environment renders three

128\times 128

RGB-D cameras and each AIR-VLA environment renders four

640\times 480

RGB cameras. Points show means over

10

seeds, and error bars indicate

95\%

confidence intervals. The vertical axis uses a logarithmic scale. At their highest-throughput tested configurations (

1024

vs.

64

environments), AeroManip-VLA achieves

5.4\times

the throughput of AIR-VLA while using

41\%

of its GPU memory.

Outcome	Payload-aware	Nominal
Success	106 (92.2%)	47 (40.9%)
Grasp lost	3	33
Collision (during pick / transport)	0 / 0	14 / 5
Timeout	6	16

Appendix

Table 2: Outcomes of the expert on 115 TidyHouse episodes with and without payload-aware control (same instances, seeds and start states).

Figure 10: One tomato-can episode (0.37 kg) with payload-aware control (blue) and the nominal controller (orange), from the same start state, aligned at the grasp. From top to bottom: object height above its initial rest height, vertical integrator state

\xi_{z}

(the horizontal line marks the

\pm 2

limit), object slip in the jaws, and airframe tilt.

\times

marks the loss of the grasp.

Figure 11: Per object class, sorted by mass, with the same instances in both conditions (TidyHouse, 14 instances per class, minus initialization rejections). Left to right: median time from grasp to lift-off (open marker: the median episode never lifts the object), mean

|\xi_{z}|

during the carry (vertical line: integrator limit), RMS tilt rate during the carry, and success rate. Carry metrics are averaged over episodes that reach the carry, so they are missing where no nominal episode does (master chef can).

Figure 12: Overall effect of payload-aware control on TidyHouse. Compared with the nominal controller, payload-aware control achieves higher success rate, faster grasp-to-lift-off, lower mean vertical tracking error during transport, and lower RMS tilt rate. Values are aggregated over the evaluated episodes.

Figure 13: Power spectral density (PSD) of airframe tilt rate during TidyHouse carries lasting at least

6.4

s (

115

payload-aware and

101

nominal episodes). PSDs are estimated per episode using Welch’s method with

6.4

s segments sampled at

20

Hz. Solid lines show pointwise medians across episodes; shaded bands indicate the 25th–75th percentiles. Payload-aware control attenuates the pronounced nominal peak near

1.4

Hz, while exhibiting higher PSD over

2

–

5

Hz.

Figure 14: Error propagation across skill boundaries. (a) Failure rate of the shared expert transport after a successful pick, grouped by the policy that produced the grasp and by failure cause. (b) Place success conditioned on successful transport, separated by place policy and grasp source. For PrepareGroceries , the expert-grasp result for PPO place is taken from the held-out place evaluation, whose initial states are generated by expert transport.

Figure 15: Pick-failure rate for each object class on held-out starts, grouped by policy and failure mode. Percentages are computed within each class. Timeout cases are categorized by the last stage reached: never grasped, grasped but never lifted, or lifted but never stabilized.

Figure 16: Placement behavior. (a) Drone speed when the jaws open on the first held-out start of each instance; each dot is one episode and the vertical bar is the median. (b) PPO place outcomes by object class over all held-out place starts. “Toppled after release” denotes an object that was released and followed by retreat but did not come to rest upright inside the goal region.

Figure 17: Pick mechanisms in the diagnostic rerun. (a) Drone speed at first finger contact; the vertical bar is the median. (b) Failed pick episodes grouped by mechanism. “Knocked the object” means that the object moved more than 3 cm or rotated more than 20 ∘ before being grasped; “hit the scene” denotes a collision while the object was undisturbed.

Figure 18: Grasp quality and transport robustness. (a) Object rotation relative to its pre-pick orientation at the pick-to-transport handoff; the vertical bar is the median. (b) Grasp-loss rate during expert transport as a function of handoff rotation, pooling expert and PPO picks. (c) Mechanisms of PPO grasp loss. Bowls are excluded because their planned rim grasp rotates them by design.

Figure 19: Overview of AeroManip-VLA. (a) Simulation and data construction: indoor and outdoor aerial manipulation scenes are instantiated with X500-based platforms, while hybrid expert and RL policies generate manipulation behaviors that are synchronously recorded as multi-view RGB-D observations, robot states, actions, task annotations, and provenance metadata. (b) Task suite: the benchmark covers basic manipulation tasks, including Pick , Place , and Pick–Navigate–Place , as well as long-horizon tasks such as sequential pick–navigate–place and Open–Pick–Navigate–Place . (c) Evaluation and validation: trajectories are evaluated through physical execution criteria, collision and disturbance checks, task-completion validation, and temporal/data-integrity checks, supporting IL, RL, and VLA baselines.

Figure 20: Aerial manipulation platforms in AeroManip-VLA. Left: an X500-based platform with a downward-facing gripper mounted beneath the airframe. Right: an X500-based platform with a forward-reaching, arm-mounted gripper. Both platforms use the same flight-control interface and carry three onboard RGB-D cameras.

	Gripper platform	Arm platform
Total mass $m_{0}$	2.38 kg	2.58 kg
Max thrust per rotor	8.55 N	11.5 N
Thrust-to-weight ratio	1.46	1.82
Manipulator	fixed 1-DOF gripper	3-DOF arm + 1-DOF gripper
Shoulder pitch	–	$[-0.6,1.9]$ rad, 3.0 N m
Elbow pitch	–	$\pm 2.6$ rad, 0.9 N m

Appendix

Table 3: Parameters of the two platforms. Both use the same airframe, flight interface and camera resolution; the right column lists only what the arm platform adds or changes.

Mode	Definition
Successful grasp handoff	$S$ : the Pick completion condition is satisfied.
Collision failure	$F$ and $c=\texttt{collision}$ : the UAV makes unintended contact with non-target scene geometry during execution.
Object-drop failure	$F$ and $c=\texttt{fail\_object\_drop}$ : the object violates the minimum-height condition.
Flight-envelope failure	$F$ and $c\in\{\texttt{fail\_tilt},\texttt{fail\_low},\texttt{fail\_high}\}$ : the UAV exceeds the allowed attitude or altitude envelope, corresponding to excessive tilt, altitude below the lower bound, or altitude above the upper bound, respectively.
Other failure	$F$ with any remaining cause, including workspace violations or unclassified failures.
No-grasp truncation	$T$ and $t_{g}=\infty$ : no grasp is observed before timeout.

Appendix

Table 4: Outcome categories for Pick .

c

denotes the recorded primary failure cause.

Mode	Definition
Successful placement	$S$ : placement, release, clearance, and sustained platform stability satisfy the protocol.
Collision-budget failure	$F$ with primary cause fail_collision_force : the accumulated collision-force score exceeds its configured budget.
Flight-envelope failure	$F$ with primary cause fail_tilt , fail_low , or fail_high .
Workspace failure	$F$ with primary cause fail_workspace : the UAV position violates the predefined workspace boundary during execution.
Object-drop failure	$F$ with primary cause fail_object_drop : object height falls below the floor or support-relative threshold.
Unsupported-release failure	$F$ with primary cause fail_unsupported_release : grasp loss persists for the configured duration without sufficient support evidence and precedes an accepted release.

Appendix

Table 5: Outcome categories for Place , defined from recorded progress events and task-specific failure signals.

Mode	Definition
Successful opening	$S_{\mathrm{art}}$ : the opening threshold and all completion checks are satisfied.
Airframe-contact failure	$\neg N$ : at least one airframe contact is recorded.
Clearance-check failure	$\neg C$ : the minimum sampled geometric clearance is below $0.03\,\mathrm{m}$ .
Handle-approach failure	$p=\texttt{fail\_insert}$ : the approach stage fails to satisfy its target condition within the phase limit.
Handle-pinch failure	$p=\texttt{fail\_pinch}$ : the required finger-contact condition is not satisfied.
Handle-slip failure	$p=\texttt{fail\_slip}$ : handle contact is lost before the opening goal is reached.

Appendix

Table 6: Outcome categories for Open , based on articulation progress, completion checks, and recorded execution phases.

Mode	Definition
Successful closing	$S_{\mathrm{art}}$ : the closing threshold and all completion checks are satisfied.
Airframe-contact failure	$\neg N$ : at least one airframe contact is recorded.
Clearance-check failure	$\neg C$ : the minimum sampled geometric clearance is below $0.03\,\mathrm{m}$ .
Handle-approach failure	$p=\texttt{fail\_insert}$ : the approach stage fails to satisfy its target condition within the phase limit.
Handle-pinch failure	$p=\texttt{fail\_pinch}$ : the required finger-contact condition is not satisfied.
Handle-slip failure	$p=\texttt{fail\_slip}$ : handle contact is lost before the closing goal is reached.

Appendix

Table 7: Outcome categories for Close , using the closing threshold and the same completion and safety checks as Open .

Explore similar work

From Local Whole-Body VLA Behaviors to Scene-Scale Aerial Manipulation
Sep 30, 2026Weixiang Guo, Rui Jin, Haotian Jin +7Aerial Manipulation
School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore 639798 · College of Electronics and Information Engineering, Shanghai Institute of Intelligent Science and Technology, Tongji University, Shanghai 201804, China · School of Aeronautic Science and Engineering, Beihang University, Beijing 100191, China +1
AERMANI-VLM: Structured Prompting and Reasoning for Aerial Manipulation with Vision Language Models
Nov 3, 2025Sarthak Mishra, Rishabh Dev Yadav, Avirup Das +3Aerial Manipulation Vlm-To-Vla Capability Transfer
Robotics Research Center, IIIT Hyderabad, India · Department of Computer Science, University of Manchester, UK · Newcastle University, UK
VLA-REPLICA: A Low-Cost, Reproducible Benchmark for Real-World Evaluation of Vision-Language-Action Models
May 20, 2026Alex S. Huang, Jiahui Zhang, Shiqing Tang +1Large-Scale Robot Demonstration Datasets Evaluation Benchmarks
Intelligent Robotics and Vision Lab, University of Texas at Dallas · Allen High School

Sep 30, 2026, cs.ROJ/K move · Enter open · S save

From Local Whole-Body VLA Behaviors to Scene-Scale Aerial Manipulation

Weixiang Guo, Rui Jin, Haotian Jin, Xinhang Xu+6

School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore 639798 · College of Electronics and Information Engineering, Shanghai Institute of Intelligent Science and Technology, Tongji University, Shanghai 201804, China · School of Aeronautic Science and Engineering, Beihang University, Beijing 100191, China+1

Aerial Manipulation

Open paper AskPDF arXiv

Vision-language-action (VLA) models enable task-conditioned interaction, but extending them to scene-scale aerial manipulation remains challenging due to costly whole-body demonstrations, latency-induced action-state misalignment, and cross-site behavior composition. We present a unified framework for synthetic policy training and scene-scale execution on articulated uncrewed aerial manipulators (UAMs). A scene-reconfigurable pipeline synthesizes task-conditioned, kinodynamically feasible trajectories and synchronized multiview observations for VLA training without physical-platform demonstrations. Measured-progress-aligned realization (MPAR) aligns asynchronously returned action chunks with measured execution progress and realizes them as continuous, dynamically feasible trajectories. A relational Scene Graph grounds language goals to object instances and feasible interaction regions, while topology-guided transfer connects local behaviors across sites. Local VLA skills achieve 39/60 successes (65.0%) in simulation under oracle target and feasible-handoff conditions. Under 500-ms added latency, with and without a transient command-update stall, MPAR reduces median takeover phase error by 0.212 s over nominal-time alignment. The complete system completes 21/50 simulated multi-site missions (42.0%) and is further validated on a physical articulated UAM.

Abstract

Explore similar work

From Local Whole-Body VLA Behaviors to Scene-Scale Aerial Manipulation

AERMANI-VLM: Structured Prompting and Reasoning for Aerial Manipulation with Vision Language Models

VLA-REPLICA: A Low-Cost, Reproducible Benchmark for Real-World Evaluation of Vision-Language-Action Models