cs.CVDec 8, 2025

Towards Accurate UAV Image Perception: Guiding Vision-Language Models with Stronger Task Prompts

Authors: Mingning Guo, Mengwei Wu, Shaoxian Li, Haifeng Li, Chao Tao

Organizations: School of Geosciences and InfoPhysics, Central South University, Changsha 410083, China.

Abstract

UAV image perception with VLMs aims to identify and localize targets based on semantic and spatial task prompts. However, small, densely distributed objects and cluttered backgrounds hinder these tasks, particularly when concise prompts provide insufficient guidance. Although task-specific fine-tuning improves image-text alignment, it requires large-scale annotations and model-specific updates, increasing adaptation costs across models and tasks. To address this, we propose AerialVP, a task-adaptive, training-free agent that adapts VLMs through prompt enhancement rather than parameter optimization. AerialVP dynamically adapts its enhancement strategy to the original prompt's specificity, selectively incorporating image-derived semantics, spatial positions, and object relationships. These cues provide discriminative descriptions and spatial constraints, leveraging VLMs' pretrained alignment and reasoning capabilities to improve image-text alignment and reduce target ambiguity. Specifically, an LLM-based Task Engine coordinates modular tools for prompt analysis, visual information extraction, and prompt fusion. The enhanced prompt and original image are then provided to the unchanged backend VLM for prediction. This prompt-centric strategy enables training-free adaptation across diverse open-source and proprietary VLMs. For systematic evaluation, we introduce AerialSense, a unified benchmark covering Aerial Visual Grounding, Aerial Visual Reasoning, and Aerial Visual Question Answering across diverse UAV scenes and imaging conditions. Experiments demonstrate consistent improvements across all three tasks and evaluated VLMs. AerialVP also achieves competitive performance against LoRA-based instruction fine-tuning, validating prompt-centric adaptation. Our work is available at https://github.com/lostwolves/AerialVP.

Figures & tables

Explore similar work

CardsList
  1. Drone-PUR: From Visual Clue Discovery to Understanding and Reasoning in UAV Scenes

    Oct 29, 2026Jiahe Wu, Bing Cao, Xingyue Guo +5Visual Spatial ReasoningVLM Evaluation

  2. DroneFINE: Domain-Aware Parameter-Efficient Fine-Tuning of Vision-Language Detectors for Drone Images

    Jul 1, 2026Ke Wu, Yanan Zhang, Yingjie Gao +5Open-Vocabulary Object DetectionVLM Adaptation

  3. UAVBench and UAVIT-1M: Benchmarking and Enhancing MLLMs for Low-Altitude UAV Vision-Language Understanding

    Mar 15, 2026Yang Zhan, Yuan YuanVision-Language ModelsVLM Evaluation