Visual Prototype Conditioned Focal Region Generation for UAV-Based Object Detection

Wenhao Li; Zimeng Wu; Yu Wu; Zehua Fu; Jiaxin Chen

cs.CVApr 3, 2026

Visual Prototype Conditioned Focal Region Generation for UAV-Based Object Detection

Authors: Wenhao Li, Zimeng Wu, Yu Wu, Zehua Fu, Jiaxin Chen

Organizations: State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, China · School of Computer Science and Engineering, Beihang University, Beijing, China · Hangzhou Innovation Institute, Beihang University, Hangzhou, China

Abstract

Unmanned aerial vehicle (UAV) based object detection is a critical but challenging task, when applied in dynamically changing scenarios with limited annotated training data. Layout-to-image generation approaches have proved effective in promoting detection accuracy by synthesizing labeled images based on diffusion models. However, they suffer from frequently producing artifacts, especially near layout boundaries of tiny objects, thus substantially limiting their performance. To address these issues, we propose UAVGen, a novel layout-to-image generation framework tailored for UAV-based object detection. Specifically, UAVGen designs a Visual Prototype Conditioned Diffusion Model (VPC-DM) that constructs representative instances for each class and integrates them into latent embeddings for high-fidelity object generation. Moreover, a Focal Region Enhanced Data Pipeline (FRE-DP) is introduced to emphasize object-concentrated foreground regions in synthesis, combined with a label refinement to correct missing, extra and misaligned generations. Extensive experimental results demonstrate that our method significantly outperforms state-of-the-art approaches, and consistently promotes accuracy when integrated with distinct detectors. The source code is available at https://github.com/Sirius-Li/UAVGen.

Figures & tables

Figure 1: Illustration of different layout-to-images data generation methods. (a) In general data generation, the layout information is directly derived from real data, and synthesized data is directly used as training data for the detector. (b) Our UAVGen enhances this paradigm by introducing multi-cue, high-quality layout conditions for focal regions, as well as refining the synthesized data.

Figure 2: Architecture of Visual Prototype Conditioned Focal Region Generation. (a) Virtual Prototype Conditioned Diffusion Model (VPC-DM) generates images guided by layout images which is produced from selected visual prototypes. (b) Focal Region-Enhanced Data Pipeline (FRE-DP) synthesizes images on object-centric areas to avoid limitation of small object generation. Moreover, Label Refinement mitigates the misalignment between layouts and generated images.

Dataset	Method	FID $\downarrow$	Average Precision $\uparrow$
Dataset	Method	FID $\downarrow$	mAP	AP 50	AP 75	AP s	AP m	AP l
VisDrone [ 9 ]	Real only	-	24.5	42.1	24.5	15.4	36.1	37.9
	CopyPaste [ 11 ]	-	23.6	41.0	23.5	14.7	34.6	40.2
	GLIGEN [ 31 ]	72.68	24.8	43.0	24.6	15.6	36.3	39.0
	Geodiffusion [ 3 ]	57.96	24.7	42.9	24.7	15.4	36.1	39.1
	AeroGen [ 46 ]	48.04	24.9	43.3	24.5	15.6	36.3	43.3

Table 1: Comparison of FID scores and Average Precision (%) of UAV-based detector trained with augmented data produced by different methods on VisDrone and UAVDT. “Real only” means the detector is trained only on real data. The best results are highlighted in bold .

Figure 3: Comparison of mAP across different categories on VisDrone. Our method (w syn.) generates 738 synthesized images, yields consistent performance improvements across all categories when compared to the non-augmented baseline (w/o syn.).

Figure 4: Comparison of generated images on VisDrone. Our method exhibits superior layout-image consistency and enhanced visual fidelity of generated small-scale objects. The yellow dashed boxes denote the regions where the generated targets are inconsistent with the inputs, as well as blurry low-quality targets. For the generation of small pedestrian targets(line 1), our method achieves significantly higher clarity than the comparative methods. Furthermore, in scenarios with dense small targets(line 2, 3), our method exhibits superior consistency between objects and layout.

Method	mAP $\uparrow$	AP 50 $\uparrow$	AP 75 $\uparrow$
Real only	29.8	48.1	30.8
GLIGEN [ 31 ]	29.4	47.7	30.2
Geodiffusion [ 3 ]	29.4	47.8	30.2
AeroGen [ 46 ]	28.8	46.9	29.6
UAVGen (Ours)	30.2	48.7	31.1

Table 2: Comparison of Average Precisions (%) achieved by different methods in enhancing the state-of-the-art UAV-based object detection model RemDet-X on VisDrone. The best results are highlighted in bold .

VPC-DM			FRE-DP		mAP $\uparrow$	AP 50 $\uparrow$
Gen.	VP	LE	FR	LR	mAP $\uparrow$	AP 50 $\uparrow$
					24.5	42.1
✓					23.8	42.1
✓	✓				24.1	42.4
✓	✓	✓			25.2	43.8
✓	✓	✓	✓		25.5	44.5

Table 3: Ablation study on proposed methods in generation pipeline. ”Gen.” suggests train detector with generated images, “VP” suggests Visual Prototype, ”LE” suggests Layout Embeddings, ”FR” suggests Focal Region Generation, ”LR” suggests Label Refinement.

FR Res.	mAP $\uparrow$	AP 50 $\uparrow$	AP 75 $\uparrow$
1024	25.3	43.9	25.3
512	25.6	44.7	25.4
256	25.9	44.8	26.0

Table 4: Ablation study on resolution scale of focal region.

Figure 5: Impact of various scale of generated images on object detection model. With reduced images, our method achieves superior performance.

Scale	mAP*	mAP	AP 50	AP 75
10%	13.7	25.6	44.4	25.7
50%	19.8	25.9	45.1	25.9
100%	21.5	25.9	44.8	26.0

Table A : Ablation study on initial detector for full pipeline. “mAP*” shows initial detector mAP.

Detector	mAP*	mAP	AP 50	AP 75
w/o LR	-	25.5	44.5	25.6
YOLOv3-tiny	10.1	25.7	44.7	25.8
Faster R-CNN	21.5	25.9	44.8	26.0

Table B : Ablation study on initial detector for Label Refinement. “mAP*” shows initial detector mAP.

$t^{g}_{i}$	$t^{c_{j}}$	mAP	AP 50	AP 75
✓		24.1	42.4	24.0
	✓	25.1	43.6	25.2
✓	✓	25.2	43.8	25.1

Table C : Ablation study on text prompt

Method	mAP	AP 50	AP 75
Real Only (VisDrone)	24.5	42.1	24.5
w/ UAVGen (UAVDT)	24.8	43.3	24.8

Table D : Ablation study on cross-dataset enhancement.

Image Generation

Image generation studies models that create or edit images from text, examples, layouts, and other structured controls.

See all 463 papers

SFFNet: Synergistic Feature Fusion Network With Dual-Domain Edge Enhancement for UAV Image Object Detection
Apr 3, 2026Wenfeng Zhang, Jun Ni, Yue Meng +4Object Detection Aerial Object Detection
College of Computer and Information Science, Chongqing Normal University, Chongqing 401333, China · CETC Yizhihang (Chongqing) Technology Co., Ltd, Chongqing 400031, China · School of Computer Engineering, Weifang University, Weifang 261061, China +1
Optimizing Data Augmentation for Real-Time Small UAV Detection: A Lightweight Context-Aware Approach
Apr 21, 2026Amir Zamani, Zeinab AbediniData Augmentation Small Object Detection
Department of Computer Engineering, Comprehensive University of the Islamic Revolution, Tehran, Iran · Department of Computer Engineering, Sharif University of Technology, Tehran, Iran
DroneFINE: Domain-Aware Parameter-Efficient Fine-Tuning of Vision-Language Detectors for Drone Images
Jul 1, 2026Ke Wu, Yanan Zhang, Yingjie Gao +5Open-Vocabulary Object Detection VLM Adaptation
National College for Excellent Engineers, Beihang University, Beijing, China · School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, China · School of Computer Science and Engineering, Beihang University, Beijing, China +2

Apr 3, 2026J/K move · Enter open · S save

SFFNet: Synergistic Feature Fusion Network With Dual-Domain Edge Enhancement for UAV Image Object Detection

Wenfeng Zhang, Jun Ni, Yue Meng, Xiaodong Pei+3

College of Computer and Information Science, Chongqing Normal University, Chongqing 401333, China · CETC Yizhihang (Chongqing) Technology Co., Ltd, Chongqing 400031, China · School of Computer Engineering, Weifang University, Weifang 261061, China+1

Object Detection Aerial Object Detection Efficient Object Detection Multi-Scale Feature Fusion

Open paper AskPDF arXiv

Object detection in unmanned aerial vehicle (UAV) images remains a highly challenging task, primarily caused by the complexity of background noise and the imbalance of target scales. Traditional methods easily struggle to effectively separate objects from intricate backgrounds and fail to fully leverage the rich multi-scale information contained within images. To address these issues, we have developed a synergistic feature fusion network (SFFNet) with dual-domain edge enhancement specifically tailored for object detection in UAV images. Firstly, the multi-scale dynamic dual-domain coupling (MDDC) module is designed. This component introduces a dual-driven edge extraction architecture that operates in both the frequency and spatial domains, enabling effective decoupling of multi-scale object edges from background noise. Secondly, to further enhance the representation capability of the model's neck in terms of both geometric and semantic information, a synergistic feature pyramid network (SFPN) is proposed. SFPN leverages linear deformable convolutions to adaptively capture irregular object shapes and establishes long-range contextual associations around targets through the designed wide-area perception module (WPM). Moreover, to adapt to the various applications or resource-constrained scenarios, six detectors of different scales (N/S/M/B/L/X) are designed. Experiments on two challenging aerial datasets (VisDrone and UAVDT) demonstrate the outstanding performance of SFFNet-X, achieving 36.8 AP and 20.6 AP, respectively. The lightweight models (N/S) also maintain a balance between detection accuracy and parameter efficiency. The code will be available at https://github.com/CQNU-ZhangLab/SFFNet.

Abstract

Image Generation

Explore similar work

SFFNet: Synergistic Feature Fusion Network With Dual-Domain Edge Enhancement for UAV Image Object Detection

Optimizing Data Augmentation for Real-Time Small UAV Detection: A Lightweight Context-Aware Approach

DroneFINE: Domain-Aware Parameter-Efficient Fine-Tuning of Vision-Language Detectors for Drone Images