Leveraging Large Vision Model for Multi-UAV Co-perception in Low-Altitude Wireless Networks

Yunting Xu; Jiacheng Wang; Ruichen Zhang; Changyuan Zhao; Yinqiu Liu; Dusit Niyato; Liang Yu; Haibo Zhou

cs.CVMar 13, 2026

Leveraging Large Vision Model for Multi-UAV Co-perception in Low-Altitude Wireless Networks

Authors: Yunting Xu, Jiacheng Wang, Ruichen Zhang, Changyuan Zhao, Yinqiu Liu, Dusit Niyato, Liang Yu, Haibo Zhou,

Organizations: College of Computing and Data Science, Nanyang Technological University, Singapore, 639798 · Alibaba Cloud, Hangzhou, China, 311121 · School of Electronic Science and Engineering, Nanjing University, Nanjing, China, 210023 · Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon 16419, South Korea

Abstract

Multi-uncrewed aerial vehicle (UAV) cooperative perception has emerged as a promising paradigm for diverse low-altitude economy applications, where complementary multi-view observations are leveraged to enhance perception performance via wireless communications. However, the massive visual data generated by multiple UAVs poses significant challenges in terms of communication latency and resource efficiency. To address these challenges, this paper proposes a communication-efficient cooperative perception framework, termed Base-Station-Helped UAV (BHU), which reduces communication overhead while enhancing perception performance. Specifically, we employ a Top-K selection mechanism to identify the most informative pixels from UAV-captured RGB images, enabling sparsified visual transmission with reduced data volume and latency. The sparsified images are transmitted to a ground server via multi-user MIMO (MU-MIMO), where a Swin-large-based MaskDINO encoder extracts bird's-eye-view (BEV) features and performs cooperative feature fusion for ground vehicle perception. Furthermore, we develop a diffusion model-based deep reinforcement learning (DRL) algorithm to jointly select cooperative UAVs, sparsification ratios, and precoding matrices, achieving a balance between communication efficiency and perception utility. Simulation results on the Air-Co-Pred dataset demonstrate that, compared with traditional CNN-based BEV fusion baselines, the proposed BHU framework improves perception performance by over 5% while reducing communication overhead by 85%, providing an effective solution for multi-UAV cooperative perception under resource-constrained wireless environments.

Figures & tables

Fig. 1: Illustration of multi-UAV cooperative perception scenario. The UAV-captured aerial images are transmitted to a ground server via MU-MIMO communication, where multi-view information is fused to enable cooperative feature learning for vehicle instance and motion detection in low-altitude economy scenarios.

Fig. 2: Illustration of the proposed BHU framework for multi-UAV cooperative perception. The aerial images are first sparsified via a Top-K selection mechanism and transmitted to the ground server through wireless links. A MaskDINO-based encoder extracts multi-scale features, which are projected into BEV representations and cooperatively fused across multi-UAVs. The fused BEV features are employed for downstream perception tasks, including vehicle instance segmentation and motion flow prediction.

Fig. 3: Illustration of the proposed DDIM-based DRL framework. An MLP network is employed to determine the cooperative UAVs and Top-K sparsification ratios, while a DDIM module is employed to generate precoding actions by modeling the conditional distribution of optimal precoding vectors through a reverse denoising process.

Fig. 4: Visualization of 3D object detection results in the BEV representation.

Method	IoU (%) ↑			PQ (%) ↑			Comm. Cost (Bps) ↓
DHD [ 19 ]	Short	Long	Avg.	Short	Long	Avg.	Top-k (25%)
EfficientNet (4 UAVs)	60.58	53.14	56.86	51.80	45.89	48.85	$6.14\times 10^{7}$
EfficientNet (2 UAVs)	50.59	40.82	45.71	43.43	35.37	39.40	$2.05\times 10^{7}$
EfficientNet w/o Fusion	41.31	33.11	37.21	35.07	28.29	31.68	$0$
MaskDINO (ResNet50)	63.99	55.97	59.98	54.24	48.23	51.24	$6.14\times 10^{7}$
MaskDINO (SwinL)	65.52	57.66	61.59	54.10	50.63	52.37	$6.14\times 10^{7}$

TABLE I: Performance comparison of EfficientNet- and MaskDINO-based networks with/without BEV fusion.

Method	Metrics (%) ↑		Parameters ↓
SwinL (Backbone)	IoU	Prec.	Total	Training
EfficientNet Baseline	52.89	70.15	39.5 M	39.5 M
BHU w pretrained	57.81	77.57	0.3 B	0.3 B
BHU w/o pretrained	53.27	72.22	0.3 B	0.3 B
DHD + LoRA	55.24	73.40	$\sim$ 0.3 B	37.4 M
BHU + LoRA	55.86	74.38	$\sim$ 0.3 B	37.4 M

TABLE II: Results of backbone networks and LoRA fine-tuning.

Fig. 8: IoU versus Top-K ratios under SwinL and LoRA settings.

Method	IoU (%) ↑			PQ (%) ↑			Comm. Cost (Bps) ↓
BHU (Proposed)	Short	Long	Avg.	Short	Long	Avg.	Top-k (25%)
DHD (EfficientNet) Baseline	60.58	53.14	56.86	51.80	45.89	48.85	6.14 $\times$ $10^{7}$
BHU (EfficientNet)	57.39	52.89	55.14	47.23	45.68	46.46	$\textbf{9.22}\times 10^{6}$
MaskDINO (ResNet50)	64.00	56.32	60.16	56.05	49.40	52.73	$\textbf{9.22}\times 10^{6}$
MaskDINO (SwinL)	65.71	57.81	61.76	54.64	51.26	52.95	$\textbf{9.22}\times 10^{6}$
MaskDINO (SwinL) + LoRA	63.51	55.86	59.69	53.74	48.58	51.16	$\textbf{9.22}\times 10^{6}$

TABLE III: Comparison of the proposed BHU scheme and baselines in terms of utility and communication cost.

Communication-Efficient Collaborative Perception

Communication-efficient collaborative perception studies how networked agents share useful sensor information with minimal bandwidth while improving their joint view of the scene.

See all 26 papers

On Privacy-Preserving Image Transmission in Low-Altitude Networks: A Swin Transformer-Based Framework with Federated Learning
May 12, 2026Kexin Zhang, Lixin Li, Yuna Yan +5Semantic Communication Wireless Communications
School of Electronics and Information, Northwestern Polytechnical University, Xi’an 710129, China · DecoreX Intelligent Technologies Co., Ltd., Xi’an 710075, China · Samsung AI Center, Cambridge CB1 2JH, UK +2
WaveComm: Lightweight Communication for Collaborative Perception via Wavelet Feature Distillation
Mar 9, 2026Erdemt Bao, Jin YangCommunication-Efficient Collaborative Perception Cooperative Perception
School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan 430074, China. · National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, and Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Shaanxi 710049, China.
V2U4Real: A Real-world Large-scale Dataset for Vehicle-to-UAV Cooperative Perception
Mar 26, 2026Weijia Li, Haoen Xiang, Tianxu Wang +4Autonomous Driving Datasets Cooperative Perception
Fujian Key Laboratory of Urban Intelligent Sensing and Computing, Xiamen University, China · Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, China

May 12, 2026J/K move · Enter open · S save

On Privacy-Preserving Image Transmission in Low-Altitude Networks: A Swin Transformer-Based Framework with Federated Learning

Kexin Zhang, Lixin Li, Yuna Yan, Xin Zhang+4

School of Electronics and Information, Northwestern Polytechnical University, Xi’an 710129, China · DecoreX Intelligent Technologies Co., Ltd., Xi’an 710075, China · Samsung AI Center, Cambridge CB1 2JH, UK+2

Semantic Communication Wireless Communications UAV Communication

Open paper AskPDF arXiv

The rapid development of low-altitude economy has driven the proliferation of Unmanned Aerial Vehicle (UAV) applications, including logistics, inspection, and emergency response. However, transmitting high-volume image data from UAVs to ground stations faces significant challenges due to limited bandwidth and stringent privacy requirements. To address these issues, a Semantic Communication (SC) framework based on Federated Learning (FL) is proposed for efficient and privacy-preserving image transmission. A Swin Transformer-based Semantic Communication (STSC) architecture is designed to extract multi-scale semantic features under constrained bandwidth conditions. Dedicated communication and computing nodes are deployed on UAVs to enhance real-time coverage and flexibility. Meanwhile, a FL mechanism enables global model training across distributed devices without sharing raw data, thus preserving user privacy. Simulation experiments conducted on the CIFAR-10 dataset demonstrate that the proposed STSC framework achieves at least 5.7 dB improvement in Peak Signal-to-Noise Ratio (PSNR) compared to DeepJSCC baselines, while also showing superior convergence and generalization performance. The framework effectively integrates UAV-assisted deployment with SC and privacy protection, offering a practical solution for bandwidth-constrained image transmission in low-altitude networks.

Abstract

Communication-Efficient Collaborative Perception

Explore similar work

On Privacy-Preserving Image Transmission in Low-Altitude Networks: A Swin Transformer-Based Framework with Federated Learning

WaveComm: Lightweight Communication for Collaborative Perception via Wavelet Feature Distillation

V2U4Real: A Real-world Large-scale Dataset for Vehicle-to-UAV Cooperative Perception