Organizations: School of Computer Science, Beijing University of Posts and Telecommunications, Beijing 100876, China. · Meituan Group, Beijing 100102, China. · School of Information and Intelligent Engineering, University of Sanya, Sanya 572000, Hainan Province, China. · Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China. · College of Mechanical and Electrical Engineering, Zhoukou Normal University, Zhoukou 466001, Henan Province, China.
While Vehicle-to-Vehicle (V2V) collaboration extends sensing ranges through multi-agent data sharing, its reliability remains severely constrained by ground-level occlusions and the limited perspective of chassis-mounted sensors, which often result in critical perception blind spots. We propose OpenCOOD-Air, a novel framework that integrates UAVs as extensible platforms into V2V collaborative perception to overcome these constraints. To mitigate gradient interference from ground-air domain gaps and data sparsity, we adopt a transfer learning strategy to fine-tune UAV weights from pre-trained V2V models. To prevent the spatial information loss inherent in this transition, we formulate ground-air collaborative perception as a heterogeneous integration task with explicit altitude supervision and introduce a Cross-Domain Spatial Converter (CDSC) and a Spatial Offset Prediction Transformer (SOPT). Furthermore, we present the OPV2V-Air benchmark to validate the transition from V2V to Vehicle-to-Vehicle-to-UAV. Compared to state-of-the-art methods, our approach improves 2D and 3D [email protected] by 4% and 7%, respectively.
Figures & tables
Fig. 1: Overview of the OpenCOOD-Air framework and its motivation. (1) Naive Collaborative Training suffers from convergence difficulty due to the significant domain gap between ground vehicles and UAVs. (2) Transfer Learning from pre-trained V2V models stabilizes training but introduces spatial misalignment, as it fails to capture the unique altitude dimension of UAVs. (3) Our Method addresses these issues by inheriting V2V knowledge and employing a CDSC module for feature alignment, alongside a SOPT module to explicitly supervise and rectify altitude-induced geometric discrepancies.
Dataset
Mode
Max Vehicles/UAVs
Modality
CoPeD [ 2 ]
V2U
1 / 1
C
V2U-COO [ 3 ]
V2U
1 / 2
C
Griffin [ 4 ]
V2U
1 / 1
L/C
AGC-Drive [ 5 ]
V2V2U
2 / 1
L/C
AirV2X [ 6 ]
V2V2U
5 / 5
L/C
OPV2V-Air(Ours)
V2V2U
7 / 7
L/C
TABLE I: Comparison of Ground-Air Cooperative Perception Datasets
Fig. 2: Overview of the proposed OpenCOOD-Air framework. The architecture is implemented in three steps: (1) training base model for vehicle-to-vehicle collaboration; (2) utilizing the CDSC module and SOPT module to align aerial-ground perspectives; (3) merging trained weights and integrating V2V and V2U features for final ground-air collaborative perception.
Fig. 3: Detailed architectures of the proposed modules.(a) CDSC performs spatial alignment via bilinear interpolation and ConvNeXt-based geometric rectification; (b) SOPT utilizes Transformer encoders to regress the 3D spatial offset Δp for explicit spatial supervision.
Fig. 4: Camera data examples of the OPV2V-Air dataset. The white circle highlight the position of Vehicle 1 and Vehicle 2 in the data of UAV.
Method
Params (M) ↓
2D Eval. ↑
3D Eval. ↑
Ours (Vehicles)
-
39
27
V2X-ViT [ 15 ]
36.53
15
7
CoBEVT [ 16 ]
33.61
33
16
Where2comm [ 18 ]
31.53
41
23
V2VNet [ 13 ]
37.69
66
50
HEAL [ 25 ]
15.02
67
44
TABLE II: Quantitative comparison and model complexity on OPV2V-Air. AP is evaluated at IoU=0.7. ’Params’ denotes the total number of trainable parameters.
Veh. \ UAV
0
1
2
1
10
47
63
2
29
59
70
3
39
63
71
TABLE III: 2D Detection Performance ([email protected]) Across Vehicle-UAV Combinations
Fig. 5: Robust Experiment to pose error. Pose noise is set to N(0,σp2) on x, y location and N(0,σr2) on yaw angle.
Compression Rate
2D Eval. ↑
3D Eval. ↑
Original
71
57
4 ×
67
51
8 ×
66
50
16 ×
64
45
64 ×
42
24
TABLE IV: Impact of Compression Rate on Detection Performance ([email protected]).
Modern autonomous vehicle perception systems are often constrained by occlusions, blind spots, and limited sensing range. While existing cooperative perception paradigms, such as Vehicle-to-Vehicle (V2V) and Vehicle-to-Infrastructure (V2I), have demonstrated their effectiveness in mitigating these challenges, they remain limited to ground-level collaboration and cannot fully address large-scale occlusions or long-range perception in complex environments. To advance research in cross-view cooperative perception, we present V2U4Real, the first large-scale real-world multi-modal dataset for Vehicle-to-UAV (V2U) cooperative object perception. V2U4Real is collected by a ground vehicle and a UAV equipped with multi-view LiDARs and RGB cameras. The dataset covers urban streets, university campuses, and rural roads under diverse traffic scenarios, comprising over 56K LiDAR frames, 56K multi-view camera images, and 700K annotated 3D bounding boxes across four classes. To support a wide range of research tasks, we establish benchmarks for single-agent 3D object detection, cooperative 3D object detection, and object tracking. Comprehensive evaluations of several state-of-the-art models demonstrate the effectiveness of V2U cooperation in enhancing perception robustness and long-range awareness. The V2U4Real dataset and codebase is available at https://github.com/VjiaLi/V2U4Real.
Weijia Li, Haoen Xiang, Tianxu Wang +4
Fujian Key Laboratory of Urban Intelligent Sensing and Computing, Xiamen University, China · Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, China
Air-ground collaborative Vision-and-Language Navigation (VLN) pairs an unmanned aerial vehicle (UAV) with a global bird's-eye view and an unmanned ground vehicle (UGV) with a local first-person view, yet the setting remains largely unexplored: existing training-free methods solve single-agent tasks but offer no collaboration mechanism, and a recent CARLA-Air evaluation found no stable cooperative behavior across five state-of-the-art VLA models; naive semantic communication or bidirectional coupling even degrades performance. We establish AGC-VLN (Air-Ground Collaborative VLN), the first training-free baseline for air-ground collaborative VLN. The key insight is that training-free methods decompose navigation into VLM-based semantic reasoning and deterministic geometric execution, exposing a collaboration interface: the UAV's global view, over which it renders the UGV's reported pose and the VLM-anchored target as CAR/GOAL markers with distance labels, yielding a shared bird's-eye map. From this map, the UGV acquires global spatial context its first-person view cannot provide, plans a road-following path with a frozen VLM, and executes it under closed-loop control; in parallel, the UAV runs 3D-SPF, a spatial-search upgrade of SPF that localizes the target in the downward view and flies toward it. On 100 closed-loop episodes in CARLA-Air's Town10HD scene, AGC-VLN reaches a 77.0% joint success rate, a collaboration gain of +27.0% over the weaker individual agent (the UAV, 50.0%), and exceeds the strongest published single-agent baseline (Travel UAV, 53.0%) by 24.0 points, stemming from the complementarity of the UAV's global view and the UGV's road-following execution. Project page: https://github.com/ZSN2024/AGC-VLN.
Shuning Zhang, Liang Li, Yunheng Wang +3
Robotics and Autonomous Systems Thrust, Systems Hub, The Hong Kong University of Science and Technology (Guangzhou), China · School of Communications and Information Engineering, Nanjing University of Posts and Telecommunications, China
Air-ground collaborative perception is crucial for robust visual understanding in real-world dynamic environments. However, existing studies typically formulate collaboration as single-task cross-view fusion, overlooking the functional dependencies among localization, target association, and fine-grained parsing. In addition, the heterogeneous nature of aerial and ground views introduces substantial geometric, scale, and occlusion discrepancies, making uniform feature sharing vulnerable to negative transfer. To tackle these issues, we model air-ground perception as a progressive cross-task collaboration task and construct the Air-Ground Progressive Collaboration (AGPC) benchmark, a spatio-temporally aligned benchmark comprising more than 745K raw video frames. Built upon this benchmark, we propose Socialized Co-Perception (SCP), a coarse-to-fine framework that organizes collaboration progressively from aerial global localization to ground target association and identity-aware parsing. Its core module, the Dual-Layer Router (DLR), decouples input-side multi-scale expert selection from output-side task-conditioned modulation, enabling selective cross-view and cross-task interaction while suppressing harmful interference. Extensive experiments demonstrate the effectiveness of SCP. It achieves a 3.73% coevolutionary gain and a 7.86% improvement in average downstream performance. These results show that task-conditioned collaboration is more effective than uniform fusion for heterogeneous air-ground perception. The code is available at https://github.com/g1136639260-spec/AGSCP.
Zhoupeng Guo, Yunqi Zhu, Zhihe Fan +6
School of Automation, Southeast University, Nanjing 210096, China · School of Computer Science and Engineering, University of New South Wales, Sydney, NSW 2052, Australia · School of Sports Training, Tianjin University of Sport, Tianjin 301617, China +3