V2U4Real: A Real-world Large-scale Dataset for Vehicle-to-UAV Cooperative Perception

Weijia Li; Haoen Xiang; Tianxu Wang; Shuaibing Wu; Qiming Xia; Cheng Wang; Chenglu Wen

cs.CVMar 26, 2026

V2U4Real: A Real-world Large-scale Dataset for Vehicle-to-UAV Cooperative Perception

Authors: Weijia Li, Haoen Xiang, Tianxu Wang, Shuaibing Wu, Qiming Xia, Cheng Wang, Chenglu Wen

Organizations: Fujian Key Laboratory of Urban Intelligent Sensing and Computing, Xiamen University, China · Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, China

Abstract

Modern autonomous vehicle perception systems are often constrained by occlusions, blind spots, and limited sensing range. While existing cooperative perception paradigms, such as Vehicle-to-Vehicle (V2V) and Vehicle-to-Infrastructure (V2I), have demonstrated their effectiveness in mitigating these challenges, they remain limited to ground-level collaboration and cannot fully address large-scale occlusions or long-range perception in complex environments. To advance research in cross-view cooperative perception, we present V2U4Real, the first large-scale real-world multi-modal dataset for Vehicle-to-UAV (V2U) cooperative object perception. V2U4Real is collected by a ground vehicle and a UAV equipped with multi-view LiDARs and RGB cameras. The dataset covers urban streets, university campuses, and rural roads under diverse traffic scenarios, comprising over 56K LiDAR frames, 56K multi-view camera images, and 700K annotated 3D bounding boxes across four classes. To support a wide range of research tasks, we establish benchmarks for single-agent 3D object detection, cooperative 3D object detection, and object tracking. Comprehensive evaluations of several state-of-the-art models demonstrate the effectiveness of V2U cooperation in enhancing perception robustness and long-range awareness. The V2U4Real dataset and codebase is available at https://github.com/VjiaLi/V2U4Real.

Figures & tables

R/S	Dataset	Year	Paradigm	Vehicle Sensor		UAV Sensor		Data Details			Annotations
	Dataset	Year	Paradigm	Camera	LiDAR	Camera	LiDAR	PointCloud	Image	3D Boxes
	OPV2V [ 33 ]	2022	V2V	✓	✓	-	-	11K	44K	230K	D, T
	V2X-Sim [ 18 ]	2022	V2V & I	✓	✓	-	-	10K	60K	26.6K	D, S, T
	V2XSet [ 32 ]	2022	V2V & I	✓	✓	-	-	11.4K	44K	230K	D, T
Sim	CoP-UAVs [ 10 ]	2022	U2U	-	-	✓	✗	-	131.9K	1.94M	D, S
	UAV3D [ 25 ]	2024	U2U	-	-	✓	✗	-	500K	3.3M	D, S, T

Table 1: Comparisons of our proposed V2U4Real with existing cooperative perception datasets. R/S: Real or Sim, V: Vehicle, U: UAV, I: Infrastructure, D: Detection, S: Segmentation, T: Tracking, ‘✓’ : Supported, ‘✗’ : Not Supported, ‘-’ : Not Applicable/Specified.

Panel of Figure 2 — Figure 2: Agent motion discrepancies between ground vehicle and UAV. (a) Probability distribution of the roll angle ( $\theta_{r}$ ). (b) Probability distribution of the pitch angle ( $\theta_{p}$ ). Red color denotes the UAV. Purple color denotes the ego vehicle. $P$ denotes probability density.

Platform	Sensor	Sensor Model	Details
Vehicle	LiDAR	Ouster-1-128( $\times$ 1)	128 channels, 10Hz capture frequency, $360^{\circ}$ horizontal FOV, $-22.5^{\circ}\text{to}+22.5^{\circ}$ vertical FOV, $\leq$ 200m range;
	LiDAR	RoboSense Ruby-Plus-128 ( $\times$ 1)	128 channels, 10Hz capture frequency, $360^{\circ}$ horizontal FOV, $-25^{\circ}\text{to}+15^{\circ}$ vertical FOV, $\leq$ 250m range;
	LiDAR	RoboSense M1 Plus( $\times$ 1)	10Hz capture frequency, $120^{\circ}$ horizontal FOV, $-12.5^{\circ}\text{to}+12.5^{\circ}$ vertical FOV, $\leq$ 200m range;
	Camera	HikRobot MV-CA023-10GC ( $\times$ 3)	RGB, 10Hz capture frequency, 1920×1080 resolution;
	GPS/IMU	Quectel LC29H ( $\times$ 1) / Sigma-D4G ( $\times$ 1)	1000Hz update rate, RTK supported, Double-Precision;
UAV	LiDAR	Ouster-1-128 ( $\times$ 1)	128 channels, 10Hz capture frequency, $360^{\circ}$ horizontal FOV, $-22.5^{\circ}\text{to}+22.5^{\circ}$ vertical FOV, $\leq$ 200m range;

Table 2: Key Sensor Specifications in V2U4Real Dataset.

Panel of Figure 3 — Figure 3: Sensor overview of V2U4Real and coordinate of sensors. The x,y, and z coordinates are red, green, and blue.

Figure 4: Driving routes of two data collection agents. Urban, rural, and campus roads are represented by blue, yellow, and red lines, respectively.

Panel of Figure 5 — Figure 5: Visualization of sensor calibration and point cloud registration results. Purple points are captured by the ground vehicle. Red points are captured by the UAV. Green bounding box is the ground truth.

Panel of Figure 6 — Figure 6: Data analysis of V2U4Real. $P$ denotes probability, $\mu$ denotes mean, and $\sigma$ denotes standard deviation.

Paradigm	Method	Year	Sync. (3D AP@IoU = 0.5 / 0.7)			Async. (3D AP@IoU = 0.5 / 0.7)			AM
Paradigm	Method	Year	Overall	0–50m	50–100m	Overall	0–50m	50–100m	(MB)
Vehicle to UAV	Early Fusion	-	51.31/30.97	57.53/35.67	24.47/14.99	30.94/13.99	34.17/15.85	20.33/8.91	3.18
	Late Fusion	-	43.61/27.74	49.28/31.73	16.75/8.56	28.08/16.18	29.63/17.31	13.87/6.56	0.009
	Where2comm [ 10 ]	2022	53.85/29.71	60.47/33.14	26.73/16.59	48.99/28.36	55.30/31.83	25.51/14.46	0.65
	V2X-ViT [ 32 ]	2022	45.51/25.99	50.98/29.65	19.67/12.24	45.74/26.00	51.29/29.68	19.65/12.13	0.65
	AttFuse [ 33 ]	2022	50.20/27.33	55.31/30.41	29.33/18.39	40.51/21.92	44.71/23.41	28.51/16.98	0.65

Table 3: Cooperative 3D object detection benchmarks for vehicle category on V2U4Real. Sync. means synchronous setup ignoring communication delays. Async. implies asynchronous setup with a

(0,1000]

ms delay. The vehicle category includes car and truck.

Platform	Method	Veh. (3D AP@IoU)		Cyc. (3D AP@IoU)
Platform	Method	0.5	0.7	0.25	0.5
Vehicle	PointPillars [ 16 ]	67.06	30.22	60.45	50.12
	SECOND [ 35 ]	60.64	30.01	57.67	49.98
	CenterPoint [ 36 ]	65.79	26.96	59.89	50.48
	PV-RCNN [ 24 ]	68.18	36.23	59.51	51.31
UAV	PointPillars [ 16 ]	70.09	47.06	57.33	41.61

Table 4: Single-agent 3D object detection benchmarks on V2U4Real. The vehicle category includes car and truck.

Paradigm	Method	Year	AMOTA(↑)	AMOTP(↑)	sAMOTA(↑)	MOTA(↑)	MOTP(↑)	MT(↑)	ML(↓)
Vehicle to UAV	Early Fusion	-	19.22	39.41	56.79	59.26	64.67	67.94	23.81
	Late Fusion	-	14.82	34.39	51.12	50.64	64.90	48.41	37.30
	AttFuse [ 33 ]	2022	20.98	41.31	61.55	60.40	64.89	57.94	23.81
	Where2comm [ 10 ]	2022	20.07	41.49	58.52	62.07	63.04	65.08	15.87
	V2X-ViT [ 32 ]	2022	14.74	35.25	50.38	50.76	64.76	57.94	30.16
	CoBEVT [ 31 ]	2022	15.96	36.53	52.62	53.60	64.54	51.59	35.71

Table 5: 3D object tracking benchmarks for vehicle category on V2U4Real.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Sensor	Model	Cost ($)
LiDAR	OS-128 / RS-128 / M1-Plus	12K / 9K / 3K
Camera	HikRobot MV-CA023-10GC	700
GPS/IMU	Quectel / Sigma100 / DETA100	35 / 1K / 200

Appendix

Table 6: Sensor Cost of V2U4Real.

Panel of Figure 7 — Figure 7: LiDAR appearance of different object classes in V2U4Real. The first and second rows show the top and side views captured by the UAV-mounted LiDAR, while the third and fourth rows present the corresponding views from the vehicle-mounted LiDAR.

Panel of Figure 8 — Figure 8: Category distributions of V2U4Real.

Split	Sequence	Scene	Frames	Vehicles	Cyclists	Pedestrians	Track IDs	Duration (s)
Train	2025-07-17-16-12-1	Urban	1200	7140	1036	0	112	30.0
	2025-07-17-16-12-2	Urban	1244	9255	1386	0	138	31.1
	2025-07-17-16-12-3	Urban	1116	13865	1116	0	145	27.9
	2025-07-17-16-12-4	Urban	1208	8890	1551	0	129	30.2
	2025-07-17-16-35-3	Urban	3112	10175	18342	2730	241	77.8
	2025-07-17-16-50-1	Urban	1528	7247	14733	1953	207	38.2

Appendix

Table 7: Dataset statistics of V2U4Real.

Panel of Figure 9 — Figure 9: Target bounding box distributions of V2U4Real.

LiDAR Type	Method	Vehicle (3D AP@IoU)		Cyclist (3D AP@IoU)		Pedestrian (3D AP@IoU)
LiDAR Type	Method	0.5	0.7	0.25	0.5	0.25	0.5
Ruby-128	PointPillars [ 16 ]	60.57	29.57	59.62	45.30	38.08	27.65
	SECOND [ 35 ]	56.61	32.01	52.73	42.29	48.15	36.97
	CenterPoint [ 36 ]	58.24	26.93	56.39	42.56	58.79	43.64
	PV-RCNN [ 24 ]	65.91	41.99	67.11	53.35	45.06	37.92
M1-Plus	PointPillars [ 16 ]	41.37	20.81	53.28	41.40	15.06	9.09

Appendix

Table 8: Single-agent 3D object detection benchmarks on V2U4Real val set. The vehicle category includes car and truck.

Paradigm	Method	Year	Sync. (3D AP@IoU = 0.5 / 0.7)			Async. (3D AP@IoU = 0.5 / 0.7)			AM
Paradigm	Method	Year	Overall	0–50m	50–100m	Overall	0–50m	50–100m	(MB)
Vehicle to UAV	Early Fusion	-	45.44/20.76	49.59/22.38	20.56/13.16	24.63/9.63	26.29/9.71	16.53/11.01	3.18
	Late Fusion	-	36.76/13.20	39.89/14.06	11.53/8.29	23.48/8.14	24.62/8.38	10.22/7.37	0.009
	Where2comm [ 10 ]	2022	33.14/12.14	36.31/12.94	16.69/12.45	30.68/10.81	33.50/11.26	16.58/12.44	0.65
	V2X-ViT [ 32 ]	2022	29.69/11.65	33.24/12.63	10.92/7.93	29.72/11.62	33.23/12.76	10.87/7.80	0.65
	AttFuse [ 33 ]	2022	38.75/17.22	43.17/18.95	17.57/12.23	29.76/11.65	33.00/11.87	17.06/11.87	0.65

Appendix

Table 9: Cooperative 3D object detection benchmarks for vehicle category on V2U4Real test set. Sync. means synchronous setup ignoring communication delays. Async. implies asynchronous setup with a

(0,1000]

ms delay.

Method	Epoch	Batch	LR	LR Scheduler	Voxel Size
PointPillar [ 16 ]	40	4	0.002	OneCycle	$[0.2,0.2,8]$
SECOND [ 35 ]	40	4	0.002	OneCycle	$[0.1,0.1,0.2]$
CenterPoint [ 36 ]	40	4	0.002	OneCycle	$[0.1,0.1,0.2]$
PV-RCNN [ 24 ]	40	4	0.002	OneCycle	$[0.1,0.1,0.2]$

Appendix

Table 10: Implementation details of single-agent 3D object detection task.

Method	Batch	LR Scheduler	LiDAR Range
No Fusion	4	OneCycle	$[-100,-80,100,80]$
Early Fusion	4	MultiStep	$[-100,-80,100,80]$
Late Fusion	4	MultiStep	$[-100,-80,100,80]$
Where2comm [ 10 ]	4	CAW	$[-100,-80,100,80]$
V2X-VIT [ 32 ]	2	CAW	$[-89.6,-76.8,89.6,76.8]$
AttFuse [ 33 ]	4	MultiStep	$[-100,-80,100,80]$

Appendix

Table 11: Implementation details of cooperative 3D object detection task. CAW stands for CosineAnnealWarm.

Dataset	Localization Noise ( $\mu$ / $\sigma$ )
Dataset	X (m)	Y (m)	Z (m)	Yaw ( ∘ )	IoU (%)
V2V4Real	7.0/48.4	0.6/20.3	0.1/1.4	0.05/1.5	2.4/11.0
Ours	2.4/6.8	1.6/2.9	1.0/1.1	0.1/1.5	4.6/11.6

Appendix

Table 12: Quantitative results of localization noise.

Panel of Figure 10 — Figure 10: Visualization of 3D cooperative detection results in scene 1. Green bounding boxes indicate ground-truth annotations. Red bounding boxes indicate the model’s predictions.

Panel of Figure 11 — Figure 11: Visualization of 3D cooperative detection results scene 2. Green bounding boxes indicate ground-truth annotations. Red bounding boxes indicate the model’s predictions.

Panel of Figure 12 — Figure 12: Visualization of 3D cooperative detection results scene 3. Green bounding boxes indicate ground-truth annotations. Red bounding boxes indicate the model’s predictions.

Explore similar work

SimBEV2X: A Large-Scale Dataset and Data Generation Tool for Multi-Task Vehicle-to-Everything Cooperative Perception
Jul 27, 2026Goodarz Mehr, Sepideh Gohari, Montasir Abbas +1Autonomous Driving Datasets Synthetic Data Generation for Robotics
Autonomous Robots and Vehicles Laboratory (ARVL), College of Engineering, Virginia Commonwealth University (VCU), Richmond, Virginia, United States · Charles E. Via, Jr. Department of Civil & Environmental Engineering, Virginia Tech, Blacksburg, Virginia, United States
SV2V-RSim: A Comprehensive Benchmark for Self-Selective V2V Cooperative Perception with Near-Realistic Data
Sep 26, 2026Yulu Wu, Chao Wei, Jujun Cheng +6Cooperative Perception Autonomous Driving Datasets
Tongji University, Shanghai, China · Anhui University, Hefei, China · University of Toyama, Toyama, Japan
Long-SCOPE: Fully Sparse Long-Range Cooperative 3D Perception
Apr 10, 2026Jiahao Wang, Zikun Xu, Yuner Zhang +8Cooperative Perception Autonomous Driving Perception
Tsinghua University · University of Pennsylvania · Nanyang Technological University +1

Jul 27, 2026J/K move · Enter open · S save

SimBEV2X: A Large-Scale Dataset and Data Generation Tool for Multi-Task Vehicle-to-Everything Cooperative Perception

Goodarz Mehr, Sepideh Gohari, Montasir Abbas, Azim Eskandarian

Autonomous Robots and Vehicles Laboratory (ARVL), College of Engineering, Virginia Commonwealth University (VCU), Richmond, Virginia, United States · Charles E. Via, Jr. Department of Civil & Environmental Engineering, Virginia Tech, Blacksburg, Virginia, United States

Autonomous Driving Datasets Synthetic Data Generation for Robotics 3D Object Detection V2X Communication Cooperative Perception Autonomous Driving Scenario Generation

Open paper AskPDF arXiv

Cooperative perception through vehicle-to-everything (V2X) communication can overcome the inherent physical limitations of individual autonomous vehicles, such as occlusions and limited sensor range. However, the development of robust V2X algorithms, particularly those relying on unified spatial representations like bird's-eye view (BEV) representation, is hampered by the lack of large-scale, multi-modal, multi-task datasets. Moreover, collecting and annotating a large set of synchronized, real-world multi-agent data is prohibitively expensive. This has resulted in a landscape where existing V2X datasets are notably limited in both size and scope. To overcome this, we introduce SimBEV2X, an advanced synthetic data generation tool built on the CARLA simulator. SimBEV2X automatically creates randomized driving scenarios to collect multi-modal sensor data alongside various types of ground truth including 3D bounding boxes with unique track IDs, HD map information, BEV segmentation maps, and semantic occupancy voxel grids from both vehicles and RSUs. We also present the SimBEV2X dataset, the largest V2X perception dataset to date. The dataset comprises 258 scenes, each involving up to 8 connected vehicles and up to 4 RSUs across a variety of road networks. The SimBEV2X dataset is an order of magnitude larger than existing V2X datasets and contains 102,200 frames, 588,520 lidar point clouds, more than 3 million images, over 27 million bounding boxes, and a comprehensive set of other annotations. Finally, we establish a strong baseline on the SimBEV2X dataset using CoopDet3D and propose CoBEVFusion, a novel architecture that combines CoopDet3D with fused axial attention (FAX) for context-aware multi-agent feature aggregation, resulting in superior performance. SimBEV2X, the SimBEV2X dataset, and CoBEVFusion are available at https://simbev2x.org and https://github.com/GoodarzMehr/SimBEV2X.

Abstract

Explore similar work

SimBEV2X: A Large-Scale Dataset and Data Generation Tool for Multi-Task Vehicle-to-Everything Cooperative Perception

SV2V-RSim: A Comprehensive Benchmark for Self-Selective V2V Cooperative Perception with Near-Realistic Data

Long-SCOPE: Fully Sparse Long-Range Cooperative 3D Perception