Unsupervised Multi-agent and Single-agent Perception from Cooperative Views

Haochen Yang; Baolu Li; Lei Li; Delin Ren; Jiacheng Guo; Minghai Qin; Tianyun Zhang; Hongkai Yu

cs.CVApr 7, 2026

Unsupervised Multi-agent and Single-agent Perception from Cooperative Views

Authors: Haochen Yang, Baolu Li, Lei Li, Delin Ren, Jiacheng Guo, Minghai Qin, Tianyun Zhang, Hongkai Yu

Organizations: Cleveland State University

Abstract

The LiDAR-based multi-agent and single-agent perception has shown promising performance in environmental understanding for robots and automated vehicles. However, there is no existing method that simultaneously solves both multi-agent and single-agent perception in an unsupervised way. By sharing sensor data between multiple agents via communication, this paper discovers two key insights: 1) Improved point cloud density after the data sharing from cooperative views could benefit unsupervised object classification, 2) Cooperative view of multiple agents can be used as unsupervised guidance for the 3D object detection in the single view. Based on these two discovered insights, we propose an Unsupervised Multi-agent and Single-agent (UMS) perception framework that leverages multi-agent cooperation without human annotations to simultaneously solve multi-agent and single-agent perception. UMS combines a learning-based Proposal Purifying Filter to better classify the candidate proposals after multi-agent point cloud density cooperation, followed by a Progressive Proposal Stabilizing module to yield reliable pseudo labels by the easy-to-hard curriculum learning. Furthermore, we design a Cross-View Consensus Learning to use multi-agent cooperative view to guide detection in single-agent view. Experimental results on two public datasets V2V4Real and OPV2V show that our UMS method achieved significantly higher 3D detection performance than the state-of-the-art methods on both multi-agent and single-agent perception tasks in an unsupervised setting.

Figures & tables

Figure 1: Illustration of Benefits from Cooperative Views. (a) Point Cloud Density Benefit, (b) Cross-View Consensus Benefit. We use Vehicle-to-Vehicle (V2V) cooperative perception [ 27 ] in Connected Autonomous Vehicles (CAV) as an example here.

Figure 2: UMS Pipeline. The system jointly trains two 3D object detectors for multi-agent and single-agent perception. Candidate Proposals are first generated from two initialized weak detectors and then refined by (i) Proposal Purifying Filter (PPF), (ii) Progressive Proposal Stabilizing (PPS), and (iii) Cross-View Consensus Learning (CCL), enabling robust pseudo supervision without human annotations.

Figure 3: Illustration of the proposed modules. (a) Proposal Purifying Filter (PPF) learns an instance-level filter/classifier to remove unreliable proposals. (b) Progressive Proposal Stabilizing (PPS) maintains a memory bank and adaptively fuses historical and current pseudo labels for stability.

Figure 4: Confidence–TP/FP statistics with improved point cloud density under multi-agent setting. Based on the initialized weak detector $D_{\mathrm{m}}$ on the V2V4Real [ 26 ] training set, the distribution gap of True Positives (TP) and False Positives (FP) across confidence makes self-supervised classification possible.

Method	V2V4Real [ 26 ]				OPV2V [ 27 ]
	Multi-Agent		Single-Agent		Multi-Agent		Single-Agent
	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]
Supervised	71.35	64.75	57.40	50.17	94.80	94.11	80.01	77.89
DBSCAN [ 3 ]	14.59	7.57	11.63	6.55	32.31	24.43	26.49	21.72
OYSTER [ 29 ]	37.50	23.52	29.08	24.25	56.58	49.01	42.62	41.93
CPD [ 22 ]	40.67	30.27	37.41	30.28	59.17	50.49	44.27	43.25

Table 1: Unsupervised Multi-agent and Single-agent Perception (3D Object Detection). [email protected] / [email protected] on the V2V4Real [ 26 ] and OPV2V [ 27 ] test sets.

Figure 5: Qualitative Comparison of Pseudo Labels on V2V4Real Training Set. Green boxes: ground truths, Orange boxes: pseudo labels, Red arrows: false positives, Blue arrows: false negatives. Multi-agent dense fused point clouds are shown here.

Method	[email protected] / [email protected]
Method	0–30m	30–50m	50–100m
Supervised	80.19/75.42	41.56/34.92	10.77/8.93
DBSCAN [ 3 ]	24.23/14.67	2.13/0.73	0.00/0.00
OYSTER [ 29 ]	50.57/42.59	3.82/3.02	0.00/0.00
CPD [ 22 ]	62.35/50.18	11.57/10.06	0.00/0.00
DOtA [ 23 ]	70.15/60.73	34.77/25.00	7.77/5.23

Table 2: Unsupervised Single-agent Perception (3D Object Detection) in Different Ranges. [email protected] / [email protected] results in ranges on V2V4Real [ 26 ] . short: 0-30m, mid: 30-50m, long: 50-100m.

Method	V2V4Real [ 26 ]		OPV2V [ 27 ]
Method	Recall	Precision	Recall	Precision
DBSCAN [ 3 ]	19.59	2.51	39.54	11.83
OYSTER [ 29 ]	29.32	15.74	43.72	54.89
CPD [ 22 ]	32.15	16.46	45.80	56.23
DOtA [ 23 ]	43.91	60.42	51.87	65.74
UMS (Ours)	53.71	85.98	70.21	90.25

Table 3: Pseudo-label Quality. Recall / Precision at IoU=0.5 on V2V4Real [ 26 ] and OPV2V [ 27 ] under multi-agent setting.

PPF	PPS	CCL	V2V4Real [ 26 ]				OPV2V [ 27 ]
PPF	PPS	CCL	Multi-Agent		Single-Agent		Multi-Agent		Single-Agent
			[email protected]	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]
			17.86	16.87	12.43	10.66	28.98	19.33	16.47	14.62
✓			52.35	46.02	39.24	36.78	70.62	59.55	63.12	45.98
✓	✓		58.12	52.03	45.10	41.20	86.71	83.89	74.85	66.44
✓	✓	✓	-	-	49.72	44.27	-	-	76.31	71.30

Table 4: Ablation Study of Each Component. Unsupervised 3D object detection on V2V4Real [ 26 ] and OPV2V [ 27 ] .

Figure 6: Effect of PPS on Pseudo-Label Quality. Pseudo-label AP@0.3 / AP@0.5 curves across refinement iterations on OPV2V [ 27 ] under the multi-agent setting. — Figure 6: Effect of PPS on Pseudo-Label Quality. Pseudo-label [email protected] / [email protected] curves across refinement iterations on OPV2V [ 27 ] under the multi-agent setting.

Setting	$\tau$	[email protected]	[email protected]
Low $\tau$	0.01	77.45	71.98
High $\tau$	0.20	82.21	79.51
Dynamic $\tau$ (ours)	sigmoid	86.71	83.89

Table 5: Effect of

\tau

in PPS. Evaluated on OPV2V [ 27 ] under the multi-agent setting.

$\mu_{3}$	$0.5$	$1.0$	$1.5$	$2.0$
[email protected]	75.23	75.81	76.31	75.44
[email protected]	68.65	70.36	71.30	71.17

Table 6: Effect of

\mu_{3}

in CCL. Evaluated on OPV2V [ 27 ] under the single-agent setting.

Iteration $T$	1	5	10	20	25
[email protected]	43.07	77.76	83.50	86.71	86.91
[email protected]	36.68	71.43	79.32	83.89	83.80

Table 7: Effect of Iteration

T

. Evaluated on OPV2V [ 27 ] under the multi-agent setting.

Class	CPD [ 22 ]	DOtA [ 23 ]	UMS (Ours)
Car	27.56	34.27	40.10
Pedestrian	11.37	14.33	17.71

Table 8: Multi-Class Detection . [email protected] on V2X-Real [ 25 ] under the multi-agent setting.

Method	Normal	GPS Pose Error	Latency
Supervised	71.35 / 64.75	68.71 / 57.70	68.72 / 57.72
DBSCAN [ 3 ]	14.42 / 7.01	13.63 / 6.78	14.45 / 7.05
OYSTER [ 29 ]	36.64 / 25.87	35.81 / 24.81	36.39 / 25.48
CPD [ 22 ]	39.38 / 31.88	38.84 / 29.69	39.23 / 30.51
DOtA [ 23 ]	54.60 / 48.84	53.23 / 45.76	54.20 / 45.23
UMS (Ours)	58.12 / 52.03	56.21 / 49.05	57.67 / 48.38

Table 9: Robustness. [email protected]/0.5 on V2V4Real [ 26 ] with GPS pose errors, communication latency under the multi-agent setting.

Explore similar work

Jul 2, 2026

C2E: Boosting Ego-Only 3D Object Detection via Multi-Teacher Contrastive Knowledge Distillation

LiDAR-based 3D object detection is essential for autonomous driving systems. However, traditional Ego-only Perception (Eo-Perception) suffers from limited perspective and occlusions in a complex outdoor environment, leading to performance bottlenecks. Recently, research on multi-agent Collaborative Perception (Co-Perception) has demonstrated excellent performance, but high communication costs and accumulated pose error hinder its application. To address this, we explore a novel C2E (Co-Perception to Eo-Perception) paradigm through the Multi-to-Single (M2S) agent contrastive knowledge distillation framework. Our M2S framework first designs Multi-Level Feature Enhancement module to provide more stable features, and introduces Auxiliary Point Cloud Reconstruction and Multi-Teacher Contrastive Distillation mechanisms to mitigate domain gaps in point cloud and feature distributions within the C2E paradigm. Benefiting from this, our M2S can retain the excellent performance of collaborative perception while effectively avoiding the drawbacks, such as communication delays and positioning errors. Extensive experiments on the V2XSet, V2V4Real and DAIR-V2X datasets show the effectiveness and generalizability of our M2S framework when combined with the state-of-the-art CoSDH model and other excellent 3D detectors. Our M2S framework can deliver up to a 8.64% improvement in 3D mAP performance without introducing any communication costs.

Jinlong Wang, Xun Huang, Qiming Xia +2

Fujian Key Laboratory of Urban Intelligent Sensing and Computing, Xiamen University, China · Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, China · Zhongguancun Academy, China

Lidar-Based 3D Object Detection 3D Object Detection Cooperative Perception Knowledge Distillation

Ask PDF

Mar 26, 2026

V2U4Real: A Real-world Large-scale Dataset for Vehicle-to-UAV Cooperative Perception

Modern autonomous vehicle perception systems are often constrained by occlusions, blind spots, and limited sensing range. While existing cooperative perception paradigms, such as Vehicle-to-Vehicle (V2V) and Vehicle-to-Infrastructure (V2I), have demonstrated their effectiveness in mitigating these challenges, they remain limited to ground-level collaboration and cannot fully address large-scale occlusions or long-range perception in complex environments. To advance research in cross-view cooperative perception, we present V2U4Real, the first large-scale real-world multi-modal dataset for Vehicle-to-UAV (V2U) cooperative object perception. V2U4Real is collected by a ground vehicle and a UAV equipped with multi-view LiDARs and RGB cameras. The dataset covers urban streets, university campuses, and rural roads under diverse traffic scenarios, comprising over 56K LiDAR frames, 56K multi-view camera images, and 700K annotated 3D bounding boxes across four classes. To support a wide range of research tasks, we establish benchmarks for single-agent 3D object detection, cooperative 3D object detection, and object tracking. Comprehensive evaluations of several state-of-the-art models demonstrate the effectiveness of V2U cooperation in enhancing perception robustness and long-range awareness. The V2U4Real dataset and codebase is available at https://github.com/VjiaLi/V2U4Real.

Weijia Li, Haoen Xiang, Tianxu Wang +4

Autonomous Driving Datasets Cooperative Perception 3D Object Detection Heterogeneous Collaborative Perception 3D MOT

Ask PDF

Feb 28, 2026

CoLC: Communication-Efficient Collaborative Perception with LiDAR Completion

Collaborative perception empowers autonomous agents to share complementary information and overcome perception limitations. While early fusion offers more perceptual complementarity and is inherently robust to model heterogeneity, its high communication cost has limited its practical deployment, prompting most existing works to favor intermediate or late fusion. To address this, we propose a communication-efficient early Collaborative perception framework that incorporates LiDAR Completion to restore scene completeness under sparse transmission, dubbed as CoLC. Specifically, the CoLC integrates three complementary designs. First, each neighbor agent applies Foreground-Aware Point Sampling (FAPS) to selectively transmit informative points that retain essential structural and contextual cues under bandwidth constraints. The ego agent then employs Completion-Enhanced Early Fusion (CEEF) to reconstruct dense pillars from the received sparse inputs and adaptively fuse them with its own observations, thereby restoring spatial completeness. Finally, the Dense-Guided Dual Alignment (DGDA) strategy enforces semantic and geometric consistency between the enhanced and dense pillars during training, ensuring consistent and robust feature learning. Experiments on both simulated and real-world datasets demonstrate that CoLC achieves superior perception-communication trade-offs and remains robust under heterogeneous model settings. The code is available at https://github.com/CatOneTwo/CoLC.

Yushan Han, Hui Zhang, Qiming Xia +2

Key Laboratory of Big Data & Artificial Intelligence in Transportation, Ministry of Education · School of Computer Science and Technology, Beijing Jiaotong University · Xiamen University

Communication-Efficient Collaborative Perception Cooperative Perception

Ask PDF