Unsupervised Multi-agent and Single-agent Perception from Cooperative Views

Haochen Yang; Baolu Li; Lei Li; Delin Ren; Jiacheng Guo; Minghai Qin; Tianyun Zhang; Hongkai Yu

cs.CVApr 7, 2026

Unsupervised Multi-agent and Single-agent Perception from Cooperative Views

Authors: Haochen Yang, Baolu Li, Lei Li, Delin Ren, Jiacheng Guo, Minghai Qin, Tianyun Zhang, Hongkai Yu

Organizations: Cleveland State University

Abstract

The LiDAR-based multi-agent and single-agent perception has shown promising performance in environmental understanding for robots and automated vehicles. However, there is no existing method that simultaneously solves both multi-agent and single-agent perception in an unsupervised way. By sharing sensor data between multiple agents via communication, this paper discovers two key insights: 1) Improved point cloud density after the data sharing from cooperative views could benefit unsupervised object classification, 2) Cooperative view of multiple agents can be used as unsupervised guidance for the 3D object detection in the single view. Based on these two discovered insights, we propose an Unsupervised Multi-agent and Single-agent (UMS) perception framework that leverages multi-agent cooperation without human annotations to simultaneously solve multi-agent and single-agent perception. UMS combines a learning-based Proposal Purifying Filter to better classify the candidate proposals after multi-agent point cloud density cooperation, followed by a Progressive Proposal Stabilizing module to yield reliable pseudo labels by the easy-to-hard curriculum learning. Furthermore, we design a Cross-View Consensus Learning to use multi-agent cooperative view to guide detection in single-agent view. Experimental results on two public datasets V2V4Real and OPV2V show that our UMS method achieved significantly higher 3D detection performance than the state-of-the-art methods on both multi-agent and single-agent perception tasks in an unsupervised setting.

Figures & tables

Figure 1: Illustration of Benefits from Cooperative Views. (a) Point Cloud Density Benefit, (b) Cross-View Consensus Benefit. We use Vehicle-to-Vehicle (V2V) cooperative perception [ 27 ] in Connected Autonomous Vehicles (CAV) as an example here.

Figure 2: UMS Pipeline. The system jointly trains two 3D object detectors for multi-agent and single-agent perception. Candidate Proposals are first generated from two initialized weak detectors and then refined by (i) Proposal Purifying Filter (PPF), (ii) Progressive Proposal Stabilizing (PPS), and (iii) Cross-View Consensus Learning (CCL), enabling robust pseudo supervision without human annotations.

Figure 3: Illustration of the proposed modules. (a) Proposal Purifying Filter (PPF) learns an instance-level filter/classifier to remove unreliable proposals. (b) Progressive Proposal Stabilizing (PPS) maintains a memory bank and adaptively fuses historical and current pseudo labels for stability.

Figure 4: Confidence–TP/FP statistics with improved point cloud density under multi-agent setting. Based on the initialized weak detector $D_{\mathrm{m}}$ on the V2V4Real [ 26 ] training set, the distribution gap of True Positives (TP) and False Positives (FP) across confidence makes self-supervised classification possible.

Method	V2V4Real [ 26 ]				OPV2V [ 27 ]
	Multi-Agent		Single-Agent		Multi-Agent		Single-Agent
	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]
Supervised	71.35	64.75	57.40	50.17	94.80	94.11	80.01	77.89
DBSCAN [ 3 ]	14.59	7.57	11.63	6.55	32.31	24.43	26.49	21.72
OYSTER [ 29 ]	37.50	23.52	29.08	24.25	56.58	49.01	42.62	41.93
CPD [ 22 ]	40.67	30.27	37.41	30.28	59.17	50.49	44.27	43.25

Table 1: Unsupervised Multi-agent and Single-agent Perception (3D Object Detection). [email protected] / [email protected] on the V2V4Real [ 26 ] and OPV2V [ 27 ] test sets.

Figure 5: Qualitative Comparison of Pseudo Labels on V2V4Real Training Set. Green boxes: ground truths, Orange boxes: pseudo labels, Red arrows: false positives, Blue arrows: false negatives. Multi-agent dense fused point clouds are shown here.

Method	[email protected] / [email protected]
Method	0–30m	30–50m	50–100m
Supervised	80.19/75.42	41.56/34.92	10.77/8.93
DBSCAN [ 3 ]	24.23/14.67	2.13/0.73	0.00/0.00
OYSTER [ 29 ]	50.57/42.59	3.82/3.02	0.00/0.00
CPD [ 22 ]	62.35/50.18	11.57/10.06	0.00/0.00
DOtA [ 23 ]	70.15/60.73	34.77/25.00	7.77/5.23

Table 2: Unsupervised Single-agent Perception (3D Object Detection) in Different Ranges. [email protected] / [email protected] results in ranges on V2V4Real [ 26 ] . short: 0-30m, mid: 30-50m, long: 50-100m.

Method	V2V4Real [ 26 ]		OPV2V [ 27 ]
Method	Recall	Precision	Recall	Precision
DBSCAN [ 3 ]	19.59	2.51	39.54	11.83
OYSTER [ 29 ]	29.32	15.74	43.72	54.89
CPD [ 22 ]	32.15	16.46	45.80	56.23
DOtA [ 23 ]	43.91	60.42	51.87	65.74
UMS (Ours)	53.71	85.98	70.21	90.25

Table 3: Pseudo-label Quality. Recall / Precision at IoU=0.5 on V2V4Real [ 26 ] and OPV2V [ 27 ] under multi-agent setting.

PPF	PPS	CCL	V2V4Real [ 26 ]				OPV2V [ 27 ]
PPF	PPS	CCL	Multi-Agent		Single-Agent		Multi-Agent		Single-Agent
			[email protected]	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]
			17.86	16.87	12.43	10.66	28.98	19.33	16.47	14.62
✓			52.35	46.02	39.24	36.78	70.62	59.55	63.12	45.98
✓	✓		58.12	52.03	45.10	41.20	86.71	83.89	74.85	66.44
✓	✓	✓	-	-	49.72	44.27	-	-	76.31	71.30

Table 4: Ablation Study of Each Component. Unsupervised 3D object detection on V2V4Real [ 26 ] and OPV2V [ 27 ] .

Figure 6: Effect of PPS on Pseudo-Label Quality. Pseudo-label AP@0.3 / AP@0.5 curves across refinement iterations on OPV2V [ 27 ] under the multi-agent setting. — Figure 6: Effect of PPS on Pseudo-Label Quality. Pseudo-label [email protected] / [email protected] curves across refinement iterations on OPV2V [ 27 ] under the multi-agent setting.

Setting	$\tau$	[email protected]	[email protected]
Low $\tau$	0.01	77.45	71.98
High $\tau$	0.20	82.21	79.51
Dynamic $\tau$ (ours)	sigmoid	86.71	83.89

Table 5: Effect of

\tau

in PPS. Evaluated on OPV2V [ 27 ] under the multi-agent setting.

$\mu_{3}$	$0.5$	$1.0$	$1.5$	$2.0$
[email protected]	75.23	75.81	76.31	75.44
[email protected]	68.65	70.36	71.30	71.17

Table 6: Effect of

\mu_{3}

in CCL. Evaluated on OPV2V [ 27 ] under the single-agent setting.

Iteration $T$	1	5	10	20	25
[email protected]	43.07	77.76	83.50	86.71	86.91
[email protected]	36.68	71.43	79.32	83.89	83.80

Table 7: Effect of Iteration

T

. Evaluated on OPV2V [ 27 ] under the multi-agent setting.

Class	CPD [ 22 ]	DOtA [ 23 ]	UMS (Ours)
Car	27.56	34.27	40.10
Pedestrian	11.37	14.33	17.71

Table 8: Multi-Class Detection . [email protected] on V2X-Real [ 25 ] under the multi-agent setting.

Method	Normal	GPS Pose Error	Latency
Supervised	71.35 / 64.75	68.71 / 57.70	68.72 / 57.72
DBSCAN [ 3 ]	14.42 / 7.01	13.63 / 6.78	14.45 / 7.05
OYSTER [ 29 ]	36.64 / 25.87	35.81 / 24.81	36.39 / 25.48
CPD [ 22 ]	39.38 / 31.88	38.84 / 29.69	39.23 / 30.51
DOtA [ 23 ]	54.60 / 48.84	53.23 / 45.76	54.20 / 45.23
UMS (Ours)	58.12 / 52.03	56.21 / 49.05	57.67 / 48.38

Table 9: Robustness. [email protected]/0.5 on V2V4Real [ 26 ] with GPS pose errors, communication latency under the multi-agent setting.

Explore similar work

C2E: Boosting Ego-Only 3D Object Detection via Multi-Teacher Contrastive Knowledge Distillation
Jul 2, 2026Jinlong Wang, Xun Huang, Qiming Xia +2Lidar-Based 3D Object Detection 3D Object Detection
Fujian Key Laboratory of Urban Intelligent Sensing and Computing, Xiamen University, China · Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, China · Zhongguancun Academy, China
V2U4Real: A Real-world Large-scale Dataset for Vehicle-to-UAV Cooperative Perception
Mar 26, 2026Weijia Li, Haoen Xiang, Tianxu Wang +4Autonomous Driving Datasets Cooperative Perception
Fujian Key Laboratory of Urban Intelligent Sensing and Computing, Xiamen University, China · Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, China
CoLC: Communication-Efficient Collaborative Perception with LiDAR Completion
Feb 28, 2026Yushan Han, Hui Zhang, Qiming Xia +2Communication-Efficient Collaborative Perception Cooperative Perception
Key Laboratory of Big Data & Artificial Intelligence in Transportation, Ministry of Education · School of Computer Science and Technology, Beijing Jiaotong University · Xiamen University

Jul 2, 2026J/K move · Enter open · S save

C2E: Boosting Ego-Only 3D Object Detection via Multi-Teacher Contrastive Knowledge Distillation

Jinlong Wang, Xun Huang, Qiming Xia, Shijia Zhao+1

Fujian Key Laboratory of Urban Intelligent Sensing and Computing, Xiamen University, China · Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, China · Zhongguancun Academy, China

Lidar-Based 3D Object Detection 3D Object Detection Cooperative Perception Knowledge Distillation

Open paper AskPDF arXiv

LiDAR-based 3D object detection is essential for autonomous driving systems. However, traditional Ego-only Perception (Eo-Perception) suffers from limited perspective and occlusions in a complex outdoor environment, leading to performance bottlenecks. Recently, research on multi-agent Collaborative Perception (Co-Perception) has demonstrated excellent performance, but high communication costs and accumulated pose error hinder its application. To address this, we explore a novel C2E (Co-Perception to Eo-Perception) paradigm through the Multi-to-Single (M2S) agent contrastive knowledge distillation framework. Our M2S framework first designs Multi-Level Feature Enhancement module to provide more stable features, and introduces Auxiliary Point Cloud Reconstruction and Multi-Teacher Contrastive Distillation mechanisms to mitigate domain gaps in point cloud and feature distributions within the C2E paradigm. Benefiting from this, our M2S can retain the excellent performance of collaborative perception while effectively avoiding the drawbacks, such as communication delays and positioning errors. Extensive experiments on the V2XSet, V2V4Real and DAIR-V2X datasets show the effectiveness and generalizability of our M2S framework when combined with the state-of-the-art CoSDH model and other excellent 3D detectors. Our M2S framework can deliver up to a 8.64% improvement in 3D mAP performance without introducing any communication costs.

Abstract

Explore similar work

C2E: Boosting Ego-Only 3D Object Detection via Multi-Teacher Contrastive Knowledge Distillation

V2U4Real: A Real-world Large-scale Dataset for Vehicle-to-UAV Cooperative Perception

CoLC: Communication-Efficient Collaborative Perception with LiDAR Completion