Class-Adaptive Cooperative Perception for Multi-Class LiDAR-based 3D Object Detection in V2X Systems

Blessing Agyei Kyem; Joshua Kofi Asamoah; Armstrong Aboah

cs.CVApr 11, 2026

Class-Adaptive Cooperative Perception for Multi-Class LiDAR-based 3D Object Detection in V2X Systems

Authors: Blessing Agyei Kyem, Joshua Kofi Asamoah, Armstrong Aboah

Organizations: Department of Civil, Construction and Environmental Engineering, North Dakota State University, Fargo, ND 58102, USA

Abstract

Cooperative perception allows connected vehicles and roadside infrastructure to share sensor observations, creating a fused scene representation beyond the capability of any single platform. However, most cooperative 3D object detectors use a uniform fusion strategy for all object classes, which limits their ability to handle the different geometric structures and point-sampling patterns of small and large objects. This problem is further reinforced by narrow evaluation protocols that often emphasize a single dominant class or only a few cooperation settings, leaving robust multi-class detection across diverse vehicle-to-everything interactions insufficiently explored. To address this gap, we propose a class-adaptive cooperative perception architecture for multi-class 3D object detection from LiDAR data. The model integrates four components: multi-scale window attention with learned scale routing for spatially adaptive feature extraction, a class-specific fusion module that separates small and large objects into attentive fusion pathways, bird's-eye-view enhancement through parallel dilated convolution and channel recalibration for richer contextual representation, and class-balanced objective weighting to reduce bias toward frequent categories. Experiments on the V2X-Real benchmark cover vehicle-centric, infrastructure-centric, vehicle-to-vehicle, infrastructure-to-infrastructure, and vehicle-to-infrastructure settings under identical backbone and training configurations. The proposed method consistently improves mean detection performance over strong intermediate-fusion baselines, with the largest gains on trucks, clear improvements on pedestrians, and competitive results on cars. These results show that aligning feature extraction and fusion with class-dependent geometry and point density leads to more balanced cooperative perception in realistic vehicle-to-everything deployments.

Figures & tables

Fig. 1: Overview of the proposed class-adaptive cooperative perception architecture for multi-class 3D object detection in V2X systems. The framework processes multi-agent LiDAR inputs through: (1) Multi-agent feature extraction via PointPillars backbone, (2) Multi-scale window attention with learned class-adaptive routing that adapts receptive fields to object scales (2×2, 4×4, 8×8, 16×16 windows), (3) Class-specific fusion block with dedicated pathways for small objects (pedestrians) and large objects (cars, trucks), (4) Multi-scale BEV enhancement using atrous spatial pyramid pooling (ASPP) with dilation rates [1, 3, 6, 12] and squeeze-and-excitation (SE) modules, and (5) 3D detection head with class-balanced loss optimization. The architecture addresses scale variation and class imbalance through differentiated feature routing and gradient reweighting.

Fig. 2: Bird’s-eye-view visualization of a representative V2X-Real scene showing multi-class 3D annotations: pedestrians (red, 25), cars (green, 16), and trucks (blue, 4) relative to ego vehicle (cyan). The dataset exhibits significant class imbalance and scale variations.

Fig. 3: Per-class detection performance comparison. Values are AP@50 (%) averaged across all five cooperation modes. Our method achieves the highest accuracy in every category, with the largest margin on truck detection (+5.8 pp on average over the strongest baseline per cooperation mode), confirming that class-differentiated fusion disproportionately benefits scale-diverse objects.

Fig. 4: Per-class performance profile in VC mode (AP@50). Each polygon represents one method. Our method (red, solid) achieves the largest enclosed area, reflecting the most balanced detection across all three classes. The pronounced Truck-axis expansion (+6.7 pp over the strongest baseline, AttFuse at 31.8%) highlights the benefit of scale-adaptive fusion for large objects.

Mode	Method	Vehicle			Pedestrian			Truck			mAP
Mode	Method	AP@30	AP@50	$\Delta$	AP@30	AP@50	$\Delta$	AP@30	AP@50	$\Delta$	AP@30	AP@50
VC	F-Cooper	62.9	49.5	-	26.8	9.9	-	35.0	23.6	-	41.6	27.7
	AttFuse	58.5	55.1	-	29.4	12.9	-	37.0	31.8	-	41.6	33.3
	V2X-ViT	67.3	61.5	-	29.8	12.1	-	38.6	29.0	-	45.3	34.2
	Ours	67.8	62.8	+1.3	30.2	12.3	-0.6	44.6	38.5	+6.7	47.5	37.9
IC	F-Cooper	61.8	23.8	-	24.8	7.7	-	18.4	4.1	-	35.0	11.8

TABLE I: Comparative results on V2X-Real test set across all cooperation modes. All methods use PointPillars backbone with identical training procedures. Best results in bold , second best underlined . AP reported as percentage at IoU thresholds 0.3 and 0.5.

\Delta

indicates the difference between our method and the best baseline for AP@50 (positive values in green , negative in red ).

Fig. 5: Qualitative comparison across three cooperation modes. Each row shows a representative scene: vehicle-centric (VC, top), vehicle-to-vehicle (V2V, middle), and infrastructure-centric (IC, bottom). Columns compare F-Cooper, AttFuse, V2X-ViT, and our method on identical frames. Green boxes denote ground truth annotations. Predicted bounding boxes are color-coded by class: orange (vehicle), red (pedestrian), magenta (truck). Our method produces more accurate localizations with fewer false positives and missed detections, particularly for underrepresented pedestrians and trucks. Best viewed in color with zoom.

Method	Params (M)	Memory (MB)	Inference (ms)
F-Cooper	8.07	30.8	13.1
AttFuse	8.07	30.8	15.6
V2X-ViT	13.47	51.4	289.9
Ours	15.89	60.6	345.8

TABLE II: Computational cost comparison. Params: total learnable parameters; Memory: model memory footprint; Inference: average per-sample inference time on NVIDIA A100.

Configuration	Vehicle	Pedestrian	Truck	mAP@50
Baseline (V2X-ViT)	52.3	11.1	25.3	29.5
+M1	56.8	13.6	39.4	36.6
+M1+M2	58.7	13.2	40.9	37.6
+M1+M2+M3 (Full)	56.4	12.9	38.0	35.8
Gain vs. Baseline	+4.1	+1.8	+12.7	+6.3

TABLE III: Ablation study on V2X-Real test set showing incremental contribution of each architectural component. Results reported for V2V cooperation mode at IoU threshold 0.5. M1: Multi-scale window attention + Class-specific fusion; M2: Multi-scale BEV enhancement; M3: Class-balanced loss.

Fig. 6: Qualitative ablation on a scene in 3D perspective view for v2v collaborative mode. (a) +M1: multi-scale attention and class-specific fusion; (b) +M2: multi-scale BEV enhancement; (c) +M3: full model with class-balanced loss. Green: ground truth; orange: vehicle; red: pedestrian; magenta: truck. Configurations (a) and (b) produce scattered pedestrian false positives (left region), while (c) yields cleaner, more calibrated predictions.

Fig. 7: Per-class AP@50 as a function of BEV distance, averaged across all cooperation modes. Vehicle detection (left) degrades with range for all methods, while our variants maintain higher accuracy throughout. Pedestrian detection (center) exhibits limited range sensitivity due to inherent LiDAR sparsity. Truck detection (right) shows consistent gains for our method at all distances, with the +M1+M2 configuration (Ours v2) achieving the best far-range performance owing to ASPP-based context aggregation.

Mode	Method	0–30 m (Near)				30–60 m (Mid)				60–100 m (Far)
Mode	Method	Veh.	Ped.	Truck	mAP	Veh.	Ped.	Truck	mAP	Veh.	Ped.	Truck	mAP
VC	F-Cooper	59.9	9.9	16.7	28.8	47.2	8.8	31.8	29.3	39.8	12.3	42.7	31.6
	AttFuse	81.6	14.6	35.7	44.0	51.2	12.6	29.5	31.1	27.0	11.8	43.9	27.5
	V2X-ViT	81.0	13.7	24.9	39.8	60.6	12.2	31.7	34.8	36.5	10.8	39.9	29.0
	Ours (v1)	83.0	15.1	49.5	49.2	62.9	13.1	42.8	39.6	36.2	12.5	53.5	34.0
	Ours (v2)	85.6	15.2	44.0	48.3	60.1	13.0	46.0	39.7	41.2	11.1	59.9	37.4

TABLE IV: Range-stratified detection performance on V2X-Real test set. AP@50 (%) reported at three BEV distance ranges for all cooperation modes. Best in bold , second best underlined . Ours (v1): multi-scale attention + class-specific fusion; Ours (v2): +multi-scale BEV enhancement; Ours (Full): +class-balanced loss.

Figure 12

Cooperative Perception

Cooperative perception lets connected vehicles, robots, and aerial agents combine sensor observations to understand scenes beyond a single viewpoint.

See all 111 papers

Jul 14, 2026

ViCo3D: Empowering LiDAR-based Collaborative 3D Object Detection with Vision Foundation Models

LiDAR-based collaborative 3D perception in Vehicle-to-Everything (V2X) systems typically relies on fusing bird's-eye-view (BEV) features across agents. However, current BEV representations, typically extracted by LiDAR backbones trained from scratch, are geometry-dominated and lack general semantic priors, inherently limiting the efficacy of feature-level collaboration. Meanwhile, vision foundation models (VFMs) pretrained on large-scale image data have demonstrated strong capability in learning general-purpose and informative visual representations for 2D tasks, and have the potential to enhance agent-wise LiDAR BEV representations for collaboration. Despite this potential, adapting VFMs to LiDAR-based 3D detection remains challenging due to the substantial image-point cloud modality gap. To bridge this gap, we propose ViCo3D, a collaborative 3D object detection framework powered by VFMs. Specifically, ViCo3D adapts VFMs to LiDAR-based collaborative perception from three aspects: First, ViCo3D projects point clouds onto the BEV plane as three-channel images, enabling DINOv2 to extract BEV-space visual features from LiDAR inputs. Besides, to effectively integrate these DINOv2-derived features with LiDAR geometric features, ViCo3D introduces a multi-scale BEV fusion module within the single-agent encoder. In addition, ViCo3D adopts an ego-centric cross-agent fusion strategy to aggregate complementary information from multiple agents. Experiments on DAIR-V2X and V2XSet demonstrate that ViCo3D achieves state-of-the-art 3D detection performance. Remarkably, it delivers up to 1.8x greater collaborative gains than prior methods on DAIR-V2X. The code will be made public available for future investigation.

Haojie Ren, Songrui Luo, Lingfeng Wang +6

University of Science and Technology of China

Lidar-Based 3D Object Detection Vision Foundation Models Bird's-Eye-View Representation 3D Object Detection Cooperative Perception

Ask PDF

May 1, 2026

Robust Fusion of Object-Level V2X for Learned 3D Object Detection

Perception for automated driving is largely based on onboard environmental sensors, such as cameras and radar, which are cost-effective but limited by line-of-sight and field-of-view constraints. These inherent limitations may cause onboard perception to fail under occlusions or poor visibility conditions. In parallel, cooperative awareness via vehicle-to-everything (V2X) communication is becoming increasingly available, enabling vehicles and infrastructure to share their own state as object-level information that complements onboard perception. In this work, we study how such V2X information can be integrated into 3D object detection and how robust the resulting system is to realistic V2X imperfections. Using the nuScenes dataset, we emulate object-level cooperative awareness messages from ground truth, injecting controlled noise and object dropout to mimic real-world conditions such as latency, localization errors, and low V2X penetration rates. We convert these messages into a dedicated bird's-eye view (BEV) input and fuse them into a BEVFusion-style detector. Our results demonstrate that while object-level cooperative information can substantially improve detection performance, achieving an NDS of 0.80 under favorable conditions, models trained on idealized data become fragile and over-reliant on V2X. Conversely, our proposed noise-aware training strategy, coupled with explicit confidence encoding, enhances robustness, maintaining performance gains even under severe noise and reduced V2X penetration.

Lukas Ostendorf, Lennart Reiher, Onn Haran +1

Institute for Automotive Engineering, RWTH Aachen University, 52074 Aachen, Germany · Qualcomm Technologies, Inc., Kfar-Netter, Israel

3D Object Detection Cooperative Perception

Ask PDF

Jul 21, 2026

CoGoal3D: Collaborative 3D Object Detection with 3D-Aware Fusion and Refinement

V2X collaborative object detection features overcoming the limitations of single-vehicle systems by aggregating environmental features from multiple collaborative agents. However, existing mainstream V2X perception methods mainly focus on 2D BEV object detection. When 3D detection task is concerned, inferior results are obtained because they ignore the 3D spatial misalignment caused by differing height and attitude among the collaborators. In this paper, we propose a novel collaborative 3D object detection framework called CoGoal3D, which extracts and refines the 3D feature gradually in a two-stage pipeline. In the first stage, a multiscale 3D-aware global fusion module is designed to mitigate the 3D spatial misalignment. The resulting proposals are then refined in the second stage with an auxiliary task of 3D point reconstruction. An effective multi-agent collaborative data augmentation strategy is further proposed to enrich the training data while minimizing information loss. Extensive experiments on public real-world datasets demonstrate that our CoGoal3D achieves new state-of-the-art performance, with 3D [email protected] improvements of 10.86%, 10.34%, and 10.18% on the DAIR-V2X, V2V4Real, and V2X-Real datasets, respectively. Code is available at https://github.com/Megalo-f/CoGoal3D.

Zhihao Yang, Zhiyu Xiang, Peng Xu +5

Zhejiang University, Hangzhou, China · Zhejiang Provincial Key Laboratory of Multi-Modal Communication Networks and Intelligent Information Processing, Hangzhou, China · China Jiliang University, Hangzhou, China

Point Cloud Surface Reconstruction 3D Object Detection Cooperative Perception Feature Fusion

Ask PDF

Abstract

Cooperative Perception

Explore similar work

ViCo3D: Empowering LiDAR-based Collaborative 3D Object Detection with Vision Foundation Models

Robust Fusion of Object-Level V2X for Learned 3D Object Detection

CoGoal3D: Collaborative 3D Object Detection with 3D-Aware Fusion and Refinement