Organizations: Zhejiang University · Zhejiang Provincial Key Laboratory of Multi-Modal Communication Networks and Intelligent Information Processing · China Jiliang University
V2X collaborative perception improves single-vehicle perception by aggregating multi-view features from collaborative agents. However, existing methods typically rely on simple spatial warping to align the collaborative features, which can cause semantic misalignment during fusion. Meanwhile, these deterministic warping-based methods are highly sensitive to the positional noises. In this paper, we propose a plug-in module called Robust Semantic and Spatial Aligner (RSSA), which performs semantic and probabilistic spatial transformations for robust feature alignment. Specifically, a Semantic Transformation (SemT) block modulates feature channels based on the observed relative pose to correct semantic inconsistencies. Subsequently, a Probabilistic Spatial Transformation (P-SpaT) block performs probabilistic warping by sampling from the Gaussian-modeled relative pose space. A Relative Transformation Augmentation (RT-Aug) strategy is further introduced to augment the relative poses and facilitate the training of the RSSA. Extensive experiments on both simulated and real-world datasets demonstrate that RSSA improves the state-of-the-art methods by a large margin under the configuration of both clean and noisy positions.
Figures & tables
Figure 1: Comparison between the traditional spatial warping and our proposed RSSA for BEV features.
Figure 2: The overall collaborative perception pipeline and the architecture of our proposed RSSA.
Figure 3: Illustration of the Relative Transformation Augmentation (RT-Aug) strategy.
Table 1: Performance comparison with state-of-the-art methods on the V2XSet and V2X-Real datasets under the clean setting. The best results are highlighted in bold . H and B denote handshake and broadcast, respectively. The values in red indicate the performance improvement of our RSSA.
Figure 4: Performance comparison with state-of-the-art methods on the V2XSet (top row) and V2X-Real (bottom row) datasets under various noise levels. Columns from left to right are: Att fuse [ 18 ] , CoBEVT [ 16 ] , ERMVP [ 21 ] , CoSDH [ 15 ] , respectively. B and H denote Broadcast and Handshake, respectively.
Figure 6Figure 7
Figure 9: Visualization of detection results under the clean (left three columns) and noisy (right three columns) settings. For each setting, the upper row shows the zoomed-in view of the region highlighted by the blue circle in the lower BEV view. Green and red boxes represent ground truth and predictions, respectively.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 10: Comparison of detection results between different align methods. The yellow square represents the foreground feature, and the green box indicates the anchor. The top row (result-level alignment) yields the target result ( blue box ), whereas the bottom row (feature-level alignment) results in a misalignment ( red box ). The discrepancy confirms that spatial warping of BEV features causes semantic misalignment and leads to detection error, highlighting the importance and necessity of semantic alignment during coordinate transformation.
Figure 11: Visualization of detection results under the clean setting (zoomed-in portion of Figure 9 ). The incorrect detection result within the blue circle confirms that spatial warping causes semantic misalignment, leading to orientation and center errors. By resolving the semantic misalignment, RSSA yields more accurate detection results.
Feature
Center error (m)
Orientation error ( ∘ )
Before SemT
0.52
45.93
After SemT
0.17
10.78
Appendix
Table 2: Mean error before and after SemT on V2XSet under the clean setting.
Figure 12: Statistical analysis of relative pose distributions on the V2XSet dataset. The yaw angle distribution (right) exhibits significant data bias with peaks at 0∘,±90∘, and ±180∘ , highlighting the structured nature of urban scenes and the necessity of the RT-Aug strategy.
Table 11: Comparison of model complexity, efficiency, and performance using CoBEVT as fusion module on the V2XSet dataset.
Figure 14: Visualization of detection results on the V2X-Real dataset using CoBEVT as fusion module across clean (top) and noisy (bottom) settings. The blue circles highlight that the broadcast paradigm suffers from semantic misalignment caused by spatial warping, and the handshake paradigm is sensitive to noise, resulting in more false positives. Our RSSA provides the most robust and accurate detection by performing semantic transformation and probabilistic spatial transformation.
Collaborative perception extends the sensing range of a single vehicle by fusing observations from nearby agents, which improves the robustness of autonomous driving. In realistic deployments, however, the received collaborator messages are often affected by both communication delay and relative-pose noise, which jointly cause stale observations, spatial misalignment, and unstable feature fusion. Existing methods usually address these issues from either the spatial or temporal side, but handling them jointly in a unified and efficient manner remains challenging. In this paper, we propose CoAnchor, an anchor-centric spatio-temporal alignment framework for asynchronous collaborative perception. Instead of directly reasoning on dense BEV features, CoAnchor builds sparse object-level spatio-temporal anchors as a shared interface for pose correction and tightly connects spatial refinement, temporal propagation, and current-time verification within one unified loop, while keeping the overall correction process lightweight. Extensive experiments on both simulated and real-world datasets illustrate that CoAnchor remains competitive under clean settings and improves the robustness under joint delay and pose perturbations with a favorable practical accuracy-efficiency trade-off.
Chi Li, Rui Lin, Aobo Ji +1
State Key Laboratory of Networking and Switching Technology Beijing University of Posts and Telecommunications Beijing, China
In autonomous driving, multi-agent collaborative perception enhances sensing capabilities by enabling agents to share perceptual data. A key challenge lies in handling {\em heterogeneous} features from agents equipped with different sensing modalities or model architectures, which complicates data fusion. Existing approaches often require retraining encoders or designing interpreter modules for pairwise feature alignment, but these solutions are not scalable in practice. To address this, we propose {\em GT-Space}, a flexible and scalable collaborative perception framework for heterogeneous agents. GT-Space constructs a common feature space from ground-truth labels, providing a unified reference for feature alignment. With this shared space, agents only need a single adapter module to project their features, eliminating the need for pairwise interactions with other agents. Furthermore, we design a fusion network trained with contrastive losses across diverse modality combinations. Extensive experiments on simulation datasets (OPV2V and V2XSet) and a real-world dataset (RCooper) demonstrate that GT-Space consistently outperforms baselines in detection accuracy while delivering robust performance. Our code will be released at https://github.com/KingScar/GT-Space.
Wentao Wang, Haoran Xu, Guang Tan
Sun Yat-sen University, Shenzhen Campus, Shenzhen, China · Peng Cheng Laboratory (PCL), Shenzhen, China
Collaborative perception enables connected agents to share complementary observations for 3D object detection, extending sensing range and mitigating occlusion. Under asynchronous communication, however, cooperative features arrive with temporal delay. Existing prediction-based methods compensate for these features mainly from the transmitting agent's own history, leaving residual misalignment with the ego agent's current observation; subsequent fusion also often overlooks spatial variations in alignment quality. We propose EgoRefine, an ego-referenced predictive alignment and reliability-aware fusion framework for asynchronous collaborative perception. Its Ego-referenced Predictive Alignment module uses the current ego feature to guide cooperative trajectory-field prediction and refines the sampling offsets along an ego-referenced trajectory direction. Its Trajectory-conditioned Reliability-aware Fusion module treats the trajectory discrepancy between the ego and cooperative streams and the directional refinement magnitude as alignment cues, using them to condition the relation between aligned features and adaptively reweight the two streams before convolutional fusion. Experiments on V2V4Real and DAIR-V2X-Seq show that EgoRefine outperforms TraF-Align by 1.6 and 2.9 points on average in [email protected] and [email protected], respectively. The source code will be made publicly available at https://github.com/godk0509/EgoRefine.
Lingzhao Kong, Yongsheng Zang, Yu Kang +4
School of Computer Science and Electronic Engineering, Hunan University, Changsha 410082, China · School of Artificial Intelligence and Robotics, Hunan University, Changsha 410012, China · National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, Changsha 410082, China +1