CATNet: Collaborative Alignment and Transformation Network for Cooperative Perception
Organizations: School of Computer Science and Technology, Tianjin University · School of Cybersecurity, Tianjin University · School of Future Technology, Tianjin University
Abstract
Cooperative perception significantly enhances scene understanding by integrating complementary information from diverse agents. However, existing research often overlooks critical challenges inherent in real-world multi-source data integration, specifically high temporal latency and multi-source noise. To address these practical limitations, we propose Collaborative Alignment and Transformation Network (CATNet), an adaptive compensation framework that resolves temporal latency and noise interference in multi-agent systems. Our key innovations can be summarized in three aspects. First, we introduce a Spatio-Temporal Recurrent Synchronization (STSync) that aligns asynchronous feature streams via adjacent-frame differential modeling, establishing a temporal-spatially unified representation space. Second, we design a Dual-Branch Wavelet Enhanced Denoiser (WTDen) that suppresses global noise and reconstructs localized feature distortions within aligned representations. Third, we construct an Adaptive Feature Selector (AdpSel) that dynamically focuses on critical perceptual features for robust fusion. Extensive experiments on multiple datasets demonstrate that CATNet consistently outperforms existing methods under complex traffic conditions, proving its superior robustness and adaptability.
Figures & tables
| Method | Published | Params | OPV2V | V2XSet | DAIR-V2X | |||
|---|---|---|---|---|---|---|---|---|
| [email protected] | [email protected] | [email protected] | [email protected] | [email protected] | [email protected] | |||
| No Fusion | – | – | 0.738 | 0.509 | 0.698 | 0.516 | 0.625 | 0.446 |
| ∗ Where2comm [ 13 ] | NIPS’22 | 11.43 M | 0.699 | 0.402 | 0.757 | 0.495 | 0.654 | 0.502 |
| ∗ CoMamba [ 17 ] | Arxiv’24 | 09.15 M | 0.743 | 0.558 | 0.726 | 0.531 | 0.635 | 0.479 |
| † V2X-ViT [ 30 ] | ICRA’22 | 13.50 M | 0.817 | 0.633 | 0.797 | 0.593 | 0.696 | 0.517 |
| † CORE [ 26 ] | ICCV’23 | 09.19 M | 0.831 | 0.628 | 0.764 | 0.504 | – | |
| Method | 0ms | 0-200ms | 0-300ms | 0-400ms | 0-500ms | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| [email protected] | [email protected] | [email protected] | [email protected] | [email protected] | [email protected] | [email protected] | [email protected] | [email protected] | [email protected] | |
| CoAlign | 0.881 | 0.748 | 0.709 | 0.517 | 0.658 | 0.524 | 0.631 | 0.508 | 0.610 | 0.495 |
| V2X-ViT | 0.863 | 0.724 | 0.774 | 0.580 | 0.738 | 0.557 | 0.716 | 0.538 | 0.697 | 0.530 |
| ERMVP | 0.831 | 0.686 | 0.824 | 0.643 | 0.783 | 0.630 | 0.767 | 0.613 | 0.745 | 0.600 |
| MRCNet | 0.848 | 0.715 | 0.836 | 0.647 | 0.774 | 0.638 | 0.759 | 0.625 | 0.738 | 0.615 |
| DSRC | 0.887 | 0.752 | 0.841 | 0.598 | 0.731 | 0.552 | 0.709 | 0.522 | 0.640 | 0.508 |
| Configuration | OPV2V | DAIR-V2X |
|---|---|---|
| Baseline | 0.595 / 0.384 | 0.659 / 0.461 |
| + STSync | 0.818 / 0.678 | 0.683 / 0.496 |
| + WTDen | 0.645 / 0.461 | 0.671 / 0.482 |
| + AdpSel | 0.624 / 0.425 | 0.666 / 0.477 |
| + STSync + WTDen | 0.834 / 0.680 | 0.717 / 0.549 |
| + STSync + AdpSel | 0.822 / 0.682 | 0.708 / 0.553 |
| Proportion | OPV2V | V2XSet | DAIR-V2X |
|---|---|---|---|
| 0.1 | 0.854/0.674 | 0.873/0.717 | 0.767/0.589 |
| 0.2 | 0.854/0.674 | 0.873/0.716 | 0.774/0.587 |
| 0.3 | 0.855 / 0.691 | 0.874 / 0.717 | 0.775 /0.603 |
| 0.4 | 0.854/0.675 | 0.873/0.716 | 0.773/0.604 |
| 0.5 | 0.845/0.681 | 0.863/0.711 | 0.770/ 0.605 |
| 0.6 | 0.836/0.663 | 0.863/0.702 | 0.765/0.596 |
| Module / Metric | [email protected] | [email protected] | [email protected] |
|---|---|---|---|
| AdpSel | 0.905 | 0.897 | 0.791 |
| AdpSel (high mask) | 0.366 | 0.364 | 0.328 |
| AdpSel (low mask) | 0.791 | 0.784 | 0.707 |
| Method | 0/0 | 0.2/0.2 | 0.4/0.4 |
|---|---|---|---|
| Agent-graph | 0.872/0.790 | 0.868/0.737 | 0.838/0.588 |
| WTDen | 0.893/0.795 | 0.875/0.737 | 0.860/0.654 |
| FeaCo | 0.881/0.784 | 0.874/0.755 | 0.855/0.649 |
| CoAlign | 0.888/0.790 | 0.881/0.748 | 0.848/0.633 |
| DSRC | 0.894/0.803 | 0.887/0.752 | 0.854/0.638 |
| CATNet | 0.905/0.807 | 0.896/0.763 | 0.870/0.663 |
Cooperative Perception
Cooperative perception lets connected vehicles, robots, and aerial agents combine sensor observations to understand scenes beyond a single viewpoint.
See all 111 papers