Send Less, Perceive More: Masked Quantized Point Cloud Communication for Loss-Tolerant Collaborative Perception
Authors: Sheng Xu, Enshu Wang, Hongfei Xue, Jian Teng, Bingyi Liu, Yi Zhu, Pu Wang, Libing Wu, Chunming Qiao
Organizations: School of Cyber Science and Engineering, Wuhan University, China · University of North Carolina at Charlotte, USA · Wuhan University of Technology, China · Wayne State University, USA · University at Buffalo, USA
Collaborative perception allows connected vehicles to overcome occlusions and limited viewpoints by sharing sensory information. However, existing approaches struggle to achieve high accuracy under strict bandwidth constraints and remain highly vulnerable to random transmission packet loss. We introduce QPoint2Comm, a quantized point-cloud communication framework that dramatically reduces bandwidth while preserving high-fidelity 3D information. Instead of transmitting intermediate features, QPoint2Comm directly communicates quantized point-cloud indices using a shared codebook, enabling efficient reconstruction with lower bandwidth than feature-based methods. To ensure robustness to possible communication packet loss, we employ a masked training strategy that simulates random packet loss, allowing the model to maintain strong performance even under severe transmission failures. In addition, a cascade attention fusion module is proposed to enhance multi-vehicle information integration. Extensive experiments on both simulated and real-world datasets demonstrate that QPoint2Comm sets a new state of the art in accuracy, communication efficiency, and resilience to packet loss.
Figures & tables
Figure 1: Comparison between our proposed quantized point cloud-based framework and existing feature-based methods for collaborative perception.
Figure 2: Overview of the proposed framework, consisting of Discrete Point Cloud Representation ( Sec. 3.2 ) for point cloud encoding and decoding, Packet-loss Tolerant Design ( Sec. 3.3 ) to enable communication with missing packets, Feature Fusion and Bounding Boxes Generation ( Sec. 3.4 ) module for generating bounding boxes.
Figure 3: The Discrete Point Cloud Representation (DPR) module is trained to encode point clouds into discrete index sequences and to faithfully reconstruct them via a shared codebook.
Figure 4: The Feature Fusion and Bounding Boxes generation.
Figure 5: (a) The Pyramid-scale Fusion. (b) The Cascade Attention Fusion (CAF).
Figure 6
Figure 7: Robustness to the transmission delay and localization error on the OPV2V and DAIR-V2X.
Figure 10: Robustness to the localization error on the OPV2V and DAIR-V2X datasets.
Figure 11: Robustness to the transmission delay on the OPV2V and DAIR-V2X datasets.
Figure 12: Robustness to the heading error on the OPV2V and DAIR-V2X datasets.
Figure 13: Visualization of BEV representations on the OPV2V dataset. The top row shows the original LiDAR point cloud BEV maps, while the bottom row presents the reconstructed BEV maps obtained from the transmitted discrete indices. The high structural consistency indicates that the DPR module effectively preserves the geometric and semantic information of the original scenes.
Figure 14: Visualization of BEV representations on the DAIR-V2X dataset. The top row displays the original point cloud BEV maps, and the bottom row shows the reconstructed BEV maps derived from discrete indices. The reconstruction quality demonstrates the strong representation capability and information retention of the DPR module across different domains.
Figure 15: Additional qualitative comparison of detection results on the DAIR-V2X dataset. Green and red bounding boxes denote the ground-truth annotations and the predictions generated by the models, respectively.
Dataset
OPV2V
DAIR-V2X
Chamfer Distance (m)
0.0572
0.0516
Table 8: Chamfer Distance on OPV2V and DAIR-V2X dataset.
OPV2V
DAIR-V2X
Method
0.0s
0.1s
0.2s
0.3s
0.4s
0.0s
0.1s
0.2s
0.3s
0.4s
Ours
86.81
81.78
80.66
80.70
77.92
56.50
53.10
51.67
51.83
51.63
mmCooper
86.41
78.57
77.44
75.95
75.26
56.06
53.05
50.96
49.86
50.81
ERMVP
80.55
71.42
68.89
68.52
68.31
53.27
49.81
49.68
48.74
48.60
Where2comm
78.47
71.60
65.23
58.73
53.33
52.34
49.74
47.76
45.83
45.25
V2X-ViT
77.88
71.41
66.08
55.71
54.74
46.12
45.69
45.49
44.54
43.99
Table 9: [email protected] under different communication latency (s) on OPV2V and DAIR-V2X.
OPV2V
DAIR-V2X
Method
0.0 m
0.1 m
0.2 m
0.3 m
0.4 m
0.0 m
0.1 m
0.2 m
0.3 m
0.4 m
Ours
86.81
85.91
85.50
83.68
80.30
56.50
55.68
55.72
50.53
49.46
mmCooper
86.41
84.15
82.35
79.70
76.80
56.06
52.77
51.25
48.17
47.66
ERMVP
80.55
79.07
76.19
75.46
72.78
53.27
51.91
48.66
46.66
45.97
Where2comm
78.47
77.27
75.45
72.14
69.77
52.34
51.05
49.34
47.90
46.96
V2X-ViT
77.88
76.31
75.21
70.77
66.72
46.09
45.21
43.81
42.83
42.53
Table 10: [email protected] under different localization error (m) on OPV2V and DAIR-V2X.
OPV2V
DAIR-V2X
Method
0.0 ∘
0.1 ∘
0.2 ∘
0.3 ∘
0.4 ∘
0.0 ∘
0.1 ∘
0.2 ∘
0.3 ∘
0.4 ∘
Ours
86.81
86.43
85.64
84.33
81.81
56.50
56.37
53.76
52.70
51.34
mmCooper
86.41
84.82
83.18
81.70
79.91
56.06
54.77
53.11
50.17
49.70
ERMVP
80.55
77.07
74.32
72.46
71.23
53.27
52.81
51.38
47.66
48.48
Where2comm
78.47
77.27
75.45
72.14
69.77
52.34
51.75
50.39
49.47
48.76
V2X-ViT
75.88
75.71
75.69
74.77
73.89
46.09
45.81
45.54
45.23
44.77
Table 11: [email protected] under different heading errors ( ∘ ) on OPV2V and DAIR-V2X.
Communication-efficient collaborative perception studies how networked agents share useful sensor information with minimal bandwidth while improving their joint view of the scene.
Renmin University of China · The Hong Kong University of Science and Technology (HKUST) · Hebei Key Laboratory of Real-virtual Integrated Autonomous Systems (RIAS)
Key Laboratory of Big Data & Artificial Intelligence in Transportation, Ministry of Education · School of Computer Science and Technology, Beijing Jiaotong University · Xiamen University