Organizations: College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou 310027, China · School of Automotive Studies, Tongji University, Shanghai 201804, China · Ningbo Global Innovation Center, Zhejiang University, Ningbo 315100, China
4D radar measurements offer an affordable and weather-robust solution for 3D perception. However, the inherent sparsity and noise of radar point clouds present significant challenges for accurate 3D object detection, underscoring the need for effective and robust point clouds densification. Despite recent progress, existing densification methods often fail to address the extreme sparsity of 4D radar point clouds and exhibit limited robustness when processing scenes with a small number of points. In this paper, we propose SD4R, a novel framework that transforms sparse radar point clouds into dense representations. SD4R begins by utilizing a foreground point generator (FPG) to mitigate noise propagation and produce densified point clouds. Subsequently, a logit-query encoder (LQE) enhances conventional pillarization, resulting in robust feature representations. Through these innovations, our SD4R demonstrates strong capability in both noise reduction and foreground point densification. Extensive experiments conducted on the publicly available View-of-Delft dataset demonstrate that SD4R achieves state-of-the-art performance. Source code is available at https://github.com/lancelot0805/SD4R.
Figures & tables
Fig. 1: Performance of our SD4R in real-scene 3D detection. The sparse points refer to the original radar measurements, while the virtual points are generated only based on the foreground points. The second row shows the corresponding visualization results. Our SD4R framework demonstrates strong capability in both noise reduction and foreground point densification.
Fig. 2: (a) The proposed SD4R pipeline begins with voxelization of 4D radar point cloud, followed by processing through the VoteHead. This step predicts offsets between points and their corresponding object centers, classification logits, and point-wise features. These point-wise features are then concatenated with the logits to classify the point clouds. Subsequently, virtual points are generated at positions determined by the offsets, resulting in a densified point cloud. (b) The densified point cloud undergoes pillarization [ 24 ] to extract features. To further address the sparsity of radar data, we introduce a logit-query encoder (LQE) module, which aggregates features from neighboring points into pillars, leading to more robust representations. Finally, the detection head processes these features to generate the final detection outputs.
Fig. 3: Based on each virtual point, we select k original points. The upper branch conducts the selection and feature extraction of k points, while the lower branch assigns weights to k points based on distance. Finally, the features of k points are multiplied by their weights and summed to obtain the feature of the virtual point.
Fig. 4: Here are the details about LQE. We firstly computes the aggregation radius (R) based on the inside points, then aggregate the features of the outside points within this radius (R). Ultimately, the updated pillar features are obtained.
Method
Modality
Entire Annotated Area AP (%) ↑
Driving Corridor AP (%) ↑
FPS ↑
Car
Pedestrian
Cyclist
mAP
Car
Pedestrian
Cyclist
mAP
ImVoxelNet (WACV 2022) [ 31 ]
C
19.35
5.62
17.53
14.17
49.52
9.68
28.97
29.39
7.3
FUTR3D (CVPR 2023) [ 32 ]
R+C
46.01
35.11
65.98
49.03
78.66
43.10
86.19
69.32
11.0
BEVFusion (ICRA 2023) [ 33 ]
R+C
37.85
40.96
68.95
49.25
70.21
45.86
89.48
68.52
7.1
RCFusion (IEEE TIM 2023) [ 24 ]
R+C
41.70
38.95
68.31
49.65
71.87
47.50
88.33
69.23
9.0
RCBEVDet (CVPR 2024) [ 34 ]
R+C
40.63
38.86
70.48
49.99
72.48
49.89
87.01
69.80
-
TABLE I: Comparison with state-of-the-art approaches on the validation set of VoD [ 7 ] . In the Modality column, R denotes 4D radar and C denotes camera. The best values of the single 4D radar modality are emboldened.
Category
Method
3D
BEV
AOS
Car
Baseline
39.31
49.67
31.30
SD4R
41.04
47.47
35.79
Pedestrian
Baseline
35.07
40.47
25.94
SD4R
43.41
47.37
33.25
Cyclist
Baseline
63.63
66.36
52.37
SD4R
70.98
72.13
62.15
TABLE II: Comparison of SD4R and the baseline [ 24 ] across different categories on the validation set of VoD [ 7 ] .
Baseline
FPG
LQE
Car
Pedestrian
Cyclist
mAP
✓
39.30
35.10
63.63
46.01
✓
✓
38.59
37.17
64.16
46.64
✓
✓
✓
40.81
42.02
65.30
49.38
TABLE III: Effect of each component in our SD4R in terms of mAP on the VoD dataset [ 7 ] .
Fig. 5: Some visualization results on the VoD [ 7 ] validation. Each column corresponds to a frame of data containing radar points in BEV and an image, where the red triangle denotes the position of the ego-vehicle. Ground-truth boxes are shown in orange (perspective) and yellow (bird’s-eye), while predicted boxes appear in blue and green, respectively. The second row overlays SD4R’s image-plane predictions.
Rp
Rcyc
Rcar
Ped
Cyc
Car
mAP
0.2
0.2
0.2
33.94
50.67
32.92
39.18
0.3
0.3
0.3
35.39
55.03
35.55
41.99
0.4
0.4
0.4
32.45
52.14
40.78
38.98
0.1
0.3
0.3
32.07
51.89
35.04
39.67
0.2
0.3
0.3
37.06
52.59
33.16
40.94
0.3
0.2
0.3
34.79
54.06
34.66
41.17
TABLE IV: Ablation of radius parameters within our LQE.