Organizations: College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou 310027, China · School of Automotive Studies, Tongji University, Shanghai 201804, China · Ningbo Global Innovation Center, Zhejiang University, Ningbo 315100, China
4D radar measurements offer an affordable and weather-robust solution for 3D perception. However, the inherent sparsity and noise of radar point clouds present significant challenges for accurate 3D object detection, underscoring the need for effective and robust point clouds densification. Despite recent progress, existing densification methods often fail to address the extreme sparsity of 4D radar point clouds and exhibit limited robustness when processing scenes with a small number of points. In this paper, we propose SD4R, a novel framework that transforms sparse radar point clouds into dense representations. SD4R begins by utilizing a foreground point generator (FPG) to mitigate noise propagation and produce densified point clouds. Subsequently, a logit-query encoder (LQE) enhances conventional pillarization, resulting in robust feature representations. Through these innovations, our SD4R demonstrates strong capability in both noise reduction and foreground point densification. Extensive experiments conducted on the publicly available View-of-Delft dataset demonstrate that SD4R achieves state-of-the-art performance. Source code is available at https://github.com/lancelot0805/SD4R.
Figures & tables
Fig. 1: Performance of our SD4R in real-scene 3D detection. The sparse points refer to the original radar measurements, while the virtual points are generated only based on the foreground points. The second row shows the corresponding visualization results. Our SD4R framework demonstrates strong capability in both noise reduction and foreground point densification.
Fig. 2: (a) The proposed SD4R pipeline begins with voxelization of 4D radar point cloud, followed by processing through the VoteHead. This step predicts offsets between points and their corresponding object centers, classification logits, and point-wise features. These point-wise features are then concatenated with the logits to classify the point clouds. Subsequently, virtual points are generated at positions determined by the offsets, resulting in a densified point cloud. (b) The densified point cloud undergoes pillarization [ 24 ] to extract features. To further address the sparsity of radar data, we introduce a logit-query encoder (LQE) module, which aggregates features from neighboring points into pillars, leading to more robust representations. Finally, the detection head processes these features to generate the final detection outputs.
Fig. 3: Based on each virtual point, we select k original points. The upper branch conducts the selection and feature extraction of k points, while the lower branch assigns weights to k points based on distance. Finally, the features of k points are multiplied by their weights and summed to obtain the feature of the virtual point.
Fig. 4: Here are the details about LQE. We firstly computes the aggregation radius (R) based on the inside points, then aggregate the features of the outside points within this radius (R). Ultimately, the updated pillar features are obtained.
Method
Modality
Entire Annotated Area AP (%) ↑
Driving Corridor AP (%) ↑
FPS ↑
Car
Pedestrian
Cyclist
mAP
Car
Pedestrian
Cyclist
mAP
ImVoxelNet (WACV 2022) [ 31 ]
C
19.35
5.62
17.53
14.17
49.52
9.68
28.97
29.39
7.3
FUTR3D (CVPR 2023) [ 32 ]
R+C
46.01
35.11
65.98
49.03
78.66
43.10
86.19
69.32
11.0
BEVFusion (ICRA 2023) [ 33 ]
R+C
37.85
40.96
68.95
49.25
70.21
45.86
89.48
68.52
7.1
RCFusion (IEEE TIM 2023) [ 24 ]
R+C
41.70
38.95
68.31
49.65
71.87
47.50
88.33
69.23
9.0
RCBEVDet (CVPR 2024) [ 34 ]
R+C
40.63
38.86
70.48
49.99
72.48
49.89
87.01
69.80
-
TABLE I: Comparison with state-of-the-art approaches on the validation set of VoD [ 7 ] . In the Modality column, R denotes 4D radar and C denotes camera. The best values of the single 4D radar modality are emboldened.
Category
Method
3D
BEV
AOS
Car
Baseline
39.31
49.67
31.30
SD4R
41.04
47.47
35.79
Pedestrian
Baseline
35.07
40.47
25.94
SD4R
43.41
47.37
33.25
Cyclist
Baseline
63.63
66.36
52.37
SD4R
70.98
72.13
62.15
TABLE II: Comparison of SD4R and the baseline [ 24 ] across different categories on the validation set of VoD [ 7 ] .
Baseline
FPG
LQE
Car
Pedestrian
Cyclist
mAP
✓
39.30
35.10
63.63
46.01
✓
✓
38.59
37.17
64.16
46.64
✓
✓
✓
40.81
42.02
65.30
49.38
TABLE III: Effect of each component in our SD4R in terms of mAP on the VoD dataset [ 7 ] .
Fig. 5: Some visualization results on the VoD [ 7 ] validation. Each column corresponds to a frame of data containing radar points in BEV and an image, where the red triangle denotes the position of the ego-vehicle. Ground-truth boxes are shown in orange (perspective) and yellow (bird’s-eye), while predicted boxes appear in blue and green, respectively. The second row overlays SD4R’s image-plane predictions.
Rp
Rcyc
Rcar
Ped
Cyc
Car
mAP
0.2
0.2
0.2
33.94
50.67
32.92
39.18
0.3
0.3
0.3
35.39
55.03
35.55
41.99
0.4
0.4
0.4
32.45
52.14
40.78
38.98
0.1
0.3
0.3
32.07
51.89
35.04
39.67
0.2
0.3
0.3
37.06
52.59
33.16
40.94
0.3
0.2
0.3
34.79
54.06
34.66
41.17
TABLE IV: Ablation of radius parameters within our LQE.
How far can 3D object detection go using 4D radar alone? Despite offering weather-robust and velocity- aware sensing for autonomous perception, modern 4D radar still yields sparse, noisy, and unstable point clouds, limiting radar-only 3D detection. We present HyperDet, a detector- agnostic input enhancement pipeline that constructs task- aware hyper 4D radar point clouds by combining measured observations with completed foreground geometry. HyperDet first refines short-window surround-view radar observations through spatio-temporal accumulation and cross-sensor val- idation, while Doppler-guided motion compensation reduces dynamic object trails when motion can be estimated reliably. It then performs foreground generative enhancement using LiDAR-guided pseudo-radar supervision available only during training, enriching object geometry while preserving measured radar background and radar-native attributes. During detec- tor training, radar-aware object-level augmentation maintains Doppler consistency under geometric relocation. At inference, HyperDet requires radar input alone and can be directly paired with standard 3D detectors. Experiments on two public surround-view 4D radar datasets demonstrate consistent im- provements over matched temporal accumulation across stan- dard 3D detectors, validating input-level radar enhancement as an effective approach to radar-only 3D detection.
Yichun Xiao, Jin Jin, Runwei Guan +1
HKUST(GZ) · University of Edinburgh · University of Oxford
Automotive radar has shown promising developments in environment perception due to its cost-effectiveness and robustness in adverse weather conditions. However, the limited availability of annotated radar data poses a significant challenge for advancing radar-based perception systems. To address this limitation, we propose a novel framework to generate 4D radar point clouds for training and evaluating object detectors. Unlike image-based diffusion, our method is designed to consider the sparsity and unique characteristics of radar point clouds by applying diffusion to a latent point cloud representation. Within this latent space, generation is controlled via conditioning at either the object or scene level. The proposed 4D-RaDiff converts unlabeled bounding boxes into high-quality radar annotations and transforms existing LiDAR point cloud data into realistic radar scenes. Experiments demonstrate that incorporating synthetic radar data of 4D-RaDiff as data augmentation method during training of object detection models consistently improves performance compared to training on real data only. In addition, pre-training on our synthetic radar data achieves competitive detection performance, providing a promising, scalable approach to reduce dependence on costly manual annotations.
Jimmie Kwok, Holger Caesar, Andras Palffy
Delft University of Technology, Netherlands · Perciv AI, Netherlands
4D automotive radar is indispensable for autonomous driving due to its low cost and robustness, yet its point cloud sparsity challenges 3D object detection. Existing 4D radar-camera fusion methods focus on complex fusion strategies, trading inference speed for marginal gains. This trade-off hinders real-time deployment due to heavy computation on dense feature maps. In contrast, feature extraction from sparse radar points is less time-consuming but remains under-explored. This work uncovers that simply enhancing radar feature extraction can achieve comparable or even higher performance than elaborate fusion modules, while maintaining real-time performance. Based on this finding, we propose RCGDet3D, which centers on radar feature encoding and simplifies multi-modal fusion. Its encoder inherits from the efficient Gaussian Splatting-based Point Gaussian Encoder (PGE) in RadarGaussianDet3D with two key improvements. First, the Ray-centric PGE (R-PGE) predicts Gaussian attributes in ray-aligned coordinate systems before unifying them to Bird's-Eye View (BEV) space, significantly improving geometric consistency and reducing learning difficulty by decoupling the coordinate transformation from representation learning. Second, a Semantic Injection (SI) module incorporates visual cues from images, producing more geometrically accurate and semantically enriched radar features. Experiments on View-of-Delft (VoD) and TJ4DRadSet show that RCGDet3D outperforms state-of-the-art methods in both accuracy and speed, setting a new benchmark for real-time deployment.
Weiyi Xiong, Bing Zhu
School of Automation Science and Electrical Engineering, Beihang University, Beijing, P.R. China