Towards Multi-Object-Tracking with Radar on a Fast Moving Vehicle: On the Potential of Processing Radar in the Frequency Domain
Authors: Tim Hansen, Arturo Gomez-Chavez, Ilya Shimchik, Andreas Birk
Organizations: School of Computer Science and Engineering at Constructor University Bremen, Germany. · Constructor Knowledge Labs (CKL), Bremen, Germany.
We promote in this paper the processing of radar data in the frequency domain to achieve higher robustness against noise and structural errors, especially in comparison to feature-based methods. This holds also for high dynamics in the scene, i.e., ego-motion of the vehicle with the sensor plus the presence of an unknown number of other moving objects. In addition to the high robustness, the processing in the frequency domain has the so far neglected advantage that the underlying correlation based methods used for, e.g., registration, provide information about all moving structures in the scene. A typical automotive application case is overtaking maneuvers, which in the context of autonomous racing are used here as a motivating example. Initial experiments and results with Fourier SOFT in 2D (FS2D) are presented that use the Boreas dataset to demonstrate radar-only-odometry, i.e., radar-odometry without sensor-fusion, to support our arguments.
Figures & tables
Fig. 1 : Registration in the frequency domain, here of sonar data with FS2D, can not only deal very well with noisy data. There is also the so far neglected advantage that the underlying processing with correlation methods leads to peaks that indicate the transformation parameters for multiple moving structures in the data in parallel.
Fig. 2 : The Constructor University (CU) vehicle used for high-velocity autonomous driving.
Fig. 3 : The A2RL races feature fully autonomous driving at high velocities on the Yas Marina Formula-1 race-track in Abu Dhabi.
Fig. 4 : Autonomous overtaking includes the perception challenge of Multi-Object-Tracking (MOT). For planning and executing the maneuver, the vehicle A itself, the car B that is to be overtaken, as well as an unknown number of vehicles Ci in the vicinity ( i≥0 ) have to be tracked.
Fig. 5 : The Glen Shield route from the Boreas dataset (image from the Boreas website https://www.boreas.utias.utoronto.ca/ using OpenStreetMap).
Fig. 6 : The estimated trajectories (blue) and the ground truth (red) for the first two datasets as illustrative examples. Note that the estimated trajectories are purely based on pairwise registration of radar scans without any other localization or motion estimation, and that they include all outliers.
Fig. 7 : Three examples of the FS2D registration. On the top, an overlay of two scans in red and blue is shown using their ground truth poses. The actual motion is hence represented in the translation and rotation of the blue scan relative to the red one. Below, the same scan pairs are projected according to the motion estimates from FS2D, which are very close to the ground truth.
Dynamic object segmentation and ego-motion estimation are closely coupled problems in autonomous driving, as accurate ego-motion estimation typically requires static scene observations, while identifying static observations requires knowledge of the ego motion. We present Radar-Dot, a radar--RGB framework that exploits radar Doppler measurements to address this coupling. Radar returns are first used to estimate ego velocity through a linear Doppler constraint, with residual-based static/dynamic segmentation and robust estimation used to reduce the influence of moving objects. The estimated motion is then combined with metric depth and dense optical flow to identify image regions whose observed motion is inconsistent with the rigid scene motion. Experiments on 10 nuScenes scenes (part of nuscenes-mini) demonstrate that the resulting geometric pipeline achieves 20.24% dynamic IoU and 33.67% F1-score over 394 frame pairs, while radar-based static-point filtering improves ego-velocity estimation compared with using all radar returns. These results demonstrate the potential of radar as a modality for jointly improving ego-motion estimation and dynamic object segmentation.
Recent advances in 4D radar enable robust perception in adverse weather; however, the inherent sparsity, noise, and limited positional precision of radar point clouds pose significant challenges for registration-based odometry. In this letter, we propose RaDiVe, a 4D radar odometry framework designed to improve the accuracy and robustness of radar point-cloud registration. We introduce a distance-bounded Normal Distributions Transform (NDT), which improves optimization stability and computational efficiency by restricting the correspondence search to near-distance voxel pairs. To mitigate measurement ambiguity, we propose a velocity-discrepancy point uncertainty model that weights each input 4D radar point according to the discrepancy between its measured Doppler radial velocity and the radial velocity predicted from the estimated ego-velocity. Furthermore, we incorporate Signed Distance Function (SDF)-based surface point extraction via implicit neural mapping to construct a geometrically consistent and noise-filtered local submap. Evaluations across multiple public datasets demonstrate that RaDiVe outperforms existing 4D radar odometry baselines by 44.4% in translational Absolute Trajectory Error (ATE) and 21.3% in rotational ATE on average, while maintaining real-time performance. The source code will be made publicly available to the robotics community: https://github.com/to-be-open-sourced.
Sangwoo Jung, Dongjae Lee, Chiyun Noh +1
Department of Mechanical Engineering, SNU, Seoul, S. Korea
The challenge of 3D multi-object tracking is achieving robustness in real-world applications, for example under adverse conditions and maintaining consistency as distance increases. To overcome these challenges, sensor fusion approaches that combine LiDAR, cameras, and radar have emerged. However, existing multimodal methods usually treat radar as another learned feature inside the network. When the overall model degrades in difficult environments, the robustness advantages that radar could provide are also reduced. In this paper we propose RadarMOT, a radar-informed 3D multi-object tracking framework that explicitly uses radar point clouds as additional observations to refine state estimation and recover objects missed by the detector at long ranges. Evaluations on the MAN-TruckScenes dataset show that RadarMOT consistently improves the Average Multi-Object Tracking Accuracy (AMOTA) by 12.7% at long range and up to 10.3% in adverse weather. The code will be available at https://github.com/bingxue-xu/radarmot
Bingxue Xu, Emil Hedemalm, Ajinkya Khoche +1
KTH Royal Institute of Technology, Stockholm 10044, Sweden.