RaMOD: Moving-Object Detection from Raw mmWave Radar Cubes under Ego Motion
Authors: Yuan Fang, Long Zhuang, Lingzi Xing, Xijia Wei, Jingqi Jiang, Fangzhan Shi, Simon Julier, Kevin Chetty
Organizations: University College London, London, United Kingdom. · Southeast University, Nanjing, China, and Nanyang Technological University, Singapore. · University of Liverpool, Liverpool, United Kingdom. · Imperial College London, London, United Kingdom.
Millimeter-wave (mmWave) radar provides weather-robust sensing and direct Doppler measurements, yet automotive perception pipelines often reduce its output to sparse point clouds or two-dimensional projections, leaving raw range-azimuth-Doppler (RAD) cubes underutilized. Ego motion induces Doppler responses from world-static structures, while Doppler wrapping aliases radial velocities, complicating moving-object detection and velocity estimation. We present RaMOD (Radar-based Moving-Object Detection), a real-time framework for moving-object detection and velocity estimation from raw RAD cubes acquired by a vehicle-mounted single-chip radar. RaMOD explicitly estimates frame-wise ego motion and uses the estimated radar translational velocity to predict the Doppler region of world-static structures and suppress returns within it. It temporally encodes the suppressed cubes into multi-scale features to predict RAD-space boxes with class labels. For each detected object, it predicts radial and forward velocities under raw-Doppler and Doppler-unwrapped hypotheses, together with Doppler-wrapping probabilities. On our BRIDGE-Liv dataset, RaMOD outperforms all compared raw-radar methods across RAD-cube, RA, and RD detection settings, achieving 80.8% cube mAP at IoU 0.30, 15.7 percentage points above the strongest RAD-cube baseline. Velocity MAE is 1.128 m/s, with 99.87% Doppler-wrapping accuracy.
Figures & tables
Radar
State
Dataset
Representative Methods
Input
Location
Multi-class
Labels
Radial
Velocity
Wrap/
Unwrap
Object
Velocity
Ego
Motion
TABLE I: Comparison of representative low-level automotive radar perception methods.
Fig. 1: (a) Signal chain of the 2-TX, 4-RX TDM-MIMO FMCW radar. Each received chirp sR(t) is mixed with the transmitted chirp sT(t) to produce the IF signal sIF(t) , which the ADC samples into the raw data cube of ADC samples × chirps × antennas. (b) A 3D FFT converts it into the range–azimuth–Doppler (RAD) cube, on which RaMOD detects directly. Its RA and RD projections are shown only for visualization, revealing four road users at different azimuths with distinct radial velocities and Doppler spreads. (c) The conventional pipeline applies CFAR and direction-of-arrival (DoA) estimation to the same cube, compressing 12.5 M bins into only 262 points (157 static). Object extent and class are hard to recover from such points, and further clustering and tracking are still required.
Fig. 2: Static-consistent Doppler-region masking at two ego speeds: (a) vego≈5 m/s and (b) vego≈28 m/s. The top and bottom rows show the raw and masked RAD cubes with their corresponding RA and RD projections, respectively. The static field forms a single Doppler band in (a) and two disjoint wrapped bands in (b), with masked regions shown in gray.
Fig. 3: Moving-object Doppler wrapping over four consecutive frames ( vego≈14 m/s; RGB, RA, and RD from top to bottom). The oncoming vehicle is wrapped to the positive Doppler side in the first two frames, but returns to the negative side as its increasing azimuth reduces the radial-speed magnitude below vmax .
Fig. 4: Object-level forward-velocity recovery from a single-radar radial measurement. Car 1 lies near the radar boresight, while the off-axis Car 2 requires azimuth-based correction. For Car 3, the actual projection angle α differs from the assumed angle θo3 , resulting in a biased velocity estimate. The tangential component vtan is not directly observable from Doppler.
Fig. 5: Overview of RaMOD. The forward path consists of an Ego-Motion Branch, a Temporal Cube Encoding Branch, and Multi-Scale Radar Heads. The Ego-Motion Branch estimates frame-wise ego motion and suppresses static-consistent Doppler responses. The temporal branch encodes the resulting cube sequence into a multi-scale pyramid, from which the radar heads predict cube boxes, object-level radial and forward velocities, and CVAS outputs including RA/RD boxes and coarse object-support heatmaps.
Fig. 6: Cross-Scale Attentive Residual Fusion (CS-ARF). For each independent target path ( Ci∈{C4,C5} ), Ci supplies a query; {C3,C4,C5} supply keys and spatially aligned values. Attention-weighted aggregation, 3D refinement, and residual scaling produce an update added to Ci to obtain Cˉi .
Fig. 7: Cross-View Auxiliary Supervision. Top: the Cross-View Transformer performs cross-attention between RA and RD features before the RA and RD heads. Bottom: coarse object-support pre-filtering. Without the coarse-support prior, detector candidates accumulate in unsupported regions and overlap around true object responses; the predicted support maps suppress such candidates before final decoding and NMS.
Fig. 8: The synchronized multi-sensor vehicle platform used for collecting the BRIDGE-Liv dataset.
Method
Pred. Space
Params./FLOPs (M/G)
Cube mAP
RA mAP
RD mAP
FPS
IoU=0.1
IoU=0.3
IoU=0.5
IoU=0.1
IoU=0.3
IoU=0.5
IoU=0.1
IoU=0.3
IoU=0.5
RODNet [ 39 ]
RA
7.34/59.91
–
–
–
33.5
31.7
21.1
–
–
–
458.7
T-RODNet [ 40 ]
RA
32.31/409.79
–
–
–
28.3
23.9
12.8
–
–
–
89.7
FFT-RadNet [ 47 ]
RA
2.65/12.79
–
–
–
30.3
26.6
16.0
–
–
–
283.6
T-FFTRadNet [ 49 ]
RA
2.32/5.23
–
–
–
34.5
28.2
14.1
–
–
–
247.5
RadarFormer [ 42 ]
RA
1.95/368.62
–
–
–
29.8
24.7
13.3
–
–
–
15.9
TABLE II: Detection performance and model complexity on BRIDGE-Liv. mAP (%) is reported at IoU 0.10 , 0.30 , and 0.50 . Params./FLOPs are given in millions/billions. FPS denotes model-only FP32 throughput at batch size 1 on an RTX 5090; a dash indicates no prediction in that space.
Method
mAP
Person
Car
Large Veh.
Two-Wheeler
RADDet
57.0
50.4
75.4
72.5
29.7
Mamba-RODNet
65.1
50.4
74.7
78.8
56.5
RaMOD (ours)
80.8
66.9
93.5
97.3
65.6
TABLE III: Per-class cube AP (%) at IoU 0.30 .
Fig. 9: Qualitative RAD cube results with RGB references, annotations, and representative cube-detector predictions. RaMOD preserves spatial and Doppler extents with category and velocity labels.
Fig. 10: Qualitative RA comparison with RGB references, annotations, RA-based methods, and RaMOD. RaMOD maintains consistent projected range–azimuth localization in cluttered and multi-object scenes.
Fig. 11: Qualitative RA/RD comparison with RGB references and detections from multi-view methods and RaMOD.
Fig. 12: Coarse object-support guidance in the RA and RD views. For each scene, the raw projection is followed by the target support and predicted coarse heatmaps. The predicted support highlights target-related regions and provides the pre-filtering prior for view-wise candidate decoding.
Variant
Cube mAP (%)
Ego-State MAE
Velocity MAE (m/s)
Wrap Acc. (%)
RaMOD (stage 1)
80.85
0.129
1.465
99.86
w/o ego supervision
74.16
2.755
1.782
99.83
w/o Cross-View Transformer
75.59
0.115
1.215
99.79
w/o CS-ARF
77.50
0.109
1.335
99.80
w/o RA/RD auxiliary supervision
78.98
0.165
1.546
99.81
w/o coarse support
77.23
0.135
1.376
99.80
TABLE IV: Core component ablation under the matched λcoarse=0.05 , T=3 , and 10-epoch detector protocol. Detection entries are cube mAP (%) at IoU 0.30 . “–” denotes that no velocity output is predicted.
Sensing ego-velocity estimation is fundamental to state estimation in visually degraded environments, where camera- and LiDAR-based pipelines can become unreliable. Millimetre-wave radar is well suited to these conditions because it provides direct Doppler velocity sensing and remains robust to poor illumination, textureless scenes, and airborne particulates. However, conventional radar ego-velocity pipelines typically apply constant false alarm rate (CFAR) thresholding to convert dense radar spectra into sparse point clouds, prematurely discarding sub-threshold returns that may still retain useful Doppler motion cues. We present Dense Soft Weighting, an analytic radar front-end that maps every range-Doppler cell to a continuous confidence metric rather than enforcing a binary detection threshold. Ego-velocity is then estimated using a deterministic robust weighted least-squares formulation, while the same weighted measurements provide a closed-form, measurement-derived velocity covariance for integration with a shared inertial back-end. The method requires no platform-specific training data or learning-based uncertainty model, supporting transfer across single-chip radar configurations. Across two public datasets and one self-collected dataset, Dense Soft Weighting reduces mean absolute pose error by 31-45% relative to the strongest CFAR point-cloud baseline under an identical inertial back-end, while running in real time on embedded hardware.
Dynamic object segmentation and ego-motion estimation are closely coupled problems in autonomous driving, as accurate ego-motion estimation typically requires static scene observations, while identifying static observations requires knowledge of the ego motion. We present Radar-Dot, a radar--RGB framework that exploits radar Doppler measurements to address this coupling. Radar returns are first used to estimate ego velocity through a linear Doppler constraint, with residual-based static/dynamic segmentation and robust estimation used to reduce the influence of moving objects. The estimated motion is then combined with metric depth and dense optical flow to identify image regions whose observed motion is inconsistent with the rigid scene motion. Experiments on 10 nuScenes scenes (part of nuscenes-mini) demonstrate that the resulting geometric pipeline achieves 20.24% dynamic IoU and 33.67% F1-score over 394 frame pairs, while radar-based static-point filtering improves ego-velocity estimation compared with using all radar returns. These results demonstrate the potential of radar as a modality for jointly improving ego-motion estimation and dynamic object segmentation.
Automotive perception systems are obligated to meet high requirements. While optical sensors such as Camera and Lidar struggle in adverse weather conditions, Radar provides a more robust perception performance, effectively penetrating fog, rain, and snow. Since full Radar tensors have large data sizes and very few datasets provide them, most Radar-based approaches work with sparse point clouds or 2D projections, which can result in information loss. Additionally, deep learning methods show potential to extract richer and more dense features from low level Radar data and therefore significantly increase the perception performance. Therefore, we propose a 3D projection method for fast-Fourier-transformed 4D Range-Azimuth-Doppler-Elevation (RADE) tensors. Our method preserves rich Doppler and Elevation features while reducing the required data size for a single frame by 91.9% compared to a full tensor, thus achieving higher training and inference speed as well as lower model complexity. We introduce RADE-Net, a lightweight model tailored to 3D projections of the RADE tensor. The backbone enables exploitation of low-level and high-level cues of Radar tensors with spatial and channel-attention. The decoupled detection heads predict object center-points directly in the Range-Azimuth domain and regress rotated 3D bounding boxes from rich feature maps in the cartesian scene. We evaluate the model on scenes with multiple different road users and under various weather conditions on the large-scale K-Radar dataset and achieve a 16.7% improvement compared to their baseline, as well as 6.5% improvement over current Radar-only models. Additionally, we outperform several Lidar approaches in scenarios with adverse weather conditions. The code is available under https://github.com/chr-is-tof/RADE-Net.
Christof Leitgeb, Thomas Puchleitner, Max Peter Ronecker +1
Infineon Technologies AG, Institute of Visual Computing, Graz University of Technology, Graz, Austria · Infineon Technologies AG, Graz, Austria · Virtual Vehicle Research GmbH, Institute of Visual Computing, Graz University of Technology, Graz, Austria +1