mmWave Radar Sensing

mmWave: Millimeter Wave

Ask about this topic

Overview

Millimeter-wave radar sensing studies how to recover useful information about people and environments from reflected radio signals, including motion and distance measurements that can work without identifiable images and in poor visibility. Methods process raw I/Q signals, range-Doppler tensors, or sparse point clouds using signal processing, neural networks, and radar-camera or radar-LiDAR fusion. Applications include 4D automotive detection and mapping, human pose and activity recognition, vital-sign monitoring, and robot localization; evaluation uses driving, indoor, and health datasets, often across people, environments, and weather conditions. A central challenge is making systems reliable under sparse measurements, multipath, sensor misalignment, and distribution shifts, while balancing learned representations against physical modeling and the cost of collecting diverse real-world data.

Latest papers

CardsList

85 papers12 papers in the last four weeks, up 33% on the four weeks before. 0.1% of all new papers.

Apr 19, 2026

DGHMesh: A Large-scale Dual-radar mmWave Dataset and Generalization-focused Benchmark for Human Mesh Reconstruction

Millimeter-wave (mmWave) radar has shown great potential for contactless, privacy-preserving, and robust human sensing, yet existing mmWave-based human mesh reconstruction (HMR) studies are still limited by the lack of benchmarks for generalization analysis under configuration shifts and fair comparison of different algorithms. To address the limitation, we present DGHMesh, a large-scale dual-radar mmWave dataset and generalization-focused benchmark for HMR. It contains data from 15 subjects performing 8 actions, with 360,000 synchronized frames collected from FMCW radar, SFCW radar, RGB images, and high-precision 3D HMR annotations. In addition, the dataset provides synchronized raw I/Q data from both radar modalities and accurately calibrated radar spatial positions. The benchmark is designed to evaluate HMR methods under diverse measurement configurations, including human position shifts, human orientation shifts, subarray size variations, and cross-subject settings. Based on DGHMesh, we also propose mmPTM, a query-based multi-radar fusion framework that jointly exploits point clouds and imaging tubes for HMR. Extensive experiments are conducted against representative baselines under different settings. The results demonstrate that mmPTM consistently achieves outstanding accuracy and competitive generalization capability across multiple sub-benchmarks, validating the effectiveness of multi-radar fusion and the practical value of the proposed dataset and benchmark for mmWave-based HMR research. DGHMesh and mmPTM are publicly available at https://github.com/SPIresearch/DGHMesh.(The complete benchmark and code will be released after paper publication)

Rongxiao Guo, Qingchao Chen

Institute of Medical Technology, Peking University Health Science Center · National Institute of Health Data Science at Peking University · State Key Laboratory of General Artificial Intelligence, Peking University

3D ReconstructionHuman Mesh RecoveryMillimeter-Wave RadarmmWave Radar SensingMultimodal Sensor Fusion
AskPDF
Apr 16, 2026

4D Radar Gaussian Modeling and Scan Matching with RCS

4D millimeter-wave (mmWave) radars are increasingly used in robotics, as they offer robustness against adverse environmental conditions. Besides the usual XYZ position, they provide Doppler velocity measurements as well as Radar Cross Section (RCS) information for every point. While Doppler is widely used to filter out dynamic points, RCS is often overlooked and not usually used in modeling and scan matching processes. Building on previous 3D Gaussian modeling and scan matching work, we propose incorporating the physical behavior of RCS in the model, in order to further enrich the summarized information about the scene, and improve the scan matching process.

Fernando Amodeo, Luis Merino, Fernando Caballero

Service Robotics Laboratory – Universidad Pablo de Olavide (Seville), Spain

Millimeter-Wave RadarmmWave Radar SensingRobot Localization
AskPDF
Apr 15, 2026

UNRIO: Uncertainty-Aware Velocity Learning for Radar-Inertial Odometry

mmWave radars are robust to darkness and occlusions such as dust and smoke, and can directly constrain ego-velocity from a single frame via Doppler measurements, making them attractive sensors for odometry in visually denied conditions. However, almost all existing radar-inertial odometry systems rely on lossy radar point clouds that are highly sparse, generally concentrated in a narrow angular band, and aliased at high speeds. We propose to instead estimate ego-velocity directly from unfiltered mmWave I/Q signals. Taking advantage of a foundation model for 4D radar spectrum, we develop a system that integrates uncertainty-aware velocity predictions with IMU measurements using an uncertainty-weighted sliding-window pose graph to accurately compute odometry even when provided radar data with aliasing or an unfavorable field of view. Evaluated on public benchmark datasets, our system, UNRIO, attains the lowest relative pose error on the majority of sequences across held-out environments, despite differing chirp configurations, motion patterns, and platforms with its strongest gains coming where point clouds fail most: lateral motion in IQ1M and heavily aliased Doppler in ColoRadar.

Jui-Te Huang, Tianshu Huang, Anthony Rowe +1

Carnegie Mellon University, Pittsburgh, PA 15213, USA

Radar-Inertial OdometryRobot State EstimationmmWave Radar SensingUncertainty-Aware Multimodal Fusion
AskPDF
Apr 13, 2026

VLMaterial: Vision-Language Model-Based Camera-Radar Fusion for Physics-Grounded Material Identification

Accurate material recognition is a fundamental capability for intelligent perception systems to interact safely and effectively with the physical world. For instance, distinguishing visually similar objects like glass and plastic cups is critical for safety but challenging for vision-based methods due to specular reflections, transparency, and visual deception. While millimeter-wave (mmWave) radar offers robust material sensing regardless of lighting, existing camera-radar fusion methods are limited to closed-set categories and lack semantic interpretability. In this paper, we introduce VLMaterial, a training-free framework that fuses vision-language models (VLMs) with domain-specific radar knowledge for physics-grounded material identification. First, we propose a dual-pipeline architecture: an optical pipeline uses the segment anything model and VLM for material candidate proposals, while an electromagnetic characterization pipeline extracts the intrinsic dielectric constant from radar signals via an effective peak reflection cell area (PRCA) method and weighted vector synthesis. Second, we employ a context-augmented generation (CAG) strategy to equip the VLM with radar-specific physical knowledge, enabling it to interpret electromagnetic parameters as stable references. Third, an adaptive fusion mechanism is introduced to intelligently integrate outputs from both sensors by resolving cross-modal conflicts based on uncertainty estimation. We evaluated VLMaterial in over 120 real-world experiments involving 41 diverse everyday objects and 4 typical visually deceptive counterfeits across varying environments. Experimental results demonstrate that VLMaterial achieves a recognition accuracy of 96.08%, delivering performance on par with state-of-the-art closed-set benchmarks while eliminating the need for extensive task-specific data collection and training.

Jiangyou Zhu, He Chen

Department of Inforation Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China

Camera-Radar FusionMaterials Property PredictionmmWave Radar SensingUncertainty-Aware Multimodal Fusion
AskPDF
Apr 7, 2026

The Breakthrough of Sleep: A Contactless Approach for Accurate Sleep Stage Detection Using the Sleepal AI Lamp

Sleep staging is essential for the assessment of sleep quality and the diagnosis of sleep-related disorders. Conventional polysomnography (PSG), while considered the gold standard, is intrusive, labor-intensive, and unsuitable for long-term monitoring. This study evaluates the performance of the Sleepal AI Lamp, a contactless, radar-based consumer-grade sleep tracker, in comparison with gold-standard polysomnography (PSG), using a large-scale dataset comprising 1022 overnight recordings. We extract multi-scale respiratory and motion-related features from radar signals to train a frequency-augmented deep learning model. For the binary sleep-wake classification task, experimental results demonstrated that the model achieved an accuracy of 92.8% alongside a macro-averaged F1 score of 0.895. For four-stage classification (wake, light NREM (N1 + N2), deep NREM (N3), REM), the model achieved an accuracy of 78.5% with a Cohen's kappa coefficient of 0.695 in healthy individuals and maintained a stable accuracy of 77.2% with a kappa of 0.677 in a heterogeneous population including patients with varying severities of obstructive sleep apnea (OSA). These experimental results demonstrate that the sleep staging performance of the contactless Sleepal AI Lamp is in high agreement with expert-labeled PSG sleep stages. Our findings suggest that non-contact radar sensing, combined with advanced temporal modeling, can provide reliable sleep staging performance without requiring physical contact or wearable devices. Owing to its unobtrusive nature, ease of deployment, and robustness to long-term use, the contactless Sleepal AI Lamp shows strong potential for clinical screening, home-based sleep assessment, and continuous longitudinal sleep monitoring in real-world medical and healthcare applications.

Zhuo Diao, Yueting Li, Jianpeng Wang +9

Hong Kong Xsmart Century Technology Co., Ltd., Hong Kong, China · Urban Mobility Lab, Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Singapore-MIT Alliance for Research and Technology (SMART) Centre, Singapore 138602, Singapore +2

Sleep Stage ClassificationmmWave Radar Sensing
AskPDF
Apr 6, 2026

RAVEN: Radar Adaptive Vision Encoders for Efficient Chirp-wise Object Detection and Segmentation

This paper presents RAVEN, a computationally efficient deep learning architecture for FMCW radar perception. The method processes raw ADC data in a chirp-wise streaming manner, preserves MIMO structure through independent receiver state-space encoders, and uses a learnable cross-antenna mixing module to recover compact virtual-array features. It also introduces an early-exit mechanism so the model can make decisions using only a subset of chirps when the latent state has stabilized. Across automotive radar benchmarks, the approach reports strong object detection and BEV free-space segmentation performance while substantially reducing computation and end-to-end latency compared with conventional frame-based radar pipelines.

Anuvab Sen, Mir Sayeed Mohammad, Saibal Mukhopadhyay

Georgia Institute of Technology, Atlanta, Georgia, USA

mmWave Radar SensingEfficient Object DetectionObject Detection
AskPDF
Apr 3, 2026

Rascene: High-Fidelity 3D Scene Imaging with mmWave Communication Signals

Robust 3D environmental perception is critical for applications such as autonomous driving and robot navigation. However, optical sensors such as cameras and LiDAR often fail under adverse conditions, including smoke, fog, and non-ideal lighting. Although specialized radar systems can operate in these environments, their reliance on bespoke hardware and licensed spectrum limits scalability and cost-effectiveness. This paper introduces Rascene, an integrated sensing and communication (ISAC) framework that leverages ubiquitous mmWave OFDM communication signals for 3D scene imaging. To overcome the sparse and multipath-ambiguous nature of individual radio frames, Rascene performs multi-frame, spatially adaptive fusion with confidence-weighted forward projection, enabling the recovery of geometric consensus across arbitrary poses. Experimental results demonstrate that our method reconstructs 3D scenes with high precision, offering a new pathway toward low-cost, scalable, and robust 3D perception.

Kunzhe Song, Geo Jie Zhou, Xiaoming Liu +1

Department of Computer Science and Engineering, Michigan State University

3D ReconstructionmmWave Radar SensingIntegrated Sensing and Communication
AskPDF
Apr 2, 2026

On Learning Spatial Structure from Pre-Beamforming Per-Antenna Range-Doppler Radar Measurements

Automotive radar perception pipelines commonly construct angle-domain representations via beamforming before applying learning-based models. This work instead investigates a representational question: can meaningful spatial structure be learned directly from pre-beamforming per-antenna range-Doppler (RD) measurements? Experiments are conducted on a 6-TX x 8-RX (48 virtual antennas) commodity automotive radar employing an A/B chirp-sequence frequency-modulated continuous-wave (CS-FMCW) transmit scheme, in which the effective transmit aperture varies between chirps (single-TX vs multi-TX), enabling controlled analyses of chirp-dependent transmit configurations. We operate on pre-beamforming per-antenna RD tensors using a dual-chirp shared-weight encoder trained in an end-to-end, fully data-driven manner, and evaluate spatial recoverability using bird's-eye-view (BEV) occupancy as a geometric probe rather than a performance-driven objective. Supervision is visibility-aware and cross-modal, derived from LiDAR with explicit modeling of the radar field-of-view and occlusion-aware LiDAR observability via ray-based visibility. Through analyses of signal properties, transmit configurations (A-only, B-only, and A+B), receive aperture, and range-Doppler structure, together with physics-aligned baselines, we investigate the factors influencing spatial recoverability. The results indicate that meaningful spatial structure is recoverable from pre-beamforming per-antenna RD tensors under the studied A/B CS-FMCW radar configuration through learned spatial mixing, without relying on hand-crafted signal-processing stages.

George Sebastian, Philipp Berthold, Bianca Forkel +2

Institute for Autonomous Driving, Department of Aerospace Engineering, University of the Bundeswehr Munich, Neubiberg, Germany

Representation LearningMillimeter-Wave RadarmmWave Radar SensingAutonomous Driving Perception
AskPDF
Apr 1, 2026

mmAnomaly: Leveraging Visual Context for Robust Anomaly Detection in the Non-Visual World with mmWave Radar

mmWave radar enables human sensing in non-visual scenarios-e.g., through clothing or certain types of walls-where traditional cameras fail due to occlusion or privacy limitations. However, robust anomaly detection with mmWave remains challenging, as signal reflections are influenced by material properties, clutter, and multipath interference, producing complex, non-Gaussian distortions. Existing methods lack contextual awareness and misclassify benign signal variations as anomalies. We present mmAnomaly, a multi-modal anomaly detection framework that combines mmWave radar with RGBD input to incorporate visual context. Our system extracts semantic cues-such as scene geometry and material properties-using a fast ResNet-based classifier, and uses a conditional latent diffusion model to synthesize the expected mmWave spectrum for the given visual context. A dual-input comparison module then identifies spatial deviations between real and generated spectra to localize anomalies. We evaluate mmAnomaly on two multi-modal datasets across three applications: concealed weapon localization, through-wall intruder localization, and through-wall fall localization. The system achieves up to 94% F1 score and sub-meter localization error, demonstrating robust generalization across clothing, occlusions, and cluttered environments. These results establish mmAnomaly as an accurate and interpretable framework for context-aware anomaly detection in mmWave sensing.

Tarik Reza Toha, Shao-Jung, Lu +2

Louie · University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA

Anomaly DetectionAnomaly LocalizationMultimodal Anomaly DetectionmmWave Radar SensingCamera-Radar Fusion
AskPDF
Mar 24, 2026

Edge Radar Material Classification Under Geometry Shifts

Material awareness can improve robotic navigation and interaction, particularly in conditions where cameras and LiDAR degrade. We present a lightweight mmWave radar material classification pipeline designed for ultra-low-power edge devices (TI IWRL6432), using compact range-bin intensity descriptors and a Multilayer Perceptron (MLP) for real-time inference. While the classifier reaches a macro-F1 of 94.2% under the nominal training geometry, we observe a pronounced performance drop under realistic geometry shifts, including sensor height changes and small tilt angles. These perturbations induce systematic intensity scaling and angle-dependent radar cross section (RCS) effects, pushing features out of distribution and reducing macro-F1 to around 68.5%. We analyze these failure modes and outline practical directions for improving robustness with normalization, geometry augmentation, and motion-aware features.

Jannik Hohmann, Dong Wang, Andreas Nüchter

Department of Informatics XVII (Robotics), Julius-Maximilians-University Würzburg, 97074 Würzburg, Germany. · Zentrum für Telematik e.V., Würzburg · U2IS, ENSTA, Institut Polytechnique de Paris, France.

mmWave Radar SensingDistribution ShiftRobotic PerceptionEdge Inference
AskPDF
Mar 23, 2026

mmFHE: mmWave Sensing with End-to-End Fully Homomorphic Encryption

We present mmFHE, the first system that executes the entire cloud-side mmWave sensing pipeline including the DSP and ML inference under fully homomorphic encryption (FHE). mmFHE encrypts range profiles on an edge device after lightweight plaintext preprocessing and executes the entire mmWave signal-processing and ML inference pipeline homomorphically on a semi-honest cloud that operates exclusively on ciphertexts. At the core of mmFHE is a library of seven composable, data-oblivious FHE kernels that replace standard DSP routines with fixed arithmetic circuits for different application-specific pipelines. We demonstrate this approach on two representative tasks: vital-sign monitoring and gesture recognition. We formally prove two cryptographic guarantees for any pipeline assembled from this library: input privacy and data obliviousness. These guarantees effectively neutralize various supervised and unsupervised privacy attacks on raw data, including re-identification and data-dependent privacy leakage. Evaluation on three public radar datasets shows that encryption introduces negligible error versus the plaintext pipeline, with 84.5% gesture accuracy (vs. 84.7%). End-to-end cloud GPU latency is 1.21 s per 10 s vital-sign window and 5.76 s per 3 s gesture window. These results establish the initial feasibility of end-to-end mmWave sensing under FHE on commodity hardware.

Tanvir Ahmed, Yixuan Gao, Adnan Armouti +1

Cornell Tech

mmWave Radar SensingRemote SensingGesture RecognitionHomomorphic EncryptionPrivacy-Preserving ML
AskPDF
Mar 21, 2026

mmWave-Diffusion:A Novel Framework for Respiration Sensing Using Observation-Anchored Conditional Diffusion Model

Millimeter-wave (mmWave) radar enables contactless respiratory sensing,yet fine-grained monitoring is often degraded by nonstationary interference from body micromotions.To achieve micromotion interference removal,we propose mmWave-Diffusion,an observation-anchored conditional diffusion framework that directly models the residual between radar phase observations and the respiratory ground truth,and initializes sampling within an observation-consistent neighborhood rather than from Gaussian noise-thereby aligning the generative process with the measurement physics and reducing inference overhead. The accompanying Radar Diffusion Transformer (RDT) is explicitly conditioned on phase observations, enforces strict one-to-one temporal alignment via patch-level dual positional encodings, and injects local physical priors through banded-mask multi-head cross-attention, enabling robust denoising and interference removal in just 20 reverse steps. Evaluated on 13.25 hours of synchronized radar-respiration data, mmWave-Diffusion achieves state-of-the-art waveform reconstruction and respiratory-rate estimation with strong generalization. Code repository:https://github.com/goodluckyongw/mmWave-Diffusion.

Yong Wang, Qifan Shen, Bao Zhang +4

Key Laboratory of Underwater Acoustic Signal Processing of Ministry of Education, Southeast University, Nanjing, 211189, China · Purple Mountain Laboratories, Nanjing, 211111, China

mmWave Radar SensingConditional Diffusion ModelsDiffusion Transformer
AskPDF
Mar 17, 2026

Edge-Efficient Two-Stream Multimodal Architecture for Non-Intrusive Bathroom Fall Detection

Falls in wet bathroom environments are a major safety risk for seniors living alone. Recent work has shown that mmWave-only, vibration-only, and existing multimodal schemes, such as vibration-triggered radar activation, early feature concatenation, and decision-level score fusion, can support privacy-preserving, non-intrusive fall detection. However, these designs still treat motion and impact as loosely coupled streams, depending on coarse temporal alignment and amplitude thresholds, and do not explicitly encode the causal link between radar-observed collapse and floor impact or address timing drift, object drop confounders, and latency and energy constraints on low-power edge devices. To this end, we propose a two-stream architecture that encodes radar signals with a Motion--Mamba branch for long-range motion patterns and processes floor vibration with an Impact--Griffin branch that emphasizes impact transients and cross-axis coupling. Cross-conditioned fusion uses low-rank bilinear interaction and a Switch--MoE head to align motion and impact tokens and suppress object-drop confounders. The model keeps inference cost suitable for real-time execution on a Raspberry Pi 4B gateway. We construct a bathroom fall detection benchmark dataset with frame-level annotations, comprising more than 3~h of synchronized mmWave radar and triaxial vibration recordings across eight scenarios under running water, together with subject-independent training, validation, and test splits. On the test split, our model attains 96.1% accuracy, 94.8% precision, 88.0% recall, a 91.1% macro F1 score, and an AUC of 0.968. Compared with the strongest baseline, it improves accuracy by 2.0 percentage points and fall recall by 1.3 percentage points, while reducing latency from 35.9 ms to 15.8 ms and lowering energy per 2.56 s window from 14200 mJ to 10750 mJ on the Raspberry Pi 4B gateway.

Haitian Wang, Yiren Wang, Xinyu Wang +2

The University of Western Australia, 35 Stirling Highway, Crawley, WA 6009, Australia

Fall DetectionMultimodal Sensor FusionHuman Activity RecognitionEdge InferencemmWave Radar Sensing
AskPDF
Mar 15, 2026

Expanding mmWave Datasets for Human Pose Estimation with Unlabeled Data and LiDAR Datasets

Current millimeter-wave (mmWave) datasets for human pose estimation (HPE) are scarce and lack diversity in both point cloud (PC) attributes and human poses, hindering the generalization ability of their trained models. On the other hand, unlabeled mmWave HPE data and diverse LiDAR HPE datasets are readily available. We propose EMDUL, a novel approach to expand the volume and diversity of an existing mmWave dataset using unlabeled mmWave data and LiDAR datasets. EMDUL consists of two independent modules, namely a pseudo-label estimator to annotate unlabeled mmWave data, and a closed-form converter that translates an annotated LiDAR PC to its mmWave counterpart. Expanding the original dataset with both LiDAR-converted and pseudo-labeled mmWave PCs significantly boosts the performance and generalization ability of all the examined HPE models, reducing 15.1% and 18.9% error for in-domain and out-of-domain settings, respectively. Code is available at https://github.com/Shimmer93/EMDUL.

Zhuoxuan Peng, Boan Zhu, Xingjian Zhang +2

The Hong Kong University of Science and Technology

Human Pose EstimationData AugmentationmmWave Radar Sensing
AskPDF
Mar 9, 2026

mmGAT: Pose Estimation by Graph Attention with Mutual Features from mmWave Radar Point Cloud

Pose estimation and human action recognition (HAR) are pivotal technologies spanning various domains. While the image-based pose estimation and HAR are widely admired for their superior performance, they lack in privacy protection and suboptimal performance in low-light and dark environments. This paper exploits the capabilities of millimeter-wave (mmWave) radar technology for human pose estimation by processing radar data with Graph Neural Network (GNN) architecture, coupled with the attention mechanism. Our goal is to capture the finer details of the radar point cloud to improve the pose estimation performance. To this end, we present a unique feature extraction technique that exploits the full potential of the GNN processing method for pose estimation. Our model mmGAT demonstrates remarkable performance on two publicly available benchmark mmWave datasets and establishes new state of the art results in most scenarios in terms of human pose estimation. Our approach achieves a noteworthy reduction of pose estimation mean per joint position error (MPJPE) by 35.6% and PA-MPJPE by 14.1% from the current state of the art benchmark within this domain.

Abdullah Al Masud, Shi Xintong, Mondher Bouazizi +1

Graduate School of Science and Technology, Keio University, Kanagawa, 223-8522, Japan · Faculty of Science and Technology, Keio University, Kanagawa, 223-8522, Japan

Human Pose Estimation3D Human Pose EstimationmmWave Radar SensingGraph Attention NetworksPoint Cloud Learning
AskPDF
Mar 9, 2026

PCFEx: Point Cloud Feature Extraction for Graph Neural Networks

Graph neural networks (GNNs) have gained significant attention for their effectiveness across various domains. This study focuses on applying GNN to process 3D point cloud data for human pose estimation (HPE) and human activity recognition (HAR). We propose novel point cloud feature extraction (PCFEx) techniques to capture meaningful information at the point, edge, and graph levels of the point cloud by considering point cloud as a graph. Moreover, we introduce a GNN architecture designed to efficiently process these features. Our approach is evaluated on four most popular publicly available millimeter wave radar datasets, three for HPE and one for HAR. The results show substantial improvements, with significantly reduced errors in all three HPE benchmarks, and an overall accuracy of 98.8% in mmWave-based HAR, outperforming the existing state of the art models. This work demonstrates the great potential of feature extraction incorporated with GNN modeling approach to enhance the precision of point cloud processing.

Abdullah Al Masud, Shi Xintong, Mondher Bouazizi +1

Graduate School of Science and Technology, Keio University, Kanagawa, 223-8522, Japan · Faculty of Science and Technology, Keio University Kanagawa, 223-8522, Japan

Point Cloud Learning3D Human Pose EstimationHuman Activity RecognitionGraph Neural NetworksmmWave Radar Sensing
AskPDF
Mar 3, 2026

calibfusion: Transformer-Based Differentiable Calibration for Radar-Camera Fusion Detection in Water-Surface Environments

Millimeter-wave (mmWave) Radar--Camera fusion improves perception under adverse illumination and weather, but its performance is sensitive to Radar--Camera extrinsic calibration: residual misalignment biases Radar-to-image projection and degrades cross-modal aggregation for downstream 2D detection. Existing calibration and auto-calibration methods are mainly developed for road and urban scenes with abundant structures and object constraints, whereas water-surface environments feature large textureless regions, sparse and intermittent targets, and wave-/specular-induced Radar clutter, which weakens explicit object-centric matching. We propose CalibFusion, a calibration-conditioned Radar--Camera fusion detector that learns implicit extrinsic refinement end-to-end with the detection objective. CalibFusion builds a multi-frame persistence-aware Radar density representation with intensity weighting and Doppler-guided suppression of fast-varying clutter. A cross-modal transformer interaction module predicts a confidence-gated refinement of the initial extrinsics, which is integrated through a differentiable projection-and-splatting operator to generate calibration-conditioned image-plane Radar features. Experiments on WaterScenes and FLOW show improved fusion-based 2D detection and robustness under synthetic miscalibration, supported by sensitivity analyses and qualitative Radar-to-image overlays. Results on nuScenes indicate that the refinement mechanism transfers beyond water-surface scenarios.

Yuting Wan, Liguo Sun, Jiuwu Hao +1

Camera-Radar FusionObject DetectionCross-Modal LearningmmWave Radar Sensing
AskPDF
Mar 1, 2026

RaUF: Learning the Spatial Uncertainty Field of Radar

Millimeter-wave radar offers unique advantages in adverse weather but suffers from low spatial fidelity, severe azimuth ambiguity, and clutter-induced spurious returns. Existing methods mainly focus on improving spatial perception effectiveness via coarse-to-fine cross-modal supervision, yet often overlook the ambiguous feature-to-label mapping, which may lead to ill-posed geometric inference and pose fundamental challenges to downstream perception tasks. In this work, we propose RaUF, a spatial uncertainty field learning framework that models radar measurements through their physically grounded anisotropic properties. To resolve conflicting feature-to-label mapping, we design an anisotropic probabilistic model that learns fine-grained uncertainty. To further enhance reliability, we propose a Bidirectional Domain Attention mechanism that exploits the mutual complementarity between spatial structure and Doppler consistency, effectively suppressing spurious or multipath-induced reflections. Extensive experiments on public benchmarks and real-world datasets demonstrate that RaUF delivers highly reliable spatial detections with well-calibrated uncertainty. Moreover, downstream case studies further validate the enhanced reliability and scalability of RaUF under challenging real-world driving scenarios.

Shengpeng Wang, Kuangyu Wang, Wei Wang

Huazhong University of Science and Technology · Wuhan University

mmWave Radar SensingAutonomous Driving PerceptionUncertainty QuantificationUncertainty CalibrationObject Detection
AskPDF
Feb 25, 2026

mmWave Radar Aware Dual-Conditioned GAN for Speech Reconstruction of Signals With Low SNR

Millimeter-wave (mmWave) radar captures are band-limited and noisy, making for difficult reconstruction of intelligible full-bandwidth speech. In this work, we propose a two-stage speech reconstruction pipeline for mmWave using a Radar-Aware Dual-conditioned Generative Adversarial Network (RAD-GAN), which is capable of performing bandwidth extension on signals with low signal-to-noise ratios (-5 dB to -1 dB), captured through glass walls. We propose an mmWave-tailored Multi-Mel Discriminator (MMD) and a Residual Fusion Gate (RFG) to enhance the generator input to process multiple conditioning channels. The proposed two-stage pipeline involves pretraining the model on synthetically clipped clean speech and finetuning on fused mel spectrograms generated by the RFG. We empirically show that the proposed method, trained on a limited dataset, with no pre-trained modules, and no data augmentations, outperformed state-of-the-art approaches for this specific task. Audio examples of RAD-GAN are available online at https://rad-gan-demo-site.vercel.app/.

Jash Karani, Adithya Chittem, Deepan Roy +1

Department of Electrical and Electronics Engineering, Birla Institute of Technology and Science, Pilani, Rajasthan 333031, India · Department of Computer Science and Information Systems, Birla Institute of Technology and Science, Pilani, Rajasthan 333031, India

Generative Adversarial NetworksSpeech EnhancementmmWave Radar Sensing
AskPDF
Feb 24, 2026

Boosting Instance Awareness via Cross-View Correlation with 4D Radar and Camera for 3D Object Detection

4D millimeter-wave radar has emerged as a promising sensing modality for autonomous driving due to its robustness and affordability. However, its sparse and weak geometric cues make reliable instance activation difficult, limiting the effectiveness of existing radar-camera fusion paradigms. BEV-level fusion offers global scene understanding but suffers from weak instance focus, while perspective-level fusion captures instance details but lacks holistic context. To address these limitations, we propose SIFormer, a scene-instance aware transformer for 3D object detection using 4D radar and camera. SIFormer first suppresses background noise during view transformation through segmentation- and depth-guided localization. It then introduces a cross-view activation mechanism that injects 2D instance cues into BEV space, enabling reliable instance awareness under weak radar geometry. Finally, a transformer-based fusion module aggregates complementary image semantics and radar geometry for robust perception. As a result, with the aim of enhancing instance awareness, SIFormer bridges the gap between the two paradigms, combining their complementary strengths to address inherent sparse nature of radar and improve detection accuracy. Experiments demonstrate that SIFormer achieves state-of-the-art performance on View-of-Delft, TJ4DRadSet and NuScenes datasets. Source code is available at github.com/shawnnnkb/SIFormer.

Xiaokai Bai, Lianqing Zheng, Si-Yuan Cao +6

College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou 310027, China · School of Automotive Studies, Tongji University, Shanghai 201804, China · College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China, and also with the Jinhua Institute of Zhejiang University, Jinhua 321299, China +1

3D Object DetectionCamera-Radar FusionmmWave Radar Sensing3D ViTs
AskPDF
Feb 23, 2026

RADE-Net: Robust Attention Network for Radar-Only Object Detection in Adverse Weather

Automotive perception systems are obligated to meet high requirements. While optical sensors such as Camera and Lidar struggle in adverse weather conditions, Radar provides a more robust perception performance, effectively penetrating fog, rain, and snow. Since full Radar tensors have large data sizes and very few datasets provide them, most Radar-based approaches work with sparse point clouds or 2D projections, which can result in information loss. Additionally, deep learning methods show potential to extract richer and more dense features from low level Radar data and therefore significantly increase the perception performance. Therefore, we propose a 3D projection method for fast-Fourier-transformed 4D Range-Azimuth-Doppler-Elevation (RADE) tensors. Our method preserves rich Doppler and Elevation features while reducing the required data size for a single frame by 91.9% compared to a full tensor, thus achieving higher training and inference speed as well as lower model complexity. We introduce RADE-Net, a lightweight model tailored to 3D projections of the RADE tensor. The backbone enables exploitation of low-level and high-level cues of Radar tensors with spatial and channel-attention. The decoupled detection heads predict object center-points directly in the Range-Azimuth domain and regress rotated 3D bounding boxes from rich feature maps in the cartesian scene. We evaluate the model on scenes with multiple different road users and under various weather conditions on the large-scale K-Radar dataset and achieve a 16.7% improvement compared to their baseline, as well as 6.5% improvement over current Radar-only models. Additionally, we outperform several Lidar approaches in scenarios with adverse weather conditions. The code is available under https://github.com/chr-is-tof/RADE-Net.

Christof Leitgeb, Thomas Puchleitner, Max Peter Ronecker +1

Infineon Technologies AG, Institute of Visual Computing, Graz University of Technology, Graz, Austria · Infineon Technologies AG, Graz, Austria · Virtual Vehicle Research GmbH, Institute of Visual Computing, Graz University of Technology, Graz, Austria +1

3D Object DetectionmmWave Radar Sensing
AskPDF
Feb 21, 2026

WiCompass: Oracle-driven Data Scaling for mmWave Human Pose Estimation

Millimeter-wave Human Pose Estimation (mmWave HPE) promises privacy but suffers from poor generalization under distribution shifts. We demonstrate that brute-force data scaling is ineffective for out-of-distribution (OOD) robustness; efficiency and coverage are the true bottlenecks. To address this, we introduce WiCompass, a coverage-aware data-collection framework. WiCompass leverages large-scale motion-capture corpora to build a universal pose space ``oracle'' that quantifies dataset redundancy and identifies underrepresented motions. Guided by this oracle, WiCompass employs a closed-loop policy to prioritize collecting informative missing samples. Experiments show that WiCompass consistently improves OOD accuracy at matched budgets and exhibits superior scaling behavior compared to conventional collection strategies. By shifting focus from brute-force scaling to coverage-aware data acquisition, this work offers a practical path toward robust mmWave sensing.

Bo Liang, Chen Gong, Haobo Wang +8

School of Computer Science, Peking University · University of Electronic Science and Technology of China · University of Pittsburgh +2

Human Pose EstimationOOD GeneralizationmmWave Radar SensingData-Efficient Learning
AskPDF
Feb 12, 2026

HyperDet: 3D Object Detection with Hyper 4D Radar Point Clouds

How far can 3D object detection go using 4D radar alone? Despite offering weather-robust and velocity- aware sensing for autonomous perception, modern 4D radar still yields sparse, noisy, and unstable point clouds, limiting radar-only 3D detection. We present HyperDet, a detector- agnostic input enhancement pipeline that constructs task- aware hyper 4D radar point clouds by combining measured observations with completed foreground geometry. HyperDet first refines short-window surround-view radar observations through spatio-temporal accumulation and cross-sensor val- idation, while Doppler-guided motion compensation reduces dynamic object trails when motion can be estimated reliably. It then performs foreground generative enhancement using LiDAR-guided pseudo-radar supervision available only during training, enriching object geometry while preserving measured radar background and radar-native attributes. During detec- tor training, radar-aware object-level augmentation maintains Doppler consistency under geometric relocation. At inference, HyperDet requires radar input alone and can be directly paired with standard 3D detectors. Experiments on two public surround-view 4D radar datasets demonstrate consistent im- provements over matched temporal accumulation across stan- dard 3D detectors, validating input-level radar enhancement as an effective approach to radar-only 3D detection.

Yichun Xiao, Jin Jin, Runwei Guan +1

HKUST(GZ) · University of Edinburgh · University of Oxford

Point Cloud CompletionmmWave Radar SensingAutonomous Driving Perception3D Object Detection
AskPDF
Aug 4, 2025

Registering the 4D Millimeter Wave Radar Point Clouds Via Generalized Method of Moments

4D millimeter wave radars (4D radars) are new emerging sensors that provide point clouds of objects with both position and radial velocity measurements. Compared to LiDARs, they are more affordable and reliable sensors for robots' perception under extreme weather conditions. On the other hand, point cloud registration is an essential perception module that provides robot's pose feedback information in applications such as Simultaneous Localization and Mapping (SLAM). Nevertheless, the 4D radar point clouds are sparse and noisy compared to those of LiDAR, and hence we shall confront great challenges in registering the radar point clouds. To address this issue, we propose a point cloud registration framework for 4D radars based on Generalized Method of Moments. The method does not require explicit point-to-point correspondences between the source and target point clouds, which is difficult to compute for sparse 4D radar point clouds. Moreover, we show the consistency of the proposed method. Experiments on both synthetic and real-world datasets show that our approach achieves higher accuracy and robustness than benchmarks, and the accuracy is even comparable to LiDAR-based frameworks.

Xingyi Li, Han Zhang, Ziliang Wang +2

School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai, China · Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China · Department of Statistics, Uppsala University, Sweden

Millimeter-Wave RadarmmWave Radar SensingPoint Cloud Registration
AskPDF
Mar 11, 2025

Simulating Automotive Radar with Lidar and Camera Inputs

Low-cost millimeter automotive radar has received more and more attention due to its ability to handle adverse weather and lighting conditions in autonomous driving. However, the lack of quality datasets hinders research and development. We report a new method that is able to simulate 4D millimeter wave radar signals including pitch, yaw, range, and Doppler velocity along with radar signal strength (RSS) using camera image, light detection and ranging (lidar) point cloud, and ego-velocity. The method is based on two new neural networks: 1) DIS-Net, which estimates the spatial distribution and number of radar signals, and 2) RSS-Net, which predicts the RSS of the signal based on appearance and geometric information. We have implemented and tested our method using open datasets from 3 different models of commercial automotive radar. The experimental results show that our method can successfully generate high-fidelity radar signals. Moreover, we have trained a popular object detection neural network with data augmented by our synthesized radar. The network outperforms the counterpart trained only on raw radar data, a promising result to facilitate future radar-based research and development.

Peili Song, Dezhen Song, Yifan Yang +2

Institute of Robotics and Automatic Information System, Nankai University, Tianjin 300350, China; Tianjin Key Laboratory of Intelligent Robotics, Tianjin 300350, China; and also with TBI center, Nankai University, Tianjin 300350, China · Department of Robotics, Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi, United Arab Emirates

Synthetic Data AugmentationMillimeter-Wave RadarmmWave Radar SensingAutonomous Driving PerceptionSynthetic Data Generation for RoboticsAutonomous Driving Simulation
AskPDF