What Limits mmWave Human-State Sensing for Robot Perception?

Shuntian Zheng; Guangming Wang; Hongyang He; Shuai He; Yu Guan

cs.RONov 6, 2026

What Limits mmWave Human-State Sensing for Robot Perception?

Authors: Shuntian Zheng, Guangming Wang, Hongyang He, Shuai He, Yu Guan

Abstract

Assistive robots and search-and-rescue platforms require reliable human state sensing in smoke, darkness, and privacy-sensitive environments, where cameras and LiDAR may be unreliable Millimeter-wave radar offers an alternative that works without ambient light, penetrates smoke, and does not capture visual appearance. However, the compact arrays that fit robot size, power, and cost budgets have limited apertures, producing joint-localization errors on the order of decimeters at a few meters, large enough to compromise the safety margins and tracking accuracy a robot depends on. Existing pipelines reduce pose error through frame fusion, multiple viewpoints, return selection, or stronger estimators, but examine one technique at a time, without relating its effect to a shared measurement limit or separating gains from added information from gains from estimator completion. We address this gap with a framework that models the human and environment as one time-varying scattering field and states a Cramer-Rao type error floor for joint localization as a function of bandwidth, aperture, signal-to-noise ratio, fused frame count, phase coherence, and visibility of retained returns. We then use existing frame fusion, multi viewpoint, return selection, and pose estimation methods as matched probes of this floor rather than as competing pipelines. Under the evaluated conditions, experiments on three datasets and a live deployment show these strategies move joint localization error in the predicted directions, for example lowering mean per-joint position error from about 97 to 65 mm with a second viewpoint, while a stronger estimator mainly lowers overall error by completing joints the measurement cannot resolve. These results give the robotics community a principled basis for deciding whether to prioritize observation design or estimator capacity when deploying reliable mmWave human sensing.

Figures & tables

Fig. 1: Smoke in an industrial workspace blocks a camera, whereas a robot-mounted mmWave can still observe the worker; some joints nevertheless remain not identifiable. The right-hand callouts label the common front-end choices examined in this paper, namely frame fusion, viewpoint, and return selection, and we ask which sensing quantities set the localization limit.

Fig. 2: Operator view of pipelines. $\Phi$ applies return selection to a raw range–angle–Doppler cube $\mathbf{x}_{\mathrm{raw}}$ , yielding a clutter-suppressed ( $\Phi_{\mathrm{CS}}$ ) or environment-aware ( $\Phi_{\mathrm{EA}}$ ) observation. $\mathcal{G}$ aggregates frames, coherently or non-coherently, and optional viewpoints into a fused measurement. $\Pi$ maps that measurement to a pose; labeled joints are those identifiable from measurement information. Each operator is later used as a factor probe under one error floor.

Fig. 3: Factor-level validation. Each panel changes one operator associated with $K$ / $\eta$ , viewpoint count, or $\mathcal{V}$ , holding the others fixed. (a) Fuse: RMSE on $\mathcal{U}$ versus fused frame count $K$ , with overall MPJPE, the $K^{-1/4}$ guide, and the CRB reference evaluated at the measured $\hat{\eta}(K)$ printed below the axis; shading marks the remaining distance to that floor. (b) View: one versus two complementary radars; outer bars give MPJPE, inner bars RMSE on $\mathcal{U}$ , and azimuth RMSE is printed under each tick. (c) Vis: clutter-suppressed versus environment-aware return selection under the same inner-bar convention, including the extreme multipath tertiles (the mid tertile is in Table III ).

Fig. 4: Estimator probe under frozen $\Phi$ and $\mathcal{G}$ on mmRadPose ( $K{=}{\color[rgb]{0,0,0}10}$ , $\Phi_{\mathrm{EA}}$ ). A stronger $\Pi$ reduces overall MPJPE mainly outside $\mathcal{U}$ ; the change on $\mathcal{U}$ is not resolved, and the CRB reference is unchanged by construction because Eq. ( 8 ) does not depend on $\Pi$ .

Fig. 5: Live AWR1843 capture. (a) Hardware and three indoor environments; co-located RGB is used only for annotation. (b) Coherent RMSE on $\mathcal{U}$ versus fused frame count $K$ ; log–log slopes remain negative from one to three subjects. (c) Clutter-suppressed versus environment-aware MPJPE at the matched operating point. Dataset-replay is a same-host single-person reference rather than a copy of the public tables.

Radar-Based Human Pose Estimation

See all 8 papers

Feb 21, 2026

WiCompass: Oracle-driven Data Scaling for mmWave Human Pose Estimation

Millimeter-wave Human Pose Estimation (mmWave HPE) promises privacy but suffers from poor generalization under distribution shifts. We demonstrate that brute-force data scaling is ineffective for out-of-distribution (OOD) robustness; efficiency and coverage are the true bottlenecks. To address this, we introduce WiCompass, a coverage-aware data-collection framework. WiCompass leverages large-scale motion-capture corpora to build a universal pose space ``oracle'' that quantifies dataset redundancy and identifies underrepresented motions. Guided by this oracle, WiCompass employs a closed-loop policy to prioritize collecting informative missing samples. Experiments show that WiCompass consistently improves OOD accuracy at matched budgets and exhibits superior scaling behavior compared to conventional collection strategies. By shifting focus from brute-force scaling to coverage-aware data acquisition, this work offers a practical path toward robust mmWave sensing.

Bo Liang, Chen Gong, Haobo Wang +8

School of Computer Science, Peking University · University of Electronic Science and Technology of China · University of Pittsburgh +2

Human Pose Estimation OOD Generalization mmWave Radar Sensing Data-Efficient Learning

Ask PDF

Jul 21, 2026

Wave2Body: Rethinking mmWave Human Pose Estimation as Radar-to-Body Token Translation

Millimeter-wave (mmWave) radar enables privacy-friendly human sensing, but its sparse point clouds are physical measurements of view-dependent electromagnetic reflections and only indirectly characterize body articulation. Recovering a complete 3D pose from such partial, geometry-dependent observations is therefore under-constrained. Existing methods directly regress joint coordinates from paired radar-pose data, relying on the same limited paired supervision to learn radar perception, human-body structure, and their alignment. This coupling can encourage dataset-specific shortcuts under ambiguous radar observations. We propose Wave2Body, a radar-to-body token translation framework that decouples these learning targets using a self-supervised mmWave tokenizer, a pretrained compositional body tokenizer that defines the output space, and a lightweight translator between them. Experiments on M4Human and mmBody show that Wave2Body achieves stronger cross-domain generalization than previous methods while incurring much lower computational costs for training and inference. All the code and experiment results are publicly available at https://github.com/Galaxywalk/Wave2Body.

Bo Liang, Chen Gong, Wei Gao +1

School of Computer Science, Peking University · University of Pittsburgh · Key Laboratory of High Confidence Software Technologies, Ministry of Education (PKU)

Radar-Based Human Pose Estimation Millimeter-Wave Radar mmWave Radar Sensing 3D Human Pose Estimation Human Pose Estimation

Ask PDF

Apr 30, 2026

MAEPose: Self-Supervised Spatiotemporal Learning for Human Pose Estimation on mmWave Video

Millimetre-wave (mmWave) radar offers a more privacy-preserving alternative to RGB-based human pose estimation. However, existing methods typically rely on pre-extracted intermediate representations such as sparse point clouds or spectrogram images, where the rich spatiotemporal information naturally present in radar video streams is discarded for model learning, while such signal processing adds system complexity. In addition, existing solutions are mainly conducted in an end-to-end supervised manner without leveraging unlabelled raw video streams to learn generalized representations. In this study, we present MAEPose, a masked autoencoding-based human pose estimation approach that operates directly on mmWave spectrogram videos. MAEPose learns spatiotemporal motion-aware generalized representations from unlabelled radar video, and leverages its heatmap decoder for multi-frame pose estimation predictions. We evaluate it across three datasets based on leave-one-person-out cross-validation with rigorous statistical testing. MAEPose consistently outperforms state-of-the-art baselines by up to 22.1% in MPJPE p<0.05, and maintains robust accuracy under zero-shot bystander interference with only a 6.5% error increase. Ablation studies confirm that both the pre-training and the heatmap decoder contribute substantially, while modality analysis indicates that leveraging Range-Doppler video as input achieves better pose estimation performance than Range-Azimuth or their fusion, with lower computational cost.

Xijia Wei, Yuan Fang, Kevin Chetty +2

University College London London, UK

Radar-Based Human Pose Estimation Masked Autoencoders Self-Supervised Learning Millimeter-Wave Radar mmWave Radar Sensing 3D Human Pose Estimation+1

Ask PDF

Abstract

Radar-Based Human Pose Estimation

Explore similar work

WiCompass: Oracle-driven Data Scaling for mmWave Human Pose Estimation

Wave2Body: Rethinking mmWave Human Pose Estimation as Radar-to-Body Token Translation

MAEPose: Self-Supervised Spatiotemporal Learning for Human Pose Estimation on mmWave Video