Learning to Assess Heartbeat Observability for mmWave Heart-Rate Sensing

Yuxuan Hu; Shilin Shan; Jianfei Yang; Feng Xu

cs.AIOct 2, 2026

Learning to Assess Heartbeat Observability for mmWave Heart-Rate Sensing

Authors: Yuxuan Hu, Shilin Shan, Jianfei Yang, Feng Xu

Organizations: Key Laboratory for Information Science of Electromagnetic Waves, Ministry of Education, School of Information Science and Technology, Fudan University, Shanghai 200433, China · School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore 639798

Abstract

Contactless heart-rate sensing with millimeter-wave (mmWave) radar requires assessing whether individual measurements support reliable estimation. We study learning to assess heartbeat observability, defined as the readability of the heartbeat component in an acquired phase spectrum, for selective heart-rate estimation. Coherent superposition of scatterer returns can suppress this component even under similar macroscopic observation geometry, motivating assessment directly from acquired measurements. To obtain training supervision across different observability conditions, we develop a controllable multi-scatterer frequency-modulated continuous-wave (FMCW) simulator. Agreement between the dominant heartbeat-band peak and the known heart rate provides an automatic observability label for each simulated measurement. We propose HEAR (Heartbeat Estimation with Assessed Reliability), a compact dual-task Transformer that jointly predicts an observability score and heart rate. Its input combines spectral magnitudes with frequencies relative to the respiration fundamental, providing context for respiratory harmonics. Trained solely on simulated observations, HEAR transfers zero-shot to two public real-world datasets collected at 60 and 120 GHz from 134 subjects. The same learned score supports selective prediction with both HEAR's own heart-rate head and multiple existing estimators. On the 120 GHz dataset, score-based selection reduces the HR head's mean absolute error from 17.9 BPM at full coverage to 1.6 BPM at 50% coverage. The complete pipeline achieves an end-to-end processing latency of 50.8 ms on an edge device. Project page: https://yuxuanhu9.github.io/HEAR/.

Figures & tables

Fig. 1: Motivation and overview of HEAR. (a) Simulated observations with similar motion projections can exhibit markedly different heartbeat observability, showing that geometry alone is insufficient to determine whether the heartbeat component is readable. This motivates assessment from the acquired signal. (b) Trained solely on simulated observations, HEAR is applied to real recordings without adaptation and jointly predicts an observability score and heart rate from the phase spectrum of each 25 s window. Heart-rate estimates are reported only for windows with predicted scores at or above the selection threshold.

Fig. 2: Offline training and zero-shot inference with HEAR. (a) The multi-scatterer FMCW simulator generates paired observations across radar positions under identical respiratory and heartbeat motion. The discrepancy between the dominant heartbeat-band peak and the reference heart rate defines a soft observability label. These labels and reference heart rates supervise joint observability prediction and HR estimation. (b) Without real-data adaptation, HEAR processes the phase spectrum of each 25 s window, extracted with a 10 s hop, and jointly predicts an observability score and heart rate. The HR estimate is reported when the score meets or exceeds the selection threshold.

Fig. 3: SMPL body models and candidate radar position generation. Twelve standing, sitting, and lying postures are combined with four body types to construct 48 body models. Representative meshes illustrate variations in sex and body mass. Geometric constraints, spatial filtering, and surface-normal filtering identify candidate points on the anterior chest. Arrows indicate the surface normals used to place the radar at an outward offset of 0.40 m from each retained point.

Fig. 4: Chest-wall motion synthesis and FMCW radar simulation. (a) Chest-wall micro-motion synthesis combines respiration and heartbeat drive waveforms with spatial motion weights to generate displacement along each scatterer’s surface normal over time. (b) Radar measurement simulation uses scatterer ranges and antenna-pattern weights to generate per-scatterer FMCW mixed signals. These signals are coherently summed, combined with noise, and processed to extract the phase at the selected range bin. Simulated heartbeat frequencies span $[0.8,3.0]$ Hz, while the deployed HR output band is $[0.8,2.0)$ Hz.

Fig. 5: Signal-processing chain for a simulated observation at one radar position. Time-domain signals are shown on the left and spectra on the right, with respiration and heartbeat bands marked. The synthesized surface motion is used in FMCW simulation to generate the complex return at the selected range bin. Its phase is extracted, unwrapped, differenced between frames, Hampel-filtered, and smoothed. In this example, preprocessing makes the heartbeat peak more distinct in the spectrum of the processed phase-difference signal $\phi_{\mathrm{d}}$ . This spectrum is used to construct the network input.

Fig. 6: Network inputs and observability-label construction. Motion-projection maps and simulated phase spectra illustrate differences between geometric favorability and heartbeat observability. The projection scores are not network inputs. For each candidate heartbeat bin, HEAR forms a 2-D token $[X(b),r(b)]$ , combining the normalized spectral magnitude with the frequency ratio $r(b)=f(b)/f_{\mathrm{b}}$ , where $f_{\mathrm{b}}$ is the estimated respiration fundamental. The soft observability label is derived from the bin distance $d=|b_{\mathrm{peak}}-b_{\mathrm{true}}|$ between the dominant heartbeat-band peak and the reference HR, using $y_{\mathrm{gate}}=\exp(-d/\tau)$ with $\tau=3$ .

Fig. 7: Physics-aware dual-task network. HEAR represents each candidate heartbeat bin by a 2-D token $[X(b),r(b)]$ , containing the normalized spectral magnitude and the frequency ratio $r(b)=f(b)/f_{\mathrm{b}}$ . A lightweight Transformer models relations across HR bins, and its output tokens are mean-pooled into a shared representation. Two heads jointly predict the observability score $g$ and the HR posterior $\hat{\mathbf{p}}$ . The observability head is supervised by soft labels for all training observations, while the HR head is supervised by reference HR bins only for observations within the deployed HR band.

Fig. 8: Ground-truth HR distributions of sliding windows in the two public datasets, grouped by acquisition condition. Each window spans 25 s with a 10 s hop, and

n

denotes the number of windows in each group. In each violin, the central horizontal line marks the median, and the lower and upper caps indicate the minimum and maximum, respectively. Parralejo 60 GHz shows a broader HR distribution after exercise, reaching 144 BPM, whereas VitalSense 120 GHz remains entirely at or below 83 BPM under both resting and breath-hold conditions.

Panel of Figure 9 — Fig. 9: Geometry and heartbeat observability in simulation. (a) ROC curves within the top-10% projection-quality subset. HEAR and $Q_{k}$ achieve AUCs of 0.868 and 0.635, respectively. (b) Dominant-peak HR estimates under differential sub-wavelength range perturbations, colored by observability score. The dashed line indicates perfect agreement. Splitting at the median score yields Acc@ $\pm 1$ bin of 85% and 23% for the higher- and lower-score groups. (c) Heartbeat energy concentration across 492 positions on one body model. Color denotes the fraction of heartbeat-band energy within $\pm 2$ bins of the reference HR. (d) Heartbeat energy concentration versus $Q_{k}$ (Pearson $r=-0.03$ ).

Category	Quality predictor	Simulation $N{=}562{,}986$		60 GHz Parralejo $N{=}1408$		120 GHz VitalSense $N{=}480$
		AUC $\uparrow$	Spear. $\uparrow$	AUC $\uparrow$	Spear. $\uparrow$	AUC $\uparrow$	Spear. $\uparrow$
Geometric prior	$Q_{k}=S\cdot C$ (Eq. ( 14 ))	0.635	$+0.165$	—	—	—	—
Spectral-shape	snr_mean	0.750	$+0.284$	0.758	$+0.345$	0.852	$+0.582$
	energy_conc	0.748	$+0.277$	0.747	$+0.321$	0.843	$+0.577$
	prominence	0.742	$+0.360$	0.689	$+0.279$	0.830	$+0.511$
	neg_flatness	0.669	$+0.215$	0.737	$+0.349$	0.788	$+0.456$

TABLE I: Reliability-score comparison on simulated and real observations. AUC treats windows with a dominant HR-band peak within

\pm 1

bin of the reference HR as positive. Spear. correlates each method’s score with the negative absolute error of HEAR’s HR estimate.

Q_{k}

is evaluated only in simulation. The shaded row marks HEAR.

Fig. 10: Observability-guided selective HR estimation on real sliding windows. All HR estimates come from HEAR’s HR head. (a)–(b) Risk–coverage curves on Parralejo 60 GHz and VitalSense 120 GHz, comparing selection by HEAR scores with Oracle selection by reference-based absolute errors. The horizontal dashed line marks MAE at full coverage. (c) Accepted-window Acc@5 BPM versus the selection threshold

\eta

. (d) Empirical Acc@5 BPM within observability-score bins on both datasets.

Gate bin	Simulation $N{=}562{,}986$				60 GHz Parralejo $N{=}1408$				120 GHz VitalSense $N{=}480$
	$n$	cov	MAE $\downarrow$	acc $\uparrow$	$n$	cov	MAE $\downarrow$	acc $\uparrow$	$n$	cov	MAE $\downarrow$	acc $\uparrow$
$[0.0,0.1)$	108,718	100%	44.9	5.6%	218	100%	41.3	4%	25	100%	58.2	0%
$[0.1,0.2)$	177,403	80.7%	34.6	9.6%	375	84.5%	43.0	2%	66	94.8%	57.1	0%
$[0.2,0.3)$	63,757	49.2%	25.6	18.6%	226	57.9%	38.8	8%	38	81.0%	49.5	8%
$[0.3,0.4)$	38,523	37.9%	20.1	28.5%	172	41.8%	26.3	21%	37	73.1%	23.0	41%
$[0.4,0.5)$	29,437	31.0%	15.5	38.5%	120	29.6%	19.7	28%	24	65.4%	3.7	75%

TABLE II: HR estimation performance across 10 equal-width observability-score bins. Sample count

n

, MAE (BPM), and accuracy are computed within each bin. Coverage (cov) is the fraction of all windows with scores at or above the bin’s lower bound. Simulation uses FFT-peak estimates with accuracy measured within

\pm 1

bin (

\pm 2.4

BPM). Real datasets use HEAR’s HR-head estimates with accuracy measured within

\pm 5

BPM.

Coverage	Simulation $N{=}562{,}986$			60 GHz Parralejo $N{=}1408$			120 GHz VitalSense $N{=}480$
Coverage	HEAR	snr_m	Oracle	HEAR	snr_m	Oracle	HEAR	snr_m	Oracle
100% (no gating)	16.6 / 53	16.6 / 53	16.6 / 53	31.0 / 19	31.0 / 19	31.0 / 19	17.9 / 62	17.9 / 62	17.9 / 62
90%	16.7 / 53	16.4 / 54	11.3 / 59	29.9 / 21	29.9 / 21	27.3 / 22	13.5 / 69	15.1 / 68	13.0 / 69
70%	15.8 / 55	15.6 / 56	3.9 / 75	26.1 / 27	27.4 / 26	20.0 / 28	3.2 / 86	9.8 / 80	2.0 / 89
60%	14.8 / 57	15.1 / 58	2.0 / 88	23.5 / 31	26.4 / 28	16.1 / 32	1.9 / 91	8.2 / 84	1.1 / 100
50%	13.1 / 61	14.3 / 60	1.0 / 100	19.8 / 36	25.1 / 30	12.0 / 39	1.6 / 93	8.6 / 85	0.8 / 100

TABLE III: Risk–coverage trade-off of selective HR estimation. All selectors use the same HEAR HR-head estimates. Entries report retained-window MAE (BPM) / Acc@5 BPM (%). Full coverage denotes no selection. HEAR and snr_m select windows by observability and SNR scores, respectively. Oracle selects windows with the smallest reference-based absolute errors. The final row reports normalized AURC over 5%–100% coverage. Lower is better.

	Simulation (acc@ $\pm 1$ bin)			Parralejo 60 GHz			VitalSense 120 GHz
HR estimator	All	After gate $\uparrow$	$\Delta$	All	After gate $\uparrow$	$\Delta$	All	After gate $\uparrow$	$\Delta$
FFT peak	29.8%	86.7%	$+56.9$	32.0%	65.8%	$+33.8$	81.7%	94.1%	$+12.4$
Weighted peak	30.0%	86.8%	$+56.9$	32.7%	66.4%	$+33.7$	83.1%	93.6%	$+10.5$
Autocorrelation [ 17 ]	—	—	—	36.2%	59.7%	$+23.6$	83.3%	90.0%	$+6.6$
Harmonic product spectrum	22.4%	42.6%	$+20.2$	19.7%	37.6%	$+17.8$	69.0%	77.6%	$+8.7$
VMD [ 16 ]	—	—	—	32.7%	65.1%	$+32.4$	76.5%	90.9%	$+14.4$

TABLE IV: Selective HR estimation with a shared HEAR score. “All” denotes all windows, and “After gate” denotes windows accepted by

g\geq 0.7

, retaining approximately 17%, 11%, and 46% of windows in simulation, Parralejo, and VitalSense, respectively. Within each dataset, all estimators use the same retained windows.

\Delta

reports the gain in percentage points. Accuracy is measured within

\pm 1

bin in simulation and

\pm 5

BPM on real data. Dashes indicate unreported results.

	Simulation test set		60 GHz Parralejo			120 GHz VitalSense
	$N{=}562{,}986$		$N{=}1408$			$N{=}480$
Configuration	AUC $\uparrow$	$\Delta$ AUC	AUC $\uparrow$	SE	$\Delta$ AUC	AUC $\uparrow$	SE	$\Delta$ AUC
2-D token $[X(b),\,f/f_{\mathrm{b}}]$ (full)	0.865	—	0.864	0.013	—	0.941	0.011	—
$-$ spectral value $X(b)$	0.517	$-0.348$	0.325	0.052	$-0.539$	—	—	—
$-$ frequency-ratio encoding	0.784	$-0.081$	0.627	0.049	$-0.237$	0.832	0.027	$-0.109$

TABLE V: Input feature ablation for the 2-D token

[X(b),r(b)]

. AUC measures observability discrimination. SE denotes bootstrap standard error, and

\Delta

AUC is the ablated model’s AUC minus the full-token model’s AUC. The ratio-only variant performs poorly on VitalSense, where numerical results are not reported. Dashes mark omitted baseline differences and unreported results.

Fig. 11: Joint t-SNE visualization of 75-dimensional normalized phase spectra from simulation and real datasets. (a) Simulation versus Parralejo 60 GHz, colored by posture. (b) Simulation versus VitalSense 120 GHz, with simulated observations colored by posture and real observations by recording condition. Circles and triangles represent simulated and real observations, respectively. The two-dimensional embedding shows partial overlap between simulated and real observations together with dataset-specific variation.

Dimension	Subset	$n$	AUC $\uparrow$
HR range	$<70$ BPM	661	0.918
	70–90 BPM	533	0.877
	90–110 BPM	160	0.843
	$>110$ BPM	54	0.327
Posture	Lying	706	0.897
Posture	Sitting	702	0.771

TABLE VI: Stratified AUC of the HEAR observability score on the full Parralejo sliding-window set (

N=1408

). Windows are grouped by reference HR range, posture, physiological state, and the joint Lying

\times

Rest subset.

n

denotes the number of windows. The

>110

BPM subset includes reference HRs both within and outside the deployed HR band.

Metric	Value
Network parameters	5,295
Model weights (FP32)	20.7 KB
Network FLOPs / window	0.26 M
Signal preprocessing (per window)	47.7 ms
Network inference (per window)	3.1 ms
Total processing time (per window)	50.8 ms

TABLE VII: Computational efficiency of HEAR on an NVIDIA Jetson AGX Orin in 30 W mode using only the ARMv8 CPU. Times are reported per 25 s phase-signal window. Total processing time includes signal preprocessing and network inference. A single forward pass produces both outputs. Peak memory refers to the full runtime process.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Category	Parameter	Value
Signal proc.	Frame rate / analysis window	20 Hz / 500 frames (25 s)
	Spectrum crop	$[0,3.0)$ Hz (75 bins), $L^{2}$ -norm. after crop
	Heartbeat / respiration band	$[0.8,2.0)$ Hz (30 bins) / $[0.1,0.6]$ Hz ( $f_{\mathrm{b}}$ : band argmax)
Architecture	token / embed dim	2 / 16
Architecture	attention / layers / params	std. 4-head / 2 / 5,295
Training	Optimizer	Adam (lr $10^{-3}$ , wd $10^{-4}$ ), cosine annealing

Appendix

TABLE VIII: Implementation details and hyper-parameters. The table summarizes the signal-processing settings, network architecture, training configuration, and simulation parameters used for reproducibility.

Hop (s)	Windows	AUC $\uparrow$	Temporal cov. $\uparrow$	Acc@5 BPM $\uparrow$	Rec. $\uparrow$
2	6159	0.871	23%	70%	36%
5	2815	0.863	20%	69%	33%
10	1408	0.856	15%	68%	24%
15	970	0.864	15%	73%	23%
25	877	0.836	9%	60%	16%

Appendix

TABLE IX: Sliding-window hop sensitivity on all 440 Parralejo records with a fixed 25 s window and

\eta=0.7

. Smaller hops raise temporal coverage while gate-discrimination AUC stays stable for 2 to 15 s hops. Temporal cov. is the per-record fraction of time covered by the union of accepted 25 s windows, averaged over records, and Rec. is the fraction of records with at least one accepted window.

Explore similar work

mmWave-Diffusion:A Novel Framework for Respiration Sensing Using Observation-Anchored Conditional Diffusion Model
Mar 21, 2026Yong Wang, Qifan Shen, Bao Zhang +4mmWave Radar Sensing Conditional Diffusion Models
Key Laboratory of Underwater Acoustic Signal Processing of Ministry of Education, Southeast University, Nanjing, 211189, China · Purple Mountain Laboratories, Nanjing, 211111, China
Deep Learning for Sleep Heart Rate Estimation from Accelerometers: Toward Population-Scale Cardiac Insight Without Optical Sensors
Oct 5, 2026Tanbin Islam Rohan, Pranjol Sen Gupta, Tanusree Debi +1Transformer Wearable Sensing
Department of Information Technology Kennesaw State University Kennesaw, GA, USA · School of Computing University of Georgia Athens, GA, USA
Kapture: Capturing Cardiac Dynamics with Koopman-Governed Learning for Efficient Radar-Based Electrocardiogram Recovery
Oct 4, 2026Tong Wu, Jing Peng, Ziqi Feng +1Electrocardiography mmWave Radar Sensing
Amazon, Seattle, Washington, United States · Equitable, Waterloo, Canada · School of Psychology & Neuroscience, University of Glasgow, Glasgow, United Kingdom

Mar 21, 2026J/K move · Enter open · S save

mmWave-Diffusion:A Novel Framework for Respiration Sensing Using Observation-Anchored Conditional Diffusion Model

Yong Wang, Qifan Shen, Bao Zhang, Zijun Huang+3

Key Laboratory of Underwater Acoustic Signal Processing of Ministry of Education, Southeast University, Nanjing, 211189, China · Purple Mountain Laboratories, Nanjing, 211111, China

mmWave Radar Sensing Conditional Diffusion Models Diffusion Transformer

Open paper AskPDF arXiv

Millimeter-wave (mmWave) radar enables contactless respiratory sensing,yet fine-grained monitoring is often degraded by nonstationary interference from body micromotions.To achieve micromotion interference removal,we propose mmWave-Diffusion,an observation-anchored conditional diffusion framework that directly models the residual between radar phase observations and the respiratory ground truth,and initializes sampling within an observation-consistent neighborhood rather than from Gaussian noise-thereby aligning the generative process with the measurement physics and reducing inference overhead. The accompanying Radar Diffusion Transformer (RDT) is explicitly conditioned on phase observations, enforces strict one-to-one temporal alignment via patch-level dual positional encodings, and injects local physical priors through banded-mask multi-head cross-attention, enabling robust denoising and interference removal in just 20 reverse steps. Evaluated on 13.25 hours of synchronized radar-respiration data, mmWave-Diffusion achieves state-of-the-art waveform reconstruction and respiratory-rate estimation with strong generalization. Code repository:https://github.com/goodluckyongw/mmWave-Diffusion.

Abstract

Explore similar work

mmWave-Diffusion:A Novel Framework for Respiration Sensing Using Observation-Anchored Conditional Diffusion Model

Deep Learning for Sleep Heart Rate Estimation from Accelerometers: Toward Population-Scale Cardiac Insight Without Optical Sensors

Kapture: Capturing Cardiac Dynamics with Koopman-Governed Learning for Efficient Radar-Based Electrocardiogram Recovery