Video-based Heart Rate Estimation with Angle-guided ROI Optimization and Graph Signal Denoising
Authors: Gan Pei, Junhao Ning, Boqiu Shen, Yan Zhu, Menghan Hu
Organizations: Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University · Shanghai Changzheng Hospital, Naval Military Medical University · Chongqing Key Laboratory of Precision Optics, Chongqing Institute of East China Normal University
Remote photoplethysmography (rPPG) enables non-contact heart rate measurement from facial videos, but its performance is significantly degraded by facial motions such as speaking and head shaking. To address this issue, we propose two plug-and-play modules. The Angle-guided ROI Adaptive Optimization module quantifies ROI-Camera angles to refine motion-affected signals and capture global motion, while the Multi-region Joint Graph Signal Denoising module jointly models intra- and inter-regional ROI signals using graph signal processing to suppress motion artifacts. The modules are compatible with reflection model-based rPPG methods and validated on three public datasets. Results show that jointly use markedly reduces MAE, with an average decrease of 20.38% over the baseline, while ablation studies confirm the effectiveness of each module. The work demonstrates the potential of angle-guided optimization and graph-based denoising to enhance rPPG performance in motion scenarios.
Figures & tables
Figure 1: Challenges of reflection model-based rPPG methods for HR estimation and the proposed solutions.
Figure 2: Pipeline of HR measurement framework is shown, with variables consistent with those in Sec. 2 . Angle-guided ROI Adaptive Optimization module aims to detect global head motion based on angular features and to refine ROI signals, while Multi-Region Joint Graph Signal Denoising module purifies heart rate signals by jointly modeling and denoising ROI signals.
Figure 3: Global motion detection can effectively identifying head rotations and nods on the PURE dataset.
S-E
S-P
COHFACE
PURE
MMPD
MAE↓
MAPE↓
MAE↓
MAPE↓
MAE↓
MAPE↓
PBV & SROI
BP
7.61
11.83
6.54
8.15
20.40
22.35
PBV & AGROI
BP
5.82
6.17
5.72
6.24
19.87
21.73
PBV & AGROI
BGSD
5.68
8.29
4.52
6.13
18.94
21.59
POS & SROI
BP
5.18
7.18
5.88
6.33
16.16
18.06
POS & AGROI
BP
4.63
6.42
5.31
5.46
15.45
17.56
Table 1: Performance comparison of baseline rPPG methods with angle-guided ROI module (AGROI) and multi-region joint graph signal denoising module (GSD) on three public datasets. S-E: Signal Extraction; S-P: Signal Processing; SROI: Single ROI; BP: Bandpass; BGSD: Bandpass & GSD.
Remote photoplethysmography (rPPG) is a non-contact technique that estimates physiological signals by analyzing subtle skin color changes in facial videos. Existing rPPG methods often encounter performance degradation under facial motion and occlusion scenarios due to their reliance on static and single-view facial videos. Thus, this work focuses on tackling the motion-induced occlusion problem for rPPG measurement in unconstrained multi-view facial videos. Specifically, we introduce a Multi-View rPPG Dataset (MVRD), a high-quality benchmark dataset featuring synchronized facial videos from three viewpoints under stationary, speaking, and head movement scenarios to better match real-world conditions. We also propose MVRD-rPPG, a unified multi-view rPPG learning framework that fuses complementary visual cues to maintain robust facial skin coverage, especially under motion conditions. Our method integrates an Adaptive Temporal Optical Compensation (ATOC) module for motion artifact suppression, a Rhythm-Visual Dual-Stream Network to disentangle rhythmic and appearance-related features, and a Multi-View Correlation-Aware Attention (MVCA) for adaptive view-wise signal aggregation. Furthermore, we introduce a Correlation Frequency Adversarial (CFA) learning strategy, which jointly enforces temporal accuracy, spectral consistency, and perceptual realism in the predicted signals. Extensive experiments and ablation studies on the MVRD dataset demonstrate the superiority of our approach. In the MVRD movement scenario, MVRD-rPPG achieves an MAE of 0.90 and a Pearson correlation coefficient (R) of 0.99. The source code and dataset will be made available.
Zuxian He, Xu Cheng, Zhaodong Sun +4
Nanjing University of Information Science and Technology · University of Oulu · Xi’an Jiaotong University +1
Remote photoplethysmography (rPPG) enables contactless measurement of heart rate and other vital signs by analyzing subtle color variations in facial skin induced by cardiac pulsation. Current rPPG methods are mainly based on either end-to-end modeling from raw videos or intermediate spatial-temporal map (STMap) representations. The former preserves complete spatiotemporal information and can capture subtle heartbeat-related signals, but it also introduces substantial noise from motion artifacts and illumination variations. The latter stacks the temporal color changes of multiple facial regions of interest into compact two-dimensional representations, significantly reducing data volume and computational complexity, although some high-frequency details may be lost. To effectively integrate the mutual strengths, we propose PhysNeXt, a dual-input deep learning framework that jointly exploits video frames and STMap representations. By incorporating a spatio-temporal difference modeling unit, a cross-modal interaction module, and a structured attention-based decoder, PhysNeXt collaboratively enhances the robustness of pulse signal extraction. Experimental results demonstrate that PhysNeXt achieves more stable and fine-grained rPPG signal recovery under challenging conditions, validating the effectiveness of joint modeling of video and STMap representations. The codes will be released.
Junzhe Cao, Bo Zhao, Zhiyi Niu +5
Harbin Institute of Technology, Shenzhen · Great Bay University · Hefei University of Technology +1
Remote photoplethysmography (rPPG) enables non-contact heart rate (HR) monitoring from facial videos, but RGB-only methods are vulnerable to illumination changes, motion artifacts, and skin-tone-dependent optical reflectance. We propose CardiacMamba, a fair and robust RGB-RF fusion framework that integrates optical facial cues and radio-frequency cardiac motion cues through state space modeling. CardiacMamba introduces a Temporal Difference Mamba Module (TDMM) to enhance subtle RF temporal variations, a bidirectional SSM-based interaction mechanism to align heterogeneous RGB-RF dynamics, and a Channel-wise Fast Fourier Transform (CFFT) module for channel-domain spectral refinement. On the EquiPleth dataset, CardiacMamba achieves state-of-the-art performance with 0.96 bpm MAE, 3.06 bpm RMSE, and 0.97 Pearson correlation, while reducing the observed light-dark skin-tone MAE gap to 0.26 bpm and maintaining robustness under RGB degradation and RF-missing conditions