Design, Implementation and Evaluation of a Real-Time Remote Photoplethysmography (rPPG) Acquisition System for Non-Invasive Vital Sign Monitoring
Authors: Constantino Álvarez Casado, Sasan Sharifipour, Manuel Lage Cañellas, Nhi Nguyen, Le Nguyen, Miguel Bordallo López
Organizations: Center for Machine Vision and Signal Analysis (CMVS), University of Oulu, Oulu, Finland. · VTT Technical Research Centre of Finland, Oulu, Finland.
The growing integration of smart environments and low-power computing devices, coupled with mass-market sensor technologies, is driving advancements in remote and non-contact-based physiological monitoring. However, deploying these systems in real-time on resource-constrained platforms introduces significant challenges related to scalability, interoperability, and performance. This paper presents a computationally efficient real-time remote photoplethysmography (rPPG) system, designed to extract physiological signals, such as heart rate, respiratory rate, and oxygen saturation (SpO2), from facial video streams. The system is built on the Face2PPG pipeline, which processes video frames sequentially for rPPG signal extraction and analysis, while leveraging a multithreaded architecture to manage video capture, real-time processing, network communication, and graphical user interface updates concurrently. This design targets continuous operation at 30 frames per second (FPS), with adaptive feedback through a collaborative user interface to guide optimal signal capture conditions. The network interface includes both an HTTP server for continuous video streaming and a RESTful API for on-demand vital sign retrieval. To ensure accurate performance despite limited computational resources, we use a hybrid programming model combining Functional Reactive Programming (FRP) and the Actor Model, allowing event-driven processing and efficient task parallelization. We evaluate HR accuracy on three public datasets (COHFACE, LGI-PPGI, and UBFC-RPPG), and conduct preliminary evaluations of RR on COHFACE and SpO2 on PURE. We also assess the processing time and end-to-end operation of the system on two computers, including live HR and SpO2 agreement with a medical-grade fingertip pulse oximeter across five 10-minute sessions. Our work addresses key challenges in real-time contactless biosignal monitoring.
Figures & tables
Figure 1: Architecture overview of the proposed real-time rPPG application, comprised of several interconnected components, each executing a specific task.
Figure 2: Implementation design of the proposed real-time rPPG application.
Face2PPG-RT
Face2PPG-RT
Face2PPG-Server
Face2PPG-Server
Module
Config. 1
Config. 2
Normalized
Multiregion
Face Detection
Dlib HOG
OpenCV YuNet
OpenCV SSD
OpenCV SSD
Face Alignment
Dlib ERT GTX
Dlib ERT GTX
DAN
DAN
Face Normalization
Mesh (131 triangles)
Mesh (131 triangles)
Mesh (131 triangles)
Mesh (131 triangles)
Skin Segmentation
Geometrical (Patches)
Geometrical (Patches)
Geometrical (Whole Face)
Geometrical (Best Regions)
RGB to BVP
Lab
Lab
POS
POS
Table 1: Configuration setups and modules for real-time and cloud-based rPPG systems. In the C++ application used for the speed evaluation, the face normalization uses a mesh of 122 triangles, and the spectral analysis of both configurations uses the FFT with a Hann window.
LGI-PPGI
COHFACE
UBFC1
UBFC2
Pipeline
MAE ± SD (bpm)
PCC
MAE ± SD (bpm)
PCC
MAE ± SD (bpm)
PCC
MAE ± SD (bpm)
PCC
Face2PPG-Server Normalized
8.7 ± 8.4
0.50
9.4 ± 4.8
0.10
1.2 ± 0.4
0.94
1.4 ± 1.5
0.95
Face2PPG-Server Multiregion
4.5 ± 3.3
0.57
8.0 ± 4.4
0.06
0.9 ± 0.4
0.96
0.9 ± 0.9
0.98
Face2PPG-RT Configuration 1
6.4 ± 6.8
0.45
10.9 ± 5.6
-0.04
1.7 ± 0.6
0.80
3.8 ± 4.7
0.72
Face2PPG-RT Configuration 2
5.9 ± 8.0
0.49
11.4 ± 7.3
-0.01
2.1 ± 1.1
0.83
6.0 ± 6.3
0.54
Table 2: Heart rate (HR) estimation error comparison between the four different configurations of the proposed rPPG system on three databases, with UBFC-RPPG divided into the UBFC1 and UBFC2 subsets. The table includes two configurations of the real-time system and two configurations for high-performance computers without any constraint. MAE and SD are expressed in beats per minute (bpm); PCC is dimensionless.
Face2PPG-RT Configuration 1
Face2PPG-RT Configuration 2
Configuration 2 − Configuration 1
Dataset
Rec. (Part.)
MAE [95% CI]
RMSE
MAE [95% CI]
RMSE
Δ MAE [95% CI]
p
dz
LGI-PPGI
24 (6)
6.4 [4.3, 8.6]
8.4
5.9 [3.6, 9.1]
7.6
− 0.5 [ − 1.5, 0.7]
0.44
− 0.34
COHFACE
160 (40)
10.9 [9.8, 12.0]
13.5
11.4 [9.8, 13.0]
14.2
0.5 [ − 0.6, 1.7]
0.41
0.13
UBFC1
8 (8)
1.7 [1.3, 2.1]
2.7
2.1 [1.5, 2.9]
3.3
0.4 [ − 0.1, 1.2]
0.30
0.40
UBFC2
42 (42)
3.8 [2.5, 5.4]
6.1
6.0 [4.1, 8.0]
9.0
2.2 [0.8, 3.7]
0.004
0.45
Table 3: Statistical comparison of the HR estimation of the two real-time configurations, evaluated on the same recordings. Rec. (Part.): number of recordings (participants). MAE: mean of the per-recording MAE, with its 95% confidence interval (CI) computed with a participant-level bootstrap. RMSE: mean of the per-recording RMSE. Δ MAE: paired difference between Configuration 2 and Configuration 1, with its 95% CI. p : participant-level sign-flip permutation test. dz : effect size. MAE, RMSE and Δ MAE are expressed in bpm. In UBFC1, each recording is treated as a separate participant.
Face2PPG-RT Configuration 1
Face2PPG-RT Configuration 2
Dataset
Scenario
Rec.
MAE [95% CI]
PCC
MAE [95% CI]
PCC
LGI-PPGI
Resting
6
2.0 [0.5, 4.6]
0.67
1.7 [0.6, 3.7]
0.66
Talking
6
4.4 [3.1, 5.9]
0.32
4.9 [2.2, 8.1]
0.42
Rotation
6
7.2 [3.8, 10.8]
0.26
5.6 [3.7, 7.7]
0.27
Gym
6
11.9 [5.1, 20.8]
0.56
11.3 [3.4, 23.5]
0.60
COHFACE
Studio lighting
80
9.7 [8.6, 10.9]
0.00
9.7 [8.4, 11.1]
0.00
Table 4: HR estimation error of the two real-time configurations for the different scenarios of LGI-PPGI and the two lighting conditions of COHFACE. Rec.: number of recordings. MAE: mean of the per-recording MAE in bpm, with its 95% CI computed with a participant-level bootstrap. PCC: mean of the per-recording PCC (dimensionless).
User equipment configurations
Cloud configurations
RT Config. 1
RT Config. 2
Server-Normalized
Server-Multiregion
Module
ms/frame
ms/frame
ms/frame
ms/frame
Face Detection
5.48
1.10
20.35
20.35
Face Alignment
4.48
3.74
82.52
82.52
Face Normalization
3.67
4.37
12.48
12.48
Skin Segmentation
8.23 us
8.23 us
95.57 us
105.41
Table 5: Performance speed for individual modules in various Face2PPG pipeline configurations, as well as the overall speed per face and frame. Times are given in ms per frame, except where indicated in microseconds (us). Total FPS is the inverse of the total processing time of the core modules, and it does not include camera capture, GUI rendering or network transmission. The Spectral Analysis row includes the computation of HR, RR and SpO2 at each update of the vital signs, although the SpO2 is computed without spectral analysis.
Figure 3: Setup of the live evaluation: the application during live operation on the laptop, in the first session, with the USB webcam on top of the screen, showing the facial landmarks, the HR, SpO2 and RR values and the pulse and respiratory signals, next to the fingertip pulse oximeter (Contec CMS50E) used as reference, connected to the same computer. The terminals behind the application window run the measurement script and a system monitor (htop), whose meters, at the top right, show the processing load distributed across the cores of the processor by the threads of the application.
Laptop (Intel Core i7-6700HQ)
Desktop (AMD Ryzen 7 3700X)
No clients
API + 8 streams
All phases
No clients
API + 8 streams
All phases
Sessions
3
1
3
2
1
2
Duration (s)
1800
180
2520
1200
180
1920
Frame rate and timing
Loop iterations per second
31.25
31.25
31.25
31.25
31.24
31.25
New camera frames per second
29.56
29.51
29.54
29.81
29.80
29.83
Table 6: Live operation of the complete system with a live camera at 30 FPS and one participant at rest, on the laptop of the speed evaluation (Intel Core i7-6700HQ; three sessions, one of them with network load) and on a desktop computer (AMD Ryzen 7 3700X; two sessions, one of them with network load). No clients: phases without network clients. API + 8 streams: requests to the RESTful API at 10 per second together with eight clients of the MJPEG streams, on the same computer. All phases: all the phases of all the sessions, including those with the API requests alone and with one and four stream clients. Values are medians with the 99th percentile in brackets, unless otherwise indicated. Latencies are measured from the timestamp of the camera driver. Loop work: time from the start of a loop iteration to the end of its network step. Agreement: differences with a fingertip pulse oximeter, one paired value per second.
Figure 4: Agreement of the HR of the application with the fingertip pulse oximeter during live operation. (a) HR displayed by the application, pulse rate displayed by the oximeter and beat-to-beat pulse rate obtained from the pulse wave of the oximeter, in the first 300 s of the third session on the laptop. (b) Bland-Altman plot of the 4,440 paired values, one per second, of the five sessions, with the bias (solid lines) and the 95% limits of agreement (dashed lines) of each computer; the 24 differences larger than 20 bpm are drawn at the border.
Remote photoplethysmography (rPPG) estimates the blood volume pulse (BVP) signal from facial videos, enabling contact-free health monitoring. Conventional clip-wise approaches, which use video clips as input, require capturing over one hundred frames before inference, thus introducing several seconds of delay and hindering real-time use. Meanwhile, frame-wise approaches struggle to capture long-range temporal and periodic features of physiological rhythms, and therefore lead to reduced estimation accuracy. To overcome these issues, we propose StreamPPG, a unified architecture that enables low-latency frame-wise physiological signal estimation while achieving competitive accuracy compared with clip-wise approaches. StreamPPG is trained under a consistent privileged learning (CPL) strategy, which leverages ground-truth rPPG signals as privileged information to enhance the model's representation capability. Extensive experiments demonstrate that StreamPPG achieves state-of-the-art accuracy across multiple datasets while maintaining real-time throughput on edge devices.
Remote photoplethysmography (rPPG) enables contactless measurement of heart rate and other vital signs by analyzing subtle color variations in facial skin induced by cardiac pulsation. Current rPPG methods are mainly based on either end-to-end modeling from raw videos or intermediate spatial-temporal map (STMap) representations. The former preserves complete spatiotemporal information and can capture subtle heartbeat-related signals, but it also introduces substantial noise from motion artifacts and illumination variations. The latter stacks the temporal color changes of multiple facial regions of interest into compact two-dimensional representations, significantly reducing data volume and computational complexity, although some high-frequency details may be lost. To effectively integrate the mutual strengths, we propose PhysNeXt, a dual-input deep learning framework that jointly exploits video frames and STMap representations. By incorporating a spatio-temporal difference modeling unit, a cross-modal interaction module, and a structured attention-based decoder, PhysNeXt collaboratively enhances the robustness of pulse signal extraction. Experimental results demonstrate that PhysNeXt achieves more stable and fine-grained rPPG signal recovery under challenging conditions, validating the effectiveness of joint modeling of video and STMap representations. The codes will be released.
Junzhe Cao, Bo Zhao, Zhiyi Niu +5
Harbin Institute of Technology, Shenzhen · Great Bay University · Hefei University of Technology +1
Remote photoplethysmography (rPPG) enables contactless physiological monitoring by capturing subtle skin-color variations from facial videos. However, most existing methods predominantly rely on time-domain modeling, making them vulnerable to motion artifacts and illumination fluctuations, where weak physiological clues are easily overwhelmed by noise. To address these challenges, we propose FreqPhys, a frequency-guided rPPG framework that explicitly leverages physiological frequency priors for robust signal recovery. Specifically, FreqPhys first applies a Physiological Bandpass Filtering module to suppress out-of-band interference, and then performs Physiological Spectrum Modulation together with adaptive spectral selection to emphasize pulse-related frequency components while suppress residual in-band noise. A Cross-domain Representation Learning module further fuses these spectral priors with deep time-domain features to capture informative spatial--temporal dependencies. Finally, a frequency-aware conditional diffusion process progressively reconstructs high-fidelity rPPG signals. Extensive experiments on six benchmarks demonstrate that FreqPhys yields significant improvements over state-of-the-art approaches, particularly under challenging motion conditions. It highlights the importance of explicitly modeling physiological frequency priors. The source code will be released.
Wei Qian, Dan Guo, Jinxing Zhou +3
Hefei University of Technology · Hefei Comprehensive National Science Center · MBZUAI +2