Design, Implementation and Evaluation of a Real-Time Remote Photoplethysmography (rPPG) Acquisition System for Non-Invasive Vital Sign Monitoring
Authors: Constantino Álvarez Casado, Sasan Sharifipour, Manuel Lage Cañellas, Nhi Nguyen, Le Nguyen, Miguel Bordallo López
Organizations: Center for Machine Vision and Signal Analysis (CMVS), University of Oulu, Oulu, Finland. · VTT Technical Research Centre of Finland, Oulu, Finland.
The growing integration of smart environments and low-power computing devices, coupled with mass-market sensor technologies, is driving advancements in remote and non-contact-based physiological monitoring. However, deploying these systems in real-time on resource-constrained platforms introduces significant challenges related to scalability, interoperability, and performance. This paper presents a computationally efficient real-time remote photoplethysmography (rPPG) system, designed to extract physiological signals, such as heart rate, respiratory rate, and oxygen saturation (SpO2), from facial video streams. The system is built on the Face2PPG pipeline, which processes video frames sequentially for rPPG signal extraction and analysis, while leveraging a multithreaded architecture to manage video capture, real-time processing, network communication, and graphical user interface updates concurrently. This design targets continuous operation at 30 frames per second (FPS), with adaptive feedback through a collaborative user interface to guide optimal signal capture conditions. The network interface includes both an HTTP server for continuous video streaming and a RESTful API for on-demand vital sign retrieval. To ensure accurate performance despite limited computational resources, we use a hybrid programming model combining Functional Reactive Programming (FRP) and the Actor Model, allowing event-driven processing and efficient task parallelization. We evaluate HR accuracy on three public datasets (COHFACE, LGI-PPGI, and UBFC-RPPG), and conduct preliminary evaluations of RR on COHFACE and SpO2 on PURE. We also assess the processing time and end-to-end operation of the system on two computers, including live HR and SpO2 agreement with a medical-grade fingertip pulse oximeter across five 10-minute sessions. Our work addresses key challenges in real-time contactless biosignal monitoring.
Figures & tables
Figure 1: Architecture overview of the proposed real-time rPPG application, comprised of several interconnected components, each executing a specific task.
Figure 2: Implementation design of the proposed real-time rPPG application.
Face2PPG-RT
Face2PPG-RT
Face2PPG-Server
Face2PPG-Server
Module
Config. 1
Config. 2
Normalized
Multiregion
Face Detection
Dlib HOG
OpenCV YuNet
OpenCV SSD
OpenCV SSD
Face Alignment
Dlib ERT GTX
Dlib ERT GTX
DAN
DAN
Face Normalization
Mesh (131 triangles)
Mesh (131 triangles)
Mesh (131 triangles)
Mesh (131 triangles)
Skin Segmentation
Geometrical (Patches)
Geometrical (Patches)
Geometrical (Whole Face)
Geometrical (Best Regions)
RGB to BVP
Lab
Lab
POS
POS
Table 1: Configuration setups and modules for real-time and cloud-based rPPG systems. In the C++ application used for the speed evaluation, the face normalization uses a mesh of 122 triangles, and the spectral analysis of both configurations uses the FFT with a Hann window.
LGI-PPGI
COHFACE
UBFC1
UBFC2
Pipeline
MAE ± SD (bpm)
PCC
MAE ± SD (bpm)
PCC
MAE ± SD (bpm)
PCC
MAE ± SD (bpm)
PCC
Face2PPG-Server Normalized
8.7 ± 8.4
0.50
9.4 ± 4.8
0.10
1.2 ± 0.4
0.94
1.4 ± 1.5
0.95
Face2PPG-Server Multiregion
4.5 ± 3.3
0.57
8.0 ± 4.4
0.06
0.9 ± 0.4
0.96
0.9 ± 0.9
0.98
Face2PPG-RT Configuration 1
6.4 ± 6.8
0.45
10.9 ± 5.6
-0.04
1.7 ± 0.6
0.80
3.8 ± 4.7
0.72
Face2PPG-RT Configuration 2
5.9 ± 8.0
0.49
11.4 ± 7.3
-0.01
2.1 ± 1.1
0.83
6.0 ± 6.3
0.54
Table 2: Heart rate (HR) estimation error comparison between the four different configurations of the proposed rPPG system on three databases, with UBFC-RPPG divided into the UBFC1 and UBFC2 subsets. The table includes two configurations of the real-time system and two configurations for high-performance computers without any constraint. MAE and SD are expressed in beats per minute (bpm); PCC is dimensionless.
Face2PPG-RT Configuration 1
Face2PPG-RT Configuration 2
Configuration 2 − Configuration 1
Dataset
Rec. (Part.)
MAE [95% CI]
RMSE
MAE [95% CI]
RMSE
Δ MAE [95% CI]
p
dz
LGI-PPGI
24 (6)
6.4 [4.3, 8.6]
8.4
5.9 [3.6, 9.1]
7.6
− 0.5 [ − 1.5, 0.7]
0.44
− 0.34
COHFACE
160 (40)
10.9 [9.8, 12.0]
13.5
11.4 [9.8, 13.0]
14.2
0.5 [ − 0.6, 1.7]
0.41
0.13
UBFC1
8 (8)
1.7 [1.3, 2.1]
2.7
2.1 [1.5, 2.9]
3.3
0.4 [ − 0.1, 1.2]
0.30
0.40
UBFC2
42 (42)
3.8 [2.5, 5.4]
6.1
6.0 [4.1, 8.0]
9.0
2.2 [0.8, 3.7]
0.004
0.45
Table 3: Statistical comparison of the HR estimation of the two real-time configurations, evaluated on the same recordings. Rec. (Part.): number of recordings (participants). MAE: mean of the per-recording MAE, with its 95% confidence interval (CI) computed with a participant-level bootstrap. RMSE: mean of the per-recording RMSE. Δ MAE: paired difference between Configuration 2 and Configuration 1, with its 95% CI. p : participant-level sign-flip permutation test. dz : effect size. MAE, RMSE and Δ MAE are expressed in bpm. In UBFC1, each recording is treated as a separate participant.
Face2PPG-RT Configuration 1
Face2PPG-RT Configuration 2
Dataset
Scenario
Rec.
MAE [95% CI]
PCC
MAE [95% CI]
PCC
LGI-PPGI
Resting
6
2.0 [0.5, 4.6]
0.67
1.7 [0.6, 3.7]
0.66
Talking
6
4.4 [3.1, 5.9]
0.32
4.9 [2.2, 8.1]
0.42
Rotation
6
7.2 [3.8, 10.8]
0.26
5.6 [3.7, 7.7]
0.27
Gym
6
11.9 [5.1, 20.8]
0.56
11.3 [3.4, 23.5]
0.60
COHFACE
Studio lighting
80
9.7 [8.6, 10.9]
0.00
9.7 [8.4, 11.1]
0.00
Table 4: HR estimation error of the two real-time configurations for the different scenarios of LGI-PPGI and the two lighting conditions of COHFACE. Rec.: number of recordings. MAE: mean of the per-recording MAE in bpm, with its 95% CI computed with a participant-level bootstrap. PCC: mean of the per-recording PCC (dimensionless).
User equipment configurations
Cloud configurations
RT Config. 1
RT Config. 2
Server-Normalized
Server-Multiregion
Module
ms/frame
ms/frame
ms/frame
ms/frame
Face Detection
5.48
1.10
20.35
20.35
Face Alignment
4.48
3.74
82.52
82.52
Face Normalization
3.67
4.37
12.48
12.48
Skin Segmentation
8.23 us
8.23 us
95.57 us
105.41
Table 5: Performance speed for individual modules in various Face2PPG pipeline configurations, as well as the overall speed per face and frame. Times are given in ms per frame, except where indicated in microseconds (us). Total FPS is the inverse of the total processing time of the core modules, and it does not include camera capture, GUI rendering or network transmission. The Spectral Analysis row includes the computation of HR, RR and SpO2 at each update of the vital signs, although the SpO2 is computed without spectral analysis.
Figure 3: Setup of the live evaluation: the application during live operation on the laptop, in the first session, with the USB webcam on top of the screen, showing the facial landmarks, the HR, SpO2 and RR values and the pulse and respiratory signals, next to the fingertip pulse oximeter (Contec CMS50E) used as reference, connected to the same computer. The terminals behind the application window run the measurement script and a system monitor (htop), whose meters, at the top right, show the processing load distributed across the cores of the processor by the threads of the application.
Laptop (Intel Core i7-6700HQ)
Desktop (AMD Ryzen 7 3700X)
No clients
API + 8 streams
All phases
No clients
API + 8 streams
All phases
Sessions
3
1
3
2
1
2
Duration (s)
1800
180
2520
1200
180
1920
Frame rate and timing
Loop iterations per second
31.25
31.25
31.25
31.25
31.24
31.25
New camera frames per second
29.56
29.51
29.54
29.81
29.80
29.83
Table 6: Live operation of the complete system with a live camera at 30 FPS and one participant at rest, on the laptop of the speed evaluation (Intel Core i7-6700HQ; three sessions, one of them with network load) and on a desktop computer (AMD Ryzen 7 3700X; two sessions, one of them with network load). No clients: phases without network clients. API + 8 streams: requests to the RESTful API at 10 per second together with eight clients of the MJPEG streams, on the same computer. All phases: all the phases of all the sessions, including those with the API requests alone and with one and four stream clients. Values are medians with the 99th percentile in brackets, unless otherwise indicated. Latencies are measured from the timestamp of the camera driver. Loop work: time from the start of a loop iteration to the end of its network step. Agreement: differences with a fingertip pulse oximeter, one paired value per second.
Figure 4: Agreement of the HR of the application with the fingertip pulse oximeter during live operation. (a) HR displayed by the application, pulse rate displayed by the oximeter and beat-to-beat pulse rate obtained from the pulse wave of the oximeter, in the first 300 s of the third session on the laptop. (b) Bland-Altman plot of the 4,440 paired values, one per second, of the five sessions, with the bias (solid lines) and the 95% limits of agreement (dashed lines) of each computer; the 24 differences larger than 20 bpm are drawn at the border.