Organizations: VNU University of Engineering and Technology, Hanoi, Vietnam · College of Engineering and Computer Science, VinUniversity, Hanoi, Vietnam · VinUni-Illinois Smart Health Center, VinUniversity, Hanoi, Vietnam · Center for Innovations in Health Sciences, VinUniversity, Hanoi, Vietnam
Remote photoplethysmography (rPPG) enables contactless physiological sensing from facial videos by analyzing subtle appearance variations induced by blood circulation. However, modeling the temporal dynamics of these signals remains challenging, as many deep learning methods rely on temporal shifting or convolutional operators that aggregate information primarily from neighboring frames, resulting in predominantly local temporal modeling and limited temporal receptive fields. To address this limitation, we propose BTS-rPPG, a temporal modeling framework based on Orthogonal Butterfly Temporal Shifting (BTS). Inspired by the butterfly communication pattern in the Fast Fourier Transform (FFT), BTS establishes structured frame interactions via an XOR-based butterfly pairing schedule, progressively expanding the temporal receptive field and enabling efficient propagation of information across distant frames. Furthermore, we introduce an orthogonal feature transfer mechanism (OFT) that filters the source feature with respect to the target context before temporal shifting, retaining only the orthogonal component for cross-frame transmission. This reduces redundant feature propagation and encourages complementary temporal interaction. Extensive experiments on multiple benchmark datasets demonstrate that BTS-rPPG improves long-range temporal modeling of physiological dynamics and consistently outperforms existing temporal modeling strategies for rPPG estimation.
Figures & tables
Figure 1: Comparison of temporal communication patterns between (a) existing methods and (b) the proposed BTS-rPPG. Existing methods mainly exchange features between adjacent frames, limiting long-range temporal interaction. In contrast, BTS-rPPG employs a butterfly-inspired hierarchical communication scheme, analogous to FFT-style sparse multi-stage mixing, to progressively expand temporal interaction from local to global scales. This structure is well suited to rPPG signals, which exhibit both short-term continuity and longer-range periodic dependency.
Figure 2: Overview of the proposed BTS-rPPG. I) The framework comprises (a) an input representation module, (b) a BTS-enhanced Swin Transformer backbone, and (c) a predictor head. RGB frames and their normalized difference frames are fused and embedded into patch-level tokens, which are then processed by Swin Transformer stages equipped with BTS and finally regressed to the target rPPG waveform. II) OFT suppresses redundant transferred information by preserving only the feature component orthogonal to the target context, thereby promoting complementary temporal exchange. III) BTS establishes a butterfly-inspired hierarchical communication pattern, where temporal interactions are progressively expanded across stages to enable efficient propagation from short-range to long-range dependencies.
Method
PURE
UBFC-rPPG
MMPD
MAE ↓
MAPE ↓
RMSE ↓
r↑
MAE ↓
MAPE ↓
RMSE ↓
r↑
MAE ↓
MAPE ↓
RMSE ↓
r↑
LGI [ 15 ]
3.59
3.37
14.66
0.79
5.39
11.08
15.09
0.81
16.63
18.77
23.06
0.11
CHROM [ 3 ]
5.39
11.08
15.09
0.81
4.06
3.34
8.83
0.89
13.73
16.95
18.88
0.15
POS [ 23 ]
0.36
0.50
0.93
1.00
4.08
3.93
7.72
0.92
15.61
18.28
21.40
0.14
DeepPhys [ 2 ]
3.33
2.91
14.45
0.90
0.76
0.79
1.09
0.99
23.73
25.63
28.25
-0.06
PhysNet [ 26 ]
0.93
1.40
2.08
0.99
2.25
2.37
4.81
0.94
4.81
4.84
11.83
0.60
Table 1: Intra-dataset evaluation results on PURE [ 21 ] , UBFC-rPPG [ 1 ] , and MMPD [ 22 ] . Lower is better for MAE, MAPE, and RMSE, while higher is better for Pearson correlation r . Best results are highlighted in bold and second-best results are underlined.
Model
PURE → UBFC-rPPG
UBFC-rPPG → PURE
PURE → MMPD
UBFC-rPPG → MMPD
MAE ↓
MAPE ↓
RMSE ↓
r↑
MAE ↓
MAPE ↓
RMSE ↓
r↑
MAE ↓
MAPE ↓
RMSE ↓
r↑
MAE ↓
MAPE ↓
RMSE ↓
r↑
DeepPhys [ 2 ]
1.21
1.42
2.90
0.99
8.06
13.67
19.71
0.61
16.92
18.54
24.61
0.05
17.50
19.27
25.00
0.05
PhysNet [ 26 ]
0.98
1.12
2.48
0.99
3.69
3.39
13.80
0.82
13.22
14.73
19.61
0.23
10.24
12.46
16.54
0.29
TSCAN [ 9 ]
1.30
1.50
2.87
0.99
12.92
23.92
24.36
0.47
13.94
15.14
21.61
0.20
14.01
15.48
21.04
0.24
PhysFormer [ 27 ]
1.44
1.66
3.77
0.98
12.92
23.92
24.36
0.47
14.57
16.73
20.71
0.15
12.10
15.41
17.79
0.17
EfficientPhys [ 10 ]
2.07
2.10
6.32
0.94
5.47
5.40
17.04
0.71
14.03
15.32
21.62
0.17
13.78
15.15
22.25
0.09
Table 2: Cross-dataset test results. PURE → UBFC-rPPG means the model is trained on PURE and tested on UBFC-rPPG. Lower is better for MAE/MAPE/RMSE, while higher is better for Pearson correlation r .
Figure 3: Ablation study on intra-dataset MMPD [ 22 ] . This experiment aims to examine how OFT and BTS each contribute to improving the baseline TCS [ 8 ] framework.
Figure 4: Sensitivity analysis of the OFT fold ratio on (a) PURE [ 21 ] and (b) UBFC-rPPG [ 1 ] datasets. We report MAE and RMSE, and find that the best performance is achieved at Cs/d=1/4 , which provides sufficient cross-frame transfer while retaining adequate target-specific representation.
Method
#Param (M) ↓
MACs (G) ↓
Throughput (Kfps) ↑
Latency (ms) ↓
MTTS-CAN [ 9 ]
4.33
45.85
3.99
50.03
EfficientPhys [ 10 ]
2.16
22.96
7.21
27.72
PhysFormer [ 27 ]
7.38
60.72
2.27
84.50
LSTS [ 6 ]
6.37
71.06
1.30
147.68
Reperio-rPPG [ 14 ]
6.50
71.02
1.47
130.18
BTS-rPPG (Ours)
5.98
69.29
2.05
98.44
Table 3: Efficiency comparison of BTS-rPPG against representative baselines. We report the number of parameters, computational cost measured by MACs, throughput, and inference latency under the same benchmarking setting.
Variant
s1
s2
s3
s4
s5
s6
RMSE ↓
Local-only
1
1
1
1
1
1
14.60
Linear
1
2
3
4
5
6
12.18
Reverse Butterfly
32
16
8
4
2
1
11.03
Proposed Butterfly
1
2
4
8
16
32
9.74
Table 4: Design analysis of BTS under different temporal pairing schedules on the intra-dataset MMPD [ 22 ] benchmark.
Remote photoplethysmography (rPPG) estimates the blood volume pulse (BVP) signal from facial videos, enabling contact-free health monitoring. Conventional clip-wise approaches, which use video clips as input, require capturing over one hundred frames before inference, thus introducing several seconds of delay and hindering real-time use. Meanwhile, frame-wise approaches struggle to capture long-range temporal and periodic features of physiological rhythms, and therefore lead to reduced estimation accuracy. To overcome these issues, we propose StreamPPG, a unified architecture that enables low-latency frame-wise physiological signal estimation while achieving competitive accuracy compared with clip-wise approaches. StreamPPG is trained under a consistent privileged learning (CPL) strategy, which leverages ground-truth rPPG signals as privileged information to enhance the model's representation capability. Extensive experiments demonstrate that StreamPPG achieves state-of-the-art accuracy across multiple datasets while maintaining real-time throughput on edge devices.
Remote photoplethysmography (rPPG) enables contactless physiological monitoring by capturing subtle skin-color variations from facial videos. However, most existing methods predominantly rely on time-domain modeling, making them vulnerable to motion artifacts and illumination fluctuations, where weak physiological clues are easily overwhelmed by noise. To address these challenges, we propose FreqPhys, a frequency-guided rPPG framework that explicitly leverages physiological frequency priors for robust signal recovery. Specifically, FreqPhys first applies a Physiological Bandpass Filtering module to suppress out-of-band interference, and then performs Physiological Spectrum Modulation together with adaptive spectral selection to emphasize pulse-related frequency components while suppress residual in-band noise. A Cross-domain Representation Learning module further fuses these spectral priors with deep time-domain features to capture informative spatial--temporal dependencies. Finally, a frequency-aware conditional diffusion process progressively reconstructs high-fidelity rPPG signals. Extensive experiments on six benchmarks demonstrate that FreqPhys yields significant improvements over state-of-the-art approaches, particularly under challenging motion conditions. It highlights the importance of explicitly modeling physiological frequency priors. The source code will be released.
Wei Qian, Dan Guo, Jinxing Zhou +3
Hefei University of Technology · Hefei Comprehensive National Science Center · MBZUAI +2
Remote photoplethysmography (rPPG) enables contactless measurement of heart rate and other vital signs by analyzing subtle color variations in facial skin induced by cardiac pulsation. Current rPPG methods are mainly based on either end-to-end modeling from raw videos or intermediate spatial-temporal map (STMap) representations. The former preserves complete spatiotemporal information and can capture subtle heartbeat-related signals, but it also introduces substantial noise from motion artifacts and illumination variations. The latter stacks the temporal color changes of multiple facial regions of interest into compact two-dimensional representations, significantly reducing data volume and computational complexity, although some high-frequency details may be lost. To effectively integrate the mutual strengths, we propose PhysNeXt, a dual-input deep learning framework that jointly exploits video frames and STMap representations. By incorporating a spatio-temporal difference modeling unit, a cross-modal interaction module, and a structured attention-based decoder, PhysNeXt collaboratively enhances the robustness of pulse signal extraction. Experimental results demonstrate that PhysNeXt achieves more stable and fine-grained rPPG signal recovery under challenging conditions, validating the effectiveness of joint modeling of video and STMap representations. The codes will be released.
Junzhe Cao, Bo Zhao, Zhiyi Niu +5
Harbin Institute of Technology, Shenzhen · Great Bay University · Hefei University of Technology +1