PhysNeXt: Next-Generation Dual-Branch Structured Attention Fusion Network for Remote Photoplethysmography Measurement

Junzhe Cao; Bo Zhao; Zhiyi Niu; Dan Guo; Yue Sun; Haochen Liang; Yong Xu; Zitong YU

cs.CVMar 20, 2026

PhysNeXt: Next-Generation Dual-Branch Structured Attention Fusion Network for Remote Photoplethysmography Measurement

Authors: Junzhe Cao, Bo Zhao, Zhiyi Niu, Dan Guo, Yue Sun, Haochen Liang, Yong Xu, Zitong YU

Organizations: Harbin Institute of Technology, Shenzhen · Great Bay University · Hefei University of Technology · Macao Polytechnic University

Abstract

Remote photoplethysmography (rPPG) enables contactless measurement of heart rate and other vital signs by analyzing subtle color variations in facial skin induced by cardiac pulsation. Current rPPG methods are mainly based on either end-to-end modeling from raw videos or intermediate spatial-temporal map (STMap) representations. The former preserves complete spatiotemporal information and can capture subtle heartbeat-related signals, but it also introduces substantial noise from motion artifacts and illumination variations. The latter stacks the temporal color changes of multiple facial regions of interest into compact two-dimensional representations, significantly reducing data volume and computational complexity, although some high-frequency details may be lost. To effectively integrate the mutual strengths, we propose PhysNeXt, a dual-input deep learning framework that jointly exploits video frames and STMap representations. By incorporating a spatio-temporal difference modeling unit, a cross-modal interaction module, and a structured attention-based decoder, PhysNeXt collaboratively enhances the robustness of pulse signal extraction. Experimental results demonstrate that PhysNeXt achieves more stable and fine-grained rPPG signal recovery under challenging conditions, validating the effectiveness of joint modeling of video and STMap representations. The codes will be released.

Figures & tables

Figure 1: The comparison between (a) Video input, (b) STMap input, and (c) our proposed Video and STMap Dual-Stream input network.

Figure 2: The overall framework of PhysNeXt. PhysNeXt is a dual-stream network that processes video and STMap inputs in parallel. The video and STMap branches use PhysNet and RhythmNet as their backbones, respectively. In the STMap branch, Spatio-Temporal Difference Modeling Units (SDMUs) are embedded at multiple layers to enhance spatio-temporal feature extraction. The two branches interact at multiple temporal scales through Confidence-Gated Exchange Blocks, enabling feature alignment and information exchange. Finally, a Structured Attention Fusion Decoder fuses features from both streams to predict accurate rPPG signals.

Method	UBFC-RPPG			PURE			BUAA-MIHR			MMPD
	$\mathrm{MAE}\!\downarrow$	$\mathrm{RMSE}\!\downarrow$	$R\!\uparrow$	$\mathrm{MAE}\!\downarrow$	$\mathrm{RMSE}\!\downarrow$	$R\!\uparrow$	$\mathrm{MAE}\!\downarrow$	$\mathrm{RMSE}\!\downarrow$	$R\!\uparrow$	$\mathrm{MAE}\!\downarrow$	$\mathrm{RMSE}\!\downarrow$	$R\!\uparrow$
GREEN △ [ 28 ]	19.73	31.00	0.37	10.09	23.85	0.34	6.89	10.39	0.60	21.68	27.69	-0.01
ICA △ [ 19 ]	16.00	25.65	0.44	4.77	16.07	0.72	–	–	–	18.60	24.30	0.01
CHROM △ [ 4 ]	4.06	8.83	0.89	5.77	14.93	0.81	–	–	–	13.66	18.76	0.08
POS △ [ 31 ]	4.08	7.72	0.92	3.67	11.82	0.88	–	–	–	12.36	17.71	0.18
Meta-rPPG ⋆ [ 7 ]	5.97	7.42	0.57	2.52	4.63	0.98	–	–	–	–	–	–

Table 1: Intra-dataset testing results on UBFC-RPPG [ 1 ] , PURE [ 22 ] , BUAA-MIHR [ 33 ] , and MMPD [ 26 ] . The symbols △ and ⋆ denote the traditional and deep learning-based methods, respectively. Best results are marked in bold .

Method	Other $\rightarrow$ U			Other $\rightarrow$ P			Other $\rightarrow$ B			Other $\rightarrow$ M
	$\mathrm{MAE}\!\downarrow$	$\mathrm{RMSE}\!\downarrow$	$R\!\uparrow$	$\mathrm{MAE}\!\downarrow$	$\mathrm{RMSE}\!\downarrow$	$R\!\uparrow$	$\mathrm{MAE}\!\downarrow$	$\mathrm{RMSE}\!\downarrow$	$R\!\uparrow$	$\mathrm{MAE}\!\downarrow$	$\mathrm{RMSE}\!\downarrow$	$R\!\uparrow$
NEST [ 15 ]	17.24	20.58	0.06	19.26	26.11	-0.06	9.19	12.38	0.19	13.97	18.20	0.15
GREIP [ 40 ]	17.50	20.42	0.17	5.07	14.50	0.78	7.94	10.93	-0.03	13.02	17.11	0.14
EfficientPhys [ 12 ]	12.87	18.80	0.19	7.15	15.04	0.23	32.30	34.00	-0.03	12.87	18.80	0.19
PhysFormer [ 39 ]	10.29	18.13	0.60	19.75	24.30	0.24	22.09	26.21	0.03	13.90	19.30	0.06
PhysNet [ 36 ]	13.83	23.66	0.35	33.23	35.25	-0.15	12.75	16.37	0.08	13.37	16.64	0.29

Table 2: Multi-domain generalization evaluation under the leave-one-out protocol. U=UBFC-RPPG [ 1 ] , P=PURE [ 22 ] , B=BUAA-MIHR [ 33 ] , and M=MMPD [ 26 ] . Best results are marked in bold .

Method	U+P $\rightarrow$ B			M+P $\rightarrow$ B			U+M $\rightarrow$ B			Average
	$\mathrm{MAE}\!\downarrow$	$\mathrm{RMSE}\!\downarrow$	$R\!\uparrow$	$\mathrm{MAE}\!\downarrow$	$\mathrm{RMSE}\!\downarrow$	$R\!\uparrow$	$\mathrm{MAE}\!\downarrow$	$\mathrm{RMSE}\!\downarrow$	$R\!\uparrow$	$\mathrm{MAE}\!\downarrow$	$\mathrm{RMSE}\!\downarrow$	$R\!\uparrow$
PhysNet [ 36 ]	15.34	21.48	-0.29	20.97	24.75	0.01	11.40	16.72	0.14	15.90	20.98	-0.05
PhysFormer [ 39 ]	18.23	22.17	0.07	14.86	18.26	0.03	10.87	16.20	0.08	14.65	18.88	0.06
EfficientPhys [ 12 ]	4.60	8.06	0.72	4.15	7.14	0.77	3.00	5.18	0.89	3.92	6.79	0.79
RhythmFormer [ 41 ]	3.90	6.51	0.82	4.32	6.70	0.82	6.20	11.23	0.49	4.81	8.15	0.71
NEST [ 15 ]	11.19	14.36	-0.12	8.67	11.38	0.05	9.14	12.09	-0.04	9.67	12.60	-0.04

Table 3: Limited-source domain generalization evaluation on BUAA-MIHR. Models are trained on limited source domains. U=UBFC-RPPG [ 1 ] , P=PURE [ 22 ] , B=BUAA-MIHR [ 33 ] , and M=MMPD [ 26 ] . Best results are marked in bold .

Method	U+P $\rightarrow$ M			B+P $\rightarrow$ M			U+B $\rightarrow$ M			Average
	$\mathrm{MAE}\!\downarrow$	$\mathrm{RMSE}\!\downarrow$	$R\!\uparrow$	$\mathrm{MAE}\!\downarrow$	$\mathrm{RMSE}\!\downarrow$	$R\!\uparrow$	$\mathrm{MAE}\!\downarrow$	$\mathrm{RMSE}\!\downarrow$	$R\!\uparrow$	$\mathrm{MAE}\!\downarrow$	$\mathrm{RMSE}\!\downarrow$	$R\!\uparrow$
PhysNet [ 36 ]	11.00	17.30	0.28	13.20	16.70	0.23	13.50	17.00	0.09	12.57	17.00	0.20
PhysFormer [ 39 ]	11.40	17.50	0.23	13.90	18.60	0.21	13.20	16.50	0.12	12.83	17.53	0.19
EfficientPhys [ 12 ]	11.80	18.90	0.22	11.90	18.50	0.21	15.50	20.80	0.03	13.07	19.40	0.15
RhythmFormer [ 41 ]	10.50	16.72	0.28	13.98	19.46	0.12	12.57	17.45	0.15	12.35	17.88	0.18
NEST [ 15 ]	14.01	18.12	0.12	12.82	17.26	0.22	13.29	16.58	0.16	13.37	17.32	0.17

Table 4: Limited-source domain generalization evaluation on MMPD. Models are trained on limited combinations of source domains. Best results are marked in bold .

Figure 3: Ablations on PhysNeXt architecture, covering dual-stream configurations, temporal exchange module (DCEB), decoder structure, and the number of query tokens.

Figure 4: Visualization results on the MMPD dataset. Left: Scatter plot of predicted heart rate versus ground truth. Upper right: Time-domain comparison between the predicted and ground truth rPPG waveforms. Lower right: Frequency-domain power spectral density (PSD) comparison.

Method	Params (M)	MACs (G)
DeepPhys [ 2 ]	7.50	96.0
EfficientPhys [ 12 ]	7.40	45.6
PhysFormer [ 39 ]	7.38	40.5
RhythmFormer [ 41 ]	4.21	28.8
Contrast-Phys+ [ 25 ]	0.85	145.7
PhysNet [ 36 ]	0.77	56.1

Table 5: Efficiency of different rPPG methods.

Remote Photoplethysmography

Remote photoplethysmography (rPPG) estimates pulse and other physiological signals from video without contact sensors.

See all 46 papers

FreqPhys: Repurposing Implicit Physiological Frequency Prior for Robust Remote Photoplethysmography
Apr 1, 2026Wei Qian, Dan Guo, Jinxing Zhou +3Remote Photoplethysmography Frequency-Domain Feature Learning
Hefei University of Technology · Hefei Comprehensive National Science Center · MBZUAI +2
Rhythm-Structured Predictive Learning for Remote Photoplethysmography
Jun 30, 2026Ba-Thinh Nguyen, Huu-Dung Nguyen, Thi-Duyen Ngo +1Remote Photoplethysmography Mamba
VNU University of Engineering and Technology, Hanoi, Vietnam
PhysFlow: Frequency Decoupled with Dual-Field Rectified Flow for Remote Photoplethysmography
Jun 22, 2026Zixu Li, jianjun Qian, Hang Shao +2Remote Photoplethysmography
PCA Lab, Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China · College of Computer Science and Technology, Qingdao University, Qingdao 266071, China

Apr 1, 2026J/K move · Enter open · S save

FreqPhys: Repurposing Implicit Physiological Frequency Prior for Robust Remote Photoplethysmography

Wei Qian, Dan Guo, Jinxing Zhou, Bochao Zou+2

Hefei University of Technology · Hefei Comprehensive National Science Center · MBZUAI+2

Remote Photoplethysmography Frequency-Domain Feature Learning

Open paper AskPDF arXiv

Remote photoplethysmography (rPPG) enables contactless physiological monitoring by capturing subtle skin-color variations from facial videos. However, most existing methods predominantly rely on time-domain modeling, making them vulnerable to motion artifacts and illumination fluctuations, where weak physiological clues are easily overwhelmed by noise. To address these challenges, we propose FreqPhys, a frequency-guided rPPG framework that explicitly leverages physiological frequency priors for robust signal recovery. Specifically, FreqPhys first applies a Physiological Bandpass Filtering module to suppress out-of-band interference, and then performs Physiological Spectrum Modulation together with adaptive spectral selection to emphasize pulse-related frequency components while suppress residual in-band noise. A Cross-domain Representation Learning module further fuses these spectral priors with deep time-domain features to capture informative spatial--temporal dependencies. Finally, a frequency-aware conditional diffusion process progressively reconstructs high-fidelity rPPG signals. Extensive experiments on six benchmarks demonstrate that FreqPhys yields significant improvements over state-of-the-art approaches, particularly under challenging motion conditions. It highlights the importance of explicitly modeling physiological frequency priors. The source code will be released.

Abstract

Remote Photoplethysmography

Explore similar work

FreqPhys: Repurposing Implicit Physiological Frequency Prior for Robust Remote Photoplethysmography

Rhythm-Structured Predictive Learning for Remote Photoplethysmography

PhysFlow: Frequency Decoupled with Dual-Field Rectified Flow for Remote Photoplethysmography