Pixel-wise Exposure for Highly Robust In-Vehicle Remote-PPG
Authors: Jieying Wang, Xinqi Cai, Caifeng Shan, Wenjin Wang
Organizations: College of Computer Science and Engineering, Shandong University of Science and Technology, Qingdao 266590, China · Department of Biomedical Engineering, College of Engineering, Southern University of Science and Technology, Shenzhen 518000, China · State Key Laboratory for Novel Software Technology and School of Intelligence Science and Technology, Nanjing University, Nanjing 210023, China
Remote photoplethysmography (rPPG) offers a promising non-contact solution for heart rate monitoring, yet its real-world robustness is fundamentally limited by an inherent hardware limitation: existing camera exposure control paradigms, whether fixed or auto-exposure, impose a uniform exposure time across all pixels within a frame. In high-dynamic-range scenes such as automotive cabins with strong directional sunlight, this spatially invariant exposure constraint inevitably leads to localized facial overexposure or underexposure, irreversibly corrupting the subtle pulsatile signals essential for rPPG at the point of capture, a physical degradation that no downstream algorithm can recover. To overcome this bottleneck, we propose PixExpo (Pixel-wise Exposure), a "temporal-for-spatial" framework that sequentially captures frames under a predefined cyclic exposure schedule and performs non-iterative pixel-wise fusion. At each pixel location, PixExpo selects the observation closest to an rPPG-motivated target intensity. This criterion seeks to reduce local saturation and severe underexposure rather than optimize perceptual appearance. PixExpo requires no sensor modification but assumes programmable frame-level exposure control. We validate the proposed PixExpo framework using our newly introduced MEX-Drive dataset, comprising 48 participants under real-world driving conditions. Experimental results demonstrate that PixExpo outperforms manufacture-default auto-exposure methods, reducing the mean absolute error (MAE) by 7.21 bpm (from 13.94 to 6.73 bpm) and increasing the success rate by 37.29 percentage points (from 25.95% to 63.24%) across challenging driving scenarios.
Figures & tables
Fig. 1 : Overview of illumination challenges for in-vehicle rPPG and the proposed PixExpo framework. (a) Representative high-dynamic-range (HDR) driving scenarios from the MEX-Drive dataset. (b) Multi-exposure acquisition and pixel-wise fusion pipeline of PixExpo.
TABLE I : Comparison of Existing Exposure Control Strategies and the Proposed PixExpo
Fig. 2 : Experimental setup and characteristics of MEX-Drive. (a) Dual-camera deployment and reference ECG acquisition; (b) Representative high-dynamic-range driving conditions; (c) Age and reference-HR distributions of the 48 participants.
Fig. 3 : Participant-level distributions of MAE, SNR, and SR for four exposure and fusion strategies ( n=48 ). The internal box plots show the median (red line), mean (white dot), and interquartile range.
Fig. 4 : Visual and time-frequency comparison of four exposure and fusion strategies. The first row shows representative frames, and the second row shows the corresponding time-frequency representations. The black dashed curve denotes the estimated HR, and the white curve denotes the ECG-derived reference HR. HR MAE (bpm), SNR (dB), and SR (%) are reported below each method.
PixExpo
Fixed Exposure
Full-frame AE
Estimator
MAE ↓
RMSE ↓
SR ↑
MAE ↓
RMSE ↓
SR ↑
MAE ↓
RMSE ↓
SR ↑
EfficientPhys [ 21 ]
8.30
13.30
57.91
10.80
15.38
43.87
12.05
16.72
38.21
FactorizePhys [ 12 ]
4.11
7.76
79.44
7.09
11.36
61.31
8.91
13.36
51.04
PhysFormer [ 43 ]
8.12
12.93
58.14
10.56
15.82
47.10
10.52
15.24
45.27
iBVPNet [ 13 ]
8.38
13.51
61.76
11.08
15.99
46.12
12.10
16.71
40.61
TABLE II : Cross-dataset HR estimation performance on all 48 MEX-Drive participants using PURE-pretrained rPPG models without fine-tuning. Bold values indicate the best result among the three acquisition strategies for each estimator and metric.
Fig. 5 : Performance comparison of Fixed Exposure, Face-ROI AE, Skin-ROI AE, and PixExpo using POS, CHROM, and ICA in the additional seven-participant experiment. Light lines connect paired participant-level results, and black markers show the mean ± SEM.
Fig. 6 : Participant-level comparison of fixed exposure, global adaptive exposure, and PixExpo in Scenario II ( n=48 ). Grey lines connect paired results from the same participant; violin plots show the corresponding distributions of MAE and SR.
Fig. 7 : Mean MAE and SR of fixed exposure, global adaptive exposure, and PixExpo under rainy and sunny conditions.
Fig. 8 : Representative video frames and corresponding rPPG spectrograms under four in-vehicle illumination conditions. Rows 1–4 show sunset, sunny with head rotation, overcast, and rainy conditions, respectively. Columns (a)–(c) show fixed exposure, global adaptive exposure, and PixExpo. The MAE and SR are reported below each spectrogram.
N
Total rate (fps)
Exposure times (ms)
MAE (bpm)
SNR (dB)
SR (%)
2
30
16, 32
3.75
4.92
90.15
3
45
7, 14, 21
4.59
3.30
81.76
6
90
1.8×{1,…,6}
6.99
1.19
67.76
TABLE III : Effect of Exposure Channel Count under Controlled Low-Light Conditions
Fusion Strategy
MAE (bpm) ↓
SR (%) ↑
SNR (dB) ↑
One-hot
6.73 ± 3.56
63.24 ± 18.95
0.60 ± 3.27
Gaussian decay
7.56 ± 3.77
57.90 ± 19.25
0.15 ± 3.23
Inverse weighting
9.83 ± 4.51
47.43 ± 20.73
−1.34±3.52
Friedman p
<0.001
<0.001
<0.001
Kendall’s W
0.365
0.730
0.875
TABLE IV : Participant-level Comparison of Fusion Weighting Strategies ( n=48 )
Fig. 9 : Visual and time–frequency comparison of fusion weighting strategies.
Fig. 10 : Representative comparison under transient driving disturbances. The six frames shown above are representative frames selected from the time interval highlighted by the red dashed rectangle in each row.
Fig. 11 : Visual effect of inter-frame registration on Mertens fusion and PixExpo in a representative example.
Fig. 12 : Spatial and spectral characteristics of exposure switching in a representative example. (a) PSR map, showing higher switching rates around facial boundaries and other high-gradient regions and lower switching rates over the forehead and cheeks. (b) PSDs of the recovered rPPG signal, FSR, and mean selection index. The recovered rPPG signal peaks at 1.52 Hz (91.0 bpm), close to the mean reference HR of 1.56 Hz.
Remote photoplethysmography (rPPG) holds great promise for continuous heart-rate monitoring of drivers in intelligent vehicles. However, its performance is severely degraded by the highly dynamic illumination changes. A critical yet overlooked factor is the lack of exposure controlling during video acquisition -- most existing systems rely on either fixed exposure settings or camera build-in auto-exposure, both of which fail to maintain stable facial brightness under rapidly changing lighting conditions during driving. To address this gap, we propose a highly-adaptive exposure controlling framework that proactively adjusts exposure parameters based on predictive modeling of historical skin reflections. Unlike standard auto-exposure, our method is specifically optimized for rPPG measurement, ensuring the skin region of interest (ROI) remains within the optimal dynamic range for rPPG signal extraction. As an important contribution of this study, we introduce ExpDrive, a public in-vehicle physiological monitoring dataset comprising synchronized facial video and reference ECG from 48 subjects captured under real driving conditions. Extensive experiments demonstrate that our method consistently outperforms fixed exposure and standard auto-exposure strategies. Specifically, it reduces the Mean Absolute Error (MAE) by 6.31 bpm (from 14.1 to 7.79 bpm) and significantly increases the success rate by 32.3 percentage points (p < 0.001) (from 24.9% to 57.2%) across challenging driving scenarios. Notably, it clearly improved the performance of non-contact heart-rate monitoring in both low-light (rainy) and high-glare (sunny) conditions, validating the efficacy of exposure-aware acquisition design.
Jieying Wang, Xinqi Cai, Caifeng Shan +1
Department of Biomedical Engineering, College of Engineering, Southern University of Science and Technology, Shenzhen 518000, China · State Key Laboratory for Novel Software Technology and School of Intelligence Science and Technology, Nanjing University, Nanjing 210023, China
Remote photoplethysmography (rPPG) is a camera-based technique for measuring physiological signals, particularly cardiac activity. From the remotely measured signals, heart rate can be estimated, which is crucial for health monitoring. In this study, we investigate a driver health monitoring system based on remote heart rate estimation. However, driving environments represent uncontrolled settings where videos are subject to varying illumination conditions and frequent head movements. We introduce MS-rPPG, a multi-spectral framework that combines RGB with near-infrared (NIR) face video to alleviate rPPG estimation under challenging driving conditions. To combine the complementary features from two spectral videos, we propose a cross-spectral linear modulation (CSLM) strategy based on frequency-domain analysis. Moreover, we introduce MS-Mamba, a novel state space model designed to effectively model long-range temporal dependencies while jointly capturing cross-channel interactions between multi-spectral features. We collected a real-world dataset called MS-Drive, which was recorded from 50 participants while driving the vehicle. The proposed method was evaluated on the MR-NIRP Car dataset and MS-Drive datasets. The experimental results indicate that MS-rPPG shows better robustness and heart rate estimation accuracy than previous methods, highlighting its promise for driver health monitoring. The codes are available at github.com/ziiho08/MS-rPPG.
Jiho Choi, Sang Jun Lee
Division of Electronics and Information Engineering, Jeonbuk National University, Republic of Korea
Physiological awareness is important for service, social, and assistive robots that interact with humans in everyday environments. Remote photoplethysmography (rPPG) enables non-contact heart-rate (HR) estimation from an RGB camera, making it a promising sensing modality for robot-mounted vision systems. However, illumination variation remains a major barrier to robust deployment. This paper presents an end-to-end spatial-temporal transformer framework for remote HR estimation on a new dataset with varied illumination. Our estimator integrates PRNet-based 3D face alignment, clip-level illumination augmentation, the Residual Temporal Standardization Module, and controlled hybrid temporal-frequency supervision. The training objective combines a Soft-Shifted Pearson waveform loss with a spectral Kullback-Leibler divergence loss, where a tuned weight (β) controls the contribution of frequency-domain heart-rate guidance. Experiments on a static all-level mix protocol covering three illumination levels show that β=5 provides the strongest result among the tested beta settings, achieving a best-run HR mean absolute error (MAE) of 0.79 bpm and an HR correlation of 0.982. Compared with the PhysFormer baseline evaluated on our dataset, our estimator reduces HR MAE by 93.6 %, while increasing HR correlation from 0.088 to 0.982, making it usable when illumination varies.
Zhi Wei Xu, Torbjörn E. M. Nordling
Department of Mechanical Engineering, National Cheng Kung University, Tainan, Taiwan.