Pixel-wise Exposure for Highly Robust In-Vehicle Remote-PPG
Authors: Jieying Wang, Xinqi Cai, Caifeng Shan, Wenjin Wang
Organizations: College of Computer Science and Engineering, Shandong University of Science and Technology, Qingdao 266590, China · Department of Biomedical Engineering, College of Engineering, Southern University of Science and Technology, Shenzhen 518000, China · State Key Laboratory for Novel Software Technology and School of Intelligence Science and Technology, Nanjing University, Nanjing 210023, China
Remote photoplethysmography (rPPG) offers a promising non-contact solution for heart rate monitoring, yet its real-world robustness is fundamentally limited by an inherent hardware limitation: existing camera exposure control paradigms, whether fixed or auto-exposure, impose a uniform exposure time across all pixels within a frame. In high-dynamic-range scenes such as automotive cabins with strong directional sunlight, this spatially invariant exposure constraint inevitably leads to localized facial overexposure or underexposure, irreversibly corrupting the subtle pulsatile signals essential for rPPG at the point of capture, a physical degradation that no downstream algorithm can recover. To overcome this bottleneck, we propose PixExpo (Pixel-wise Exposure), a "temporal-for-spatial" framework that sequentially captures frames under a predefined cyclic exposure schedule and performs non-iterative pixel-wise fusion. At each pixel location, PixExpo selects the observation closest to an rPPG-motivated target intensity. This criterion seeks to reduce local saturation and severe underexposure rather than optimize perceptual appearance. PixExpo requires no sensor modification but assumes programmable frame-level exposure control. We validate the proposed PixExpo framework using our newly introduced MEX-Drive dataset, comprising 48 participants under real-world driving conditions. Experimental results demonstrate that PixExpo outperforms manufacture-default auto-exposure methods, reducing the mean absolute error (MAE) by 7.21 bpm (from 13.94 to 6.73 bpm) and increasing the success rate by 37.29 percentage points (from 25.95% to 63.24%) across challenging driving scenarios.
Figures & tables
Fig. 1 : Overview of illumination challenges for in-vehicle rPPG and the proposed PixExpo framework. (a) Representative high-dynamic-range (HDR) driving scenarios from the MEX-Drive dataset. (b) Multi-exposure acquisition and pixel-wise fusion pipeline of PixExpo.
TABLE I : Comparison of Existing Exposure Control Strategies and the Proposed PixExpo
Fig. 2 : Experimental setup and characteristics of MEX-Drive. (a) Dual-camera deployment and reference ECG acquisition; (b) Representative high-dynamic-range driving conditions; (c) Age and reference-HR distributions of the 48 participants.
Fig. 3 : Participant-level distributions of MAE, SNR, and SR for four exposure and fusion strategies ( n=48 ). The internal box plots show the median (red line), mean (white dot), and interquartile range.
Fig. 4 : Visual and time-frequency comparison of four exposure and fusion strategies. The first row shows representative frames, and the second row shows the corresponding time-frequency representations. The black dashed curve denotes the estimated HR, and the white curve denotes the ECG-derived reference HR. HR MAE (bpm), SNR (dB), and SR (%) are reported below each method.
PixExpo
Fixed Exposure
Full-frame AE
Estimator
MAE ↓
RMSE ↓
SR ↑
MAE ↓
RMSE ↓
SR ↑
MAE ↓
RMSE ↓
SR ↑
EfficientPhys [ 21 ]
8.30
13.30
57.91
10.80
15.38
43.87
12.05
16.72
38.21
FactorizePhys [ 12 ]
4.11
7.76
79.44
7.09
11.36
61.31
8.91
13.36
51.04
PhysFormer [ 43 ]
8.12
12.93
58.14
10.56
15.82
47.10
10.52
15.24
45.27
iBVPNet [ 13 ]
8.38
13.51
61.76
11.08
15.99
46.12
12.10
16.71
40.61
TABLE II : Cross-dataset HR estimation performance on all 48 MEX-Drive participants using PURE-pretrained rPPG models without fine-tuning. Bold values indicate the best result among the three acquisition strategies for each estimator and metric.
Fig. 5 : Performance comparison of Fixed Exposure, Face-ROI AE, Skin-ROI AE, and PixExpo using POS, CHROM, and ICA in the additional seven-participant experiment. Light lines connect paired participant-level results, and black markers show the mean ± SEM.
Fig. 6 : Participant-level comparison of fixed exposure, global adaptive exposure, and PixExpo in Scenario II ( n=48 ). Grey lines connect paired results from the same participant; violin plots show the corresponding distributions of MAE and SR.
Fig. 7 : Mean MAE and SR of fixed exposure, global adaptive exposure, and PixExpo under rainy and sunny conditions.
Fig. 8 : Representative video frames and corresponding rPPG spectrograms under four in-vehicle illumination conditions. Rows 1–4 show sunset, sunny with head rotation, overcast, and rainy conditions, respectively. Columns (a)–(c) show fixed exposure, global adaptive exposure, and PixExpo. The MAE and SR are reported below each spectrogram.
N
Total rate (fps)
Exposure times (ms)
MAE (bpm)
SNR (dB)
SR (%)
2
30
16, 32
3.75
4.92
90.15
3
45
7, 14, 21
4.59
3.30
81.76
6
90
1.8×{1,…,6}
6.99
1.19
67.76
TABLE III : Effect of Exposure Channel Count under Controlled Low-Light Conditions
Fusion Strategy
MAE (bpm) ↓
SR (%) ↑
SNR (dB) ↑
One-hot
6.73 ± 3.56
63.24 ± 18.95
0.60 ± 3.27
Gaussian decay
7.56 ± 3.77
57.90 ± 19.25
0.15 ± 3.23
Inverse weighting
9.83 ± 4.51
47.43 ± 20.73
−1.34±3.52
Friedman p
<0.001
<0.001
<0.001
Kendall’s W
0.365
0.730
0.875
TABLE IV : Participant-level Comparison of Fusion Weighting Strategies ( n=48 )
Fig. 9 : Visual and time–frequency comparison of fusion weighting strategies.
Fig. 10 : Representative comparison under transient driving disturbances. The six frames shown above are representative frames selected from the time interval highlighted by the red dashed rectangle in each row.
Fig. 11 : Visual effect of inter-frame registration on Mertens fusion and PixExpo in a representative example.
Fig. 12 : Spatial and spectral characteristics of exposure switching in a representative example. (a) PSR map, showing higher switching rates around facial boundaries and other high-gradient regions and lower switching rates over the forehead and cheeks. (b) PSDs of the recovered rPPG signal, FSR, and mean selection index. The recovered rPPG signal peaks at 1.52 Hz (91.0 bpm), close to the mean reference HR of 1.56 Hz.
Department of Biomedical Engineering, College of Engineering, Southern University of Science and Technology, Shenzhen 518000, China · State Key Laboratory for Novel Software Technology and School of Intelligence Science and Technology, Nanjing University, Nanjing 210023, China