cs.CVSep 22, 2026

Heartian: Physiology-Aware Relightable Gaussian Head Avatar

Authors: Xiaoyue Fan, Jose Echevarria, Akshay Paruchuri, Kaan Akşit

Organizations: University College London London, UK · Adobe Research New York, NY, USA · Stanford University Stanford, CA, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Gaussian head avatars typically model intrinsic facial appearance as temporally static, omitting subtle cardiac-induced skin-color variation. We propose Heartian, a physiology-aware modulation framework that learns cardiac-cycle-dependent per-frame albedo modulation of facial skin-region Gaussians within a relightable head avatar to encode remote photoplethysmography (rPPG) signals. Using synchronized contact PPG supervision, Heartian models the prescribed cardiac waveform as the sum of two Gaussian functions and learns per-frame spatial residuals via a lightweight MLP. Across 152 stationary recordings from UBFC-rPPG, PURE, and MMPD, attribute-space recovery of the supplied signal achieves a pooled recording-level heart-rate MAE of 0.29 bpm and MAPE of 0.38%. The signals remain detectable after rendering by benchmark rPPG methods, with the best tested configuration - a motion-augmented TS-CAN decoder pretrained on UBFC-rPPG - recovering heart rate from the rendered MMPD avatars at 0.97 bpm MAE and 1.21% MAPE. Meanwhile, Heartian maintains reconstruction quality comparable to the baseline, with negligible average PSNR degradation of 0.005 dB. Overall, our work embeds recoverable rPPG signals as controllable material attributes to subject-specific Gaussian head avatars while retaining the reconstruction quality.

Figures & tables

Figure 1. Skin Gaussians Gs\mathcal{G}^{s} (red) overlaid on G\mathcal{G} (blue).
Figure 2. A typical PPG waveform with systolic (blue) and diastolic waves (red) in the time (a) and phase domains (b).
  Test Set
  UBFC-rPPGPUREMMPD
MethodTrain SetMAE ↓\downarrowMAPE ↓\downarrowMAE ↓\downarrowMAPE ↓\downarrowMAE ↓\downarrowMAPE ↓\downarrow
POS-1.23 ±\pm 0.771.01 ±\pm 0.6214.23 ±\pm 6.7829.34 ±\pm 13.981.53 ±\pm 0.632.66 ±\pm 1.20
TS-CANUBFC-rPPG--10.01 ±\pm 4.5217.89 ±\pm 8.682.59 ±\pm 0.523.89 ±\pm 0.94
PURE9.22 ±\pm 7.477.41 ±\pm 5.56--2.51 ±\pm 0.573.87 ±\pm 1.03
TS-CAN (MA)UBFC-rPPG--4.74 ±\pm 4.225.23 ±\pm 4.670.97 ±\pm 0.271.21 ±\pm 0.31
Table 1. Benchmark Evaluation Results. Performance of benchmark methods on our rendered Heartian videos, evaluated across datasets. Green highlights indicate metrics where embedded signals remain recoverable on par with the original benchmark results on the same selected subsets.
 BaselineOurs
DatasetMAEMAPESNRMAE ↓\downarrowMAPE ↓\downarrowSNR ↑\uparrow
UBFC-rPPG59.0053.81-21.520.000.001.45
PURE21.5325.37-7.130.170.273.47
MMPD39.1134.81-11.470.330.424.04
Table 2. Attribute-level Evaluation. rPPG signal metrics extracted from the Gaussian albedo of baseline and Heartian.
 POSTS-CAN (MA)FactorizePhys
 MAE ↓\downarrowMAPE ↓\downarrowMAE ↓\downarrowMAPE ↓\downarrowMAE ↓\downarrowMAPE ↓\downarrow
Skin tone      
31.01 ±\pm 0.821.80 ±\pm 1.521.04 ±\pm 0.441.35 ±\pm 0.522.89 ±\pm 1.484.66 ±\pm 2.38
41.91 ±\pm 1.442.33 ±\pm 1.822.17 ±\pm 1.112.35 ±\pm 1.112.89 ±\pm 1.883.61 ±\pm 2.41
53.80 ±\pm 2.347.44 ±\pm 4.760.62 ±\pm 0.340.87 ±\pm 0.454.90 ±\pm 2.899.50 ±\pm 5.81
60.47 ±\pm 0.210.53 ±\pm 0.250.10 ±\pm 0.100.12 ±\pm 0.110.10 ±\pm 0.070.12 ±\pm 0.09
Table 3. Lights and skin tones comparison within MMPD ( Tang et al., 2023 ) , tested with models pretrained on UBFC-rPPG ( Bobbia et al., 2019 ) .
  Test Set
  UBFC-rPPGPUREMMPD
MethodTrain SetMAERMSEMAPEρ\rhoSNRMAERMSEMAPEρ\rhoSNRMAERMSEMAPEρ\rhoSNR
POS-1.23 ±\pm 0.772.75 ±\pm 2.431.01 ±\pm 0.620.98 ±\pm 0.050.26 ±\pm 0.9214.23 ±\pm 6.7825.75 ±\pm 18.0429.34 ±\pm 13.980.48 ±\pm 0.302.93 ±\pm 0.561.53 ±\pm 0.637.43 ±\pm 5.532.66 ±\pm 1.200.82 ±\pm 0.044.20 ±\pm 0.25
TS-CANUBFC-rPPG-----10.01 ±\pm 4.5217.46 ±\pm 12.5417.89 ±\pm 8.680.68 ±\pm 0.25-3.31 ±\pm 0.662.59 ±\pm 0.526.60 ±\pm 4.603.89 ±\pm 0.940.85 ±\pm 0.04-1.71 ±\pm 0.17
PURE9.22 ±\pm 7.4725.37 ±\pm 24.627.41 ±\pm 5.560.23 ±\pm 0.34-4.9 ±\pm 1.14-----2.51 ±\pm 0.577.02 ±\pm 4.853.87 ±\pm 1.030.83 ±\pm 0.04-1.76 ±\pm 0.16
TS-CAN (MA)UBFC-rPPG-----4.74 ±\pm 4.2214.18 ±\pm 13.805.23 ±\pm 4.670.82 ±\pm 0.19-0.28 ±\pm 0.960.97 ±\pm 0.273.35 ±\pm 2.631.21 ±\pm 0.310.96 ±\pm 0.02-0.40 ±\pm 0.15
Table 1. Additional Benchmark Evaluation Results. Performance of benchmark methods on our rendered Heartian avatar videos, evaluated across the UBFC-rPPG ( Bobbia et al., 2019 ) , PURE ( Stricker et al., 2014 ) , and MMPD ( Tang et al., 2023 ) datasets. Green highlights indicate metrics where signals remain recoverable on par with the benchmark results on the same subsets.
Figure 1. Visualization of Recovered Heart Rate Accuracy. Scatter plots of rPPG estimates from our rendered Heartian videos versus ground truth using POS for UBFC-rPPG and MMPD, and motion-augmented TS-CAN for PURE and MMPD.
  Test Set
  UBFC-rPPGPUREMMPD
MethodTrain SetMAERMSEMAPEρ\rhoSNRMAERMSEMAPEρ\rhoSNRMAERMSEMAPEρ\rhoSNR
POS-+ 0.87+ 1.15+ 0.87- 0.01- 0.94+ 0.52+ 0.05+ 0.62- 0.01- 0.22+ 3.04+ 5.45+ 5.62- 0.24- 2.09
TS-CAN (MA)UBFC-rPPG-----0.000.000.000.00+ 0.06- 0.30- 1.90+ 0.30+ 0.01+ 0.04
FactorizePhysUBFC-rPPG------ 0.17- 0.30- 0.38+ 0.01- 0.11- 0.05- 0.01- 0.060.00- 0.10
PURE0.000.000.000.00- 0.05------ 1.37- 4.47- 2.90+ 0.13- 0.07
Table 2. Background Compositing Ablation. Metric changes from background compositing over the default black background.
Figure 2. Relighting Ablation Visualization. From left to right: white, cool, warm, and real-world hospital lighting ( Source Link ); two subjects from PURE (left) and UBFC-rPPG (right). Solid color environment maps are synthesized with matched luminance.
Figure 3. Ablation on Spatial Residual of Our Modulation Components. Both ours (blue) and GT (orange) are filtered and normalized. Full modulation shows a better alignment of the prescribed signals, with 0.05 MACC improved in this case.

Explore similar work

CardsList

  1. PhysAgent: A Multi-Agent Framework for Reliable Remote Heart Rate Estimation

    Jul 29, 2026Yehui Yang, Bo Zhao, Junzhe Cao +4Remote PhotoplethysmographyHeart Rate

    1Great Bay University · 2Shenzhen University of Advanced Technology · 3Macao Polytechnic University +1

  2. CanonicalPhys: Pose-Robust Remote Photoplethysmography via Canonical-Space Priors

    Jul 17, 2026Hui Wei, Seyedata Jodeiri Seyedian, Xiaobai Li +1Remote PhotoplethysmographyMean Squared Error

    Center for Machine Vision and Signal Analysis (CMVS), University of Oulu · ELLIS Institute Finland · Zhejiang University

  3. Template Collapse and Information-Theoretic Limits in Camera rPPG Pulse Morphology Restoration

    Jun 2, 2026Achraf Ben AhmedRemote PhotoplethysmographyCardiovascular Risk Prediction

    PlesmoSense SARL