A Lightweight Differentiable Method for Real-Time Skin Rendering and Digitization
Organizations: Universidad Rey Juan Carlos, Spain · Politecnico di Milano, Milan, Italy · Euro-Mediterranean Center on Climate Change (CMCC), Lecce, Italy · RFF-CMCC European Institute on Economics and the Environment (EIEE), Milan, Italy
Abstract
Inverse rendering for skin typically relies on offline path tracing to estimate physical properties, which fail to map directly to the approximate shaders utilized in interactive applications, such as game engines. We bridge this gap by directly estimating subsurface scattering parameters tailored for real-time rendering pipelines. We introduce a differentiable Screen-Space Subsurface Scattering (SSSS) framework that enables the joint optimization of spatially varying base textures and scattering distances. To account for local variations in skin thickness, we estimate these parameters across distinct facial regions, generating a blended profile map ready for production runtime engines. Our pipeline utilizes a low-cost, single-light, single-view capture setup designed for accessible and practical appearance acquisition. Optimization is driven by a masked loss function targeting regions where subsurface scattering effects are dominant. We validate our approach on a real-world dataset of individuals across the Monk Skin Tone Scale, ensuring demographic generalization. Furthermore, we complement this evaluation with synthetic data, a multi-light setup validating our single-light assumption, and a tilted-view configuration confirming angular consistency. Results demonstrate that our method effectively recovers scattering parameters from lightweight inputs, yielding realistic skin appearance directly compatible with standard real-time shaders.
Figures & tables
| Configuration | Stage 1 | Stage 2 | Stage 3 |
| Optimized Parameters | |||
| Frozen Parameters | |||
| Iterations | 700 | 500 | 600 |
| Learning Rate | |||
| Active Mask ( ) | Full Image ( ) | Full Image ( ) | Shadow Mask ( ) |
| Roughness Prior ( ) | None |
| o r *4c Monk Range | FLIP | LPIPS | SSIM | PSNR |
| Monk 1-3 | ||||
| Monk 4-6 | ||||
| Monk 7-10 |
| o l *4c Metric | Stage 1 | Stage 2 | Stage 3 | Overall |
| Execution Time (s) | 51.1 | 38.3 | 109.0 | 198.4 |
| Iterations | 700 | 500 | 600 | 1800 |
| Time per Iteration (ms) | 72.9 | 76.6 | 181.7 | 110.2 |
| Peak VRAM (GB) | 2.5 | 2.6 | 9.4 | 9.4 |
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Inverse Rendering
Inverse rendering estimates scene geometry, materials, lighting, and other causes of images from observed visual or sensor data.
See all 77 papers