Skullptor: High Fidelity 3D Head Reconstruction in Seconds with Multi-View Normal Prediction
Organizations: Ubisoft La Forge · University of Toronto · ÉTS Montréal · Vector Institute
Abstract
Reconstructing high-fidelity 3D head geometry from images is critical for a wide range of applications, yet existing methods face fundamental limitations. Traditional photogrammetry achieves exceptional detail but requires extensive camera arrays (25-200+ views), substantial computation, and manual cleanup in challenging areas like facial hair. Recent alternatives present a fundamental trade-off: foundation models enable efficient single-image reconstruction but lack fine geometric detail, while optimization-based methods achieve higher fidelity but require dense views and expensive computation. We bridge this gap with a hybrid approach that combines the strengths of both paradigms. Our method introduces a multi-view surface normal prediction model that extends monocular foundation models with cross-view attention to produce geometrically consistent normals in a feed-forward pass. We then leverage these predictions as strong geometric priors within an inverse rendering optimization framework to recover high-frequency surface details. Our approach outperforms state-of-the-art single-image and multi-view methods, achieving high-fidelity reconstruction on par with dense-view photogrammetry while reducing camera requirements and computational cost.
Figures & tables
| Dataset | Method | Avg. Angular Error | Avg. Normal Grad. Err. | <$$$ (%) \uparrow$ | <$$$ (%) \uparrow$ | <$$$ (%) \uparrow$ | Time (s) |
|---|---|---|---|---|---|---|---|
| Multiface | Sapiens 0.3B | 10.3 | 0.265 | 61.9 | 85.3 | 94.6 | 7.2 |
| Sapiens 2B | 9.23 | 0.257 | 69.3 | 86.3 | 94.2 | 41.3 | |
| DAViD | 9.16 | 0.250 | 69.7 | 86.7 | 94.7 | 1.1 | |
| Skullptor (Ours) | 9.13 | 0.234 | 69.2 | 86.7 | 94.6 | 1.5 | |
| NPHM | Sapiens 0.3B | 7.33 | 0.194 | 78.4 | 93.4 | 97.8 | 8.2 |
| Sapiens 2B | 6.86 | 0.185 | 80.0 | 94.4 | 98.3 | 43.7 |
| Dataset | Method | Depth Error | Avg. Angular | Avg. Normal | $<$$$ | $<$$$ | $<$$$ | Runtime | # Views |
|---|---|---|---|---|---|---|---|---|---|
| (mm) | Error | Gradient Error | (%) | (%) | (%) | (min) | |||
| Multiface* | Meshroom (Photogrammetry) | 0.467 | 5.43 | 0.143 | 85.1 | 95.8 | 99.0 | 7.8 | 26 |
| 2D GS | 5.73 | 10.7 | 0.206 | 68.6 | 85.4 | 92.5 | 50 | 26 | |
| SuGaR | 5.54 | 11.5 | 0.324 | 66.1 | 83.8 | 91.1 | 42 | 26 | |
| Skullptor (Ours) | 2.43 | 6.17 | 0.156 | 91.5 | 97.6 | 99.0 | 0.67 | 26 | |
| Skullptor (Ours, 10 Views) | 2.99 | 6.46 | 0.157 | 86.8 | 95.8 | 97.9 | 0.48 | 10 |
| Normal Metrics | Mesh Metrics | |||||
|---|---|---|---|---|---|---|
| Method | Ang. | Grad. | Depth | Ang. | Grad. | |
| NPHM | DAViD | 7.80 | 0.188 | 2.68 | 5.68 | 0.115 |
| DAViD_ft_mono | 8.43 | 0.174 | 3.53 | 6.04 | 0.116 | |
| DAViD_ft_multi | 7.13 | 0.176 | 2.87 | 5.98 | 0.113 | |
| Skullptor | 7.05 | 0.167 | 2.44 | 5.89 | 0.110 | |
| Multiface | DAViD | 8.61 | 0.238 | 3.95 | 6.83 | 0.163 |
| Dataset | Normal Estimator | Depth (mm) | Angular Err. | Avg. Normal Grad. Err. |
|---|---|---|---|---|
| Multiface | Sapiens 0.3B | 5.28 | 8.75 | 0.177 |
| Sapiens 2B | 3.79 | 6.76 | 0.167 | |
| DAViD | 3.74 | 6.73 | 0.163 | |
| Skullptor (Ours) | 3.20 | 6.45 | 0.157 | |
| NPHM | Sapiens 0.3B | 6.38 | 8.23 | 0.124 |
| Sapiens 2B | 6.76 | 7.91 | 0.121 |
| Male | Female | |
|---|---|---|
| Number of subjects | 18 | 32 |
| Age range | 18–64 | 18–75 |
| Mean age | 37.2 | 42.8 |
| Age 25 | 9 (50%) | 14 (44%) |
| Age 25–65 | 8 (44%) | 12 (38%) |
| Age 65 | 1 (6%) | 6 (19%) |
| Camera Sampling | |
|---|---|
| Pitch angle | |
| Yaw angle | |
| Initial radius | 2.0 |
| Extra 3D translation | |
| Scale factor | |
| Projection Matrix | |
3D Face Reconstruction
3D face reconstruction estimates facial shape and appearance from images or video to create usable models and avatars.
See all 54 papers