Modeling Uncertainty Propagation in 3D Morphable Models

Megh Shukla; Edo Collins; Prashanth Chandran; Karel Petránek; Sergio Orts-Escolano; Mathieu Salzmann; Alexandre Alahi

cs.CVOct 17, 2026

Modeling Uncertainty Propagation in 3D Morphable Models

Authors: Megh Shukla, Edo Collins, Prashanth Chandran, Karel Petránek, Sergio Orts-Escolano, Mathieu Salzmann, Alexandre Alahi

Organizations: EPFL · Google · EPFL, Waabi

Abstract

We study the problem of mapping parameter-space variance of a 3D Morphable Model (3DMM) to per-vertex variance. Traditionally, per-vertex variance has been used to visually interpret the learnt parameter uncertainty. However, mapping parameter-space variance to per-vertex using sampling or error propagation is computationally expensive. We address this limitation with Closed-form Local Uncertainty Estimation (CLUE), which transforms the parameter-space distribution of 3DMMs to per-vertex through a closed-form approximation. CLUE emerges from applying the law of total variance to the 3DMM formulation and is modular, computationally inexpensive, and accurate. Subsequently, we study integrating CLUE for in-the-wild human mesh recovery training and inference. We note that CLUE uses the geometry of the mesh as an inductive bias when predicting the variance, leading to improved accuracy. Through extensive qualitative and quantitative experiments, we show that when used in training, CLUE (1) outpeforms probabilistic and deterministic baselines on metrics evaluating joints, mesh and uncertainty, and (2) has the highest uncertainty-to-error spearman correlation, with applications including data mining and interpretable uncertainty. Finally, we share SAM 3D Body (CLUE) which augments the regressor with high resolution uncertainty maps. We make our code available at https://github.com/vita-epfl/CLUE

Figures & tables

Figure 1: a) CLUE : C losed-form L ocal U ncertainty E stimation is a novel transformation that maps parameter-space distribution to local, per-vertex distribution for 3D Morphable Models. b) Accurate and Real-time : CLUE closely approximates sampling-based methods, while being $\mathbf{\sim 40\times}$ faster. c) In-the-wild : We study in-the-wild training and inference, and share SAM 3D Body (CLUE) , augmenting a frozen SAM 3D Body model with high-resolution uncertainty maps.

Figure 2: FLAME Error Map : We visualize the per-vertex error between the ground truth variance (computed with $N=10,000$ samples) and the estimated variance from different baselines. To do so, we synthesize different faces by setting randomly chosen parameters and their associated variance to non-zero. We observe that sampling with N=1000, Jacobian based error propagation and CLUE have low errors. Unlike sampling, Jacobian and CLUE may overestimate the variance due to first order assumptions. Additional visualizations in Fig. 7 .

Figure 3: Per-vertex error maps. Each row shows the input, the reference mesh (yellow), and the predictions of different baselines, coloured by the per-vertex error. The largest baseline errors sit on limbs that are occluded or having complex poses. CLUE uses the mesh as an inductive bias when predicting the variance, allowing it to have lower errors when dealing with ambiguity. Additional visualizations are shown in Fig. 8 .

Figure 4: Correlations across vertex groups. We compute the Spearman correlation between the predicted per-vertex variance and the rigidly aligned vertex error for each of the 19 MHR vertex groups. We observe that CLUE has a higher spearman correlation for each of the vertex groups, enabling applications in data mining and uncertainty based decision making.

Method	Time (ms)	Mem. (MB)	GFLOPs
Sampling $N{=}100$	138.7	14.2	1.43
Sampling $N{=}1000$	1379.4	140.9	14.26
Jacobian	226.4	410.4	6.01
CLUE (ours)	3.4	4.3	0.039

Table 1: FLAME compute. Median of 10 trials after 3 warm-ups. Additional results in Tab. 5 .

Figure 5: Uncertainty based data-mining. Per-vertex variance is computed empirically by sampling N=100 for all other baselines, and using CLUE for itself. Similar to spearman correlation, for each method we normalize a vertex’s uncertainty based on its per-vertex percentile for that method. These images were retrieved algorithmically by querying for images with the highest uncertainty discrepancy between a limb and the body. In contrast to CLUE, the baselines’ uncertainty on the same images is body-wide or diffused and lacks fine-grained resolution. Additional visualizations are shown in Fig. 9 .

Panel of Table 2 — Table 2: 3DPW and EMDB (MoCap SMPL). All metrics in mm ( $\downarrow$ ). The first row measures SAM 3D Body on our evaluation pipeline. The other rows compare deterministic and probabilistic losses under identical training and evaluation protocol.

	Joints		Surface		Uncertainty
Method	MPJPE	PA-MPJPE	PVE	PA-PVE	vNLL	$\rho$	AUSE
Deterministic	31.6	19.7	35.4	24.2	–	–	–
Param-NLL	32.0	18.5	35.0	22.3	2.35	0.68	0.078
Matrix-Fisher	35.9	20.5	38.9	24.4	2.74	0.56	0.129
Normalizing Flow	32.0	18.1	35.3	22.2	3.08	0.69	0.077
CLUE	27.5	17.7	30.4	21.7	2.13	0.75	0.067

Table 3: Pseudo-ground-truth evaluation , averaged over five in-the-wild datasets (H36M, MPI-INF-3DHP, PoseTrack, CrowdPose, OCHuman): joint and surface error (mm,

\downarrow

), vertex NLL (

\downarrow

), Pearson correlation between predicted uncertainty and error (

\uparrow

), and the area under the sparsification error curve (AUSE,

\downarrow

; per person, ranking by the mean predicted vertex sigma). Per-dataset results are in the appendix (Tab. 4 ).

Figure 6: Uncertainty for downstream decision making. Top: frames from a headset’s egocentric camera with the mouth progressively occluded. Bottom: the tracked 3DMM per-vertex standard deviation (mm). The uncertainty concentrates on the occluded region, rising monotonically with the degree of occlusion. With this signal, a system can suspend or down-weight vision-based face tracking during occlusion, or defer to another modality, instead of tracking blindly.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Table 4: Pseudo-ground-truth evaluation on five in-the-wild datasets : (a) joint accuracy, (b) surface accuracy (mm,

\downarrow

), (c) uncertainty quality — per-vertex NLL (

\downarrow

) and the Pearson correlation between predicted

\sigma

and vertex error (

\uparrow

), and (d) the area under the sparsification error curve (AUSE,

\downarrow

; per person, ranking by the mean predicted vertex

\sigma

, normalised by each dataset’s full-set error).

Method	ms	MB	GFLOPs	ms	MB	GFLOPs	ms	MB	GFLOPs
	FLAME			SMPL-X			MHR
Sampling $N{=}100$	141.6	14.2	1.43	368.5	29.5	5.42	681.7	44.5	1.71
Sampling $N{=}1000$	1404.7	140.9	14.33	3733.6	293.6	54.20	6802.4	442.9	17.09
Jacobian	228.6	410.4	6.08	177.0	383.6	10.23	3272.2	1984.4	6.40
CLUE (ours)	3.5	4.3	0.039	4.0	9.5	0.015	10.9	16.9	0.055

Appendix

Table 5: Compute cost across body models. Time (ms), memory (MB) and GFLOPs of one variance evaluation. Median of 10 trials after 3 warm-ups on one CPU (Intel i7-8700).

Panel of Figure 7 — Figure 7: Complete error maps for FLAME, SMPLX, MHR. Both, Jacobian based linear error propagation and CLUE overestimate the variance due to first order assumptions. However when used in training, the learnt parameter variances compensate for them.

Figure 8: Error Maps for in-the-wild Human Mesh Recovery. We show that CLUE has higher accuracy when dealing with occlusions or complex poses.

Figure 9: Uncertainty based data-mining.

Figure 10: Augmenting SAM 3D Body with probabilistic reasoning using CLUE (Part 1).

Figure 11: Augmenting SAM 3D Body with probabilistic reasoning using CLUE (Part 2).

UQ for 3D Reconstruction

See all 18 papers

Revar3r: gauge-aware perturbation uncertainty for feed-forward 3d reconstruction
Oct 6, 2026Sammam Mahdi, Fariha Binta Salim, Rakin Bin Rabbani +1Uncertainty Quantification 3D Reconstruction
BRAC University · Mahidol University
Meta-Learned Adaptive Optimization for Robust Human Mesh Recovery with Uncertainty-Aware Parameter Updates
Mar 27, 2026Shaurjya Mandal, Nutan Sharma, John GaleottiHuman Mesh Recovery Test-Time Optimization
Trust It or Not: Evidential Uncertainty for Feed-Forward 3D Reconstruction with Trust3R
May 19, 2026Zihao Zhu, Wenyuan Zhao, Nuo Chen +2Evidential Deep Learning 3D Reconstruction
Department of Electrical and Computer Engineering, Texas A&M University, College Station, TX, USA

Oct 6, 2026J/K move · Enter open · S save

Revar3r: gauge-aware perturbation uncertainty for feed-forward 3d reconstruction

Sammam Mahdi, Fariha Binta Salim, Rakin Bin Rabbani, Aniqua Nusrat Zereen

BRAC University · Mahidol University

Uncertainty Quantification 3D Reconstruction UQ for 3D Reconstruction Feed-Forward 3D Reconstruction

Open paper AskPDF arXiv

A correctly reconstructed distant point appears uncertain even when a frozen 3D model processes equivalent inputs because its output frame rotates fractionally. This exposes a weakness of trainingfree perturbation uncertainty: when outputs contain an unobserved symmetry, run-to-run variation potentially reflects symmetry rather than error. Existing alternatives have trade-offs: built-in confidence is outperformed in most evaluated conditions, while trained evidential heads require modelspecific supervision. For point maps, this research derives a closed-form, error-independent variance term that grows with scene extent and potentially overwhelms the desired signal. Simulation reproduces the effect; all 30 real VGGT view-sets tested exhibit its predicted

\|x_p\|^2

signature. ReVar3R robustly registers predictions to a common similarity frame before computing per-point variance, without retraining or modifying the frozen model. Optional calibration and fusion use a held-out split. Across VGGT, π3, and MASt3R on six datasets, the same estimator on every backbone lowers AUSE below built-in confidence in 15 of 18 conditions. The staged evaluation yields 11 of 18 wins for the label-free core, 12/18 for label-free equal-weight fusion, 14/18 with held-out weights, and 15/18 when the built-in signal is included. Against a trained evidential head, the result is a trade-off: the head calibrates magnitude better and leads in its training domain, whereas ReVar3R transfers across backbones without adaptation. Its ranking improves point filtering, but it does not detect stable systematic bias, aid novel-view synthesis, or transfer calibration across domains.

Abstract

UQ for 3D Reconstruction

Explore similar work

Revar3r: gauge-aware perturbation uncertainty for feed-forward 3d reconstruction

Meta-Learned Adaptive Optimization for Robust Human Mesh Recovery with Uncertainty-Aware Parameter Updates

Trust It or Not: Evidential Uncertainty for Feed-Forward 3D Reconstruction with Trust3R