We study the problem of mapping parameter-space variance of a 3D Morphable Model (3DMM) to per-vertex variance. Traditionally, per-vertex variance has been used to visually interpret the learnt parameter uncertainty. However, mapping parameter-space variance to per-vertex using sampling or error propagation is computationally expensive. We address this limitation with Closed-form Local Uncertainty Estimation (CLUE), which transforms the parameter-space distribution of 3DMMs to per-vertex through a closed-form approximation. CLUE emerges from applying the law of total variance to the 3DMM formulation and is modular, computationally inexpensive, and accurate. Subsequently, we study integrating CLUE for in-the-wild human mesh recovery training and inference. We note that CLUE uses the geometry of the mesh as an inductive bias when predicting the variance, leading to improved accuracy. Through extensive qualitative and quantitative experiments, we show that when used in training, CLUE (1) outpeforms probabilistic and deterministic baselines on metrics evaluating joints, mesh and uncertainty, and (2) has the highest uncertainty-to-error spearman correlation, with applications including data mining and interpretable uncertainty. Finally, we share SAM 3D Body (CLUE) which augments the regressor with high resolution uncertainty maps. We make our code available at https://github.com/vita-epfl/CLUE
Figures & tables
Figure 1: a) CLUE : C losed-form L ocal U ncertainty E stimation is a novel transformation that maps parameter-space distribution to local, per-vertex distribution for 3D Morphable Models. b) Accurate and Real-time : CLUE closely approximates sampling-based methods, while being ∼40× faster. c) In-the-wild : We study in-the-wild training and inference, and share SAM 3D Body (CLUE) , augmenting a frozen SAM 3D Body model with high-resolution uncertainty maps.
Figure 2: FLAME Error Map : We visualize the per-vertex error between the ground truth variance (computed with N=10,000 samples) and the estimated variance from different baselines. To do so, we synthesize different faces by setting randomly chosen parameters and their associated variance to non-zero. We observe that sampling with N=1000, Jacobian based error propagation and CLUE have low errors. Unlike sampling, Jacobian and CLUE may overestimate the variance due to first order assumptions. Additional visualizations in Fig. 7 .
Figure 3: Per-vertex error maps. Each row shows the input, the reference mesh (yellow), and the predictions of different baselines, coloured by the per-vertex error. The largest baseline errors sit on limbs that are occluded or having complex poses. CLUE uses the mesh as an inductive bias when predicting the variance, allowing it to have lower errors when dealing with ambiguity. Additional visualizations are shown in Fig. 8 .
Figure 4: Correlations across vertex groups. We compute the Spearman correlation between the predicted per-vertex variance and the rigidly aligned vertex error for each of the 19 MHR vertex groups. We observe that CLUE has a higher spearman correlation for each of the vertex groups, enabling applications in data mining and uncertainty based decision making.
Method
Time (ms)
Mem. (MB)
GFLOPs
Sampling N=100
138.7
14.2
1.43
Sampling N=1000
1379.4
140.9
14.26
Jacobian
226.4
410.4
6.01
CLUE (ours)
3.4
4.3
0.039
Table 1: FLAME compute. Median of 10 trials after 3 warm-ups. Additional results in Tab. 5 .
Figure 5: Uncertainty based data-mining. Per-vertex variance is computed empirically by sampling N=100 for all other baselines, and using CLUE for itself. Similar to spearman correlation, for each method we normalize a vertex’s uncertainty based on its per-vertex percentile for that method. These images were retrieved algorithmically by querying for images with the highest uncertainty discrepancy between a limb and the body. In contrast to CLUE, the baselines’ uncertainty on the same images is body-wide or diffused and lacks fine-grained resolution. Additional visualizations are shown in Fig. 9 .
Table 2: 3DPW and EMDB (MoCap SMPL). All metrics in mm ( ↓ ). The first row measures SAM 3D Body on our evaluation pipeline. The other rows compare deterministic and probabilistic losses under identical training and evaluation protocol.
Joints
Surface
Uncertainty
Method
MPJPE
PA-MPJPE
PVE
PA-PVE
vNLL
ρ
AUSE
Deterministic
31.6
19.7
35.4
24.2
–
–
–
Param-NLL
32.0
18.5
35.0
22.3
2.35
0.68
0.078
Matrix-Fisher
35.9
20.5
38.9
24.4
2.74
0.56
0.129
Normalizing Flow
32.0
18.1
35.3
22.2
3.08
0.69
0.077
CLUE
27.5
17.7
30.4
21.7
2.13
0.75
0.067
Table 3: Pseudo-ground-truth evaluation , averaged over five in-the-wild datasets (H36M, MPI-INF-3DHP, PoseTrack, CrowdPose, OCHuman): joint and surface error (mm, ↓ ), vertex NLL ( ↓ ), Pearson correlation between predicted uncertainty and error ( ↑ ), and the area under the sparsification error curve (AUSE, ↓ ; per person, ranking by the mean predicted vertex sigma). Per-dataset results are in the appendix (Tab. 4 ).
Figure 6: Uncertainty for downstream decision making. Top: frames from a headset’s egocentric camera with the mouth progressively occluded. Bottom: the tracked 3DMM per-vertex standard deviation (mm). The uncertainty concentrates on the occluded region, rising monotonically with the degree of occlusion. With this signal, a system can suspend or down-weight vision-based face tracking during occlusion, or defer to another modality, instead of tracking blindly.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Table 4: Pseudo-ground-truth evaluation on five in-the-wild datasets : (a) joint accuracy, (b) surface accuracy (mm, ↓ ), (c) uncertainty quality — per-vertex NLL ( ↓ ) and the Pearson correlation between predicted σ and vertex error ( ↑ ), and (d) the area under the sparsification error curve (AUSE, ↓ ; per person, ranking by the mean predicted vertex σ , normalised by each dataset’s full-set error).
FLAME
SMPL-X
MHR
Method
ms
MB
GFLOPs
ms
MB
GFLOPs
ms
MB
GFLOPs
Sampling N=100
141.6
14.2
1.43
368.5
29.5
5.42
681.7
44.5
1.71
Sampling N=1000
1404.7
140.9
14.33
3733.6
293.6
54.20
6802.4
442.9
17.09
Jacobian
228.6
410.4
6.08
177.0
383.6
10.23
3272.2
1984.4
6.40
CLUE (ours)
3.5
4.3
0.039
4.0
9.5
0.015
10.9
16.9
0.055
Appendix
Table 5: Compute cost across body models. Time (ms), memory (MB) and GFLOPs of one variance evaluation. Median of 10 trials after 3 warm-ups on one CPU (Intel i7-8700).
Figure 7: Complete error maps for FLAME, SMPLX, MHR. Both, Jacobian based linear error propagation and CLUE overestimate the variance due to first order assumptions. However when used in training, the learnt parameter variances compensate for them.
Figure 8: Error Maps for in-the-wild Human Mesh Recovery. We show that CLUE has higher accuracy when dealing with occlusions or complex poses.
Figure 9: Uncertainty based data-mining.
Figure 10: Augmenting SAM 3D Body with probabilistic reasoning using CLUE (Part 1).
Figure 11: Augmenting SAM 3D Body with probabilistic reasoning using CLUE (Part 2).
A correctly reconstructed distant point appears uncertain even when a frozen 3D model processes equivalent inputs because its output frame rotates fractionally. This exposes a weakness of trainingfree perturbation uncertainty: when outputs contain an unobserved symmetry, run-to-run variation potentially reflects symmetry rather than error. Existing alternatives have trade-offs: built-in confidence is outperformed in most evaluated conditions, while trained evidential heads require modelspecific supervision. For point maps, this research derives a closed-form, error-independent variance term that grows with scene extent and potentially overwhelms the desired signal. Simulation reproduces the effect; all 30 real VGGT view-sets tested exhibit its predicted ∥xp∥2 signature. ReVar3R robustly registers predictions to a common similarity frame before computing per-point variance, without retraining or modifying the frozen model. Optional calibration and fusion use a held-out split. Across VGGT, π3, and MASt3R on six datasets, the same estimator on every backbone lowers AUSE below built-in confidence in 15 of 18 conditions. The staged evaluation yields 11 of 18 wins for the label-free core, 12/18 for label-free equal-weight fusion, 14/18 with held-out weights, and 15/18 when the built-in signal is included. Against a trained evidential head, the result is a trade-off: the head calibrates magnitude better and leads in its training domain, whereas ReVar3R transfers across backbones without adaptation. Its ranking improves point filtering, but it does not detect stable systematic bias, aid novel-view synthesis, or transfer calibration across domains.
Sammam Mahdi, Fariha Binta Salim, Rakin Bin Rabbani +1
Human mesh recovery from single images remains challenging due to inherent depth ambiguity and limited generalization across domains. While recent methods combine regression and optimization approaches, they struggle with poor initialization for test-time refinement and inefficient parameter updates during optimization. We propose a novel meta-learning framework that trains models to produce optimization-friendly initializations while incorporating uncertainty-aware adaptive updates during test-time refinement. Our approach introduces three key innovations: (1) a meta-learning strategy that simulates test-time optimization during training to learn better parameter initializations, (2) a selective parameter caching mechanism that identifies and freezes converged joints to reduce computational overhead, and (3) distribution-based adaptive updates that sample parameter changes from learned distributions, enabling robust exploration while quantifying uncertainty. Additionally, we employ stochastic approximation techniques to handle intractable gradients in complex loss landscapes. Extensive experiments on standard benchmarks demonstrate that our method achieves state-of-the-art performance, reducing MPJPE by 10.3 on 3DPW and 8.0 on Human3.6M compared to strong baselines. Our approach shows superior domain adaptation capabilities with minimal performance degradation across different environmental conditions, while providing meaningful uncertainty estimates that correlate with actual prediction errors. Combining meta-learning and adaptive optimization enables accurate mesh recovery and robust generalization to challenging scenarios.
Geometric foundation models hold promise for unconstrained dense geometry prediction from uncalibrated images. However, in current feed-forward designs, their predicted confidence scores are heuristic, lack probabilistic interpretation, and often fail to indicate where and how much the predicted geometry can be trusted. To address this gap, we present Trust3R, a lightweight evidential uncertainty framework for feed-forward 3D reconstruction. Trust3R combines gated residual mean refinement with a Normal-Inverse-Wishart evidential head, yielding a closed-form multivariate Student-t distribution for per-point geometric uncertainty. This design provides probabilistically grounded pointmap uncertainty estimates while adding moderate inference overhead. We evaluate on diverse indoor and outdoor benchmarks and compare against MASt3R's built-in confidence map as well as common uncertainty-aware baselines spanning single-pass heteroscedastic regression and sampling-based methods such as MC dropout and deep ensembles. Experimental results show that Trust3R consistently improves risk-coverage and sparsification, and generally improves geometric accuracy. These gains are reflected in stronger uncertainty ranking across benchmarks, with 25% lower AURC and 41% lower AUSE on ScanNet++, providing a practical reliability signal for uncertainty-aware weighting in downstream geometry pipelines. The project page and code are available at https://trust3r-z.github.io/.
Zihao Zhu, Wenyuan Zhao, Nuo Chen +2
Department of Electrical and Computer Engineering, Texas A&M University, College Station, TX, USA