cs.CVMar 27, 2026

Meta-Learned Adaptive Optimization for Robust Human Mesh Recovery with Uncertainty-Aware Parameter Updates

Authors: Shaurjya Mandal, Nutan Sharma, John Galeotti

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Human mesh recovery from single images remains challenging due to inherent depth ambiguity and limited generalization across domains. While recent methods combine regression and optimization approaches, they struggle with poor initialization for test-time refinement and inefficient parameter updates during optimization. We propose a novel meta-learning framework that trains models to produce optimization-friendly initializations while incorporating uncertainty-aware adaptive updates during test-time refinement. Our approach introduces three key innovations: (1) a meta-learning strategy that simulates test-time optimization during training to learn better parameter initializations, (2) a selective parameter caching mechanism that identifies and freezes converged joints to reduce computational overhead, and (3) distribution-based adaptive updates that sample parameter changes from learned distributions, enabling robust exploration while quantifying uncertainty. Additionally, we employ stochastic approximation techniques to handle intractable gradients in complex loss landscapes. Extensive experiments on standard benchmarks demonstrate that our method achieves state-of-the-art performance, reducing MPJPE by 10.3 on 3DPW and 8.0 on Human3.6M compared to strong baselines. Our approach shows superior domain adaptation capabilities with minimal performance degradation across different environmental conditions, while providing meaningful uncertainty estimates that correlate with actual prediction errors. Combining meta-learning and adaptive optimization enables accurate mesh recovery and robust generalization to challenging scenarios.

Figures & tables

Figure 1: Overview of our meta-learned adaptive optimization framework for human mesh recovery.
MethodTypeBackbone3DPWHuman3.6M
MPJPE↓PA-MPJPE↓PVE↓MPJPE↓PA-MPJPE↓
Regression-based
HMR ( Kanazawa et al. 2018 )RegResNet-50130.081.3-88.056.8
CLIFF ( Li et al. 2022 )RegHRNet-W4869.043.081.247.132.7
PyMAF ( Zhang et al. 2021 )RegHRNet-W4874.245.387.054.237.2
METRO ( Lin et al. 2021 )RegHRNet-W6477.147.988.254.036.7
Table 1: Quantitative comparison with state-of-the-art methods on 3DPW ( Von Marcard et al. 2018 ) and Human3.6M ( Ionescu et al. 2013 ) datasets. Best results in bold .
Components3DPWHuman3.6M
Meta-LearningCachingAdaptive UpdatesMPJPE↓PA-MPJPE↓MPJPE↓PA-MPJPE↓
✗✗✗69.043.047.132.7
✓✗✗64.240.143.529.8
✓✓✗61.538.741.228.1
✓✗✓62.839.442.628.9
✓✓✓58.737.239.126.8
Table 2: Ablation study on core components. Each innovation contributes to overall performance improvement.
Variance StrategyOpt StepsMPJPE↓PA-MPJPE↓
Fixed ( σ=0.01\sigma=0.01 )562.138.9
Fixed ( σ=0.05\sigma=0.05 )561.438.5
Adaptive (Eq. 9 )560.237.8
Adaptive (Eq. 9 )1059.137.4
Adaptive (Eq. 9 )1558.737.2
Table 3: Impact of variance adaptation strategy and optimization steps on 3DPW dataset.
Training DomainTest DomainMethodMPJPE↓PA-MPJPE↓Δ\Delta MPJPE
Indoor → Outdoor Transfer
Human3.6M3DPWCLIFF78.448.2+9.4
Human3.6M3DPWEFT72.644.8+12.5
Human3.6M3DPWMetaHMR68.942.1+6.5
Human3.6M3DPWOurs64.239.7+4.1
Outdoor → Indoor Transfer
Table 4: Out-of-domain generalization analysis. Δ\Delta MPJPE represents performance degradation compared to in-domain training.
Figure 2: Convergence analysis of our adaptive optimization framework.
More in this topic

Human Mesh Recovery

Human mesh recovery estimates articulated 3D body shape and pose from images, video, or other sensors for animation, tracking, and physical analysis.

See all 51 papers

Explore similar work

CardsList
Mar 20, 2026

Monocular Models are Strong Learners for Multi-View Human Mesh Recovery

Multi-view human mesh recovery (HMR) is broadly deployed in diverse domains where high accuracy and strong generalization are essential. Existing approaches can be broadly grouped into geometry-based and learning-based methods. However, geometry-based methods (e.g., triangulation) rely on cumbersome camera calibration, while learning-based approaches often generalize poorly to unseen camera configurations due to the lack of multi-view training data, limiting their performance in real-world scenarios. To enable calibration-free reconstruction that generalizes to arbitrary camera setups, we propose a training-free framework that leverages pretrained single-view HMR models as strong priors, eliminating the need for multi-view training data. Our method first constructs a robust and consistent multi-view initialization from single-view predictions, and then refines it via test-time optimization guided by multi-view consistency and anatomical constraints. Extensive experiments demonstrate state-of-the-art performance on standard benchmarks, surpassing multi-view models trained with explicit multi-view supervision.

Haoyu Xie, Shengkai Xu, Cheng Guo +6

University of North Carolina at Charlotte · University of Central Florida

Human Mesh RecoveryMulti-View 3D Reconstruction
AskPDF
May 6, 2026

InterMesh: Explicit Interaction-Aware End-to-End Multi-Person Human Mesh Recovery

Humans constantly interact with their surroundings. Existing end-to-end multi-person human mesh recovery methods, typically based on the DETR framework, capture inter-human relationships through self-attention across all human queries. However, these approaches model interactions only implicitly and lack explicit reasoning about how humans interact with objects and with each other. In this paper, we propose InterMesh, a simple yet effective framework that explicitly incorporates human-environment interaction information into human mesh recovery pipeline. By leveraging a human-object interaction detector, InterMesh enriches query representations with structured interaction semantics, enabling more accurate pose and shape estimation. We design lightweight modules, Contextual Interaction Encoder and Interaction-Guided Refiner, to integrate these features into existing HMR architectures with minimal overhead. We validate our approach through extensive experiments on 3DPW, MuPoTS, CMU Panoptic, Hi4D, and CHI3D datasets, demonstrating remarkable improvements over state-of-the-art methods. Notably, InterMesh reduces MPJPE by 9.9% on CMU Panoptic and 8.2% on Hi4D, highlighting its effectiveness in scenarios with complex human-object and inter-human interactions. Code and models are released at https://github.com/Kelly510/InterMesh.

Kaili Zheng, Kaiwen Wang, Xun Zhu +2

Department of Electronic Engineering, Tsinghua University, Beijing 100084, China · College of AI, Tsinghua University, Beijing 100084, China · Beijing National Research Center for Information Science and Technology

3D ReconstructionHuman Mesh Recovery3D Human Pose Estimation3D Mesh Reconstruction3D Human Reconstruction
AskPDF
May 25, 2026

MuNet: A Mutualistic Network for Joint 3D Human Mesh Recovery and 3D Clothed Human Reconstruction from Single Images

3D human mesh recovery and 3D clothed human reconstruction are inherently related, yet they have long been studied in isolation, thereby overlooking the potential gains of joint optimization. To overcome this limitation, we propose to address these two tasks within a unified framework, which allows their mutual dependencies to be effectively exploited. Building on this idea, we propose MuNet, a mutualistic network for joint 3D human mesh recovery and 3D clothed human reconstruction from single images. First, we adopt 2-manifold graphs as a unified representation for all 3D models, enabling consistent modeling across 3D human mesh recovery and clothed human reconstruction. Second, we design an end-to-end graph convolutional network that progressively deforms an initial graph into a 3D human mesh and refines it into a detailed 3D clothed human model. Third, we introduce a mutualistic mechanism that allows reciprocal interaction between the two tasks {during training}, where 3D human mesh recovery provides guidance for 3D clothed human reconstruction, and reconstruction feedback refines the 3D human mesh recovery. We extensively evaluate MuNet on six benchmark datasets for 3D human mesh recovery and 3D clothed human reconstruction, including Human3.6M, 3DPW, MPI-INF-3DHP, THuman2.0, CAPE, and RenderPeople. Experimental results demonstrate that MuNet achieves state-of-the-art performance on both tasks across all datasets. The code of MuNet is released for research purposes at https://github.com/starVisionTeam/MuNet.

Yunqi Gao, Leyuan Liu, Yuhan Li +2

National Engineering Research Center for E-Learning Central China Normal University, Wuhan, 430079, Hubei, China · National Engineering Research Center of Educational Big Data Central China Normal University, Wuhan, 430079, Hubei, China · School of Electronic Information and Communications Huazhong University of Science and Technology, Wuhan, 430073, Hubei, China +1

3D ReconstructionHuman Mesh Recovery3D Mesh Reconstruction3D Human ReconstructionSingle-View 3D Reconstruction3D Garment Reconstruction
AskPDF