In this paper, we present a 3D human pose estimator that predicts metrically accurate human joint positions in the camera coordinate frame. This task requires recovering the configuration of the body joints, scale, and metric camera-local placement. In monocular images, body size and camera distance are coupled through perspective projection and therefore cannot be determined uniquely from image alone. This ambiguity can be reduced by combining three complementary sources of information: learned RGB priors, direct depth measurements, and geometric constraints (e.g., ground contact) from the environment. We extend a pretrained full-body estimator with optional metric depth and geometry-processing modules that improve camera-distance estimation and depth-guided body refinement while preserving RGB-only operation. The resulting unified model supports RGB, RGB-Ground, and RGB-D using shared learned parameters. Unlike pelvis-relative error, camera-local error also measures placement relative to the camera. Compared with the existing foundation model SAM 3D Body, our methods with RGB, RGB-Ground, and RGB-D modes reduce camera-local joint-position error by 12.16%, 31.00% and 54.61% respectively.
Figures & tables
Fig. 1: Monocular scale–distance ambiguity. (a) Input image and 2D keypoints. (b) Different body scales and distances yield similar projections. (c) Camera-coordinate visualization exposes errors in physical scale, distance, and ground contact, which metric depth and ground geometry help resolve.
Fig. 2: Unified architecture for RGB, RGB-Ground, and RGB-D with shared weights. SAM 3D Body provides the body and camera estimate; optional ground geometry applies a ground-contact correction, while optional depth adds feature fusion, joint-level pose refinement, and depth-surface metric correction.
Method
Intrinsics
3DPW (14) MPJPE / PA / Cam.
EMDB (24) MPJPE / PA / Cam.
RICH (24) MPJPE / PA / Cam.
CameraHMR
–
56.0 / 35.1 / –
70.3 / 43.3 / –
55.7 / 48.1 / –
PromptHMR
–
58.7 / 36.1 / –
71.7 / 41.0 / –
56.6 / 37.3 / –
NLF-L+fit ∗
–
54.9 / 33.6 / –
68.4 / 40.9 / –
51.0 / 28.7 / –
Fast 3DB
–
59.5 / 30.4 / –
64.3 / 37.3 / –
58.8 / 33.8 / –
SAM 3D Body
GT
55.0 / 33.7 / 248.4
63.1 / 38.8 / 123.4
52.8 / 31.8 / 342.2
MoGe-2
55.1 / 33.7 / 511.2
63.1 / 38.8 / 139.4
54.9 / 32.2 / 607.7
TABLE I: Results on 3DPW, EMDB, and RICH: MPJPE / PA-MPJPE / camera-local MPJPE (mm; ↓ lower is better).
Method
BEDLAM2 MPJPE / PA / Cam.
CMU Panoptic MPJPE / PA / Cam.
HuMMan MPJPE / PA / Cam.
EgoBody MPJPE / PA / Cam.
3DB
63.8 / 38.0 / 266.7
60.9 / 33.6 / 122.9
41.2 / 31.0 / 76.7
57.5 / 32.1 / 97.4
3DB + depth correction
63.8 / 38.0 / 184.5
60.9 / 33.6 / 102.0
41.2 / 31.0 / 52.7
57.5 / 32.1 / 70.2
Ours (RGB)
50.6 / 31.2 / 188.2
47.1 / 28.2 / 104.6
43.6 / 31.4 / 81.2
49.8 / 30.4 / 90.1
Ours (RGB-Ground)
49.4 / 31.2 / 147.0
44.0 / 28.2 / 78.5
40.2 / 31.4 / 50.1
52.5 / 30.4 / 96.0
Ours (RGB-D)
50.4 / 30.2 / 166.1
47.7 / 28.0 / 98.5
37.9 / 30.4 / 39.7
46.8 / 29.7 / 47.5
TABLE II: Results on depth-enabled benchmarks: MPJPE / PA-MPJPE / camera-local MPJPE (mm; ↓ ). Best per metric in bold .
Fig. 3: Isaac Sim distance evaluation for a 1.12m -tall avatar. (a) Experiment setup. (b)–(d) Distance MAE at camera heights of 0.5m0.8m1.2m .
Method
Toddler MPJPE / PA / Cam.
Adult MPJPE / PA / Cam.
3DB
218 / 35 / 2001
58 / 29 / 108
Anny-Fit
117 / 41 / 1093
82 / 41 / 219
Ours (RGB)
196 / 36 / 2027
41 / 26 / 58
Ours (RGB-Ground)
78 / 36 / 451
43 / 26 / 74
Ours (RGB-D)
60 / 36 / 97
41 / 25 / 49
TABLE III: Results on CMU Panoptic toddler sequences.
Method
Views
MPJPE
PA-MPJPE
Cam.-local MPJPE
OpenCap
Multi
38.9
30.7
41.1
3DB
Single
48.8
34.6
138.1
Ours (RGB)
Single
46.3
32.3
133.9
Ours (RGB-Ground)
Single
42.8
32.4
96.3
TABLE IV: Comparison on the OpenCap dataset (mm; ↓ lower is better).
Disabled module
HuMMan Δ MPJPE / PA / Local
EMDB Δ MPJPE / PA / Local
Joint refiner
+2.10 / +0.62 / +1.55
+3.22 / +1.10 / +2.41
Backbone fusion
+0.52 / +0.25 / +0.43
+1.71 / +0.95 / +1.37
Depth-surface correction
+2.87 / 0.00 / +40.99
+3.90 / 0.00 / +49.46
TABLE V: Test-time depth-pathway knockouts. Entries are changes in MPJPE / PA-MPJPE / camera-local MPJPE after removal (mm); positive values indicate that the pathway is beneficial.