In this paper, we present a 3D human pose estimator that predicts metrically accurate human joint positions in the camera coordinate frame. This task requires recovering the configuration of the body joints, scale, and metric camera-local placement. In monocular images, body size and camera distance are coupled through perspective projection and therefore cannot be determined uniquely from image alone. This ambiguity can be reduced by combining three complementary sources of information: learned RGB priors, direct depth measurements, and geometric constraints (e.g., ground contact) from the environment. We extend a pretrained full-body estimator with optional metric depth and geometry-processing modules that improve camera-distance estimation and depth-guided body refinement while preserving RGB-only operation. The resulting unified model supports RGB, RGB-Ground, and RGB-D using shared learned parameters. Unlike pelvis-relative error, camera-local error also measures placement relative to the camera. Compared with the existing foundation model SAM 3D Body, our methods with RGB, RGB-Ground, and RGB-D modes reduce camera-local joint-position error by 12.16%, 31.00% and 54.61% respectively.
Figures & tables
Fig. 1: Monocular scale–distance ambiguity. (a) Input image and 2D keypoints. (b) Different body scales and distances yield similar projections. (c) Camera-coordinate visualization exposes errors in physical scale, distance, and ground contact, which metric depth and ground geometry help resolve.
Fig. 2: Unified architecture for RGB, RGB-Ground, and RGB-D with shared weights. SAM 3D Body provides the body and camera estimate; optional ground geometry applies a ground-contact correction, while optional depth adds feature fusion, joint-level pose refinement, and depth-surface metric correction.
Method
Intrinsics
3DPW (14) MPJPE / PA / Cam.
EMDB (24) MPJPE / PA / Cam.
RICH (24) MPJPE / PA / Cam.
CameraHMR
–
56.0 / 35.1 / –
70.3 / 43.3 / –
55.7 / 48.1 / –
PromptHMR
–
58.7 / 36.1 / –
71.7 / 41.0 / –
56.6 / 37.3 / –
NLF-L+fit ∗
–
54.9 / 33.6 / –
68.4 / 40.9 / –
51.0 / 28.7 / –
Fast 3DB
–
59.5 / 30.4 / –
64.3 / 37.3 / –
58.8 / 33.8 / –
SAM 3D Body
GT
55.0 / 33.7 / 248.4
63.1 / 38.8 / 123.4
52.8 / 31.8 / 342.2
MoGe-2
55.1 / 33.7 / 511.2
63.1 / 38.8 / 139.4
54.9 / 32.2 / 607.7
TABLE I: Results on 3DPW, EMDB, and RICH: MPJPE / PA-MPJPE / camera-local MPJPE (mm; ↓ lower is better).
Method
BEDLAM2 MPJPE / PA / Cam.
CMU Panoptic MPJPE / PA / Cam.
HuMMan MPJPE / PA / Cam.
EgoBody MPJPE / PA / Cam.
3DB
63.8 / 38.0 / 266.7
60.9 / 33.6 / 122.9
41.2 / 31.0 / 76.7
57.5 / 32.1 / 97.4
3DB + depth correction
63.8 / 38.0 / 184.5
60.9 / 33.6 / 102.0
41.2 / 31.0 / 52.7
57.5 / 32.1 / 70.2
Ours (RGB)
50.6 / 31.2 / 188.2
47.1 / 28.2 / 104.6
43.6 / 31.4 / 81.2
49.8 / 30.4 / 90.1
Ours (RGB-Ground)
49.4 / 31.2 / 147.0
44.0 / 28.2 / 78.5
40.2 / 31.4 / 50.1
52.5 / 30.4 / 96.0
Ours (RGB-D)
50.4 / 30.2 / 166.1
47.7 / 28.0 / 98.5
37.9 / 30.4 / 39.7
46.8 / 29.7 / 47.5
TABLE II: Results on depth-enabled benchmarks: MPJPE / PA-MPJPE / camera-local MPJPE (mm; ↓ ). Best per metric in bold .
Fig. 3: Isaac Sim distance evaluation for a 1.12m -tall avatar. (a) Experiment setup. (b)–(d) Distance MAE at camera heights of 0.5m0.8m1.2m .
Method
Toddler MPJPE / PA / Cam.
Adult MPJPE / PA / Cam.
3DB
218 / 35 / 2001
58 / 29 / 108
Anny-Fit
117 / 41 / 1093
82 / 41 / 219
Ours (RGB)
196 / 36 / 2027
41 / 26 / 58
Ours (RGB-Ground)
78 / 36 / 451
43 / 26 / 74
Ours (RGB-D)
60 / 36 / 97
41 / 25 / 49
TABLE III: Results on CMU Panoptic toddler sequences.
Method
Views
MPJPE
PA-MPJPE
Cam.-local MPJPE
OpenCap
Multi
38.9
30.7
41.1
3DB
Single
48.8
34.6
138.1
Ours (RGB)
Single
46.3
32.3
133.9
Ours (RGB-Ground)
Single
42.8
32.4
96.3
TABLE IV: Comparison on the OpenCap dataset (mm; ↓ lower is better).
Disabled module
HuMMan Δ MPJPE / PA / Local
EMDB Δ MPJPE / PA / Local
Joint refiner
+2.10 / +0.62 / +1.55
+3.22 / +1.10 / +2.41
Backbone fusion
+0.52 / +0.25 / +0.43
+1.71 / +0.95 / +1.37
Depth-surface correction
+2.87 / 0.00 / +40.99
+3.90 / 0.00 / +49.46
TABLE V: Test-time depth-pathway knockouts. Entries are changes in MPJPE / PA-MPJPE / camera-local MPJPE after removal (mm); positive values indicate that the pathway is beneficial.
Even in multi-camera setups, occlusions can leave individual body joints with only a single usable 2D observation. Such an observation defines a viewing ray but leaves the joint's depth undetermined. We introduce Body of Evidence (BoE), a multi-view 3D human pose estimator that predicts per-camera joint depths alongside 2D joint locations. To reduce monocular depth ambiguity, we propose geometry-conditioned depth prediction, which combines image features with previously reconstructed 3D body geometry as contextual depth cues. We further introduce learnable depth-aware triangulation (LDAT), which augments differentiable algebraic triangulation with predicted depth constraints, enabling reconstruction even from a single contributing camera while preserving the benefits of geometric triangulation when multiple views are available. The reconstructed 3D poses subsequently guide depth prediction in the next iteration, allowing depth estimates and 3D geometry to progressively refine each other. Experiments on CMU Panoptic demonstrate state-of-the-art in-domain performance and superior average performance across evaluated camera subsets and unseen camera arrangements. BoE is particularly effective under sparse camera coverage, outperforming the previous state of the art by 10.6 AP25 points in the challenging three-camera setting.
Head-mounted stereo fisheye cameras provide a compact view of the wearer's body, but strong distortion, partial visibility and a short baseline make 3D pose estimation difficult. Existing refinement decoders sample image features independently in each view, without requiring the samples to correspond to a common 3D hypothesis. We introduce EgoRayFormer, a causal pose estimator that couples stereo feature sampling and geometric refinement in ray space. For each joint, the decoder projects shared 3D hypotheses along and around its viewing ray into both images. A geometric branch then corrects the projected joint locations and intersects the resulting rays; a learned gate blends this geometric estimate with a learned pose update. This formulation introduces an explicit stereo constraint while retaining flexibility when the rays provide weak depth constraints. In local evaluations on the official device-relative test splits, EgoRayFormer achieves three-seed mean MPJPEs of 17.64 mm on UnrealEgo2 and 49.55 mm on UnrealEgo-RW, improving to 47.57 mm with UnrealEgo2 pre-training. In paired validation ablations, removing the geometric branch increases MPJPE in all three seeds at both input resolutions. With a common ResNet-18 encoder and input resolution, our system reduces RW test MPJPE by 9.09 mm relative to the device-relative EgoPoseFormer baseline. A separate protocol analysis quantifies the difference between device-relative and oracle-root evaluation.
We introduce HYPERPOSE, a novel 3D human pose estimation framework that performs spatio-temporal reasoning entirely within the Lorentz model of hyperbolic space Hd to natively preserve the hierarchical tree topology of the human skeleton. Current state-of-the-art pose estimators aim to capture complex joint dynamics by relying on transformers and graph convolutional networks. Since these architectures operate exclusively in Euclidean space which fundamentally mismatches the inherent tree structure of the human body, these methods inevitably suffer from exponential volume distortion and struggle to maintain structural coherence. To this end, we depart from flat spaces and aim to improve geometric fidelity with Hyperbolic Kinematic Phase-Space Attention (HKPSA), natively embedding complex joint relationships without distortion, alongside a multi-scale windowed hyperbolic attention mechanism that efficiently models temporal dynamics in O(TW) complexity. Furthermore, to overcome the well-known instability of training non-Euclidean manifolds, HYPERPOSE introduces a novel Riemannian loss suite and an uncertainty-weighted curriculum, enforcing physical geodesic constraints like bone length and velocity consistency. Extensive evaluations on the Human3.6M and MPI-INF-3DHP datasets demonstrate that HYPERPOSE achieves state-of-the-art structural and temporal coherence, significantly reducing both volume distortion and velocity error, while establishing new state-of-the-art benchmarks in overall positional accuracy.