We present DuoMo, a generative method that recovers human motion in world-space coordinates from unconstrained videos with noisy or incomplete observations. Reconstructing such motion requires solving a fundamental trade-off: generalizing from diverse and noisy video inputs while maintaining global motion consistency. Our approach addresses this problem by factorizing motion learning into two diffusion models. The camera-space model first estimates motion from videos in camera coordinates. The world-space model then lifts this initial estimate into world coordinates and refines it to be globally consistent. Together, the two models can reconstruct motion across diverse scenes and trajectories, even from highly noisy or incomplete observations. Moreover, our formulation is general, generating the motion of mesh vertices directly and bypassing parametric models. DuoMo achieves state-of-the-art performance. On EMDB, our method obtains a 16% reduction in world-space reconstruction error while maintaining low foot skating. On RICH, it obtains a 30% reduction in world-space error. Project page: https://yufu-wang.github.io/duomo/
Figures & tables
Figure 2: Method overview. (A) In the first stage, our camera-space model encodes video features and generates camera-space human motion. This motion is lifted to the world coordinates using estimated camera poses, becoming the initial proposal for world-space human motion. Some predictions are missing due to subject out of frame. In the second stage, the world-space model encodes the noisy world-space motion and generates globally consistent world-space motion. Plots at the bottom visualize the pelvis depth in the world coordinates. (B) Camera-space model architecture. (C) World-space model architecture.
Figure 3: Height conditioning . Our camera-space model can generate predictions based on input body heights. As shown at the bottom row, height impacts distance from camera and thus plays an important role in world-space accuracy.
EMDB (24)
RICH (24)
Models
PA-MPJPE
MPJPE
PVE
PA-MPJPE
MPJPE
PVE
per-frame
HMR2.0 [ 19 ]
61.5
97.8
120.0
60.7
98.3
120.8
ReFit [ 14 ]
58.6
88.0
104.5
47.9
80.7
92.9
CameraHMR [ 53 ]
43.3
70.2
81.7
34.0
55.7
64.4
NLF [ 60 ]
41.2
69.6
82.4
–
–
–
PromptHMR [ 77 ]
41.0
71.7
84.5
37.3
56.6
65.5
Table 1: Camera-space reconstruction on EMDB [ 28 ] and RICH [ 24 ] , with the number of joints in parenthesis. Our method does not use test-time flip augmentation. All metrics are in mm .
Visible segment
Full segment
Models
W-MPJPE
RTE
W-MPJPE
RTE
W-MPJPE-Occ
RTE-Occ
PHMR [ 77 ]
271.1
2.3
357.2
3.3
885.0
12.0
GVHMR [ 61 ]
189.1
3.9
209.4
3.9
384.1
6.1
Cam-model + Lifting
95.3
0.7
174.2
1.9
688.1
10.4
DuoMo
99.7
1.5
109.3
1.6
193.1
2.6
DuoMo w/ guidance
90.4
1.0
101.3
1.1
175.4
1.7
Table 2: Robust reconstruction on the Egobody [ 88 ] , evaluating both visible and invisible segments (e.g. person out of frame). All methods use ground truth camera poses in this evaluation.
Methods
WA-MPJPE
W-MPJPE
RTE
Jitter
Foot Skating
World-space model (one stage)
153.5
445.1
6.7
9.1
4.8
Cam-space model + Lifting
67.0
180.2
1.3
32.6
9.2
DuoMo
66.0
167.1
1.1
8.7
3.7
Table 3: Ablation for DuoMo design on EMDB [ 28 ] . DuoMo has higher accuracy and motion quality than using only one model.
EMDB (24)
RICH (24)
Models
WA-MPJPE
W-MPJPE
RTE
Jitter
Foot-sliding
WA-MPJPE
W-MPJPE
RTE
Jitter
Foot-sliding
TRACE [ 68 ]
529.0
1702.3
17.7
2987.6
370.7
238.1
925.4
610.4
1578.6
230.7
GLAMR [ 83 ]
280.8
726.6
11.4
46.3
20.7
129.4
236.2
3.8
49.7
18.1
SLAHMR [ 82 ]
326.9
776.1
10.2
31.3
14.5
98.1
186.4
28.9
34.3
5.1
WHAM [ 62 ]
135.6
354.8
6.0
22.5
4.4
109.9
184.6
4.1
19.7
3.3
GVHMR [ 61 ]
111.0
276.5
2.0
16.7
3.5
78.8
126.3
2.4
12.8
3.0
Table 4: World-space reconstruction on the EMDB [ 28 ] and RICH [ 24 ] datasets. We do not use test-time flip augmentation. Our method uses the same estimated camera motion as TRAM [ 76 ] and GENMO [ 38 ] , providing a meaningful comparison.
Figure 4: Guided sampling on Egobody [ 44 ] . Our proposed guidances correct for drifting and improve world-space trajectory accuracy.
DuoMo w/
WA-MPJPE
W-MPJPE
RTE
Jitter
Foot Skating
World-Model-SMPL
70.1
182.5
1.3
7.4
4.8
World-Model-Mesh
65.7
164.8
1.1
8.5
3.8
Table 5: Ablation for motion representation on EMDB [ 28 ] . Directly generating sparse mesh is very competitive in terms of motion reconstruction accuracy.
Figure 5: Qualitative comparison on Egobody [ 88 ] . All methods use the ground truth camera poses. We observe that results from GVHMR [ 61 ] are smooth but drift under shaky camera motion. PromptHMR [ 77 ] has better position accuracy but is not robust to occlusion and depth ambiguity. Our results show both accuracy and robustness.
Figure 6: Impact of camera error . Comparison of methods under different camera motion noise levels.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Sparse mesh . (a) Topology of our sparse mesh, with the red box showing the details of the hands. (b) An example from the BEDLAM [ 5 ] dataset with its annotation converted to the sparse mesh format.
Figure 8: Architecture (sparse mesh to SMPLX) . This network performs iterative refinement to predict SMPLX parameters from a target sparse mesh.
Figure 9: Sparse mesh to SMPLX estimation . Examples of the predicted SMPLX mesh (black) overlaid on the target sparse mesh (white).
Figure 10: Dense keypoint detection on the EMDB dataset [ 28 ] . Edges added for visualization only.
Figure 11: Limitation 1 . Because our models do not incorporate 3D scene information, the results exhibit inconsistencies in fine-grained details.
Figure 12: Limitation 2 . In challenging poses, the generated sparse meshes sometime exhibit unrealistic deformation. Converting them to SMPL meshes (sec A.3 ) partially improves the results.