A Diagnostic Study of World-Space Human Recovery Under High Camera Dynamics

Hao-Jen Chien; Wei-Lun Chao; Yu-Lun Liu

cs.CVOct 14, 2026

A Diagnostic Study of World-Space Human Recovery Under High Camera Dynamics

Authors: Hao-Jen Chien, Wei-Lun Chao, Yu-Lun Liu

Organizations: National Yang Ming Chiao Tung University · Boston University

Abstract

Monocular world-space human recovery methods excel on current benchmarks, but these rarely include fast camera motion. We introduce the K-Pop stress test, broadcast dance videos that combine fast, large camera motion with dense multi-person choreography. Leading methods produce plausible per-frame meshes on these videos, yet their world-space trajectories drift far from the performers' stage positions. Through a systematic diagnosis, we show that the camera and human estimates each remain self-consistent, and that the drift appears when they are combined at mismatched scales. Each is recovered at its own scale, so a mismatch between the two can turn camera motion into human drift that grows with camera displacement and becomes severe in broadcast dance videos. We derive a closed-form correction that estimates this mismatch and applies one scale to all performers, removing the drift while keeping their relative positions in one shared world frame. It requires no retraining and works with existing backends. Combined with physical constraints, the correction yields stable world-space reconstruction for choreography modeling, scene editing, data generation, and virtual reshooting. Project page: https://chien90190.github.io/worldhmr-fastcam-website/

Figures & tables

Figure 2: World-space recovery that succeeds on existing benchmarks fails under high camera dynamics. On EMDB ( Kaufmann et al., 2023 ) (left) , PromptHMR ( Wang et al., 2025 ) recovers an accurate trajectory with aligned reprojection. On K-Pop broadcast video (right) , its trajectories drift in the top view, and its meshes misalign in the reprojection. As the yellow boxes show, apparent size stays nearly constant on EMDB but changes rapidly under broadcast camera motion. Existing benchmarks seldom cover this regime, let alone combined with dense multi-person choreography.

Figure 3: Performer trajectories before and after drift correction. Left: World-space performer trajectories (colored) and the camera path (black) in the $x$ - $z$ plane, with the camera’s start (crossed box) and end (star). Before correction, the trajectories follow the camera’s sweep; after, they do not. Tiles are 1 m. Right: Start and end frames, with the stage center marked. Camera-space mesh overlays stay plausible despite the world-space drift.

Figure 4: The K-Pop stress test has faster, more abrupt camera motion than EMDB ( Kaufmann et al., 2023 ) and WorldPose ( Jiang et al., 2024 ) , and denser scenes than EMDB. (a) K-Pop cameras move about 6 to 8 times faster on average. The unit body is one body length, making the three sets comparable despite different framing. (b) The stress test spans a broader focal-length range and up to 12 performers per frame, against EMDB’s near-single-person clips.

Figure 5: ViPE ( Huang et al., 2025 ) camera trajectories on K-Pop clips remain self-consistent under the time-reversal shuttle test. Each clip is shown as a top-down view on the ground plane (

x

against

z

) and a side view against vertical height (

y

against

z

). The solid line is the forward pass and the dashed line the reversed pass, after a single similarity alignment. The rightmost panel is near at the K-Pop mean S-ATE of 1.35.

Figure 6: Scale mismatch turns camera motion into world-space drift. VO and HMR give two estimates of the same camera translation, $\Delta T$ and $\Delta T^{\prime}$ (Eq. 3 ). If they agree, the static performer stays in place (left) . If $\Delta T^{\prime}=s\Delta T$ with $s\neq 1$ , the performer drifts by $(1-s)\Delta T$ (right) . HMR sees the performer’s distance shrink by 5, but VO moves the camera by only 3. To keep the 5-unit shrink, composition moves the static performer to the orange position, 2 units closer to the camera. $\mathbf{X}_{2}$ is the performer’s camera-space position after the camera moves.

Method	Tracked	WA-MPJPE	W-MPJPE	PA-MPJPE	Pair dist.	Formation	Ratio
Method	(of 78) $\uparrow$	(cm) $\downarrow$	(cm) $\downarrow$	(cm) $\downarrow$	(cm) $\downarrow$	(cm) $\downarrow$	$\rightarrow 1$
TRAM ( Wang et al., 2024a )	72	57.80	99.08	6.48	43.97	30.28	0.77
PromptHMR ( Wang et al., 2025 )	66	49.25	70.60	7.74	42.07	34.67	0.80
GVHMR ( Shen et al., 2024 )	65	62.64	68.16	8.76	50.67	53.72	0.64
HAC ( Yang et al., 2025 )	72	64.77	116.30	6.33	57.52	35.74	0.66
Ours	72	39.89	47.02	6.20	29.01	32.97	0.90

Table 1: Ours is closest to the reference. Methods run on the broadcast view and are compared against the fixed-camera reference. Tracked counts recovered performers, and Ratio is defined in Section 4 . HAC ( Yang et al., 2025 ) shares our backbones. Red and orange mark the best and second-best results.

Figure 7: A clip-level fit exposes sliding that single frames hide. Sliding or a misplaced performer spreads a method’s positions wider than the reference (gray), lowering its ratio even when one moment looks close. Rows show the first and last frames. TRAM ( Wang et al., 2024a ) and HAC ( Yang et al., 2025 ) look the right size but slide, and one PromptHMR ( Wang et al., 2025 ) performer lands meters away (arrow). Tiles are 1 m.

Method	2D Reprojection		3D World Space (mm)
Method	PCK $\uparrow$	Mask IoU $\uparrow$	Jitter $\downarrow$	Foot Skating $\downarrow$
TRAM ( Wang et al., 2024a )	0.921	0.848	312.39	131.29
GVHMR ( Shen et al., 2024 )	0.210	0.524	30.46	31.57
PromptHMR ( Wang et al., 2025 )	0.802	0.581	$4.49\times 10^{6}$	$8.91\times 10^{5}$
HAC ( Yang et al., 2025 )	0.899	0.747	17.56	50.48
Ours	0.868	0.720	26.87	17.94

Table 2: On the K-Pop stress test, ours is in the top two on both world-space metrics. Red , orange , and yellow mark the top three. TRAM ( Wang et al., 2024a ) and HAC ( Yang et al., 2025 ) only rigidly transform, so their reprojection cannot reveal wrong scales. HAC is furthest from the reference (Table 1 ). TRAM is unstable; HAC skates 2.8

\times

more on the same backbones; GVHMR ( Shen et al., 2024 ) reprojects poorly; PromptHMR ( Wang et al., 2025 ) drifts severely.

Figure 8: Existing methods fail at 2D alignment, world-space stability, or both, while ours achieves both. Each method shows mesh reprojection, with a close-up, and world space with each performer’s path. One performer is in red. Tiles are 1 m. TRAM ( Wang et al., 2024a ) has erratic trajectories. GVHMR ( Shen et al., 2024 ) misplaces performers in world space, and its reprojection leaves the frame or misaligns. PromptHMR ( Wang et al., 2025 ) misaligns, misplaces, and drifts. HAC ( Yang et al., 2025 ) aligns but slides. Ours keeps both. Video comparisons: https://chien90190.github.io/worldhmr-fastcam-website/ .

Original methods with their native odometry
HMR	Odometry	2D Reprojection		3D World Space (mm)
HMR	Odometry	PCK ( $\uparrow$ )	Mask IoU ( $\uparrow$ )	Jitter ( $\downarrow$ )	Foot Skating ( $\downarrow$ )
PromptHMR ( Wang et al., 2025 )	masked DROID-SLAM ( Teed and Deng, 2021 )	0.802	0.581	$4.49\times 10^{6}$	$8.91\times 10^{5}$
+ drift correction		0.859	0.720	27.53	18.65
TRAM ( Wang et al., 2024a ) §		0.921	0.848	312.39	131.29
+ drift correction ‡		0.849	0.730	47.38	22.03

Table 3: Drift correction transfers across visual odometry and HMR backbones. Bold marks the better of each pair. † GVHMR in Table 2 keeps its own trajectory, aligned to DPVO ( Teed et al., 2023 ) once per track, so per-frame camera noise does not enter and its jitter there is lower. ‡ Contact confidence comes from PromptHMR-Vid, as TRAM predicts none. § TRAM ( Wang et al., 2024a ) only rigidly transforms camera-frame predictions, so reprojection returns the per-frame prediction, while its world-space motion inherits camera jitter and drift. On TRAM, our correction trades a small loss in 2D fit for a large gain in world-space stability.

Figure 9: Drift correction is needed even with PGO. (a) Adding drift correction improves every metric. (b) For stationary performers, each top-view trajectory should collapse to a point. Raw trajectories drift far (left) . PGO partially shortens the drift but leaves meshes misaligned (middle) . With drift correction, positions stay fixed, and meshes align (right) (Appendix E.2 ).

Figure 10: Under long occlusion, our method places a performer meters behind the group, and the placement recovers once the occlusion ends. Each panel is a top view with the camera at the arrow. The circled performer (red) is shown during and after the occlusion. All panels share one scale. Tiles are 1 m. PromptHMR ( Wang et al., 2025 ) and HAC ( Yang et al., 2025 ) fail the same way. GVHMR ( Shen et al., 2024 ) keeps the performer far. TRAM ( Wang et al., 2024a ) still jitters.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 11: Temporal repulsion improves multi-person consistency by preventing inter-performer collapse. Without it, nearby performers penetrate and form physically implausible configurations that persist over time. The term maintains separation between bodies and produces a more coherent world-space motion across frames.

Figure 12: Ablation of visual odometry. All three methods produce plausible 2D reprojections with reasonable subject alignment, but differ in their estimated camera trajectories. Based on the input video without ground truth, the camera starts from the right, pulls back, and pans toward the center of the group. DA3 captures lateral movement but misses the pull-back, while masked DROID-SLAM shows minimal lateral movement. ViPE most accurately recovers this trajectory, estimating both the backward pull and the pan toward the group center.

Method	3DPW (14)			RICH (24)			EMDB-2 (24)
Method	PA-MPJPE	MPJPE	PVE	PA-MPJPE	MPJPE	PVE	WA-MPJPE100	W-MPJPE100	Jitter	Foot Skating
WHAM ( Shin et al., 2024 )	37.5	59.8	71.5	44.3	80.0	91.2	135.6	354.8	22.5	4.4
TRAM ( Wang et al., 2024a )	35.6	59.3	69.6	—	—	—	76.4	222.4	18.5	23.4
GVHMR ( Shen et al., 2024 )	37.0	56.6	68.7	39.5	66.0	74.4	111.0	276.5	16.7	3.5
PromptHMR-Vid ( Wang et al., 2025 )	35.5 ∗	56.9 ∗	67.3 ∗	37.0 ∗	57.4 ∗	65.8 ∗	71.0	216.5	16.3	3.5

Appendix

Table 4: Camera-space HMR methods perform strongly on existing benchmarks, consistent with the frame-to-frame plausibility we observe on K-Pop broadcast video. Quantitative comparison of representative methods on 3DPW Von Marcard et al. (2018) , RICH ( Huang et al., 2022 ) , and EMDB-2 ( Kaufmann et al., 2023 ) , with the number of joints shown in parentheses. EMDB-2 results are in world space and are reported after alignment. All metrics are in millimeters. ∗ The 3DPW and RICH entries for PromptHMR-Vid ( Wang et al., 2025 ) are those reported for the single-image PromptHMR, which is the variant its authors evaluate in camera space.

Dataset	Setting	Domain	Camera motion	Annotation		Statistics
Dataset	Setting	Domain	Camera motion	GT Pose	Source	People/Frame	Frames	Pose Instances
H3.6M ( Ionescu et al., 2013 )	Controlled	Everyday	S	Yes	Calibrated	1	630k	630k
3DPW ( Von Marcard et al., 2018 )	In-the-wild	Everyday	M	Yes	IMU video	1–2	53k	75k
EMDB ( Kaufmann et al., 2023 )	In-the-wild	Everyday	M	Yes	EM sensors	1	105k	105k
BEDLAM-2 ( Tesch et al., 2025 )	Synthetic	Everyday	M	Yes	Synthetic	$>1$	380k	1M
AIST++ ( Li et al., 2021b )	Controlled	Dance	S	Yes	Calibrated	$\sim$ 1	10.1M ∗	—

Appendix

Table 5: The K-Pop stress test is the only in-the-wild benchmark of broadcast camera work. Camera motion is categorized as static (S), moving (M), and zooming (Z). AIST++ ( Li et al., 2021b ) reports 10.1M images from 9 views, so this count is not directly comparable to monocular video frame totals. DCM ( Wang et al., 2024c ) reports 193 minutes at 30 FPS, and DanceTrack ( Sun et al., 2022a ) provides bounding boxes but not pose annotations.

Figure 13: Paired-camera reference set. Four time-synchronized frames from one pair, numbered in the upper right. The fixed venue camera (top) holds its framing on the whole group, while the broadcast camera (bottom) translates, zooms, and crops performers out of frame over the same span.

Figure 14: Drift scale under a rescaled focal length. We multiply the backend-estimated focal length by a fixed multiplier and rerun the correction. Each curve is one clip and shows the change in estimated drift fraction relative to the unperturbed focal length. The change is linear in the multiplier, as expected when focal error acts as a scale degree of freedom, and its slope differs between clips because each carries a different scale.

Admitted intervals	$a^{*}$ (closed form)	$\Delta a$ (PGO)	$a$ (final)
125	0.310	$+0.008$	0.318
3	0.443	$-0.123$	0.320

Appendix

Table 6: Cutting the pooled intervals from 125 to 3 changes the closed-form estimate but not the final result. Both rows are the same sequence, and values are drift fractions

a=1-s

a^{*}

is the closed-form estimate of Eq. 6 ,

\Delta a

is the shift PGO applies through its learned residual scale, and

a

is the value after both stages.

Scale model	2D Reprojection		3D World Space (mm)
Scale model	PCK $\uparrow$	Mask IoU $\uparrow$	Jitter $\downarrow$	Foot Skating $\downarrow$
Constant scale	0.868	0.720	26.87	17.94
Time-varying scale	0.862	0.737	131.57	32.58

Appendix

Table 7: Constant against time-varying scale. A time-varying scale is the more expressive choice, yet it destabilizes the recovered world-space motion, and the damage falls on the temporal metrics while the 2D metrics hold. Bold marks the best value in each column. Both rows use ViPE ( Huang et al., 2025 ) and PromptHMR-Vid ( Wang et al., 2025 ) .

Contact threshold $\tau_{c}$	2D Reprojection		3D World Space (mm)
Contact threshold $\tau_{c}$	PCK $\uparrow$	Mask IoU $\uparrow$	Jitter $\downarrow$	Foot Skating $\downarrow$
0.3	0.866	0.718	26.34	16.83
0.5 (default)	0.868	0.720	26.87	17.94
0.8	0.872	0.726	26.38	19.09

Appendix

Table 8: The reconstruction is robust to the contact threshold

\tau_{c}

. The threshold sets which intervals enter the estimate. Across the tested range, with ViPE ( Huang et al., 2025 ) and PromptHMR-Vid ( Wang et al., 2025 ) , the 2D metrics and jitter barely move, and foot skating rises modestly. Every setting stays best on foot skating and second on jitter among the methods of Table 2 , behind only HAC ( Yang et al., 2025 ) . We use

\tau_{c}=0.5

Configuration	World-Space Displacement (m)		Stability (mm)
Configuration	Path length $\downarrow$	Net displacement $\downarrow$	Jitter $\downarrow$	Foot Skating $\downarrow$
Raw trajectory	5.206	2.334	105.128	47.269
PGO	0.849	0.581	12.465	7.568
PGO + drift correction	0.461	0.153	12.379	4.312

Appendix

Table 9: Drift correction reduces displacement. On 10 clips covering 60 performers who hold their stage positions, the expected world-space displacement is near zero, so smaller is closer to the truth. Jitter is unchanged between the last two rows, which is why it carries no bold entry. Bold marks the best value in each column. All rows use ViPE ( Huang et al., 2025 ) and PromptHMR-Vid ( Wang et al., 2025 ) .

Figure 15: User study. Share (%) of all votes each method received on overall quality and camera motion, pooled over 24 pairwise comparisons from 34 participants. Each baseline (TRAM ( Wang et al., 2024a ) , GVHMR ( Shen et al., 2024 ) , and PromptHMR ( Wang et al., 2025 ) ) appears in one third of the comparisons.

3D Human Reconstruction

3D Human Reconstruction recovers human shape, pose, appearance, and motion from images, video, or other sensor data.

See all 105 papers

DuoMo: Dual Motion Diffusion for World-Space Human Reconstruction
Mar 3, 2026Yufu Wang, Evonne Ng, Soyong Shin +83D Human Motion Reconstruction 3D Human Reconstruction
Meta Reality Labs · University of Pennsylvania · Carnegie Mellon University
Scene and Human in One World: Reconstruction in a Feedforward Pass
Jun 26, 2026Boao Shi, Qiao Feng, Yiming Huang +1Feed-Forward 3D Reconstruction Human Mesh Recovery
University of Pennsylvania, Philadelphia, Pennsylvania, USA
4D Human-Scene Reconstruction from Low-Overlap Captures
Jul 10, 2026Minhyuk Hwang, Sangmin Kim, Seunguk Do +24D Reconstruction Dynamic Scene Reconstruction
Seoul National University, Republic of Korea

Mar 3, 2026J/K move · Enter open · S save

DuoMo: Dual Motion Diffusion for World-Space Human Reconstruction

Yufu Wang, Evonne Ng, Soyong Shin, Rawal Khirodkar+7

Meta Reality Labs · University of Pennsylvania · Carnegie Mellon University

3D Human Motion Reconstruction 3D Human Reconstruction Markerless Motion Capture 3D Reconstruction Motion Diffusion Models

Open paper AskPDF arXiv

We present DuoMo, a generative method that recovers human motion in world-space coordinates from unconstrained videos with noisy or incomplete observations. Reconstructing such motion requires solving a fundamental trade-off: generalizing from diverse and noisy video inputs while maintaining global motion consistency. Our approach addresses this problem by factorizing motion learning into two diffusion models. The camera-space model first estimates motion from videos in camera coordinates. The world-space model then lifts this initial estimate into world coordinates and refines it to be globally consistent. Together, the two models can reconstruct motion across diverse scenes and trajectories, even from highly noisy or incomplete observations. Moreover, our formulation is general, generating the motion of mesh vertices directly and bypassing parametric models. DuoMo achieves state-of-the-art performance. On EMDB, our method obtains a 16% reduction in world-space reconstruction error while maintaining low foot skating. On RICH, it obtains a 30% reduction in world-space error. Project page: https://yufu-wang.github.io/duomo/

Abstract

3D Human Reconstruction

Explore similar work

DuoMo: Dual Motion Diffusion for World-Space Human Reconstruction

Scene and Human in One World: Reconstruction in a Feedforward Pass

4D Human-Scene Reconstruction from Low-Overlap Captures