PHUMA: Physically Reliable Humanoid Locomotion Dataset

Kyungmin Lee; Sibeen Kim; Youngdo Lee; Minho Park; Hyunseung Kim; Dongyoon Hwang; Donghu Kim; Hojoon Lee

cs.ROOct 30, 2025

PHUMA: Physically Reliable Humanoid Locomotion Dataset

Authors: Kyungmin Lee, Sibeen Kim, Youngdo Lee, Minho Park, Hyunseung Kim, Dongyoon Hwang, Donghu Kim, Hojoon Lee, +1 more

Organizations: KAIST

Abstract

Motion imitation is a promising approach for humanoid locomotion, enabling agents to acquire humanlike behaviors. Existing methods typically rely on high-quality motion capture datasets such as AMASS, but these are scarce and expensive, limiting scalability and diversity. Recent studies attempt to scale data collection by converting large-scale internet videos, exemplified by Humanoid-X. However, they often suffer from physical artifacts such as floating, penetration, and foot skating, which hinder stable imitation. To address this, we introduce PHUMA, a Physically Reliable HUMAnoid locomotion dataset produced by a two-stage pipeline combining physics-aware curation and physics-constrained retargeting, aggregating both motion capture and internet video into a physically reliable, 73-hour corpus. On motion tracking benchmarks, PHUMA-trained policies achieve higher success rates than those trained on AMASS and Humanoid-X, and successfully transfer zero-shot to a real Unitree G1. The code is available at https://davian-robotics.github.io/PHUMA.

Figures & tables

Figure 1: Physical Reliability of Humanoid-X vs. PHUMA . Each column illustrates four failure modes: joint violation, floating, penetration, and skating. Humanoid-X [ 1 ] (top row) often exhibits these issues due to direct video-to-motion conversion, while PHUMA (bottom row) mitigates those violations through careful data curation and physically reliable retargeting.

Figure 2: Overview of datasets and performance. (a) Composition of feasible and infeasible human motion sources in each dataset. (b) Physical reliability of each dataset; AMASS is retargeted using a standard learning-based inverse kinematics method. (c) Success rate on unseen motions. (d) Motion tracking error on real Unitree G1.

Figure 3: Overview of the PHUMA pipeline. Our four-stage pipeline for motion imitation learning includes: (1) Motion Curation, where we filter out problematic motions from a diverse dataset; (2) Motion Retargeting, where the filtered motions are retargeted to the humanoid using PhySINK; (3) Policy Learning, where a policy is trained to imitate the retargeted motions; and (4) Inference, where the trained policy is used to control the humanoid, enabling it to imitate motions from unseen videos processed by a video-to-motion model.

	Motion Fidelity (%)	Joint Feasibility (%)	Non-Floating (%)	Non-Penetration (%)	Non-Skating (%)
IK	27.6	91.7	55.6	47.8	59.7
GMR	56.3	81.8	14.7	100.0	67.7
SINK	94.8	95.9	96.4	14.9	55.4
+ Joint Feasibility Loss	94.9	100.0	96.4	14.8	55.6
+ Grounding Loss	94.9	100.0	99.9	97.2	53.6
+ Skating Loss = PhySINK	94.8	100.0	99.9	96.8	89.7

Table 1: Quantitative comparison and ablation study of retargeting methods. We evaluate performance on the G1 humanoid, showing the progressive impact of adding each of our proposed physical constraint losses. (H1-2 results in Appendix G )

		Curation Filter				PHUMA Test					Unseen Video
Variant	Hours	Jerk	FC	Height	BoS	Total	Stat.	Ang.	Vert.	Horiz.	Total	Stat.	Ang.	Vert.	Horiz.
Humanoid-X	231	✗	✗	✗	✗	50.6	78.4	43.0	26.0	31.8	39.1	78.0	39.6	23.0	6.5
+ Jerk only	141	✓	✗	✗	✗	83.0	91.5	77.6	72.3	88.5	77.6	100.0	78.8	62.8	61.5
+ FC only	123	✗	✓	✗	✗	82.9	91.3	77.8	70.4	89.1	78.2	99.1	79.3	61.7	65.9
+ Height only	136	✗	✗	✓	✗	80.6	90.7	74.6	69.3	85.1	74.0	98.3	76.4	61.7	50.5
+ BoS only	110	✗	✗	✗	✓	81.3	90.2	76.2	72.0	84.4	74.8	96.6	79.8	54.3	57.1

Table 2: Imitation Performance: Effect of Physics-Aware Curation. MaskedMimic policies trained on Humanoid-X subsets with varying curation filters, on Unitree G1. All variants use SINK retargeting to isolate filtering effect. Hours: remaining dataset size after filtering.

	PHUMA Test					Unseen Video
Retarget	Total	Stationary	Angular	Vertical	Horizontal	Total	Stationary	Angular	Vertical	Horizontal
IK	52.8	75.3	43.9	24.3	44.2	54.6	89.3	54.6	32.7	43.3
GMR	61.7	80.2	51.7	32.5	74.0	59.3	87.1	64.5	31.9	40.7
SINK	76.2	88.5	72.1	56.8	66.8	70.2	90.7	75.0	62.7	44.1
PhySINK	79.5	89.9	76.1	61.1	69.5	72.8	93.3	78.2	65.6	47.3

Table 3: Motion tracking performance across retargeting approaches. Success rates of policies trained on AMASS data retargeted by IK, GMR, SINK, and PhySINK, evaluated on the G1 humanoid robot using two test sets: PHUMA Test and Unseen Video. (H1-2 results in Appendix G )

		PHUMA Test					Unseen Video
Dataset	Hours	Total	Stationary	Angular	Vertical	Horizontal	Total	Stationary	Angular	Vertical	Horizontal
LaFAN1	2.4	46.1	66.1	36.2	24.0	42.5	28.4	46.9	28.4	19.6	10.5
AMASS	20.9	76.2	88.5	72.1	56.8	66.8	78.2	90.7	75.0	62.7	44.1
Humanoid-X	231.4	50.6	78.4	43.0	26.0	31.8	39.1	78.0	39.6	23.0	6.5
PHUMA	73.0	92.7	95.6	91.7	86.0	85.6	82.9	96.7	88.0	71.8	67.1

Table 4: Motion tracking performance across datasets. Success rates of policies trained on LaFAN1, AMASS, Humanoid-X, and PHUMA, evaluated across motion categories on the G1 humanoid robot using two test sets: PHUMA Test and Unseen Video. (H1-2 results in Appendix G )

	$E_{\mathrm{mpjpe}}$ (mm) $\downarrow$					$E_{\mathrm{dof\_vel}}$ (deg/frame) $\downarrow$					$E_{\mathrm{dof\_acc}}$ (deg/frame 2 ) $\downarrow$
Dataset	Total	Stat.	Ang.	Vert.	Hor.	Total	Stat.	Ang.	Vert.	Hor.	Total	Stat.	Ang.	Vert.	Hor.
AMASS	41.0	27.3	47.5	49.9	39.4	3.20	1.81	4.10	3.14	3.77	1.50	0.77	2.09	1.33	1.80
PHUMA	34.3	23.2	40.4	42.6	31.1	2.69	1.41	3.07	2.68	3.60	1.29	0.58	1.43	1.18	1.96

Table 5: Real-World Tracking Results. Tracking errors of policies trained on AMASS and PHUMA, evaluated on 6 motions per category from the PHUMA test set, on the Unitree G1.

Figure 4: Real-World Deployment . Each row shows the reference motion (top), the motion tracked by the AMASS-trained policy (middle), and the motion tracked by the PHUMA-trained policy (bottom), demonstrating that PHUMA achieves closer alignment with the reference across all sequences.

Appendix figures & tables44 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: SMPL-X Foot Vertices for Ground-Contact Detection . This figure illustrates the selected foot vertices on the SMPL-X model used to detect ground contact. Blue and green points denote the left heel and left toe, while yellow and red represent the right heel and right toe, respectively. The remaining foot vertices are shown in light-gray. The clusters of colored points correspond to the specific parts of the foot that are used to check for contact with the ground, making the process more accurate and robust than using a single point.

Region	Vertex indices
Left heel	8888, 8889, 8891, 8909, 8910, 8911, 8913, 8914, 8915, 8916, 8917,
	8918, 8919, 8920, 8921, 8922, 8923, 8924, 8925, 8929, 8930, 8934
Left toe	5773, 5781, 5782, 5791, 5793, 5805, 5808, 5816, 5817, 5830, 5831,
	5859, 5860, 5906, 5907, 5908, 5909, 5912, 5914, 5915, 5916, 5917
Right heel	8676, 8677, 8679, 8697, 8698, 8699, 8701, 8702, 8703, 8704, 8705,
	8706, 8707, 8708, 8709, 8710, 8711, 8712, 8713, 8714, 8715, 8716

Appendix

Table 6: SMPL-X foot vertex indices used for ground–contact detection.

Figure 6: Procedure of Majority-Vote Ground Estimation. (1) shows how the foot-height distribution is obtained across all frames of a single human motion trajectory, and (2) shows how the ground height is selected from this distribution by majority vote.

Figure 7: Examples of Calculating the Ground-Contact Ratio. This figure illustrates how the ground-contact ratio is computed, which is used in the foot-contact score for physics-aware curation and in the ground and skating losses for PhySINK.

Metric	Threshold
Root jerk	$<50$ m/s 3
Foot contact score	$>0.6$
Minimum pelvis height	$>0.6$ m
Maximum pelvis height	$<1.5$ m
Pelvis distance to base of support	$<6$ cm
Spine1 distance to base of support	$<11$ cm

Appendix

Table 7: Physics-aware data filtering metrics and thresholds.

Figure 8: Distribution of Each Curation Metric over Humanoid-X. The dashed line marks each threshold with its percentile, and motions in the shaded region marked with a cross are discarded.

Panel of Figure 9 — Figure 9: Comparison of Motion Distributions. (a) Comparison between MoCap and Video sources in PHUMA. (b) Comparison with other existing datasets.

Dataset	# Clip	# Frame	Duration	Source	Scene
LocoMuJoCo [ 21 ]	0.78K	0.93M	0.86h	Motion Capture	Indoor
GRAB [ 85 ]	1.73K	0.20M	1.88h	Motion Capture	Indoor
EgoBody [ 20 ]	2.12K	0.24M	2.19h	Motion Capture	Indoor
LAFAN1 [ 12 ]	2.18K	0.26M	2.40h	Motion Capture	Indoor
AMASS [ 13 ]	21.73K	2.25M	20.86h	Motion Capture	Indoor
HAA500 [ 29 ]	1.76K	0.11M	1.01h	Human Video	Outdoor

Appendix

Table 8: Composition of the PHUMA dataset. A summary of the number of clips and duration for each sub-dataset, categorized by source and scene. PHUMA aggregates these diverse sub-datasets, resulting in 73.0 hours of physically reliable motion clips.

Figure 10: Qualitative Comparison of Retargeting Methods . This figure provides a visual comparison of human motion retargeted to a humanoid robot using the IK, SINK, and PhySINK methods. The top row shows the original human motion from the SMPL model, while the rows below show the resulting motions for each retargeting method.

Figure 11: Qualitative Comparison of Motion Retargeting: GMR vs. PhySINK. Comparison showing the limitations of GMR when handling extreme human heights. The top row illustrates retargeting from a short subject, where GMR causes excessive motion compression and crouching. The bottom row illustrates retargeting from a tall subject, where GMR generates infeasible targets, resulting in artifacts like joint locking (e.g., rigid knees) and loss of ground contact. PhySINK maintains kinematic feasibility in both cases.

	PHUMA Test					Unseen Video
Dataset	Total	Stationary	Angular	Vertical	Horizontal	Total	Stationary	Angular	Vertical	Horizontal
GMR	84.0	92.1	77.8	77.1	89.1	75.2	99.1	77.8	61.7	52.7
PhySINK	89.9	94.2	87.6	84.2	91.8	81.7	97.4	86.7	61.7	71.4

Appendix

Table 9: Imitation Performance: GMR vs. PhySINK Retargeting on Unitree G1. We evaluate the imitation performance of the MaskedMimic policy trained using datasets retargeted with GMR and PhySINK on the Unitree G1.

Retarget	FPS
IK	38.4
GMR	25.3
SINK	0.95
PhySINK	0.92

Appendix

Table 10: Retargeting Throughput. Frames per second of each retargeting method on a single RTX 3090 GPU.

Retarget	Open-loop SR (%)
GMR $+$ Sampling (SPIDER)	4.6
PhySINK $+$ Sampling	10.7

Appendix

Table 11: PhySINK as Initialization for Sampling-Based Retargeting. Open-loop success rate (%) on the Unseen Video motions when the sampling-based optimization of SPIDER starts from GMR, as in SPIDER, or from PhySINK.

(a) Proprioceptive State
	Dimension
State	G1	H1-2
Root height	1	1
Body position	$32\times 3$	$24\times 3$
Body rotation	$33\times 6$	$25\times 6$
Body velocity	$33\times 3$	$25\times 3$

Appendix

Table 12: Observation Space Dimensions of MaskedMimic.

(a) Proprioceptive State
State	Dimension (G1)
Previous action	29
Base angular velocity	3
DoF position	29
DoF velocity	29
(b) Goal State

Appendix

Table 13: Observation Space Dimensions of BeyondMimic.

Future	IsaacSim					MuJoCo
Future	SR $\uparrow$	$E_{\text{g-mpjpe}}\downarrow$	$E_{\text{l-mpjpe}}\downarrow$	$E_{\text{acc}}\downarrow$	$E_{\text{vel}}\downarrow$	SR $\uparrow$	$E_{\text{g-mpjpe}}\downarrow$	$E_{\text{l-mpjpe}}\downarrow$	$E_{\text{acc}}\downarrow$	$E_{\text{vel}}\downarrow$
(a) PHUMA test
✗	47.15	115.78	32.70	1.15	4.29	46.91	156.64	32.92	1.22	4.36
✓	78.77	112.00	28.28	1.18	4.14	76.02	142.79	28.98	1.24	4.18
(b) Unseen Video
✗	47.82	123.71	31.93	1.07	4.93	47.42	122.89	30.60	1.09	4.85

Appendix

Table 14: Sim-to-Sim Results of PHUMA-trained BeyondMimic Policies. This table demonstrates the effect of the future-trajectory observation on the policy for the Unitree G1, evaluated on the PHUMA Test and Unseen Video splits. SR is the whole-body success rate at

0.5\,\text{m}

(%);

E_{\text{g-mpjpe}}

E_{\text{l-mpjpe}}

E_{\text{acc}}

, and

E_{\text{vel}}

are tracking errors (mm), following the convention of [ 37 , 9 , 11 ] and averaged over successful motions.

Term	Expression	Weight
(a) Task
Global body position	$\exp(-100\cdot\lVert p_{t}-\hat{p}_{t}\rVert^{2}_{2})$	0.5
Root height	$\exp(-100\cdot(h^{\text{root}}_{t}-\hat{h}^{\text{root}}_{t})^{2})$	0.2
Global body rotation	$\exp(-10\cdot\lVert\theta_{t}\ominus\hat{\theta}_{t}\rVert^{2}_{2})$	0.3
Global body velocity	$\exp(-0.5\cdot\lVert v_{t}-\hat{v}_{t}\rVert^{2}_{2})$	0.1
Global body angular velocity	$\exp(-0.1\cdot\lVert\omega_{t}-\hat{\omega}_{t}\rVert^{2}_{2})$	0.1

Appendix

Table 15: Reward Function Terms of MaskedMimic.

Term	Expression	Weight
(a) Motion Tracking
Reference body position	$\exp(-\lVert p^{\text{ref}}_{t}-\hat{p}^{\text{ref}}_{t}\rVert^{2}_{2}\,/\,0.3^{2})$	0.5
Reference body rotation	$\exp(-\lVert\theta^{\text{ref}}_{t}\ominus\hat{\theta}^{\text{ref}}_{t}\rVert^{2}_{2}\,/\,0.4^{2})$	0.5
Global body linear velocity	$\exp\!\left(-\tfrac{1}{B}\sum_{b}\lVert v^{b}_{t}-\hat{v}^{b}_{t}\rVert^{2}_{2}\,/\,1.0^{2}\right)$	1.0
Global body angular velocity	$\exp\!\left(-\tfrac{1}{B}\sum_{b}\lVert\omega^{b}_{t}-\hat{\omega}^{b}_{t}\rVert^{2}_{2}\,/\,\pi^{2}\right)$	1.0
Relative body position	$\exp\!\left(-\tfrac{1}{B}\sum_{b}\lVert p^{b}_{t}-\hat{p}^{b}_{t}\rVert^{2}_{2}\,/\,0.3^{2}\right)$	1.0

Appendix

Table 16: Reward Function Terms of BeyondMimic.

Hyperparameter	Value
Optimizer	Adam
Num envs	8192
Mini Batches	32
Learning epochs	1
Entropy coefficient	0.0
Value loss coefficient	0.5

Appendix

Table 17: PPO Hyperparameter Values for Model Training of MaskedMimic.

Hyperparameter	Value
Optimizer	AdamW
Num envs	8192
Steps per env	24
Mini batches	4
Learning epochs	8
Entropy coefficient	0.005

Appendix

Table 18: PPO Hyperparameter Values for Model Training of BeyondMimic.

Parameter	Range
P-gain scale $s_{P}$	$\mathcal{U}(0.9,1.1)$
D-gain scale $s_{D}$	$\mathcal{U}(0.9,1.1)$
DoF position bias	$\mathcal{U}(-0.025,0.025)\,\text{rad}$
Base CoM offset $x$	$\mathcal{U}(-0.025,0.025)\,\text{m}$
Base CoM offset $y,z$	$\mathcal{U}(-0.05,0.05)\,\text{m}$
Static friction	$\mathcal{U}(0.3,1.6)$

Appendix

Table 19: Domain Randomization Ranges of BeyondMimic.

Figure 12: Overview of the Self-collected Data Pipeline . This figure illustrates the three main steps of our data collection pipeline: (left) a self-recorded video of a human motion, (center) the motion extracted using a video-to-motion model, and (right) the final motion retargeted to a humanoid robot.

(a) G1
		Success Threshold=0.15m					Success Threshold=0.5m
Dataset	Hours	Total	Stationary	Angular	Vertical	Horizontal	Total	Stationary	Angular	Vertical	Horizontal
LaFAN1	2.4	46.1	66.1	36.2	24.0	42.5	74.8	87.8	69.2	47.1	72.6
AMASS	20.9	76.2	88.5	72.1	56.8	66.8	90.2	95.0	87.9	81.1	83.7
Humanoid-X	231.4	50.6	78.4	43.0	26.0	31.8	78.4	91.3	72.9	59.5	65.9
PHUMA	73.0	92.7	95.6	91.7	86.0	85.6	97.1	98.7	96.5	94.4	92.5

Appendix

Table 20: Performance Comparison based on Success Threshold in PHUMA Test.

(a) G1
		Success Threshold=0.15m					Success Threshold=0.5m
Dataset	Hours	Total	Stationary	Angular	Vertical	Horizontal	Total	Stationary	Angular	Vertical	Horizontal
LaFAN1	2.4	28.4	46.9	28.4	19.6	10.5	78.2	85.5	70.8	76.3	80.8
AMASS	20.9	70.2	90.7	75.0	62.7	44.1	92.3	99.2	92.1	82.1	88.0
Humanoid-X	231.4	39.1	78.0	39.6	23.0	6.5	84.1	98.3	79.9	76.0	76.2
PHUMA	73.0	82.9	96.7	88.0	71.8	67.1	93.7	100.0	96.8	85.9	84.7

Appendix

Table 21: Performance Comparison based on Success Threshold in Unseen Video.

	PHUMA Test					Unseen Video
Dataset	Total	Stationary	Angular	Vertical	Horizontal	Total	Stationary	Angular	Vertical	Horizontal
AMASS	77.8	90.8	71.2	58.3	84.7	83.5	100.0	83.3	74.5	72.5
PHUMA	91.0	96.4	88.3	87.6	90.5	87.1	99.1	89.2	77.7	76.9

Appendix

Table 22: Imitation Performance of the KungfuBot Teacher Policy in Isaac Gym. We evaluate the imitation performance of KungfuBot teacher policy with unseen motions on the Unitree G1.

	PHUMA Test					Unseen Video
Dataset	Total	Stationary	Angular	Vertical	Horizontal	Total	Stationary	Angular	Vertical	Horizontal
AMASS	66.6	86.7	58.8	41.8	68.4	67.7	97.4	68.0	59.6	37.4
PHUMA	82.9	93.5	78.5	74.2	81.7	73.8	97.4	76.9	63.8	47.3

Appendix

Table 23: Imitation Performance of the KungfuBot Student Policy in Isaac Gym. We evaluate the imitation performance of KungfuBot student policy with unseen motions on the Unitree G1.

	PHUMA Test					Unseen Video
Dataset	Total	Stationary	Angular	Vertical	Horizontal	Total	Stationary	Angular	Vertical	Horizontal
AMASS	62.1	81.4	54.2	38.8	64.3	64.3	86.2	68.5	54.3	37.4
PHUMA	75.0	87.6	69.3	61.6	76.3	70.0	87.9	78.8	59.6	38.5

Appendix

Table 24: Sim-to-Sim transfer performance across motion dataset. We evaluate the zero-shot motion tracking success rate of policies trained on PHUMA and AMASS when transferred from the source simulator (Isaac Gym) to the target simulator (MuJoCo). The results are demonstrated on the G1 humanoid to assess robustness against domain shifts in physics engines.

Retarget	PHUMA Test (GMR)	Unseen Video (GMR)
IK	80.3 ( $\pm$ 0.2)	81.6 ( $\pm$ 0.6)
GMR	80.0 ( $\pm$ 0.1)	81.1 ( $\pm$ 0.3)
SINK	81.6 ( $\pm$ 0.2)	79.8 ( $\pm$ 0.6)
PhySINK	83.4 ( $\pm$ 0.2)	83.7 ( $\pm$ 0.4)

Appendix

Table 25: Imitation Performance on GMR-Retargeted References: Retargeting Methods. Success rate (%) at the 0.5m threshold of policies trained on AMASS retargeted by each method, evaluated on the PHUMA Test and Unseen Video motions retargeted with GMR. Mean (

\pm

std) over three seeds.

Dataset	PHUMA Test (GMR)	Unseen Video (GMR)
LaFAN1	76.5 ( $\pm$ 0.1)	81.6 ( $\pm$ 0.9)
Humanoid-X	77.7 ( $\pm$ 0.1)	79.6 ( $\pm$ 0.5)
AMASS (SINK)	81.6 ( $\pm$ 0.2)	79.8 ( $\pm$ 0.6)
AMASS (PhySINK)	83.4 ( $\pm$ 0.2)	83.7 ( $\pm$ 0.4)
PHUMA	95.5 ( $\pm$ 0.2)	89.4 ( $\pm$ 0.4)

Appendix

Table 26: Imitation Performance on GMR-Retargeted References: Datasets. Success rate (%) at the 0.5m threshold of policies trained on each dataset, evaluated on the PHUMA Test and Unseen Video motions retargeted with GMR. Mean (

\pm

std) over three seeds.

Dataset	PHUMA Test FR (%) $\downarrow$	Unseen Video FR (%) $\downarrow$
Humanoid-X	18.6	13.1
AMASS (SINK)	5.7	4.2

Appendix

Table 27: Fall Rate in Simulation. Percentage of episodes in which the pelvis height drops below 0.3m, for policies trained on Humanoid-X and on AMASS retargeted with SINK. Lower is better.

	Motion Fidelity (%)	Joint Feasibility (%)	Non-Floating (%)	Non-Penetration (%)	Non-Skating (%)
MoCap	96.7	100.0	99.9	94.3	92.1
Video	93.8	100.0	99.9	98.0	88.3

Appendix

Table 28: Performance Evaluation of PHUMA based on Data Source (MoCap vs. Video). We present a quantitative comparison evaluating the performance achieved using PHUMA data derived from motion capture (MoCap) versus video, concluding that both sources offer competitive results.

	PHUMA Test					Unseen Video
Dataset	Total	Stationary	Angular	Vertical	Horizontal	Total	Stationary	Angular	Vertical	Horizontal
MoCap	75.2	88.2	68.4	49.0	86.9	73.0	96.6	73.9	56.4	58.2
Video	85.7	92.9	81.8	75.5	89.5	76.2	100.0	76.4	59.6	62.6

Appendix

Table 29: Imitation Performance of PHUMA based on Data Source (MoCap vs. Video). We evaluate the imitation performance of MaskedMimic policy trained with PHUMA data derived from motion capture (MoCap) versus video on the Unitree G1.

Data Source	PHUMA Test (GMR)	Unseen Video (GMR)
MoCap Only	82.2 ( $\pm$ 0.2)	86.0 ( $\pm$ 0.2)
$+$ Video (33%)	82.1 ( $\pm$ 0.1)	80.1 ( $\pm$ 0.8)
$+$ Video (66%)	92.8 ( $\pm$ 0.1)	86.7 ( $\pm$ 0.5)
$+$ Video (100%)	94.6 ( $\pm$ 0.1)	90.2 ( $\pm$ 0.9)

Appendix

Table 30: Imitation Performance When Adding Video to Mocap. Success rate (%) at the 0.5m threshold on GMR-retargeted references, for policies trained on the mocap-sourced motions plus a growing fraction of the video-sourced motions. Mean (

\pm

std) over three seeds.

Figure 13: Dataset Statistics After Physics-based Filtering. Distribution of motion sequences after applying physics-based filtering to the combined Humanoid-X, LaFAN1, and LocoMuJoCo datasets.

Motion Source	Hours	Motion Fidelity (%)	Joint Feasibility (%)	Non-Floating (%)	Non-Penetration (%)	Non-Skating (%)
Humanoid-X	237.2	70.6	100.0	98.7	92.2	90.1
Jerk Filter	141.1	76.9	100.0	99.7	96.5	92.0
Foot Contact Filter	123.1	77.1	100.0	99.8	96.8	90.6
Height Filter	135.7	87.0	100.0	99.4	94.7	89.8
BoS Filter	110.3	90.7	100.0	99.4	94.9	90.0
All Filter	62.2	94.8	100.0	99.9	96.7	89.7

Appendix

Table 31: PhySINK Retargeting Robustness to Noisy Motion Sources. This figure presents an experiment to evaluate how robustly the PhySINK method retargets various noisy Humanoid-X motion sources. The six distinct motion source groups used for retargeting are compared: (1) Original Humanoid-X, and Humanoid-X motion sources sequentially refined by applying (2) Jerk filtering, (3) Foot Contact filtering, (4) Height filtering, (5) BoS filtering, and (6) All filtering (PHUMA).

Curation Filter	Hours	PHUMA Test (GMR)	Unseen Video (GMR)
All Filters	62	94.6 ( $\pm$ 0.1)	90.2 ( $\pm$ 0.9)
No Jerk Filter	75	91.8 ( $\pm$ 0.1)	89.5 ( $\pm$ 0.3)
No FC Filter	77	87.5 ( $\pm$ 0.1)	80.0 ( $\pm$ 0.5)
No Height Filter	73	89.2 ( $\pm$ 0.3)	85.6 ( $\pm$ 0.1)
No BoS Filter	88	89.5 ( $\pm$ 0.1)	87.4 ( $\pm$ 0.7)

Appendix

Table 32: Leave-One-Out Ablation of Curation Filters. Success rate (%) at the 0.5m threshold on GMR-retargeted references, for policies trained on motions curated with all filters or with one filter removed, all retargeted with PhySINK. Hours give the curated volume. Mean (

\pm

std) over three seeds.

Reconstruction	Acceptance	Retarget	Retargeting Quality
Reconstruction	Rate (%)	Retarget	MF (%)	JF (%)	NF (%)	NP (%)	NS (%)
TRAM	41.9	SINK	95.9	63.6	100.0	27.2	40.1
TRAM	41.9	PhySINK	95.9	100.0	100.0	99.7	93.8
GVHMR	77.4	SINK	95.6	78.7	99.9	27.3	64.8
GVHMR	77.4	PhySINK	95.6	100.0	100.0	100.0	86.6

Appendix

Table 33: Sensitivity to the Video Reconstruction Model. Curation acceptance rate on the Unseen Video motions and retargeting quality of SINK and PhySINK, for motions reconstructed with TRAM and GVHMR.

	PHUMA Test					Unseen Video
Dataset	Total	Stationary	Angular	Vertical	Horizontal	Total	Stationary	Angular	Vertical	Horizontal
IK	70.5	85.9	63.3	47.6	77.3	68.5	96.6	72.4	43.6	49.5
GMR	84.0	92.1	77.8	77.1	89.1	75.2	99.1	77.8	61.7	52.7
SINK	89.1	94.0	86.0	84.9	90.7	79.0	100.0	81.8	62.8	62.6
+ Joint Feasibility Loss	87.0	92.1	83.6	79.8	90.9	78.6	94.8	84.7	56.4	67.0
+ Grounding Loss	90.0	93.5	87.9	85.6	92.0	80.4	98.3	86.2	60.6	64.8

Appendix

Table 34: Ablation Studies of Imitation Performance on Retargeting Loss. We evaluate the imitation performance of MaskedMimic policies trained with and without the physical constraint loss (Table 1 ) using the Unitree G1 robot.

Retarget	Retargeting Quality					Motion Tracking Performance
Retarget	MF (%)	JF (%)	NF (%)	NP (%)	NS (%)	PHUMA Test (GMR)	Unseen Video (GMR)
PhySINK	94.8	100.0	99.9	96.8	89.7	94.6 ( $\pm$ 0.1)	90.2 ( $\pm$ 0.9)
No Joint Feasibility Loss	91.7	95.8	99.9	96.4	89.4	91.0 ( $\pm$ 0.1)	85.8 ( $\pm$ 0.6)
No Grounding Loss	91.8	100.0	96.4	14.6	91.7	90.5 ( $\pm$ 0.1)	84.5 ( $\pm$ 0.8)
No Skating Loss	94.9	100.0	99.9	97.2	53.6	94.2 ( $\pm$ 0.1)	88.2 ( $\pm$ 0.8)

Appendix

Table 35: Leave-One-Out Ablation of PhySINK Losses. Retargeting quality, and success rate (%) at the 0.5m threshold on GMR-retargeted references, with each loss removed from the full objective. Mean (

\pm

std) over three seeds.

(a) G1
	PHUMA Test					Unseen Video
Dataset	Total	Stationary	Angular	Vertical	Horizontal	Total	Stationary	Angular	Vertical	Horizontal
AMASS	60.5	85.6	60.1	51.4	66.5	54.8	83.6	66.5	33.0	27.5
PHUMA	84.5	94.6	86.1	83.7	90.2	74.6	98.3	83.3	54.3	57.1
(a) H1-2
AMASS	60.4	84.0	62.8	43.6	78.7	72.3	96.6	77.3	52.1	72.5

Appendix

Table 36: Pelvis path following peformance across motion dataset. We evaluate the success rate of pelvis path-following control for policies trained on the AMASS and PHUMA datasets across various pelvis trajectories from the PHUMA Test and Unseen Video.

Figure 14: Path following on running motion. We visualize the robot’s trajectory in a running motion. The target pelvis path is visualized with a green line. Top row presents results from a policy trained on AMASS, while bottom row presents results from a policy trained on PHUMA.

	Motion Fidelity (%)	Joint Feasibility (%)	Non-Floating (%)	Non-Penetration (%)	Non-Skating (%)
IK	36.3	80.9	57.7	45.2	56.1
GMR	58.0	71.8	6.0	100.0	62.3
SINK	93.9	15.3	42.2	81.4	47.9
+ Joint Feasibility Loss	94.0	99.5	44.4	79.9	50.7
+ Grounding Loss	93.9	99.9	99.8	98.1	49.3
+ Skating Loss = PhySINK	93.9	99.9	97.7	99.7	87.7

Appendix

Table 37: Quantitative comparison and ablation study of retargeting methods on H1-2. Progressive impact of adding each physical constraint loss.

		PHUMA Test					Unseen Video
Retarget	Total	Stationary	Angular	Vertical	Horizontal	Total	Stationary	Angular	Vertical	Horizontal
IK	45.3	70.9	35.7	15.2	35.0	54.2	78.0	60.7	30.1	28.6
GMR	56.6	75.7	48.1	24.8	65.9	64.3	80.2	65.5	52.1	53.8
SINK	54.4	74.9	45.9	17.2	49.6	64.3	87.3	59.7	46.0	63.9
PhySINK	64.3	83.6	57.0	27.7	55.9	72.4	99.2	66.3	57.4	63.1

Appendix

Table 38: Motion tracking performance across retargeting approaches on H1-2. Success rates using two test sets.

		PHUMA Test					Unseen Video
Dataset	Hours	Total	Stationary	Angular	Vertical	Horizontal	Total	Stationary	Angular	Vertical	Horizontal
LaFAN1	2.4	62.0	79.3	54.7	26.6	58.9	70.8	92.4	66.7	56.4	68.2
AMASS	20.9	54.4	74.9	45.9	17.2	49.6	64.3	87.3	59.7	46.0	63.9
Humanoid-X	231.4	49.7	74.6	40.4	17.0	37.3	60.5	88.3	60.0	48.7	39.7
PHUMA	73.0	82.7	91.5	79.5	68.1	68.4	78.6	97.5	76.8	74.5	63.8

Appendix

Table 39: Motion tracking performance across datasets on H1-2. Success rates evaluated on two test sets.

Explore similar work

HiPHI: A Large-Scale Benchmark for High-Precision Human Motion and Object-Interaction
Aug 17, 2026Jiahao Ji, Ji Ma, Runhan Zhang +12Humanoid Robotics Robot Learning
Noitom Robotics · National University of Singapore · The University of Hong Kong +2
Booster Lab: A Data-Centric Pipeline for Learning Deployable Humanoid Locomotion Policies
Jun 26, 2026Penghui Chen, Tinglong Zheng, Yufeng Zhang +1Humanoid Robot Locomotion Robot Skill Learning
Department of Automation, Tsinghua University, Beijing, China · Booster Robotics Technology Co., Ltd, Beijing, China · School of Mechanical and Electronic Control Engineering, Beijing Jiaotong University, Beijing, China
Iterative Closed-Loop Motion Synthesis for Scaling the Capabilities of Humanoid Control
Feb 25, 2026Weisheng Xu, Qiwei Wu, Jiaxi Zhang +7Humanoid Robot Control Robot Motion Generation
Hong Kong University of Science and Technology (Guangzhou) · University of Oxford

Aug 17, 2026J/K move · Enter open · S save

HiPHI: A Large-Scale Benchmark for High-Precision Human Motion and Object-Interaction

Jiahao Ji, Ji Ma, Runhan Zhang, Runyi Yu+11

Noitom Robotics · National University of Singapore · The University of Hong Kong+2

Humanoid Robotics Robot Learning Human-Object Interaction

Open paper AskPDF arXiv

Humanoid intelligence requires learning over an extremely diverse space of whole-body motions and physically grounded interactions. However, existing embodied datasets remain fundamentally limited: internet-scale video data lack precise physical states and interaction grounding, while laboratory motion datasets provide high fidelity but only narrow behavioral coverage. This mismatch creates a critical bottleneck for scalable humanoid policy learning. We present HiPHI, a 600+ hour scale high-fidelity whole-body human motion dataset designed to systematically maximize coverage of the human motion and interaction manifold. HiPHI is theoretically guided by FrameNet, a linguistic framework organizing human primitives. Created using an optical motion capture pipeline, HiPHI provides sub-millimeter spatial marker tracking accuracy for full-body human motion and mesh-level object trajectories. We further introduce a benchmark suite evaluating motion-space diversity, interaction grounding, object consistency, and physical AI applications. Our analyses demonstrate that HiPHI significantly expands motion coverage compared to existing motion datasets while maintaining high-fidelity interaction quality, and establishes a scalable data foundation for training, evaluating, and generalizing humanoid policies in real-world embodied tasks, where similar extensions are also applicable to motion prior models in computer graphics. Project page: https://noitom-robotics.github.io/hiphi/

Abstract

Explore similar work

HiPHI: A Large-Scale Benchmark for High-Precision Human Motion and Object-Interaction

Booster Lab: A Data-Centric Pipeline for Learning Deployable Humanoid Locomotion Policies

Iterative Closed-Loop Motion Synthesis for Scaling the Capabilities of Humanoid Control