Whole-body human-object interaction (HOI) generation requires coordinating body and finger motion, object trajectories, and time-varying contacts. Existing methods typically rely on iterative diffusion or test-time optimization, resulting in high inference costs. Direct one-step generation is highly desired in downstream applications due to efficient inference. However, it is challenging because these HOI components differ substantially in structure and scale. We present MFlowHOI, an efficient one-step framework for whole-body HOI generation. A Contact-Aware Multipart RVQVAE organizes body, hand, object, and contact signals into a structured latent space, with a shared decoder preserving their global coordination. An ST-DiT then models temporal and cross-part dependencies in this space and generates a complete interaction in a single MeanFlow evaluation. Predicted object-local contact correspondences further enable geometric refinement and interaction-preserving transfer to photorealistic 3D Gaussian avatars with different body proportions. Experiments demonstrate that MFlowHOI outperforms existing base generators in motion quality and text alignment while substantially reducing inference time.
Figures & tables
Figure 1: MFlowHOI jointly generates body and finger motions, object trajectories, and contacts in a single transport pass, enabling refinement and retargeting to 3D Gaussian avatars with diverse body proportions.
Figure 2: MFlowHOI encodes part-level motion, object motion, object-body part contacts, and contact states into a structured latent. Conditional MeanFlow transports noise to a complete HOI latent in one ST-DiT transport evaluation; the frozen decoder recovers motion for direct use, optional refinement, or 3DGS avatar deployment.
Figure 3: Direct pose transfer across different body shapes may cause grasp failure. Our coarse arm retargeting first restores reachability, after which fine hand-Gaussian SDF optimization suppresses local hand–object penetration while preserving the generated object trajectory.
Method
FID ↓
R@3 ↑
Div. →
IV ↓
ID ↓
CR ↑
Time ↓
GT
0.00
0.55
13.24
3.97
3.49
0.64
–
Dex2HOI
3.44
0.30
10.15
5.14
1.97
0.33
0.19
DiffGrasp-U
5.15
0.27
10.62
5.93
5.29
0.38
1.63
DiffGrasp-G
4.26
0.31
10.70
5.96
5.75
0.40
363.47
HOIDiNi
5.81
0.34
11.47
4.14
1.21
0.06
0.19
HOIDiNi(DNO)
5.42
0.25
11.55
4.79
4.47
0.64
379.50
Table 1: Comparison on GRAB. U and G denote unguided and guided generation, respectively; Obj. Ref. denotes object refinement, and UB-DNO denotes upper-body DNO.
Figure 4: Qualitative comparison results. Columns show two prompts, and rows compare different methods using matched frames and hand–object close-ups.
Method
FID ↓
R@3 ↑
Div.
FS ↓
CR ↑
Dex2HOI space
7.2769
0.3359
9.8175
1.0859
0.0815
HOIDiNi space
13.1630
0.2109
9.2114
3.4212
0.0300
MFlowHOI w/o ST Attn.
3.1960
0.3828
11.9047
0.1021
0.0524
MFlowHOI
2.5732
0.4531
11.2474
0.0929
0.0930
Table 2: One-step MeanFlow results using different transport representations. All variants perform one full-interval transport evaluation without DNO. FS denotes foot skating.
Table 4: Comparison with Animate Anyone. OBJ-CLIP measures object appearance similarity; LMD-B/H are body/hand landmark errors in pixels. Perceptual scores are averaged over 16 participants on a 1–5 scale and assess overall realism, appearance consistency (App.), and motion smoothness (Mot.) for the human (H) and object (O). Each method uses its respective 2D or 3D assets.
Figure 5: Reconstruction results. Panels (a)–(d) compare ground truth with reconstructed frames and hand–object close-ups; panel (e) compares the interaction sequences.
Figure 6: Qualitative comparison of Animate Anyone (first and third columns) and our 3DGS pipeline (second and fourth columns) at two matched instants under their respective asset regimes. Ours better preserves object identity, temporal coherence, and hand–object alignment.