ULIFLOW: Unified Latent Human-Object Interaction Flow

Wenqi Cai; Chunzhi Gu; Weichen Zhou; Yawen Zou; Yijin Wei; Ran Dong; Haoran Xie; Chao Zhang

cs.CVOct 16, 2026

ULIFLOW: Unified Latent Human-Object Interaction Flow

Authors: Wenqi Cai, Chunzhi Gu, Weichen Zhou, Yawen Zou, Yijin Wei, Ran Dong, Haoran Xie, Chao Zhang

Organizations: University of Toyama · University of Fukui · Chukyo University · Japan Advanced Institute of Science and Technology

Abstract

Text-conditioned 3D human--object interaction (HOI) generation remains challenging because individually plausible human and object motions do not necessarily form a coherent interaction. We argue that a key limitation lies in conventional skeleton-based representations, which compactly describe body kinematics while leaving surface-level interaction geometry largely implicit, often requiring additional contact-specific regularizers. We propose ULIFlow, a framework that explicitly models continuous human--object relations within a unified latent space. To this end, we derive continuous marker--object relations from sparse body-surface markers and treat them as an explicit interaction modality alongside human motion and object dynamics. A Mixture-of-Transformers (MoT) autoencoder is then designed to jointly model these heterogeneous streams while preserving their modality-specific structures. Importantly, we let the relation stream directly participate in human and object reconstruction, encouraging the shared latent representation to preserve both individual dynamics and the interaction geometry that coordinates them. To further achieve variable-length generation, we introduce a blockwise Flow Forcing scheme with global sequence-progress conditioning, enabling progressive continuation from fixed-length training windows while maintaining consistent long-horizon dynamics. Extensive experiments demonstrate that ULIFlow consistently outperforms prior work in overall generation quality and human--object coordination, while preserving coherent interactions over requested durations beyond the training horizon.

Figures & tables

Figure 1: Given a textual prompt and a requested duration, ULIFlow jointly generates coherent human and object motions as a variable-length human–object interaction (HOI) sequence.

Figure 2: Overview of ULIFlow . Left: Unified latent modeling. The query-based MoT autoencoder jointly encodes and reconstructs human, object, and relation streams in a structured latent space. Right: Conditional generation. A conditional latent DiT uses independently sampled blockwise flow times and sliding-window inference to support generation at any requested duration.

Figure 3: Qualitative comparison of variable-length generation with prior works. ULIFlow preserves coherent interaction beyond fixed generation windows.

Method	FID $\downarrow$	R-Precision $\uparrow$			MM-Dist $\downarrow$	Diversity $\rightarrow$	MModality $\uparrow$
Method	FID $\downarrow$	Top 1	Top 2	Top 3	MM-Dist $\downarrow$	Diversity $\rightarrow$	MModality $\uparrow$
Ground Truth	$0.000\,{\scriptstyle\pm 0.000}$	$0.869\,{\scriptstyle\pm 0.003}$	$0.952\,{\scriptstyle\pm 0.001}$	$0.971\,{\scriptstyle\pm 0.002}$	$4.012\,{\scriptstyle\pm 0.000}$	$11.939\,{\scriptstyle\pm 0.000}$	–
Text2HOI ( Cha et al., 2024 )	$24.521\,{\scriptstyle\pm 0.035}$	$0.425\,{\scriptstyle\pm 0.001}$	$0.606\,{\scriptstyle\pm 0.008}$	$0.697\,{\scriptstyle\pm 0.004}$	$6.268\,{\scriptstyle\pm 0.012}$	$6.108\,{\scriptstyle\pm 0.023}$	$2.544\,{\scriptstyle\pm 0.119}$
CHOIS ( Li et al., 2024 )	$6.914\,{\scriptstyle\pm 0.147}$	$0.644\,{\scriptstyle\pm 0.005}$	$0.870\,{\scriptstyle\pm 0.007}$	$0.958\,{\scriptstyle\pm 0.010}$	$5.278\,{\scriptstyle\pm 0.006}$	$9.378\,{\scriptstyle\pm 0.028}$	$2.758\,{\scriptstyle\pm 0.017}$
HOI-Diff ( Peng et al., 2025 )	$14.445\,{\scriptstyle\pm 0.056}$	$0.134\,{\scriptstyle\pm 0.007}$	$0.263\,{\scriptstyle\pm 0.016}$	$0.362\,{\scriptstyle\pm 0.008}$	$10.230\,{\scriptstyle\pm 0.010}$	$6.252\,{\scriptstyle\pm 0.044}$	$0.059\,{\scriptstyle\pm 0.005}$
InterAct ( Xu et al., 2025 )	$1.768\,{\scriptstyle\pm 0.019}$	$0.749\,{\scriptstyle\pm 0.014}$	$0.933\,{\scriptstyle\pm 0.011}$	$0.968\,{\scriptstyle\pm 0.005}$	$\bm{4.507\,{\scriptstyle\pm 0.003}}$	$11.416\,{\scriptstyle\pm 0.038}$	$3.253\,{\scriptstyle\pm 0.031}$

Table 1: Quantitative comparison of generation quality on the InterAct dataset ( Xu et al., 2025 ) .

Method	128-frame subsequences						Boundary-aligned 64-frame windows
	FID $\downarrow$	R-Precision $\uparrow$			Div $\rightarrow$	MM-Dist $\downarrow$	FID $\downarrow$	Div $\rightarrow$	Peak jerk $\rightarrow$	AUJ $\rightarrow$
	FID $\downarrow$	Top 1	Top 2	Top 3	Div $\rightarrow$	MM-Dist $\downarrow$	FID $\downarrow$	Div $\rightarrow$	Peak jerk $\rightarrow$	AUJ $\rightarrow$
Ground Truth	$0.000\,{\scriptstyle\pm 0.000}$	$0.920\,{\scriptstyle\pm 0.000}$	$0.969\,{\scriptstyle\pm 0.000}$	$0.987\,{\scriptstyle\pm 0.000}$	$11.104\,{\scriptstyle\pm 0.000}$	$3.946\,{\scriptstyle\pm 0.000}$	$0.000\,{\scriptstyle\pm 0.000}$	$10.860\,{\scriptstyle\pm 0.000}$	$1.530\,{\scriptstyle\pm 0.000}$	$32.841\,{\scriptstyle\pm 0.000}$
InterAct ( Xu et al., 2025 )	$3.097\,{\scriptstyle\pm 0.180}$	$0.698\,{\scriptstyle\pm 0.014}$	$0.856\,{\scriptstyle\pm 0.026}$	$0.918\,{\scriptstyle\pm 0.022}$	$\bm{10.928\,{\scriptstyle\pm 0.036}}$	$6.296\,{\scriptstyle\pm 0.049}$	$3.713\,{\scriptstyle\pm 0.160}$	$\bm{10.670\,{\scriptstyle\pm 0.036}}$	$1.734\,{\scriptstyle\pm 0.025}$	$10.416\,{\scriptstyle\pm 0.143}$
LIGHT ( Wang et al., 2026b )	$5.658\,{\scriptstyle\pm 0.128}$	$0.583\,{\scriptstyle\pm 0.009}$	$0.752\,{\scriptstyle\pm 0.013}$	$0.818\,{\scriptstyle\pm 0.008}$	$10.795\,{\scriptstyle\pm 0.125}$	$7.276\,{\scriptstyle\pm 0.028}$	$4.914\,{\scriptstyle\pm 0.069}$	$10.664\,{\scriptstyle\pm 0.039}$	$\bm{1.445\,{\scriptstyle\pm 0.077}}$	$18.012\,{\scriptstyle\pm 0.192}$
ULIFlow	$\bm{2.686\,{\scriptstyle\pm 0.060}}$	$\bm{0.753\,{\scriptstyle\pm 0.018}}$	$\bm{0.909\,{\scriptstyle\pm 0.007}}$	$\bm{0.955\,{\scriptstyle\pm 0.009}}$	$10.782\,{\scriptstyle\pm 0.051}$	$\bm{6.023\,{\scriptstyle\pm 0.031}}$	$\bm{3.415\,{\scriptstyle\pm 0.086}}$	$10.592\,{\scriptstyle\pm 0.027}$	$1.203\,{\scriptstyle\pm 0.024}$	$\bm{39.300\,{\scriptstyle\pm 0.356}}$

Table 2: Quantitative comparison of long-horizon generation on the InterAct dataset ( Xu et al., 2025 ) . The two column groups are computed over non-overlapping 128-frame subsequences and 64-frame windows centered at continuation boundaries, respectively.

Method	Human jerk			Object jerk
Method	Mean $\rightarrow$	Peak $\rightarrow$	AUJ $\rightarrow$	Mean $\rightarrow$	Peak $\rightarrow$	AUJ $\rightarrow$
Ground Truth	$0.546\,{\scriptstyle\pm 0.000}$	$2.202\,{\scriptstyle\pm 0.000}$	$98.692\,{\scriptstyle\pm 0.000}$	$0.796\,{\scriptstyle\pm 0.000}$	$5.736\,{\scriptstyle\pm 0.000}$	$144.689\,{\scriptstyle\pm 0.000}$
InterAct ( Xu et al., 2025 )	$0.216\,{\scriptstyle\pm 0.003}$	$3.196\,{\scriptstyle\pm 0.042}$	$38.742\,{\scriptstyle\pm 0.340}$	$0.548\,{\scriptstyle\pm 0.034}$	$9.094\,{\scriptstyle\pm 0.500}$	$91.701\,{\scriptstyle\pm 3.920}$
LIGHT ( Wang et al., 2026b )	$0.342\,{\scriptstyle\pm 0.001}$	$\bm{2.722\,{\scriptstyle\pm 0.080}}$	$66.963\,{\scriptstyle\pm 0.306}$	$0.536\,{\scriptstyle\pm 0.005}$	$4.089\,{\scriptstyle\pm 0.078}$	$105.587\,{\scriptstyle\pm 1.291}$
ULIFlow	$\bm{0.608\,{\scriptstyle\pm 0.003}}$	$1.442\,{\scriptstyle\pm 0.027}$	$\bm{113.770\,{\scriptstyle\pm 0.418}}$	$\bm{0.846\,{\scriptstyle\pm 0.007}}$	$\bm{4.466\,{\scriptstyle\pm 0.252}}$	$\bm{156.460\,{\scriptstyle\pm 1.902}}$

Table 3: Quantitative comparison of motion stability on the InterAct dataset ( Xu et al., 2025 ) . Jerk is reported in cm/frame 3 .

Method	Ratio $\downarrow$	Mean $\downarrow$	P95 $\downarrow$	Max $\downarrow$
Ground Truth	$0.003\,{\scriptstyle\pm 0.000}$	$0.622\,{\scriptstyle\pm 0.000}$	$1.366\,{\scriptstyle\pm 0.000}$	$1.585\,{\scriptstyle\pm 0.000}$
InterAct ( Xu et al., 2025 )	$0.006\,{\scriptstyle\pm 0.001}$	$0.905\,{\scriptstyle\pm 0.145}$	$2.002\,{\scriptstyle\pm 0.227}$	$2.421\,{\scriptstyle\pm 0.182}$
LIGHT ( Wang et al., 2026b )	$0.006\,{\scriptstyle\pm 0.001}$	$0.905\,{\scriptstyle\pm 0.068}$	$2.037\,{\scriptstyle\pm 0.140}$	$2.513\,{\scriptstyle\pm 0.180}$
ULIFlow	$\bm{0.003\,{\scriptstyle\pm 0.001}}$	$\bm{0.845\,{\scriptstyle\pm 0.088}}$	$\bm{1.805\,{\scriptstyle\pm 0.214}}$	$\bm{2.163\,{\scriptstyle\pm 0.215}}$

Table 4: Quantitative comparison of physical plausibility on the InterAct dataset ( Xu et al., 2025 ) . Body–object penetration depths are reported in cm.

Figure 4: Qualitative comparison of contact and motion coupling. ULIFlow achieves more physically plausible hand–object contact.

MoT structure	Latent regularization	Relation branch	Parameters (M)	Marker (cm) $\downarrow$	Object translation (cm) $\downarrow$	Object rotation (deg) $\downarrow$	Effective rank (%) $\uparrow$
$\times$	$\checkmark$	$\checkmark$	60.66	3.805	3.174	10.393	6.96
$\checkmark$	$\checkmark$	$\times$	60.41	1.297	1.757	1.512	72.73
$\checkmark$	$\times$	$\checkmark$	60.40	1.367	1.148	1.516	47.80
$\checkmark$	$\checkmark$	$\checkmark$	60.40	1.348	1.263	1.520	82.15

Table 5: Ablation study of latent autoencoder designs on the InterAct dataset ( Xu et al., 2025 ) .

Figure 5: Qualitative comparison of long-horizon rollout designs. Left : ULIFlow without progress conditioning, Mid : ULIFlow without blockwise forcing, and Right : the complete model. Blockwise forcing and progress conditioning jointly preserve coordinated motion during late rollout.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Frames	Blocks	Velocity evaluations	Time (s)	FPS	Peak VRAM (GiB)
128	4	65	$2.817\,{\scriptstyle\pm 0.004}$	45.43	3.267
256	8	109	$4.827\,{\scriptstyle\pm 0.007}$	53.04	3.270
384	12	153	$6.927\,{\scriptstyle\pm 0.038}$	55.43	3.274
512	16	197	$9.367\,{\scriptstyle\pm 0.312}$	54.66	3.277

Appendix

Table A: Quantitative comparison of inference efficiency across requested sequence lengths on the InterAct dataset ( Xu et al., 2025 ) . Runtime and peak memory are measured on one NVIDIA GeForce RTX 5080.

BEHAVE subset
Method	FID $\downarrow$	R-Precision $\uparrow$			MM-Dist $\downarrow$	Diversity $\rightarrow$
Method	FID $\downarrow$	Top 1	Top 2	Top 3	MM-Dist $\downarrow$	Diversity $\rightarrow$
Ground Truth	$0.000\,{\scriptstyle\pm 0.000}$	$0.887\,{\scriptstyle\pm 0.000}$	$0.956\,{\scriptstyle\pm 0.000}$	$0.969\,{\scriptstyle\pm 0.000}$	$3.242\,{\scriptstyle\pm 0.000}$	$10.805\,{\scriptstyle\pm 0.000}$
HOI-Diff ( Peng et al., 2025 )	$14.440\,{\scriptstyle\pm 0.008}$	$0.138\,{\scriptstyle\pm 0.000}$	$0.256\,{\scriptstyle\pm 0.006}$	$0.358\,{\scriptstyle\pm 0.004}$	$10.230\,{\scriptstyle\pm 0.000}$	$6.253\,{\scriptstyle\pm 0.002}$
InterAct ( Xu et al., 2025 )	$5.080\,{\scriptstyle\pm 0.079}$	$0.698\,{\scriptstyle\pm 0.010}$	$0.865\,{\scriptstyle\pm 0.010}$	$0.917\,{\scriptstyle\pm 0.018}$	$5.805\,{\scriptstyle\pm 0.069}$	$10.679\,{\scriptstyle\pm 0.036}$
LIGHT ( Wang et al., 2026b )	$4.607\,{\scriptstyle\pm 0.151}$	$0.556\,{\scriptstyle\pm 0.017}$	$0.769\,{\scriptstyle\pm 0.027}$	$0.881\,{\scriptstyle\pm 0.033}$	$6.252\,{\scriptstyle\pm 0.043}$	$10.518\,{\scriptstyle\pm 0.038}$

Appendix

Table B: Quantitative comparison of generation quality on the BEHAVE ( Bhatnagar et al., 2022 ) and OMOMO ( Li et al., 2023 ) subsets of the InterAct dataset with prior works.

Frames	Progress condition	Blockwise forcing	FID $\downarrow$	R-Precision $\uparrow$			MM-Dist $\downarrow$	Diversity $\rightarrow$
Frames	Progress condition	Blockwise forcing	FID $\downarrow$	Top 1	Top 2	Top 3	MM-Dist $\downarrow$	Diversity $\rightarrow$
128	Ground Truth		$0.000\,{\scriptstyle\pm 0.000}$	$0.635\,{\scriptstyle\pm 0.000}$	$0.781\,{\scriptstyle\pm 0.000}$	$0.875\,{\scriptstyle\pm 0.000}$	$6.738\,{\scriptstyle\pm 0.000}$	$11.171\,{\scriptstyle\pm 0.000}$
128	$\times$	$\checkmark$	$7.319\,{\scriptstyle\pm 0.219}$	$0.691\,{\scriptstyle\pm 0.024}$	$\bm{0.830\,{\scriptstyle\pm 0.012}}$	$\bm{0.892\,{\scriptstyle\pm 0.026}}$	$6.172\,{\scriptstyle\pm 0.032}$	$11.419\,{\scriptstyle\pm 0.066}$
128	$\checkmark$	$\times$	$\bm{6.837\,{\scriptstyle\pm 0.281}}$	$0.688\,{\scriptstyle\pm 0.028}$	$0.806\,{\scriptstyle\pm 0.033}$	$0.875\,{\scriptstyle\pm 0.021}$	$6.099\,{\scriptstyle\pm 0.058}$	$\bm{11.229\,{\scriptstyle\pm 0.029}}$
128	$\checkmark$	$\checkmark$	$7.712\,{\scriptstyle\pm 0.166}$	$\bm{0.694\,{\scriptstyle\pm 0.006}}$	$\bm{0.830\,{\scriptstyle\pm 0.012}}$	$0.878\,{\scriptstyle\pm 0.016}$	$\bm{6.093\,{\scriptstyle\pm 0.025}}$	$11.236\,{\scriptstyle\pm 0.035}$
256	Ground Truth		$0.000\,{\scriptstyle\pm 0.000}$	$0.563\,{\scriptstyle\pm 0.000}$	$0.719\,{\scriptstyle\pm 0.000}$	$0.811\,{\scriptstyle\pm 0.000}$	$6.849\,{\scriptstyle\pm 0.000}$	$11.076\,{\scriptstyle\pm 0.000}$

Appendix

Table C: Quantitative comparison of long-horizon generation across requested sequence lengths on the InterAct dataset ( Xu et al., 2025 ) . Metrics are computed over non-overlapping 128-frame subsequences.

Frames	Progress condition	Blockwise forcing	FID $\downarrow$	Diversity $\rightarrow$	Peak jerk $\rightarrow$	AUJ $\rightarrow$
256	Ground Truth		$0.000\,{\scriptstyle\pm 0.000}$	$11.066\,{\scriptstyle\pm 0.000}$	$0.852\,{\scriptstyle\pm 0.000}$	$17.320\,{\scriptstyle\pm 0.000}$
256	$\times$	$\checkmark$	$5.938\,{\scriptstyle\pm 0.136}$	$11.203\,{\scriptstyle\pm 0.070}$	$1.275\,{\scriptstyle\pm 0.044}$	$40.060\,{\scriptstyle\pm 0.491}$
256	$\checkmark$	$\times$	$\bm{4.853\,{\scriptstyle\pm 0.169}}$	$11.056\,{\scriptstyle\pm 0.015}$	$1.322\,{\scriptstyle\pm 0.023}$	$38.877\,{\scriptstyle\pm 0.076}$
256	$\checkmark$	$\checkmark$	$4.917\,{\scriptstyle\pm 0.146}$	$\bm{11.027\,{\scriptstyle\pm 0.054}}$	$\bm{1.192\,{\scriptstyle\pm 0.004}}$	$\bm{36.580\,{\scriptstyle\pm 0.165}}$
384	Ground Truth		$0.000\,{\scriptstyle\pm 0.000}$	$11.073\,{\scriptstyle\pm 0.000}$	$0.861\,{\scriptstyle\pm 0.000}$	$17.204\,{\scriptstyle\pm 0.000}$
384	$\times$	$\checkmark$	$4.487\,{\scriptstyle\pm 0.117}$	$11.210\,{\scriptstyle\pm 0.030}$	$1.263\,{\scriptstyle\pm 0.033}$	$40.310\,{\scriptstyle\pm 0.419}$

Appendix

Table D: Quantitative comparison of continuation-boundary quality across requested sequence lengths on the InterAct dataset ( Xu et al., 2025 ) . Metrics are computed over 64-frame windows centered at continuation boundaries.

Decoder	$\sigma$	Marker (cm) $\downarrow$	Object trans. (cm) $\downarrow$	Object rot. (deg) $\downarrow$	Acceleration	Jerk
Original	0.00	1.348	1.263	1.520	0.786	1.328
Noise-robust	0.00	1.344	1.457	1.606	0.462	0.682
Original	0.05	2.153	2.459	2.285	1.002	1.700
Noise-robust	0.05	1.805	2.039	2.245	0.496	0.732
Original	0.10	3.530	4.339	3.567	1.388	2.370
Noise-robust	0.10	2.688	3.149	3.350	0.578	0.856

Appendix

Table E: Quantitative comparison of latent-noise robustness on the InterAct dataset ( Xu et al., 2025 ) . Latent perturbations are applied in the whitened space before decoding.

BEHAVE subset
Method	FID $\downarrow$	R-Precision $\uparrow$			MM-Dist $\downarrow$	Diversity $\rightarrow$
Method	FID $\downarrow$	Top 1	Top 2	Top 3	MM-Dist $\downarrow$	Diversity $\rightarrow$
Ground Truth	$0.000\,{\scriptstyle\pm 0.000}$	$0.887\,{\scriptstyle\pm 0.000}$	$0.956\,{\scriptstyle\pm 0.000}$	$0.969\,{\scriptstyle\pm 0.000}$	$3.242\,{\scriptstyle\pm 0.000}$	$10.805\,{\scriptstyle\pm 0.000}$
ULIFlow w/o MoT structure	$5.251\,{\scriptstyle\pm 0.093}$	$0.660\,{\scriptstyle\pm 0.026}$	$0.827\,{\scriptstyle\pm 0.014}$	$0.908\,{\scriptstyle\pm 0.007}$	$5.973\,{\scriptstyle\pm 0.008}$	$10.454\,{\scriptstyle\pm 0.021}$
ULIFlow w/o relation branch	$\bm{3.025\,{\scriptstyle\pm 0.140}}$	$0.717\,{\scriptstyle\pm 0.007}$	$0.865\,{\scriptstyle\pm 0.010}$	$0.923\,{\scriptstyle\pm 0.004}$	$5.442\,{\scriptstyle\pm 0.057}$	$10.719\,{\scriptstyle\pm 0.039}$
ULIFlow w/o latent regularization	$3.347\,{\scriptstyle\pm 0.035}$	$0.740\,{\scriptstyle\pm 0.010}$	$0.890\,{\scriptstyle\pm 0.004}$	$\bm{0.950\,{\scriptstyle\pm 0.006}}$	$5.348\,{\scriptstyle\pm 0.050}$	$10.728\,{\scriptstyle\pm 0.048}$

Appendix

Table F: Quantitative comparison of latent autoencoder variants on the BEHAVE ( Bhatnagar et al., 2022 ) and OMOMO ( Li et al., 2023 ) subsets of the InterAct dataset.

Method	Pen. ratio (%) $\downarrow$	Mean depth (cm) $\downarrow$	P95 depth (cm) $\downarrow$	Max depth (cm) $\downarrow$
Ground Truth	$0.295\,{\scriptstyle\pm 0.000}$	$0.977\,{\scriptstyle\pm 0.000}$	$2.082\,{\scriptstyle\pm 0.000}$	$2.339\,{\scriptstyle\pm 0.000}$
ULIFlow w/o relation branch	$0.583\,{\scriptstyle\pm 0.043}$	$1.364\,{\scriptstyle\pm 0.474}$	$2.650\,{\scriptstyle\pm 0.526}$	$3.116\,{\scriptstyle\pm 0.541}$
ULIFlow	$\bm{0.379\,{\scriptstyle\pm 0.042}}$	$\bm{1.312\,{\scriptstyle\pm 0.572}}$	$\bm{2.186\,{\scriptstyle\pm 0.688}}$	$\bm{2.516\,{\scriptstyle\pm 0.713}}$

Appendix

Table G: Quantitative comparison of physical coordination for the relation-pathway ablation on the InterAct dataset ( Xu et al., 2025 ) . Contact metrics use a 2-cm threshold, and unsupported travel is the fraction of object displacement occurring without body contact at either endpoint.

Threshold (cm)	Method	Precision (%) $\uparrow$	Recall (%) $\uparrow$	F1 (%) $\uparrow$	Hand F1 (%) $\uparrow$
1	ULIFlow w/o relation branch	$38.106\,{\scriptstyle\pm 0.673}$	$28.172\,{\scriptstyle\pm 0.496}$	$32.395\,{\scriptstyle\pm 0.568}$	$33.304\,{\scriptstyle\pm 0.555}$
1	ULIFlow	$\bm{39.658\,{\scriptstyle\pm 0.015}}$	$\bm{44.461\,{\scriptstyle\pm 0.393}}$	$\bm{41.922\,{\scriptstyle\pm 0.182}}$	$\bm{43.098\,{\scriptstyle\pm 0.094}}$
2	ULIFlow w/o relation branch	$61.607\,{\scriptstyle\pm 0.276}$	$51.128\,{\scriptstyle\pm 0.275}$	$55.880\,{\scriptstyle\pm 0.266}$	$54.041\,{\scriptstyle\pm 0.233}$
2	ULIFlow	$\bm{64.903\,{\scriptstyle\pm 0.101}}$	$\bm{69.680\,{\scriptstyle\pm 0.122}}$	$\bm{67.207\,{\scriptstyle\pm 0.054}}$	$\bm{67.464\,{\scriptstyle\pm 0.272}}$
5	ULIFlow w/o relation branch	$82.087\,{\scriptstyle\pm 0.056}$	$78.855\,{\scriptstyle\pm 0.455}$	$80.438\,{\scriptstyle\pm 0.263}$	$75.167\,{\scriptstyle\pm 0.298}$
5	ULIFlow	$\bm{84.974\,{\scriptstyle\pm 0.081}}$	$\bm{88.498\,{\scriptstyle\pm 0.333}}$	$\bm{86.700\,{\scriptstyle\pm 0.194}}$	$\bm{84.930\,{\scriptstyle\pm 0.122}}$

Appendix

Table H: Quantitative comparison of contact-threshold sensitivity on the InterAct dataset ( Xu et al., 2025 ) . All contact decisions compare the generated human with its generated object; ground truth is used only to define the reference contact frames.

HOI Generation

Human-object interaction generation creates 3D motion, grasps, and videos of people manipulating or coordinating with objects.

See all 80 papers

Oct 12, 2026

MFlowHOI: Whole-Body Human-Object Interaction Generation via Part-Aware MeanFlow

Whole-body human-object interaction (HOI) generation requires coordinating body and finger motion, object trajectories, and time-varying contacts. Existing methods typically rely on iterative diffusion or test-time optimization, resulting in high inference costs. Direct one-step generation is highly desired in downstream applications due to efficient inference. However, it is challenging because these HOI components differ substantially in structure and scale. We present MFlowHOI, an efficient one-step framework for whole-body HOI generation. A Contact-Aware Multipart RVQVAE organizes body, hand, object, and contact signals into a structured latent space, with a shared decoder preserving their global coordination. An ST-DiT then models temporal and cross-part dependencies in this space and generates a complete interaction in a single MeanFlow evaluation. Predicted object-local contact correspondences further enable geometric refinement and interaction-preserving transfer to photorealistic 3D Gaussian avatars with different body proportions. Experiments demonstrate that MFlowHOI outperforms existing base generators in motion quality and text alignment while substantially reducing inference time.

Qiang He, Yonghao Zhang, Yinda Zhang +7

Institute of Software, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Google

HOI Generation One-Step Generative Modeling Flow-Based Generative Modeling Motion Representation Learning

Ask PDF

Sep 26, 2026

Harnessing Coupled Stream Completion For Human-Object Interaction Modeling

Text-conditioned human-object interaction (HOI) generation requires body motion, object trajectories & rotations, and hand articulation to remain coordinated. These components differ in scale and dynamics, but must agree on contact, relative pose, and timing. A shared representation may limit the distinct structure of each stream, while independent generation prevents each stream from responding to changes in the others. Latent supervision alone also does not directly constrain contact after decoding. We propose TRACE, a continuous latent framework that keeps stream states separate and couples their updates. TRACE encodes body, object, and hand motion into separate latents and predicts each stream velocity from the complete current interaction state. Geometric losses on decoded motion further constrain contact and object-relative motion over time. The same model supports completion of any single absent stream from the other two. Frozen flow features also serve as input to a language model for HOI understanding. Experiments on InterAct, OMOMO, and BEHAVE show that joint completion training improves generation and that frozen flow features improve understanding over raw-motion encoding. On InterAct, TRACE achieves the highest contact precision, recall, and F1 among the compared methods.

Dawei Guan, Di Yang, Jiangtao Wang

School of Artificial Intelligence and Data Science, University of Science and Technology of China

HOI Generation Motion Representation Learning Human-Object Interaction

Ask PDF

Apr 12, 2026

HO-Flow: Generalizable Hand-Object Interaction Generation with Latent Flow Matching

Generating realistic 3D hand-object interactions (HOI) is a fundamental challenge in computer vision and robotics, requiring both temporal coherence and high-fidelity physical plausibility. Existing methods remain limited in their ability to learn expressive motion representations for generation and perform temporal reasoning. In this paper, we present HO-Flow, a framework for synthesizing realistic hand-object motion sequences from texts and canoncial 3D objects. HO-Flow first employs an interaction-aware variational autoencoder to encode sequences of hand and object motions into a unified latent manifold by incorporating hand and object kinematics, enabling the representation to capture rich interaction dynamics. It then leverages a masked flow matching model that combines auto-regressive temporal reasoning with continuous latent generation, improving temporal coherence. To further enhance generalization, HO-Flow predicts object motions relative to the initial frame, enabling effective pre-training on large-scale synthetic data. Experiments on the GRAB, OakInk, and DexYCB benchmarks demonstrate that HO-Flow achieves state-of-the-art performance in both physical plausibility and motion diversity for interaction motion synthesis.

Zerui Chen, Rolandos Alexandros Potamias, Shizhe Chen +3

Imperial College London · Inria, CNRS, ENS-PSL

Style-Conditioned Motion Generation HOI Generation Human Motion Generation Motion Representation Learning

Ask PDF

Abstract

HOI Generation

Explore similar work

MFlowHOI: Whole-Body Human-Object Interaction Generation via Part-Aware MeanFlow

Harnessing Coupled Stream Completion For Human-Object Interaction Modeling

HO-Flow: Generalizable Hand-Object Interaction Generation with Latent Flow Matching