Unleashing Guidance Without Classifiers for Human-Object Interaction Animation

Ziyin Wang; Sirui Xu; Chuan Guo; Bing Zhou; Jiangshan Gong; Jian Wang; Yu-Xiong Wang; Liang-Yan Gui

cs.CVMar 26, 2026

Unleashing Guidance Without Classifiers for Human-Object Interaction Animation

Authors: Ziyin Wang, Sirui Xu, Chuan Guo, Bing Zhou, Jiangshan Gong, Jian Wang, Yu-Xiong Wang, Liang-Yan Gui

Organizations: University of Illinois Urbana-Champaign · Snap Inc.

Abstract

Generating realistic human-object interaction (HOI) animations remains challenging because it requires jointly modeling dynamic human actions and diverse object geometries. Prior diffusion-based approaches often rely on hand-crafted contact priors or human-imposed kinematic constraints to improve contact quality. We propose LIGHT, a data-driven alternative in which guidance emerges from the denoising pace itself, reducing dependence on manually designed priors. Building on diffusion forcing, we factor the representation into modality-specific components and assign individualized noise levels with asynchronous denoising schedules. In this paradigm, cleaner components guide noisier ones through cross-attention, yielding guidance without auxiliary classifiers. We find that this data-driven guidance is inherently contact-aware, and can be enhanced when training is augmented with a broad spectrum of synthetic object geometries, encouraging invariance of contact semantics to geometric diversity. Extensive experiments show that pace-induced guidance more effectively mirrors the benefits of contact priors than conventional classifier-free guidance, while achieving higher contact fidelity, more realistic HOI generation, and stronger generalization to unseen objects and tasks.

Figures & tables

Figure 2: Overview of LIGHT. Left: Training. We form different modalities, e.g., body, hand, and object, each diffused with its own noise level. After adding modal-wise and frame-wise rotary positional encodings, the tokens are processed by a shared Transformer decoder and an MLP head to predict clean motion. Right: Inference. We compare a uniform schedule that denoises all modalities synchronously with a staged schedule that keeps one modality cleaner from the uniform run.

Method

R-Precision ↑

FID ↓

MM Dist ↓

Diversity →

FSR ↓

Pene ↓

Contact →

Interaction ↑

Top 1

Top 2

Top 3

C_{prec}

C_{rec}

C_{F1}

Ground Truth

0.600 ±0.004

0.834 ±0.001

0.909 ±0.001

0.000 ±0.000

1.475 ±0.003

7.781 ±0.140

0.083 ±0.204

0.076 ±0.000

0.208 ±0.000

1.000 ±0.000

HOI-Diff ( Peng et al., 2023 )

0.413 ±0.010

0.624 ±0.009

0.740 ±0.011

0.689 ±0.031

3.029 ±0.007

7.620 ±0.085

0.072 ±0.160

0.103 ±0.010

0.084 ±0.006

0.722 ±0.002

0.447 ±0.026

0.501 ±0.022

CHOIS ( Li et al., 2023a )

0.439 ±0.005

0.660 ±0.002

0.766 ±0.003

0.572 ±0.008

2.781 ±0.017

7.717 ±0.115

0.119 ±0.234

0.131 ±0.013

0.115 ±0.002

0.710 ±0.012

0.520 ±0.004

0.541 ±0.007

InterDiff ( Xu et al., 2023b )

0.501 ±0.009

0.722 ±0.003

0.824 ±0.007

0.215 ±0.001

2.461 ±0.013

7.717 ±0.117

0.092 ±0.178

0.116 ±0.011

0.124 ±0.003

0.715 ±0.000

0.562 ±0.000

0.584 ±0.003

Text2HOI ( Cha et al., 2024 )

0.428 ±0.003

0.637 ±0.003

0.745 ±0.000

0.331 ±0.003

2.665 ±0.002

7.631 ±0.000

0.055 ±0.123

0.105 ±0.003

0.102 ±0.001

0.711 ±0.001

0.488 ±0.000

0.532 ±0.002

Table 1: Quantitative comparisons on the InterAct dataset ( Xu et al., 2025a ) between our method and baseline approaches. We report R-Precision with batch sizes 256.

hand–body separation

human–object separation

R-Precision ↑

FID ↓

MM Dist ↓

Diversity →

FSR ↓

Pene ↓

Contact →

Interaction ↑

Top 1

Top 2

Top 3

C_{prec}

C_{rec}

C_{F1}

\checkmark

\checkmark

0.421 ±0.014

0.637 ±0.016

0.754 ±0.016

0.148 ±0.014

2.756 ±0.018

7.712 ±0.050

0.078 ±0.197

0.132 ±0.001

0.731 ±0.002

0.615 ±0.016

0.627 ±0.011

–

\checkmark

0.414 ±0.001

0.611 ±0.003

0.717 ±0.007

0.157 ±0.017

2.863 ±0.009

7.749 ±0.125

0.089 ±0.186

0.131 ±0.000

0.124 ±0.003

0.750 ±0.001

0.576 ±0.014

0.611 ±0.009

\checkmark

–

0.409 ±0.003

0.590 ±0.005

0.693 ±0.002

0.155 ±0.010

2.917 ±0.030

7.768 ±0.067

0.063 ±0.155

0.105 ±0.002

0.107 ±0.004

0.742 ±0.004

0.524 ±0.009

0.572 ±0.007

Table 2: Ablation study of token-separation strategies on the InterAct dataset ( Xu et al., 2025a ) . We report R-Precision with batch size 256.

Figure 4: Qualitative comparison between our method using body and hand merged into a single token ( left ) versus separating body and hand into distinct tokens ( right ). Unrealistic grasping artifacts produced by the single-token approach are highlighted in red dashed boxes. Our separate-token strategy yields better results.

Figure 5: Qualitative comparison. Left : our LIGHT without guidance. Right : our full method with guidance, which markedly enhances generation quality.

Augmented

Unseen

R-Precision ↑

FID ↓

MM Dist ↓

Diversity →

FSR ↓

Pene ↓

Contact →

Interaction ↑

Top 1

Top 2

Top 3

C_{prec}

C_{rec}

C_{F1}

\times

In-category

0.216 ±0.004

0.316 ±0.005

0.396 ±0.001

0.151 ±0.015

2.151 ±0.033

6.347 ±0.022

0.031 ±0.095

0.281 ±0.013

0.193 ±0.001

0.640 ±0.008

0.503 ±0.011

0.809 ±0.006

\checkmark

In-category

0.279 ±0.012

0.386 ±0.004

0.466 ±0.007

0.133 ±0.025

2.271 ±0.006

6.696 ±0.045

0.059 ±0.137

0.305 ±0.016

0.190 ±0.002

0.651 ±0.018

0.512 ±0.012

0.833 ±0.006

\times

Cross-category

0.022 ±0.001

0.044 ±0.015

0.064 ±0.016

5.118 ±0.018

5.078 ±0.060

5.655 ±0.055

0.226 ±0.259

0.098 ±0.005

0.049 ±0.001

0.388 ±0.003

0.077 ±0.002

0.351 ±0.005

\checkmark

Cross-category

0.022 ±0.007

0.043 ±0.003

0.071 ±0.009

4.924 ±0.046

4.788 ±0.032

5.509 ±0.010

0.032 ±0.092

0.203 ±0.025

0.116 ±0.000

0.447 ±0.007

0.184 ±0.001

0.560 ±0.008

Table 3: Ablation study. We compare models trained with and without data augmentation on the InterAct dataset ( Xu et al., 2025a ) . Experiments on unseen objects include in-category and cross-category objects never observed during training. We report R-Precision with batch size 256.

Figure 6: Ablation study on the schedule-based guidance weight

\omega_{2}

and denoising preceding offset

\delta

. Left:

\delta

fixed at 200 while varying

\omega_{2}

. Right:

\omega_{2}

fixed at 4 while varying

\delta

Dataset	< $\bm{g}_{\text{CFG}},\bm{g}_{\text{GT}}$ >	< $\bm{g}_{\text{LIGHT}},\bm{g}_{\text{GT}}$ >	< $\bm{g}_{\text{CFG}},\nabla L_{\text{pen}}$ >	< $\bm{g}_{\text{LIGHT}},\nabla L_{\text{pen}}$ >
InterAct ( Xu et al., 2025a )	0.401	0.401 + 0.002	0.217	0.217 + 0.035
OMOMO ( Li et al., 2023a )	0.395	0.395 + 0.029	0.239	0.239 + 0.032

Table 4: Comparison of the mean gradient similarity with penetration-descending direction, direction towards GT distribution.

\delta=250

is used and mean gradient similarity is calculated on all the guiding steps. we define

\langle\mathbf{a},\mathbf{b}\rangle\;=\;\frac{\mathbf{a}}{\|\mathbf{a}\|}\cdot\frac{\mathbf{b}}{\|\mathbf{b}\|}

Figure A: We visualize our augmentation results, where original MoCap sequences ( left ) are enriched by introducing novel objects ( right ) sourced from ShapeNet ( Chang et al., 2015 ) and Objaverse ( Deitke et al., 2023 ) . Through optimization, human and object motions are aligned at corresponding contact points.

Dataset	Data source	Pene ↓	Floating ↓	$CH_{prec}^{\uparrow}$	$CH_{rec}^{\uparrow}$	$CH_{F1}^{\uparrow}$
InterAct ( Xu et al., 2025a )	Ground Truth	0.008	0.005	0.947	0.911	0.920
InterAct ( Xu et al., 2025a )	Augmentation	0.010	0.011	0.980	0.980	0.980
GRAB ( Taheri et al., 2020 )	Ground Truth	0.003	0.000	1.000	1.000	1.000
GRAB ( Taheri et al., 2020 )	Augmentation	0.004	0.000	0.994	0.998	0.996

Table A: Quantitative evaluation of augmented data quality relative to ground truth annotations.

$m_{1}$	$m_{2}$	FID ↓	Diversity →	FSR ↓	Pene ↓	Contact →	Interaction ↑
$m_{1}$	$m_{2}$	FID ↓	Diversity →	FSR ↓	Pene ↓	Contact →	$C_{prec}$	$C_{rec}$	$C_{F1}$
–	–	0.989 ±0.231	6.678 ±0.163	0.165 ±0.214	0.123 ±0.004	0.162 ±0.001	0.640 ±0.007	0.610 ±0.013	0.585 ±0.003
$\bm{o},\bm{h}$	$\bm{b}$	0.958 ±0.229	6.715 ±0.141	0.172 ±0.221	0.122 ±0.011	0.164 ±0.003	0.643 ±0.005	0.612 ±0.024	0.585 ±0.008
$\bm{b},\bm{h}$	$\bm{o}$	0.957 ±0.258	6.672 ±0.224	0.165 ±0.217	0.122 ±0.005	0.167 ±0.001	0.640 ±0.003	0.617 ±0.013	0.588 ±0.003
$\bm{b},\bm{o}$	$\bm{h}$	0.963 ±0.240	6.694 ±0.180	0.173 ±0.221	0.122 ±0.006	0.169 ±0.002	0.646 ±0.013	0.622 ±0.020	0.593 ±0.002
$\bm{b}$	$\bm{o},\bm{h}$	0.976 ±0.238	6.691 ±0.201	0.172 ±0.222	0.122 ±0.009	0.170 ±0.004	0.640 ±0.009	0.624 ±0.023	0.593 ±0.005

Table B: Quantitative evaluation of unconditional generation on the BEHAVE dataset ( Bhatnagar et al., 2022 ) . For R-Precision we adopt a batch size of 64. We don’t retrain our model for unconditional generation.

Figure B: User study results. 15 participants evaluates our method comparing baselines, and our guidance mechanism. Percentages indicate participant preference.

Dataset

\omega_{1}

m_{1}

m_{2}

R-Precision ↑

FID ↓

MM Dist ↓

Diversity →

FSR ↓

Pene ↓

Contact →

Interaction ↑

Top 1

Top 2

Top 3

C_{prec}

C_{rec}

C_{F1}

BEHAVE

0.5

–

0.258 ±0.000

0.447 ±0.003

0.543 ±0.027

0.493 ±0.008

3.297 ±0.080

6.878 ±0.139

0.164 ±0.239

0.144 ±0.004

0.162 ±0.002

0.737 ±0.017

0.711 ±0.015

0.694 ±0.015

\bm{o},\bm{h}

\bm{b}

0.275 ±0.003

0.455 ±0.003

0.535 ±0.022

0.485 ±0.028

3.315 ±0.070

6.942 ±0.133

0.170 ±0.244

0.150 ±0.003

0.164 ±0.001

0.732 ±0.010

0.712 ±0.014

0.693 ±0.013

\bm{b},\bm{h}

\bm{o}

0.254 ±0.005

0.451 ±0.003

0.537 ±0.030

0.491 ±0.017

3.305 ±0.070

6.892 ±0.133

0.165 ±0.238

0.146 ±0.002

0.163 ±0.002

0.732 ±0.012

0.710 ±0.013

0.691 ±0.013

\bm{b},\bm{o}

\bm{h}

0.268 ±0.003

0.449 ±0.016

0.543 ±0.022

0.480 ±0.032

3.308 ±0.065

6.941 ±0.127

0.170 ±0.245

0.144 ±0.002

0.167 ±0.001

0.728 ±0.011

0.713 ±0.018

0.694 ±0.016

\bm{b}

\bm{o},\bm{h}

0.268 ±0.003

0.453 ±0.005

0.545 ±0.019

0.478 ±0.028

3.314 ±0.069

6.951 ±0.125

0.169 ±0.245

0.148 ±0.002

0.167 ±0.002

0.727 ±0.010

0.714 ±0.019

0.694 ±0.016

Table C: Ablation of modality combinations

m_{1},m_{2}

on InterAct ( Xu et al., 2025a ) , BEHAVE ( Bhatnagar et al., 2022 ) , and OMOMO ( Li et al., 2023a ) datasets. We report R-Precision with batch size 256.

Method

R-Precision ↑

FID ↓

MM Dist ↓

Diversity →

FSR ↓

Pene ↓

Contact →

Interaction ↑

Top 1

Top 2

Top 3

C_{prec}

C_{rec}

C_{F1}

Ground Truth

0.318 ±0.003

0.560 ±0.001

0.731 ±0.001

0.000 ±0.000

1.499 ±0.000

7.400 ±0.047

0.012 ±0.039

0.067 ±0.000

0.262 ±0.000

1.000 ±0.000

HOI-Diff Peng et al. (2023)

0.221 ±0.005

0.403 ±0.001

0.542 ±0.001

0.987 ±0.023

2.520 ±0.022

7.248 ±0.066

0.021 ±0.058

0.131 ±0.005

0.135 ±0.002

0.862 ±0.000

0.696 ±0.000

0.739 ±0.001

CHOIS Li et al. (2023a)

0.291 ±0.000

0.515 ±0.009

0.676 ±0.008

0.132 ±0.016

1.746 ±0.002

7.417 ±0.104

0.019 ±0.053

0.096 ±0.004

0.230 ±0.001

0.937 ±0.000

0.900 ±0.004

0.910 ±0.002

InterDiff Xu et al. (2023b)

0.312 ±0.003

0.548 ±0.004

0.718 ±0.004

0.163 ±0.009

1.587 ±0.006

7.387 ±0.045

0.025 ±0.062

0.100 ±0.002

0.206 ±0.000

0.940 ±0.016

0.863 ±0.016

0.890 ±0.017

Text2HOI Cha et al. (2024)

0.286 ±0.001

0.513 ±0.001

0.655 ±0.001

0.168 ±0.000

1.832 ±0.006

7.359 ±0.034

0.020 ±0.056

0.127 ±0.002

0.163 ±0.000

0.919 ±0.001

0.757 ±0.002

0.807 ±0.002

Table D: Quantitative comparisons on the OMOMO dataset ( Li et al., 2023a ) between our method and baseline approaches. We report R-Precision with batch size 256.

Method

R-Precision ↑

FID ↓

MM Dist ↓

Diversity →

FSR ↓

Pene ↓

Contact →

Interaction ↑

Top 1

Top 2

Top 3

C_{prec}

C_{rec}

C_{F1}

Ground Truth

0.559 ±0.005

0.910 ±0.000

0.926 ±0.000

-0.000 ±0.000

1.483 ±0.018

6.940 ±0.153

0.181 ±0.224

0.094 ±0.001

0.204 ±0.001

1.000 ±0.000

HOI-Diff Peng et al. (2023)

0.413 ±0.010

0.624 ±0.009

0.740 ±0.011

0.689 ±0.031

3.029 ±0.007

7.620 ±0.085

0.072 ±0.160

0.103 ±0.010

0.084 ±0.006

0.722 ±0.002

0.447 ±0.026

0.501 ±0.022

CHOIS Li et al. (2023a)

0.219 ±0.000

0.396 ±0.003

0.492 ±0.022

0.705 ±0.027

3.423 ±0.045

6.863 ±0.061

0.191 ±0.253

0.128 ±0.011

0.170 ±0.003

0.717 ±0.018

0.662 ±0.000

0.650 ±0.005

InterDiff Xu et al. (2023b)

0.258 ±0.027

0.443 ±0.019

0.531 ±0.027

0.630 ±0.033

3.219 ±0.040

6.862 ±0.066

0.208 ±0.264

0.143 ±0.011

0.173 ±0.005

0.723 ±0.001

0.682 ±0.013

0.676 ±0.004

Text2HOI Cha et al. (2024)

0.135 ±0.003

0.242 ±0.000

0.322 ±0.008

1.291 ±0.073

3.815 ±0.026

6.544 ±0.013

0.177 ±0.233

0.179 ±0.003

0.128 ±0.001

0.703 ±0.011

0.579 ±0.007

0.587 ±0.008

Table E: Quantitative comparisons on the BEHAVE dataset Bhatnagar et al. (2022) between our method and baseline approaches. We report R-Precision with batch size 256.

hand–body separation

human–object separation

R-Precision ↑

FID ↓

MM Dist ↓

Diversity →

FSR ↓

Pene ↓

Contact →

Interaction ↑

Top 1

Top 2

Top 3

C_{prec}

C_{rec}

C_{F1}

\checkmark

\checkmark

0.277 ±0.011

0.475 ±0.014

0.561 ±0.041

0.481 ±0.087

3.277 ±0.076

6.961 ±0.066

0.171 ±0.245

0.140 ±0.004

0.170 ±0.001

0.731 ±0.006

0.720 ±0.011

0.697 ±0.007

–

\checkmark

0.297 ±0.005

0.533 ±0.008

0.643 ±0.003

0.515 ±0.005

2.991 ±0.053

6.963 ±0.068

0.173 ±0.250

0.126 ±0.003

0.157 ±0.004

0.731 ±0.001

0.701 ±0.012

0.692 ±0.006

\checkmark

–

0.273 ±0.022

0.471 ±0.035

0.574 ±0.038

0.498 ±0.014

3.162 ±0.000

6.898 ±0.022

0.164 ±0.246

0.135 ±0.000

0.166 ±0.001

0.713 ±0.006

0.690 ±0.006

0.676 ±0.005

Table F: Ablation study of token-separation strategies on the BEHAVE dataset Xu et al. (2025a) . We report R-Precision with batch size 256.

hand–body separation

human–object separation

R-Precision ↑

FID ↓

MM Dist ↓

Diversity →

FSR ↓

Pene ↓

Contact →

Interaction ↑

Top 1

Top 2

Top 3

C_{prec}

C_{rec}

C_{F1}

\checkmark

\checkmark

0.302 ±0.004

0.538 ±0.015

0.707 ±0.011

0.099 ±0.018

1.657 ±0.010

7.443 ±0.043

0.013 ±0.045

0.100 ±0.001

0.194 ±0.000

0.940 ±0.000

0.851 ±0.008

0.881 ±0.005

–

\checkmark

0.320 ±0.000

0.551 ±0.005

0.711 ±0.005

0.093 ±0.008

1.483 ±0.006

7.366 ±0.071

0.017 ±0.004

0.062 ±0.006

0.223 ±0.005

0.840 ±0.003

0.775 ±0.005

0.798 ±0.005

\checkmark

–

0.322 ±0.003

0.553 ±0.003

0.715 ±0.000

0.106 ±0.003

1.499 ±0.013

7.378 ±0.193

0.031 ±0.004

0.079 ±0.002

0.236 ±0.003

0.835 ±0.003

0.790 ±0.000

0.804 ±0.003

Table G: Ablation study of token-separation strategies on the OMOMO dataset ( Li et al., 2023a ) . We report R-Precision with batch size 256.

HOI Generation

Human-object interaction generation creates 3D motion, grasps, and videos of people manipulating or coordinating with objects.

See all 84 papers

ViHOI: Human-Object Interaction Synthesis with Visual Priors
Mar 25, 2026Songjin Cai, Linjie Zhong, Ling Guo +1HOI Generation Human Motion Generation
South China University of Technology
JointHOI: Jointly Generating Contact Maps Enhances Hand Object Interaction Generation
Jul 2, 2026Mingyeong Song, Jungbin Cho, Jisoo Kim +5HOI Generation Diffusion Models
Ewha Womans University, Seoul, Korea · Yonsei University, Seoul, Korea · Carnegie Mellon University, Pittsburgh PA, USA +1
GenHOI: Contact-Aware Humanoid-Object Interaction by Imitating Generated Videos without Task-Specific Training
Jun 11, 2026Zhihai Bi, Qiang Zhang, Guoyang Zhao +8Contact-Rich Robotic Manipulation Robot Motion Generation
The Hong Kong University of Science and Technology (Guangzhou) · Artificial General Intelligence Institute, University of Science and Technology of China · The University of Hong Kong +1

Mar 25, 2026J/K move · Enter open · S save

ViHOI: Human-Object Interaction Synthesis with Visual Priors

Songjin Cai, Linjie Zhong, Ling Guo, Changxing Ding

South China University of Technology

HOI Generation Human Motion Generation Diffusion Models for Motion Generation

Open paper AskPDF arXiv

Generating realistic and physically plausible 3D Human-Object Interactions (HOI) remains a key challenge in motion generation. One primary reason is that describing these physical constraints with words alone is difficult. To address this limitation, we propose a new paradigm: extracting rich interaction priors from easily accessible 2D images. Specifically, we introduce ViHOI, a novel framework that enables diffusion-based generative models to leverage rich, task-specific priors from 2D images to enhance generation quality. We utilize a large Vision-Language Model (VLM) as a powerful prior-extraction engine and adopt a layer-decoupled strategy to obtain visual and textual priors. Concurrently, we design a Q-Former-based adapter that compresses the VLM's high-dimensional features into compact prior tokens, which significantly facilitates the conditional training of our diffusion model. Our framework is trained on motion-rendered images from the dataset to ensure strict semantic alignment between visual inputs and motion sequences. During inference, it leverages reference images synthesized by a text-to-image generation model to improve generalization to unseen objects and interaction categories. Experimental results demonstrate that ViHOI achieves state-of-the-art performance, outperforming existing methods across multiple benchmarks and demonstrating superior generalization.

Abstract

HOI Generation

Explore similar work

ViHOI: Human-Object Interaction Synthesis with Visual Priors

JointHOI: Jointly Generating Contact Maps Enhances Hand Object Interaction Generation

GenHOI: Contact-Aware Humanoid-Object Interaction by Imitating Generated Videos without Task-Specific Training