cs.CVOct 19, 2026

BUMP: Body-driven Unified Multi-object Physics for HOI Synthesis

Authors: Wonjung Woo, Hyeonwoo Kim, Hanbyul Joo

Organizations: Seoul National University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Generating human-object interactions (HOIs) with multiple objects is challenging because the resulting motions arise from different underlying causes. Human motion reflects semantic intent, intentionally manipulated objects must remain coordinated with the human, while surrounding objects move primarily as physical consequences of contact and collision. We present BUMP, a text-driven framework for multi-object HOI generation based on a simple principle: learn intentional motion and simulate physical consequences. Our method first generates full-body human motion from text and scene context, then predicts trajectories for a small number of intentionally manipulated driver objects conditioned on the human motion. We then obtain responder object motions through physics simulation, allowing the framework to handle varying numbers and configurations of surrounding objects without expanding the learned trajectory output space. To address the scarcity of paired full-body multi-object HOI data, we combine heterogeneous supervision from human-only motion, full-body HOI, and hand-centric manipulation data augmented with plausible body motion. We further introduce outcome-conditioned interaction editing, which infers a local modification of the causal driver object trajectory from a desired responder object behavior without requiring an explicit target driver object trajectory. Experiments demonstrate strong performance on both single-object and multi-object HOI benchmarks. Qualitative results showcase diverse scenarios involving inter-object relationships and edits, highlighting the advantage of our unified model formulation.

Figures & tables

Figure 1: Overview of BUMP. Given a text prompt and an initial scene, BUMP generates intentional human and driver object motion while obtaining physically induced responder object motion through simulation.
Figure 2: Overview of the generation and editing pipeline. We generate human motion, predict hand-driver object contacts, and generate driver object trajectories conditioned on both. Physics simulation produces responder object motions. For outcome-conditioned editing, we locally modify the driver object trajectory, validate the resulting responder object motion through simulation, and align the human motion with the edited driver object while preserving the original motion.
Figure 3: Outcome-conditioned interaction editing. We compare the original and edited interactions under different responder object-level goals. The edits produce the requested changes in the physical outcomes while preserving the overall interaction.
Figure 4: Qualitative comparison across different numbers of interacting objects. We compare representative generations on InterAct and HIMO using temporal snapshots and hand-object close-ups.
More in this topic

HOI Generation

Human-object interaction generation creates 3D motion, grasps, and videos of people manipulating or coordinating with objects.

See all 84 papers

Explore similar work

CardsList
May 28, 2026

Dex2HOI: Dexterous Bimanual Two-Object Interaction Generation

Recent advances in 4D Human-Object Interaction (HOI) generation have enabled increasingly realistic motion synthesis, particularly for single-object manipulation. Yet current research overlooks an inherent property of human behavior: people naturally coordinate both hands and manipulate multiple objects simultaneously. To address this gap, we present Dex2HOI, a unified diffusion model for single- and two-object HOI synthesis from text. At its core, Dex2HOI employs a Dual-Stream Diffusion approach, where each object is processed in a dedicated interaction stream and coordinated through bidirectional cross-attention. To synthesize the final motion, we introduce a Motion Fusion Network integrated with novel hand-relative object representations and contact-aware conditioning applied across the whole sequence. By sampling the diffusion process autoregressively over prefix-conditioned windows, Dex2HOI generates arbitrarily long sequences at real-time speed omitting redundant test-time optimization, achieving up to x540 inference speed-up over prior state-of-the-art methods. Extensive evaluation on both single- and two-object benchmarks demonstrates state-of-the-art quantitative results, marking a step beyond conventional single-object HOI generation and toward expressive multi-object manipulation. Project: https://cpratikaki.github.io/dex2hoi/

Chrysa Pratikaki, Pablo Ruiz-Ponce, Jiankang Deng +2

Imperial College London, UK · University of Alicante, Spain

HOI GenerationHuman Motion GenerationMultimodal Diffusion ModelsHuman-Object InteractionHand-Object Interaction
AskPDF
May 28, 2026

PhyGenHOI: Physically-Aware 4D Generation of Dynamic Human-Object Interactions

We address the task of generating physically accurate and visually faithful 4D Human-Object Interaction (HOI). Given a static 3D human and target object represented as 3D Gaussian Splats (3DGS), our goal is to synthesize dynamic scenes where the human actively engages with the object through actions, such as punching or kicking, in accordance with a given input text. To this end, we introduce PhyGenHOI, a novel framework that couples generative human motion with an explicit physical object simulation. We model the human as a semantic agent driven by a Motion Diffusion Model (MDM) and the object as a physical agent simulated via the Material Point Method (MPM), utilizing 3D Gaussians as a unified, differentiable representation. We supervise their interaction through three coupled mechanisms: (1) A Windowed Attraction Loss that temporally synchronizes generative motion to intercept the object; (2) A Contact-Driven Re-simulation step that triggers physically consistent momentum transfer upon impact; and (3) A Masked Video-SDS objective that injects video-based priors to enhance contact fidelity. Experiments show PhyGenHOI generates physically consistent 4D HOI across diverse actions, humans, and objects, outperforming baselines. Project page and videos: https://omerbenishu.github.io/PhyGenHOI/

Omer Benishu, Gal Fiebelman, Sagie Benaim

Hebrew University of Jerusalem

HOI GenerationHuman Motion GenerationPhysics-Based Simulation4D Scene GenerationHuman-Object Interaction
AskPDF
Apr 15, 2026

OneHOI: Unifying Human-Object Interaction Generation and Editing

Human-Object Interaction (HOI) modelling captures how humans act upon and relate to objects, typically expressed as <person, action, object> triplets. Existing approaches split into two disjoint families: HOI generation synthesises scenes from structured triplets and layout, but fails to integrate mixed conditions like HOI and object-only entities; and HOI editing modifies interactions via text, yet struggles to decouple pose from physical contact and scale to multiple interactions. We introduce OneHOI, a unified diffusion transformer framework that consolidates HOI generation and editing into a single conditional denoising process driven by shared structured interaction representations. At its core, the Relational Diffusion Transformer (R-DiT) models verb-mediated relations through role- and instance-aware HOI tokens, layout-based spatial Action Grounding, a Structured HOI Attention to enforce interaction topology, and HOI RoPE to disentangle multi-HOI scenes. Trained jointly with modality dropout on our HOI-Edit-44K, along with HOI and object-centric datasets, OneHOI supports layout-guided, layout-free, arbitrary-mask, and mixed-condition control, achieving state-of-the-art results across both HOI generation and editing. Code is available at https://jiuntian.github.io/OneHOI/.

Jiun Tian Hoe, Weipeng Hu, Xudong Jiang +2

Nanyang Technological University · Sun Yat-sen University · VinUniversity +1

HOI GenerationDiffusion Transformer
AskPDF