HOI Generation

HOI: Human-Object Interaction

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 50

All topics

CoMoGen: COntrollable MOtion Dynamics and Interactions with Mask-Guided Video GENeration
May 21, 2026Adil Meric, Lin Geng Foo, Mert Kiray +3HOI Generation Image-to-Video Generation
Technical University of Munich · Max Planck Institute for Informatics, Saarland Informatics Campus · Munich Center for Machine Learning (MCML) +1
iTryOn: Mastering Interactive Video Virtual Try-On with Spatial-Semantic Guidance
May 20, 2026Jun Zheng, Zhengze Xu, Mengting Chen +6HOI Generation Virtual Try-On
Shenzhen Campus of Sun Yat-sen University · Taobao & Tmall Group of Alibaba
Dynamic Full-body Motion Agent with Object Interaction via Blending Pre-trained Modular Controllers
May 12, 2026Sanghyeok Nam, Byoungjun Kim, Daehyung Park +1HOI Generation Human Motion Generation
KAIST
DexSynRefine: Synthesizing and Refining Human-Object Interaction Motion for Physically Feasible Dexterous Robot Actions
May 7, 2026Hyesung Lee, Hyunwoo Jung, Si-Hwan Heo +1HOI Generation Dexterous Manipulation
Korea Institute of Science and Technology · KAIST · Hanyang University
MaMi-HOI: Harmonizing Global Kinematics and Local Geometry for Human-Object Interaction Generation
May 7, 2026Hao Wang, Shiqi Wang, Qi LiuHOI Generation Human Motion Generation
School of Future Technology, South China University of Technology, Guangzhou, Guangdong, China.
InterPhys: Physics-aware Human Motion Synthesis in a Dynamic Scene
May 1, 2026Chaoyue Xing, Wei Mao, Miaomiao LiuHOI Generation Human Motion Generation
Australian National University, Canberra, Australia
Stability-Driven Motion Generation for Object-Guided Human-Human Co-Manipulation
Apr 22, 2026Jiahao Xu, Xiaohan Yuan, Xingchen Wu +3HOI Generation Flow Matching
1Tianjin University · 2National University of Singapore · 3Sichuan University
CoInteract: Physically-Consistent Human-Object Interaction Video Synthesis via Spatially-Structured Co-Generation
Apr 21, 2026Xiangyang Luo, Xiaozhe Xin, Tao Feng +3HOI Generation Physical Consistency in Video Generation
Tsinghua University · Alibaba Group
SynAgent: Generalizable Cooperative Humanoid Manipulation via Solo-to-Cooperative Agent Synergy
Apr 20, 2026Wei Yao, Haohan Ma, Hongwen Zhang +6Robotic Control HOI Generation
School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China · School of Artificial Intelligence, Beijing Normal University, Beijing 100875, China +2
Ego-InBetween: Generating Object State Transitions in Ego-Centric Videos
Apr 20, 2026Mengmeng Ge, Takashi Isobe, Xu Jia +7HOI Generation Action-Conditioned Video Generation
Advanced Micro Devices, Inc. · Dalian University of Technology · The Hong Kong University of Science and Technology +2
EgoSim: Egocentric World Simulator for Embodied Interaction Generation
Apr 1, 2026Jinkun Hao, Mingda Jia, Ruiyan Wang +7HOI Generation Egocentric Vision
Shanghai Jiaotong University · Shanghai AI Laboratory · The University of Hong Kong
EgoForge: Goal-Directed Egocentric World Simulator
Mar 20, 2026Yifan Shen, Jiateng Liu, Xinzhuo Li +9HOI Generation Diffusion Model Guidance
University of Illinois Urbana-Champaign · University of California San Diego
Controllable Egocentric Video Generation via Occlusion-Aware Sparse 3D Hand Joints
Mar 12, 2026Chenyangguang Zhang, Botao Ye, Boqi Chen +4HOI Generation Egocentric Vision
1ETH Zurich · 2ETH AI Center · 3MPI for Informatics +1
CustomX: Unified Character, Action, and Scene Customization in Video World Models
Dec 18, 2025Yitong Wang, Fangyun Wei, Hongyang Zhang +2HOI Generation Action-Conditioned Video Generation
Fudan University · Microsoft Research · University of Waterloo +1
Learning to Generate Human-Human-Object Interactions from Textual Descriptions
Nov 25, 2025Jeonghyeon Na, Sangwon Baik, Inhee Lee +2HOI Generation Human Motion Generation
Seoul National University
Learning to Generate Rigid Body Interactions with Video Diffusion Models
Oct 2, 2025David Romero, Ariana Bermudez, Viacheslav Iablochnikov +3HOI Generation Physical Consistency in Video Generation
MBZUAI, UAE · Picscreen, USA
ECHO: Ego-Centric modeling of Human-Object interactions
Aug 29, 2025Ilya A. Petrov, Vladimir Guzov, Riccardo Marin +5HOI Generation 3D Reconstruction
University of Tübingen, Germany · Tübingen AI Center, Germany · Technical University of Munich, Germany +3
HunyuanVideo-HOMA: Generic Human-Object Interaction in Multimodal Driven Human Animation
Jun 10, 2025Ziyao Huang, Zixiang Zhou, Juan Cao +9HOI Generation Physical Consistency in Video Generation
University of Chinese Academy of Sciences · Tencent Hunyuan
FunHOI: Annotation-Free 3D Hand-Object Interaction Generation via Functional Text Guidance
Feb 28, 2025Yongqi Tian, Xueyu Sun, Haoyuan He +2HOI Generation Robotic Grasping
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an, China
Uni-HOI:A Unified framework for Learning the Joint distribution of Text and Human-Object Interaction
Date pendingMengfei Zhang, Jinlu Zhang, Zhigang TuHOI Generation Unified Multimodal Models
Wuhan University Wuhan, China · Peking University Beijing, China

May 21, 2026, cs.CVJ/K move · Enter open · S save

CoMoGen: COntrollable MOtion Dynamics and Interactions with Mask-Guided Video GENeration

Adil Meric, Lin Geng Foo, Mert Kiray, Benjamin Busam+2

Technical University of Munich · Max Planck Institute for Informatics, Saarland Informatics Campus · Munich Center for Machine Learning (MCML)+1

HOI Generation Image-to-Video Generation Diffusion Models for Motion Generation Pose-Conditioned Video Generation Controllable Video Generation

Open paper AskPDF arXiv

We present CoMoGen, a controllable video generation framework that generates realistic interactive dynamics from a single binary mask sequence conditioned on an input image. CoMoGen introduces a lightweight MaskAdapter that encodes binary mask sequences into a latent residual signal, injected into the Multi Modal Diffusion Transformer (MMDiT) model through a cosine-weighted schedule. Unlike the hierarchical coarse-to-fine design of UNet architectures, MMDiT operates as a sequence of uniform transformer blocks, making it difficult to identify which layers are responsible for the motion generation. Therefore, we propose a novel way to determine "Motion Layers" operating in the attention space of MMDiT. We fine-tune the model by using Low-Rank Adaptation (LoRA) to the Motion Layers, without requiring any architecture change in the MMDiT. This selective adaptation enables our method to focus on motion-critical components, yielding reduced computational cost. Despite its simplicity, CoMoGen enables precise subject motion and plausible interactions with surrounding humans, objects, and scenes. Comprehensive experiments on different datasets show that CoMoGen consistently outperforms prior controllable video generation methods and achieves state-of-the-art performance in motion fidelity and perceptual realism. Project page: mericadil.github.io/CoMoGen.

Momentum

Related topics

Latest papers 50