Human-Object Interaction

Also known as HOI

Follow this topic Ask about this topic

Momentum

9 papers in the last four weeks, up 200% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 75

All topics

DeVI: Physics-based Dexterous Human-Object Interaction via Synthetic Video Imitation
Apr 22, 2026Hyeonwoo Kim, Jeonghwan Kim, Kyungwon Cho +1Imitation Learning Robotic Manipulation
Seoul National University · RLWRLD
LEXIS: LatEnt ProXimal Interaction Signatures for 3D HOI from an Image
Apr 22, 2026Dimitrije Antić, Alvaro Budria, George Paschalidis +23D Reconstruction Vector-Quantized Variational Autoencoder
University of Amsterdam, The Netherlands · Max Planck Institute for Intelligent Systems, Tübingen, Germany · Aristotle University of Thessaloniki, Greece
Stability-Driven Motion Generation for Object-Guided Human-Human Co-Manipulation
Apr 22, 2026Jiahao Xu, Xiaohan Yuan, Xingchen Wu +3HOI Generation Flow Matching
1Tianjin University · 2National University of Singapore · 3Sichuan University
CoInteract: Physically-Consistent Human-Object Interaction Video Synthesis via Spatially-Structured Co-Generation
Apr 21, 2026Xiangyang Luo, Xiaozhe Xin, Tao Feng +3HOI Generation Physical Consistency in Video Generation
Tsinghua University · Alibaba Group
SynAgent: Generalizable Cooperative Humanoid Manipulation via Solo-to-Cooperative Agent Synergy
Apr 20, 2026Wei Yao, Haohan Ma, Hongwen Zhang +6Robotic Control HOI Generation
School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China · School of Artificial Intelligence, Beijing Normal University, Beijing 100875, China +2
HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
Apr 16, 2026Jongbin Lim, Taeyun Ha, Seongho Cha +6Robotic Grasping Dexterous Manipulation
Seoul National University · RLWRLD
EgoSim: Egocentric World Simulator for Embodied Interaction Generation
Apr 1, 2026Jinkun Hao, Mingda Jia, Ruiyan Wang +7HOI Generation Egocentric Vision
Shanghai Jiaotong University · Shanghai AI Laboratory · The University of Hong Kong
World In Your Hands: A Large-Scale and Open-Source Ecosystem for Learning Human-Centric Manipulation in the Wild
Dec 30, 2025Yupeng Zheng, Jichao Peng, Weize Li +22Robot Skill Learning Robotic Data Collection
TARS Robotics
Learning to Generate Human-Human-Object Interactions from Textual Descriptions
Nov 25, 2025Jeonghyeon Na, Sangwon Baik, Inhee Lee +2HOI Generation Human Motion Generation
Seoul National University
ECHO: Ego-Centric modeling of Human-Object interactions
Aug 29, 2025Ilya A. Petrov, Vladimir Guzov, Riccardo Marin +5HOI Generation 3D Reconstruction
University of Tübingen, Germany · Tübingen AI Center, Germany · Technical University of Munich, Germany +3
LHM-Humanoid: Long-Horizon Human Motion Control for Continuous Object Transport in Cluttered Scenes
Aug 23, 2025Haozhuo Zhang, Jingkai Sun, Michele Caprio +5Human Motion Generation Goal-Conditioned RL
The University of Manchester, Manchester, UK · X-Humanoid, Beijing, China · University of Hong Kong, Hong Kong, China +1
Diffusion Models are Open-World Affordance Learners: Leveraging Generative Priors for 3D Affordance Learning
Aug 3, 2025Hanqing Wang, Zhenhao Zhang, Kaiyang Ji +12Human-Object Interaction 3D Visual Grounding
The Hong Kong University of Science and Technology, Guangzhou, China · ShanghaiTech University, Shanghai, China · Zhejiang University, Hangzhou, China +3
HunyuanVideo-HOMA: Generic Human-Object Interaction in Multimodal Driven Human Animation
Jun 10, 2025Ziyao Huang, Zixiang Zhou, Juan Cao +9HOI Generation Physical Consistency in Video Generation
University of Chinese Academy of Sciences · Tencent Hunyuan
Web2Grasp: Learning Functional Grasps from Web Images of Hand-Object Interactions
May 7, 2025Hongyi Chen, Yunchao Yao, Yufei Ye +8Robotic Grasping Human-Object Interaction
Carnegie Mellon University · Nvidia · National University of Singapore +1
Uni-HOI:A Unified framework for Learning the Joint distribution of Text and Human-Object Interaction
Date pendingMengfei Zhang, Jinlu Zhang, Zhigang TuHOI Generation Unified Multimodal Models
Wuhan University Wuhan, China · Peking University Beijing, China

Apr 22, 2026, cs.CVJ/K move · Enter open · S save

DeVI: Physics-based Dexterous Human-Object Interaction via Synthetic Video Imitation

Hyeonwoo Kim, Jeonghwan Kim, Kyungwon Cho, Hanbyul Joo

Seoul National University · RLWRLD

Imitation Learning Robotic Manipulation Human-Object Interaction

Open paper AskPDF arXiv

Recent advances in video generative models enable the synthesis of realistic human-object interaction videos across a wide range of scenarios and object categories, including complex dexterous manipulations that are difficult to capture with motion capture systems. While the rich interaction knowledge embedded in these synthetic videos holds strong potential for motion planning in dexterous robotic manipulation, their limited physical fidelity and purely 2D nature make them difficult to use directly as imitation targets in physics-based character control. We present DeVI (Dexterous Video Imitation), a novel framework that leverages text-conditioned synthetic videos to enable physically plausible dexterous agent control for interacting with unseen target objects. To overcome the imprecision of generative 2D cues, we introduce a hybrid tracking reward that integrates 3D human tracking with robust 2D object tracking. Unlike methods relying on high-quality 3D kinematic demonstrations, DeVI requires only the generated video, enabling zero-shot generalization across diverse objects and interaction types. Extensive experiments demonstrate that DeVI outperforms existing approaches that imitate 3D human-object interaction demonstrations, particularly in modeling dexterous hand-object interactions. We further validate the effectiveness of DeVI in multi-object scenes and text-driven action diversity, showcasing the advantage of using video as an HOI-aware motion planner.

Momentum

Related topics

Latest papers 75