Human-Object Interaction

Also known as HOI

Ask about this topic

Overview

Human-object interaction research studies which people interact with which objects, how those interactions unfold, and what they reveal about intent, contact, and function. For images and video, detection models use human and object features, spatial relationships, and increasingly vision-language models to recognize known or open-vocabulary actions; egocentric systems also study hand-object cues, gaze, and anticipation. For 3D understanding and generation, researchers reconstruct pose, contact, forces, and object motion, using diffusion models and physics-based simulation to produce more coherent interactions and transfer demonstrations to robots. The field is expanding beyond fixed action labels and curated scenes, but remains challenged by long, ambiguous interactions and by reconciling visually plausible predictions with physical consistency and reliable real-world performance.

Latest papers

CardsList

112 papers17 papers in the last four weeks, up 240% on the four weeks before. 0.1% of all new papers.

  1. Auditing Semantic and Relational Reliability in Open-Vocabulary Human-Object Interaction Evaluation

    Oct 19, 2026S M Asif Hossain, Ruksat Khan Shayoni, Md Kishor MorolHuman-Object InteractionSemantic Textual Similarity

  2. SituHOI: Scalable Capture of Scene-Situated Full-Body Human-Object Interaction

    Oct 13, 2026Qifan Wang, Qihang Fang, Boao Shi +4Human-Object InteractionHand-Object Interaction

    The University of Hong Kong

  3. Benchmarking Multimodal Large Language Models for Zero-Shot Human-Object Interaction

    Oct 13, 2026Oriol Rabasseda, Zenjie Li, Kamal Nasrollahi +1Human-Object InteractionMultimodal Large Language Models

    Milestone Systems A/S, Banemarksvej 50, 2605 Broendby, Denmark · Universitat de Barcelona, Gran Via de les Corts Catalanes 585, 08007, Barcelona, Spain · Aalborg University, Rendsburggade 14, 9000 Aalborg, Denmark

  4. From Video to Vision: Learning Generalizable Humanoid Carrying from Human Motion

    Oct 10, 2026Ziluo DingRL for RoboticsGeneralization in Robotic Manipulation

    X-Humanoid

  5. EgoPhys: Estimating Peak Contact Force and Mechanical Work from Egocentric Manipulation Video

    Oct 8, 2026Zhuo Dong, Jianhua Yang, Haohao Li +4Contact Force EstimationHuman-Object Interaction

    School of Artificial Intelligence, Shandong University, Jinan, China · Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Mechanical Engineering, Tianjin University, Tianjin, China

  6. OmniHOI: Dexterous Hand-Object Interaction from Monocular Human Video

    Oct 7, 2026Ting Mao, Yanming Shao, Ziheng Wang +4Hand-Object InteractionHand-Object Interaction Reconstruction

    Zhejiang University · The University of Hong Kong · Shanghai Jiao Tong University +1

  7. ECHO: Embodied Camera Observations of Human Object Carrying

    Oct 7, 2026Xuefei Sun, Lorin Achey, Kali Hamilton +4Human-Object Interaction3D Scene Understanding

    Intelligent Robotics Laboratory, Department of Computer Science, University of Colorado Boulder, Boulder, CO 80309, USA · Lockheed Martin, Advanced Technology Laboratories, USA · Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA 15213, USA

  8. iGPC: Generative Motion Priors for Object-Aware Humanoid Interaction

    Oct 6, 2026Anujith Muraleedharan, Abdul Ahad Butt, Nolan Fey +5Contact-Rich Robotic ManipulationHumanoid Robot Control

    Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi, United Arab Emirates. · Massachusetts Institute of Technology (MIT), Cambridge, MA, USA. · Minerva Humanoids, Inc.

  9. Reactive Task-Oriented Robot-Human Handovers via Generative Hypothesis Selection

    Oct 6, 2026Carmen Scheidemann, Andreea Tulbure, Pascal Burkhardt +1Human-Robot InteractionHuman-Object Interaction

    Robotic Systems Lab, ETH Zurich

  10. Harnessing Multimodal Large Language Models for Training-Free Human-Object Interaction Detection

    Oct 5, 2026Zhaolin Cai, Huiyu Duan, Liu Yang +5Multimodal Large Language ModelsMLLM Reasoning

    Shanghai Jiao Tong University · Xinjiang University · Beijing Jiaotong University +1

  11. Robotizing Human Videos with Physically Consistent Interactions

    Oct 5, 2026Ching-Lam Cheng, Shengfeng He, Bin ZhuRobotic GraspingData Augmentation

    Singapore Management University

  12. I-BFM: Reward-Conditioned Robust Humanoid Interaction via Unsupervised Reinforcement Learning

    Oct 5, 2026Ziqi Han, Yitang Li, Junhan Sun +8Representation LearningReinforcement Learning

    Tongji University · Tsinghua University · Zhejiang University +3

  13. PACT: End-to-End Learning of Human Pose, Contacts, and Forces from Video

    Sep 30, 2026Rikhat Akizhanov, Yangsong Zhang, Nikolai Kaliazin +5Contact Force EstimationHuman Pose Estimation

    MBZUAI · ETH Zürich · EPFL

  14. DynamicHOI: Coupled Dynamics for Physics-aware HOI Reconstruction

    Sep 29, 2026Wenliang Guo, Zhanbo Huang, Yu Kong3D ReconstructionMonocular 3D Reconstruction

    Michigan State University

  15. HEIR: Learning Human-Entity Interactions with Functional Roles

    Sep 28, 2026Di Wen, Wenhao Guo, Yuedong Tan +15Set PredictionHuman-Object Interaction

    Karlsruhe Institute of Technology (KIT) · Institute for Computer Science, Artificial Intelligence and Technology (INSAIT) · Hunan University +2

  16. HOI-Retarget: Contact-Centric Retargeting for Human-Object Interaction

    Sep 28, 2026Jihwan Shin, Adrià López Escoriza, Junzhe He +2Contact-Rich Robotic ManipulationSynthetic Data Generation for Robotics

    Robotic Systems Lab, ETH Zürich, 8092 Zürich, Switzerland

  17. Functional Hand Type Prior for 3D Hand Pose Estimation and Action Recognition from Egocentric View Monocular Videos

    Sep 28, 2026Wonseok Roh, Seung Hyun Lee, Won Jeong Ryoo +53D Hand Pose EstimationHand-Object Interaction Detection

    Department of Artificial Intelligence, Korea University, Seoul, Republic of Korea · Electrical and Computer Engineering, Purdue University, West Lafayette, Indiana, USA

  18. Harnessing Coupled Stream Completion For Human-Object Interaction Modeling

    Sep 26, 2026Dawei Guan, Di Yang, Jiangtao WangHOI GenerationMotion Representation Learning

    School of Artificial Intelligence and Data Science, University of Science and Technology of China

  19. HOIBlender: Blending Lightweight Detection with Vision-Language Priors for Efficient Human-Object Interaction Detection

    Sep 20, 2026Junwen Chen, Keiji YanaiEfficient Object DetectionObject Detection

    Department of Informatics, The University of Electro-Communications, Tokyo, Japan

  20. Single-Query Person-Centric Bimanual Hand-Object Interaction Detection

    Sep 10, 2026Jonghyun Kim, Junho Roh, Yubin Yoon +5Hand-Object Interaction DetectionHuman-Object Interaction

    AI Lab, CTO Division, LG Electronics, South Korea

  21. Grounding Generated Video Plans in Simulation Towards Versatile Dexterous Controllers

    Sep 9, 2026Tianyue Wu, Boyuan An, Shuqi Zhao +6Goal-Conditioned Robotic ManipulationHand-Object Interaction Detection

    Sharpa Robotics · The University of Hong Kong · University of California, Berkeley

  22. FineHOI: Part-Aware Dense Representations for Zero-Shot Human-Object Interaction Detection

    Sep 5, 2026Francesco Tonini, Lorenzo Vaquero, Mohammad Mahdi Derakhshani +3Human-Object InteractionZero-Shot Learning

    University of Trento Trento, Italy · Fondazione Bruno Kessler Trento, Italy · University of Amsterdam Amsterdam, The Netherlands +1

  23. Multi-Person Human Motion Forecasting in Complex Scenes

    Aug 27, 2026Serdar Ozsoy, Lars Doorenbos, Juergen GallHuman Motion PredictionDiffusion Models for Motion Generation

    University of Bonn, Germany · Lamarr Institute for Machine Learning and Artificial Intelligence, Germany

  24. Identity-Aware Human-Object Interaction Motion Captioning

    Aug 21, 2026Yiming Wang, Yonghao Dang, Huilai Li +2Human-Object Interaction

  25. Vision-Language Models for Egocentric Video: From Hand-Object Interaction to Embodied AI

    Aug 19, 2026Mohammad Zamani, Fatemeh ZiaeetabarHand-Object Interaction DetectionEgocentric Video Understanding

    Department of Computer Science, School of Mathematics, Statistics and Computer Science, College of Science, University of Tehran, Tehran, Iran

  26. HiPHI: A Large-Scale Benchmark for High-Precision Human Motion and Object-Interaction

    Aug 17, 2026Jiahao Ji, Ji Ma, Runhan Zhang +12Humanoid RoboticsRobot Learning

    Noitom Robotics · National University of Singapore · The University of Hong Kong +2

  27. EgoPHI: Estimating 3D Hand-Object Contact and Force from Egocentric Vision

    Aug 13, 2026Andela Ilic, Rachel Schuchert, Yijing Jiang +1Hand-Object Interaction DetectionEgocentric Vision

    Department of Computer Science, ETH Zürich, Zürich, Switzerland

  28. GESTO: Human-Centric Spatio-Temporal Memory for Reasoning in Dynamic Scenes

    Aug 11, 2026Ermanno Bartoli, Buwei He, Dennis Rotondi +6Spatiotemporal ReasoningGraph-Based Agent Memory

    Department of Robotics, Perception and Learning, KTH Royal Institute of Technology, Sweden · Socially Intelligent Robotics Lab at the Institute for Artificial Intelligence, University of Stuttgart, Germany · Intl. Max Planck Research School for Intelligent Systems (IMPRS-IS) +2

  29. bFaaaP: An Inclusive, Head-Angle Piano-Pedal Interaction that Quantitatively Reproduces a Pianist's Intended Pedalling -- Foot-Free, for Acoustic and Electronic Pianos

    Aug 1, 2026Tomoyuki Shishido, Masahiro Ootaki, Hiroyuki NarusawaHuman-Object Interaction

    Shishido & Associates, Hachioji, Tokyo, Japan · SeemeData Labs, Inc., Nakano-ku, Tokyo, Japan · Shishido & Associates, Horinouchi, Hachioji, 192-0355, Tokyo, Japan +5

  30. RF-HOI: Recognize Human-Object Interaction with Radio Frequency Signals

    Jul 31, 2026Lihao Wang, Linlu Gao, Jiacan Yu +5Synthetic Data GenerationHuman-Object Interaction

    Johns Hopkins University, USA