Human-Object Interaction

Human-object interaction research studies which people interact with which objects, how those interactions unfold, and what they reveal about intent, contact, and function. For images and video, detection models use human and object features, spatial relationships, and increasingly vision-language models to recognize known or open-vocabulary actions; egocentric systems also study hand-object cues, gaze, and anticipation. For 3D understanding and generation, researchers reconstruct pose, contact, forces, and object motion, using diffusion models and physics-based simulation to produce more coherent interactions and transfer demonstrations to robots. The field is expanding beyond fixed action labels and curated scenes, but remains challenged by long, ambiguous interactions and by reconciling visually plausible predictions with physical consistency and reliable real-world performance.