Egocentric Video Understanding

Overview

The field studies how systems can understand activities and environments from a person’s moving, first-person viewpoint, where hands, objects, and relevant events may be briefly visible or occluded. Methods combine video-language models and temporal action models with structured scene graphs, long-term memory, 3D reconstruction, and cues such as gaze, audio, and motion sensors. Benchmarks test action recognition and anticipation, procedural step understanding, spatial reasoning, and streaming recall in settings such as kitchens, workplaces, and daily life. Models can describe visible scenes but still struggle to ground answers in precise temporal evidence, maintain consistent spatial memory, and predict intent reliably enough for assistance.

Latest papers

CardsList

219 papers21 papers in the last four weeks, up 5% on the four weeks before. 0.2% of all new papers.

  1. SAW-Bench: Learning Situated Awareness in the Real World

    Feb 18, 2026Chuhan Li, Rilyn Han, Joy Hsu +5Spatial Reasoning BenchmarksEgocentric Video Understanding

  2. Integrating Affordances and Attention models for Short-Term Object Interaction Anticipation

    Feb 16, 2026Lorenzo Mur Labadia, Ruben Martinez-Cantin, Jose J. Guerrero +2Next-Action PredictionEgocentric Video Understanding

  3. Agentic Very Long Video Understanding

    Jan 26, 2026Aniket Rege, Arka Sadhu, Yuliang Li +5Question AnsweringEgocentric Video Understanding

  4. SWITCH: Benchmarking Modeling and Handling of Tangible Interfaces in Long-horizon Embodied Scenarios

    Nov 20, 2025Juntao Cheng, Wanyue Zhang, Zhiwei Yu +7Long-Horizon Agent EvaluationEgocentric Video Understanding

  5. EgoCogNav: Cognition-aware Human Egocentric Navigation

    Nov 15, 2025Zhiwen Qiu, Ziang Liu, Wenqian Niu +2Robot NavigationEgocentric Video Understanding

  6. EgoM2P: Egocentric Multimodal Multitask Pretraining

    Jun 9, 2025Gen Li, Yutong Chen, Yiqian Wu +3Masked ModelingEgocentric Video Understanding

  7. From My View to Yours: Learning Egocentric Cues from Exocentric Video using Privileged Egocentric Supervision

    Jan 10, 2025Dominick Reilly, Manish Kumar Govind, Le Xue +1Video UnderstandingVLM Adaptation

  8. EgoGen: An Egocentric Synthetic Data Generator

    Jan 16, 2024Gen Li, Kaifeng Zhao, Siwei Zhang +5Synthetic Data GenerationHuman Motion Generation