Egocentric Video Understanding

Overview

The field studies how systems can understand activities and environments from a person’s moving, first-person viewpoint, where hands, objects, and relevant events may be briefly visible or occluded. Methods combine video-language models and temporal action models with structured scene graphs, long-term memory, 3D reconstruction, and cues such as gaze, audio, and motion sensors. Benchmarks test action recognition and anticipation, procedural step understanding, spatial reasoning, and streaming recall in settings such as kitchens, workplaces, and daily life. Models can describe visible scenes but still struggle to ground answers in precise temporal evidence, maintain consistent spatial memory, and predict intent reliably enough for assistance.

Latest papers

CardsList

219 papers21 papers in the last four weeks, up 5% on the four weeks before. 0.2% of all new papers.

  1. EgoLink: Egocentric Language-Vision Interactive Network Knowledge Challenge Summary

    Nov 16, 2026Renhe Sun, Yueying Feng, Bohan Yu +12Multimodal Agent EvaluationEgocentric Video Understanding

  2. Can an LVLM Perceive Event Boundaries Like a Human?

    Nov 12, 2026Hyolim Kang, Jungsoo Lee, Yeji Song +3Change-Point DetectionEgocentric Video Understanding

  3. Sub-actions in Action: Text-Guided Hand-Role Alignment for Sub-Action Recognition

    Nov 11, 2026Sadegh Rahmaniboldaji, Filip Rybansky, Quoc Vuong +2Egocentric Video UnderstandingHand-Object Interaction Detection

  4. Hand-Object Contact Transition Detection in Egocentric Video

    Nov 10, 2026Huy Anh Nguyen, Chang Dong, Feras Dayoub +1Hand-Object InteractionHand-Object Interaction Detection

  5. EgoLabel: Zero-Shot Egocentric Auto-Labeling with Cross-Modal Consistency

    Nov 9, 2026Xiaozhi Zhang, Changjia Xu, Yaoyuan Yan +2Egocentric Video UnderstandingCross-Modal Image Registration

  6. EgoVeil: Personalized Privacy Reasoning for Egocentric Video

    Nov 7, 2026Xue Wang, Yang ZhangPrivacy Protection in VLMsEgocentric Video Understanding

  7. EventHOI: Event-Driven Resource-Efficient Hand-Object Interaction Logging on Smartglasses

    Nov 6, 2026Le Zhang, Kailai Cui, Hao Chen +5Wearable SensingHand-Object Interaction

  8. Egocentric Action Recognition with Objects and Vision-Language Affordances

    Nov 5, 2026Raúl Boix-Aldeguer, Manuel Benavent-Lledo, David Ortiz-Perez +1Video Action RecognitionEgocentric Video Understanding

  9. EgoActionBench: Benchmarking Dense Action Captioning for Egocentric Human Videos

    Nov 2, 2026Yujiao Shen, Jiahe Song, Haitian Li +2Egocentric Video UnderstandingDense Video Captioning

  10. EgoMo3R: Joint Egocentric Motion and Scene Reconstruction

    Nov 1, 2026Chaitanya Patel, Yusuke Kato, Rin Takahira +33D ReconstructionEgocentric Video Understanding

  11. DUGO: Dual-Scale Gaussian Memory for Online Temporal Action Localization in Egocentric Procedural Videos

    Oct 30, 2026Mohammed Aman Bhuiyan, Md. Sazzad Hossain Adib, Samiul Basir Bhuiyan +4Temporal Action LocalizationTemporal Video Understanding

  12. MotorSense: A Video-EMG Dataset of Motoric Representations for Action Understanding

    Oct 30, 2026Eadom Dessalene, Michael Maynord, Amir-Hossein Shahidzadeh +6Multimodal DatasetEgocentric Video Understanding

  13. Perceiving and Predicting Dynamic Sports: Spatiotemporal Intelligence in Vision Models for Egocentric Tennis Video

    Oct 27, 2026Tudor Braicu, Joseph DelPreto, Jared Boyer +4Egocentric Video UnderstandingTemporal Video Understanding

  14. Ego-CEVR: Learning Cross-Embodiment Visual Representations from Human Egocentric Videos

    Oct 27, 2026Keru Zhou, Zhirui Fang, Xinrun Xu +19Visual Representation LearningRobot Learning from Video

  15. SOMA: A Benchmark for Fine-Grained Egocentric Behavior and Body-Motion Understanding

    Oct 26, 2026Matea Tashkovska, Naël Ouerghemi, Haozhe Qi +1Human Motion UnderstandingEgocentric Video Understanding

  16. ProactEgoMem: Benchmarking Proactive Memory in Streaming Egocentric Video

    Oct 21, 2026Ruotong Liao, Meikel Kokowski, Xiaoxiao Sun +6Agent MemoryMemory Agent Benchmarks

  17. DewHand: Direct Metric World-Space Hand Motion Reconstruction from Egocentric Videos

    Oct 21, 2026Naru Suzuki, Satoshi Ikehata, Takehiko Ohkawa +23D Hand ReconstructionEgocentric Video Understanding

  18. SENRI: Understanding the Space-Event Nexus in Real-Life Videos

    Oct 18, 2026Kehan Lan, Kaining Ying, Henghui Ding +1Egocentric Video UnderstandingSpatiotemporal Reasoning

  19. EgoMatrix: Physically Faithful Dexterous Robot Demonstrations from Egocentric Human Videos

    Oct 13, 2026Ziheng Wang, Yanming Shao, Ting Mao +4Hand-Object Interaction ReconstructionRobotic Data Collection

  20. EgoOmniWorld: Unifying Egocentric Understanding via Ego-Action-World Interactions

    Oct 13, 2026Songruo Chen, Yuqian Fu, Yu Li +7Egocentric Video UnderstandingEgocentric VQA

  21. EgoPhys: Estimating Peak Contact Force and Mechanical Work from Egocentric Manipulation Video

    Oct 8, 2026Zhuo Dong, Jianhua Yang, Haohao Li +4Contact Force EstimationHuman-Object Interaction

  22. FloorSAV: Elucidating Spatial Audio-Visual Context with 2D Floormap for AV-LLMs

    Oct 8, 2026Kyeong-Rae Kim, Sungnyun Kim, Tae-Hyun Oh3D Spatial ReasoningAudio-Visual Understanding