Egocentric Video Understanding

Follow this topic Ask about this topic

Momentum

15 papers in the last four weeks, up 150% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 140

All topics

CuriosAI Submission to the CASTLE Challenge at EgoVis 2026
May 27, 2026Yuto Kanda, Hayato Tanoue, Takayuki HoriEgocentric Video QA Egocentric Video Understanding
SoftBank Corp.
Zero-Shot Object Re-Identification in Egocentric Kitchen Videos via Multi-Stage SAM3 Feature Fusion
May 25, 2026Dmytro Klepachevskyi, Alexander Wong, Sirisha Rambhatla +1Zero-Shot Learning Egocentric Video Understanding
University of Waterloo · Waterloo, Ontario, Canada
EgoAdapt: A Multi-Scene Egocentric Adaptation Method for CVPR 2026 HD-EPIC VQA Challenge
May 23, 2026Zhiwei Chen, Yupeng Hu, Zixu Li +4Test-Time Adaptation VLM Adaptation
Shandong University · Harbin Institute of Technology (Shenzhen)
EgoAction: Egocentric Action Composition with Reliability-Aware Temporal Fusion for the EPIC-KITCHENS Action Detection Challenge at CVPR 2026
May 23, 2026Zhiheng Fu, Zixu Li, Zhiwei Chen +4Egocentric Video Understanding Video Action Recognition
1Shandong University · 2Harbin Institute of Technology (Shenzhen)
OmniEgo-R $^2$ : A Routed Reasoning Framework for the 1st Cross-Domain EgoCross Challenge at CVPR 2026
May 23, 2026Zixu Li, Zhiwei Chen, Zhiheng Fu +4Egocentric Video QA Egocentric Video Understanding
Shandong University · Harbin Institute of Technology (Shenzhen)
Cross-Modal Action Recognition in Egocentric Video Using Mamba: Integrating RGB and Hand Skeleton Streams via CLS Token Fusion Strategies
May 23, 2026Juan Ignacio Bustos Gorostegui, Maria Elena BuemiEgocentric Video Understanding Multimodal Fusion
Univ. of Buenos Aires. Faculty of Exact and Natural Sciences. Dept. of Computer Science (DC). · CONICET-Univ. of Buenos Aires. Institute of Computer Sciences (ICC). Argentina.
GazeBehavior Annotation Toolkit (GBAT): AI-powered toolkit for automatic annotation of egocentric eye-tracking and video data of child-caregiver interaction
May 21, 2026Iba Baig, Kevin Li, Yanbin Xu +5Human-in-the-Loop Annotation Egocentric Video Understanding
Department of Psychology, University of Miami · Northeastern University · Ecole Normale Sup´erieure, PSL University, EHESS, CNRS +1
JFAA: Technical Report for the EPIC-KITCHENS-100 Action Anticipation Challenge at EgoVis 2026
May 20, 2026Qiaohui Chu, Haoyu Zhang, Yisen Feng +4Egocentric Video Understanding
Harbin Institute of Technology (Shenzhen) · Pengcheng Laboratory · Shandong Jianzhu University
VISTA: Technical Report for the Ego4D Short-Term Object Interaction Anticipation at EgoVis 2026
May 20, 2026Qiaohui Chu, Haoyu Zhang, Yisen Feng +4Egocentric Video Understanding
Harbin Institute of Technology (Shenzhen) · Pengcheng Laboratory · Shandong Jianzhu University
OSGNet with MLLM Reranking @ Ego4D Episodic Memory Challenge 2026
May 20, 2026Yisen Feng, Leigang Qu, Haoyu Zhang +5Temporal Video Grounding Multimodal Reranking
Harbin Institute of Technology (Shenzhen) · National University of Singapore · Pengcheng Laboratory +2
How You Move Tells What You'll Do: Trajectory-Conditioned Egocentric Prediction
May 19, 2026Sejoon Jun, Hai Nguyen-Truong, Luigi Seminara +1Next-Action Prediction Egocentric Video Understanding
Khoury College of Computer Sciences, Northeastern University, Boston · Korea Advanced Institute of Science and Technology, Daejeon · Department of Mathematics and Computer Science, University of Catania, Italy
EgoCoT-Bench: Benchmarking Grounded and Verifiable Operation-Centric Chain of Thought Reasoning for MLLMs
May 19, 2026Yang Dai, Dian Jiao, Tianwei Lin +1Egocentric Video QA CoT Reasoning
Zhejiang University · Hangzhou, Zhejiang, China
EventPrune: Cascaded Event-Assisted Token Pruning for Efficient First-Person Dynamic Spatial Reasoning
May 19, 2026Pengtao Ma, Ziliang Zhou, Ciyu Ruan +7Visual Spatial Reasoning Efficient VLM Inference
Shenzhen International Graduate School, Tsinghua University · Harbin Institute of Technology · Tsinghua University +1
EgoExoMem: Cross-View Memory Reasoning over Synchronized Egocentric and Exocentric Videos
May 18, 2026Ruiping Liu, Junwei Zheng, Yufan Chen +7Multimodal Memory Video QA
Karlsruhe Institute of Technology · ETH Zurich · University of Oxford +2
StableHand: Quality-Aware Flow Matching for World-Space Dual-Hand Motion Estimation from Egocentric Video
May 18, 2026Huajian Zeng, Chaohua Yao, Yuantai Zhang +3Flow Matching 3D Hand Pose Estimation
Mohamed bin Zayed University of Artificial Intelligence · University of Illinois at Urbana-Champaign · Imperial College London
Seeing Together: Multi-Robot Cooperative Egocentric Spatial Reasoning with Multimodal Large Language Models
May 18, 2026Kunyu Peng, Zhikun Zhou, Kailun Yang +9Visual Spatial Reasoning Multi-Robot Systems
Karlsruhe Institute of Technology · INSAIT, Sofia University “St. Kliment Ohridski” · Hunan University +4
EgoInteract: Synthetic Egocentric Videos Generation for Interaction Understanding and Anticipation
May 18, 2026Rosario Leonardi, Francesco Ragusa, Daniele Materia +4Temporal Action Segmentation Next-Action Prediction
Department of Mathematics and Computer Science, University of Catania, Italy · Next Vision s.r.l., Catania, Italy · Reality Labs Research, Meta, USA
SPATIOROUTE: Dynamic Prompt Routing for Zero-Shot Spatial Reasoning
May 18, 2026Pawat Chunhachatrachai, Gueter Josmy Faure, Hung-Ting Su +1Visual Spatial Reasoning 3D Spatial Reasoning
National Taiwan University · Delta Robotics Innovation Center
MARS: Technical Report for the CASTLE Challenge at EgoVis 2026
May 18, 2026Haoyu Zhang, Qiaohui Chu, Yisen Feng +4Egocentric Video QA Egocentric Video Understanding
Harbin Institute of Technology (Shenzhen) · Pengcheng Laboratory · Shandong Jianzhu University
EgoIntrospect: An Egocentric Dataset and Benchmark for User-Centric Internal State Reasoning
May 17, 2026Zeyu Wang, Chang Liu, Eduardus Tjitrahardja +22Multimodal Large Language Models Egocentric Video Understanding
Tsinghua University · Shanghai Qi Zhi Institute · Tongji University +5
AI-Assisted Competency Assessment from Egocentric Video in Simulation-Based Nursing Education
May 16, 2026Hanchen David Wang, Yilin Liu, Madison J. Lee +4Educational Assessment Educational Technology
Vanderbilt University
Minerva-Ego: Spatiotemporal Hints for Egocentric Video Understanding
May 14, 2026Arsha Nagrani, Jasper Uijilings, Shyamal Buch +6Egocentric Video QA Egocentric Video Understanding
Google DeepMind
Characterizing the visual representation of objects from the child's view
May 14, 2026Jane Yang, Tarun Sepuri, Alvin Wei Ming Tan +3Representation Learning Visual Representation Learning
Department of Psychology, University of California San Diego · Department of Psychology, Stanford University · Department of Computer Science, Stanford University
EARL: Towards a Unified Analysis-Guided Reinforcement Learning Framework for Egocentric Interaction Reasoning and Pixel Grounding
May 14, 2026Yuejiao Su, Xinshen Zhang, Zhen Ye +3Egocentric Video Understanding Vision-Language Grounding
Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong SAR. · Division of Emerging Interdisciplinary Areas (EMIA), The Hong Kong University of Science and Technology, Hong Kong SAR.
TouchAnything: A Dataset and Framework for Bimanual Tactile Estimation from Egocentric Video
May 13, 2026Jianyi Zhou, Ziteng Gao, Feiyang Hong +11Vision-Based Tactile Sensing Tactile Sensing
Harbin Institute of Technology, Shenzhen · Meituan Academy of Robotics · Tsinghua Shenzhen International Graduate School, Tsinghua University
EgoForce: Robust Online Egocentric Motion Reconstruction via Diffusion Forcing
May 13, 2026Inwoo Hwang, Donggeun Lim, Hojun Jang +1Markerless Motion Capture Egocentric Video Understanding
Seoul National University
EgoForce: Forearm-Guided Camera-Space 3D Hand Pose from a Monocular Egocentric Camera
May 12, 2026Christen Millerdurai, Shaoxiang Wang, Yaxu Xie +33D Hand Pose Estimation Monocular 3D Reconstruction
Deutsches Forschungszentrum für Künstliche Intelligenz (DFKI), Kaiserslautern, Germany · Max Planck Institute for Informatics (MPII), Saarbrücken, Germany
BARISTA: A Multi-Task Egocentric Benchmark for Compositional Visual Understanding
May 12, 2026Patrick Knab, Orgest Xhelili, Inis Buzi +7Video Understanding Scene Graph Generation
Ramblr.ai Research · Technical University of Clausthal
EgoMemReason: A Memory-Driven Reasoning Benchmark for Long-Horizon Egocentric Video Understanding
May 11, 2026Ziyang Wang, Yue Zhang, Shoubin Yu +6Multimodal Memory Egocentric Video Understanding
UNC Chapel Hill · NTU Singapore
EyeCue: Driver Cognitive Distraction Detection via Gaze-Empowered Egocentric Video Understanding
May 8, 2026Lang Zhang, JinYi Yoon, Matthew Corbett +2Egocentric Video Understanding Multimodal Fusion
Virginia Tech · Inha University · Army Cyber Institute at West Point

May 27, 2026, cs.CVJ/K move · Enter open · S save

CuriosAI Submission to the CASTLE Challenge at EgoVis 2026

Yuto Kanda, Hayato Tanoue, Takayuki Hori

SoftBank Corp.

Egocentric Video QA Egocentric Video Understanding Long-Video QA Multimodal KGs Temporal QA Multimodal Understanding

Open paper AskPDF arXiv

CASTLE 2026 asks 185 multiple-choice questions over 600+ hours of synchronized multi-view egocentric video. We explore two approaches on top of a shared multimodal preprocessing layer, including per-person timelines, speaker-resolved transcripts, and multi-VLM caption ensembles. Approach A, SVA: Search-Verify-Answer, is a three-stage pipeline that hierarchically narrows to a primary window, verifies sub-windows with a VLM under four anti-confabulation rules, and fuses evidence with an LLM judge under an evidence-priority hierarchy. Approach B, TMKG: Temporal-Multimodal-Knowledge-Graph, is the contrast: it builds a temporal multimodal knowledge graph, locates a primary cell via graph search, and produces the final answer with a single grounded VLM. SVA reaches a leaderboard accuracy of 0.50 and is our final challenge submission; TMKG reaches 0.35.

Momentum

Related topics

Latest papers 140