Video Understanding

Ask about this topic

Overview

Video understanding studies how systems interpret actions, objects, interactions, and events as they unfold, supporting tasks such as action recognition, temporal localization, video question answering, and summarization. Approaches range from spatiotemporal transformers and self-supervised motion learning to multimodal language models that use retrieval, memory, or visual tools to analyze long videos. Benchmarks test short clips, long-form and egocentric footage, procedural tasks, and audio-visual settings, but results can depend heavily on frame sampling, token budgets, and evaluation design. A central challenge is making models rely on temporal evidence and maintain coherent reasoning over long or multiple videos rather than exploiting static cues or struggling with context limits.

Latest papers

CardsList

116 papers21 papers in the last four weeks, up 91% on the four weeks before. 0.1% of all new papers.

  1. One Identity, Many Roles: Multimodal Entity Coreference for Enhanced Video Situation Recognition

    Apr 25, 2026Balaji Darur, Amanmeet Garg, Makarand TapaswiMultimodal GroundingVideo Understanding

    CVIT, IIIT Hyderabad, India · Amazon Prime Video, Seattle

  2. InstrAct: Towards Action-Centric Understanding in Instructional Videos

    Apr 9, 2026Zhuoyi Yang, Jiapeng Yu, Reuben Tan +2Video UnderstandingVideo-Language Models

    Pennsylvania State University, University Park, USA · Microsoft Research · Nanyang Technological University, Singapore

  3. Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understanding

    Apr 6, 2026Chaoyou Fu, Haozhi Yuan, Yuhao Dong +16Video UnderstandingVideo-Language Model Evaluation

  4. Scalable and Explainable Learner-Video Interaction Prediction using Multimodal Large Language Models

    Apr 6, 2026Dominik Glandorf, Fares Fawzi, Tanja KäserVideo UnderstandingMultimodal Model Interpretability

    EPFL, Switzerland

  5. Video Patch Pruning: Efficient Video Instance Segmentation via Early Token Reduction

    Apr 1, 2026Patrick Glandorf, Thomas Norrenbrock, Bodo RosenhahnVideo Instance SegmentationVisual Token Pruning

    Institute for Information Processing (tnt), L3S - Leibniz University Hannover, Germany

  6. GIFT: Global Irreplaceability Frame Targeting for Efficient Video Understanding

    Mar 26, 2026Junpeng Ma, Sashuai Zhou, Guanghao Li +9Video Frame SelectionVideo Understanding

    Institute of Science and Technology for Brain-inspired Intelligence, Fudan University · State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Alibaba Group Holding Limited +2

  7. EVA: Efficient Reinforcement Learning for End-to-End Video Agent

    Mar 24, 2026Yaolun Zhang, Ruohui Wang, Jiahao Wang +6Video UnderstandingTemporal Video Understanding

    SenseTime Research

  8. Reconstruction-Guided Slot Curriculum: Addressing Object Over-Fragmentation in Video Object-Centric Learning

    Mar 24, 2026WonJun Moon, Hyun Seok Seong, Jae-Pil HeoVideo Object-Centric LearningObject-Centric Representation Learning

    Sungkyunkwan University, South Korea

  9. MVPBench: A Multi-Video Perception Evaluation Benchmark for Multi-Modal Video Understanding

    Mar 24, 2026Purui Bai, Tao Wu, Jiayang Sun +3Video UnderstandingVideo-Language Model Evaluation

    MAIS & NLPR, Institute of Automation, Chinese Academy of Sciences, Beijing, China · SIST, ShanghaiTech University, Shanghai, China

  10. FedCVU: Federated Learning for Cross-View Video Understanding

    Mar 23, 2026Shenghan Zhang, Run Ling, Ke Cao +2Federated LearningVideo Understanding

    Software College, Northeastern University, Shenyang, China · University of Science and Technology of China, Hefei, China · University of Chinese Academy of Sciences, Beijing, China +1

  11. Leum-VL Technical Report

    Mar 20, 2026Yuxuan He, Chaiming Huang, Yifan Wu +4Video UnderstandingVideo-Language Models

    Hainan Sihe Data Technology Co., Ltd.

  12. From Instructions to Assistance: a Dataset Aligning Instruction Manuals with Assembly Videos for Evaluating Multimodal LLMs

    Mar 20, 2026Federico Toschi, Nicolò Brunello, Andrea Sassella +2Multimodal Large Language ModelsMultimodal Model Evaluation

    DEIB, Politecnico di Milano, Via Ponzio 34/5, 20133, Milano (MI), Italy · KASTEL, Karlsruhe Institute of Technology, Am Fasanengarten, 576131, Karlsruhe, Germany

  13. Seeking Universal Shot Language Understanding Solutions

    Mar 19, 2026Haoxin Liu, Harshavardhan Kamarthi, Zhiyuan Zhao +2Video UnderstandingVideo-Language Model Evaluation

    Georgia Institute of Technology · Dolby Laboratories

  14. Video Understanding: From Geometry and Semantics to Unified Models

    Mar 18, 2026Zhaochong An, Zirui Li, Mingqiao Ye +9Video Foundation ModelsVideo Understanding

    Department of Computer Science, University of Copenhagen, Copenhagen 2100, Denmark. · College of Computer Science, Nankai University, Tianjin 300350, China. · School of Computer and Communication Sciences, EPFL, Lausanne 1015, Switzerland. +7

  15. ConfusionBench: An Expert-Validated Benchmark for Confusion Recognition and Localization in Educational Videos

    Mar 18, 2026Lu Dong, Xiao Wang, Mark Frank +3Video UnderstandingTemporal Video Understanding

    State University of New York at Buffalo, Buffalo, NY 14260, USA

  16. A Skill-augmented Agentic Framework and Benchmark for Multi-Video Understanding

    Mar 16, 2026Yue Zhang, Liqiang Jing, Jia Li +4Video UnderstandingMultimodal QA

    University of Texas at Dallas, USA

  17. Less Data, Faster Convergence: Goal-Driven Data Optimization for Multimodal Instruction Tuning

    Mar 12, 2026Rujie Wu, Haozhe Zhao, Hai Ci +1Multimodal OptimizationInstruction-Tuning Data Selection

    School of Computer Science, Peking University, Beijing, China · Institute for Artificial Intelligence, Peking University, Beijing, China · State Key Laboratory of General Artificial Intelligence, Peking University, Beijing, China +2

  18. Beyond Single-Sample: Reliable Multi-Sample Distillation for Video Understanding

    Mar 12, 2026Songlin Li, Xin Zhu, Zechao Guan +2VLM DistillationKnowledge Distillation

    University of Electronic Science and Technology of China · Robotics Center, XPeng Motors

  19. ENIGMA-360: An Ego-Exo Dataset for Human Behavior Understanding in Industrial Scenarios

    Mar 10, 2026Francesco Ragusa, Rosario Leonardi, Michele Mazzamuto +6Video UnderstandingEgocentric Video Understanding

    Next Vision s.r.l. - Spinoff of the University of Catania, Catania, Italy.

  20. Towards Multimodal Lifelong Understanding: A Dataset and Agentic Baseline

    Mar 5, 2026Guo Chen, Lidong Lu, Yicheng Liu +17Temporal Video UnderstandingVideo Understanding

    1Nanjing University · 2NVIDIA · 3Zhejiang University +2

  21. Decoding the Hook: A Multimodal LLM Framework for Analyzing the Hooking Period of Video Ads

    Feb 25, 2026Kunpeng Zhang, Poppy Zhang, Shawndra Hill +1Video UnderstandingTemporal Video Understanding

    University of Marland, College Park College Park, Maryland, USA · Meta Platforms, Inc. New York, USA · Meta Platforms, Inc. California, USA

  22. DeepSVU: Towards In-depth Security-oriented Video Understanding via Unified Physical-world Regularized MoE

    Feb 20, 2026Yujie Jin, Wenxin Zhang, Jingjing Wang +1Video UnderstandingMixture of Experts

    School of Computer Science and Technology, Soochow University, Suzhou, China

  23. TimeBlind: A Spatio-Temporal Compositionality Benchmark for Video LLMs

    Jan 30, 2026Baiqi Li, Kangyi Zhao, Ce Zhang +3VLM EvaluationVideo Understanding

    University of North Carolina at Chapel Hill · University of Pittsburgh · Carnegie Mellon University

  24. Less is More: Label-Guided Efficient Summarization of Procedural and Instructional Videos

    Jan 18, 2026Shreya Rajpal, Michal Golovanevsky, Carsten EickhoffVideo SummarizationVideo Understanding

    Vellore Institute of Technology · Visiting student at the University of Tübingen during this work. · Brown University +1

  25. NarrativeTrack: Evaluating Entity-Centric Reasoning for Narrative Understanding

    Jan 3, 2026Hyeonjeong Ha, Jinjin Ge, Bo Feng +2VLM EvaluationVideo Understanding

    Apple · University of Illinois Urbana-Champaign

  26. From My View to Yours: Learning Egocentric Cues from Exocentric Video using Privileged Egocentric Supervision

    Jan 10, 2025Dominick Reilly, Manish Kumar Govind, Le Xue +1Video UnderstandingVLM Adaptation

    University of North Carolina at Charlotte · Elorian AI