Overview

Video understanding studies how systems interpret actions, objects, interactions, and events as they unfold, supporting tasks such as action recognition, temporal localization, video question answering, and summarization. Approaches range from spatiotemporal transformers and self-supervised motion learning to multimodal language models that use retrieval, memory, or visual tools to analyze long videos. Benchmarks test short clips, long-form and egocentric footage, procedural tasks, and audio-visual settings, but results can depend heavily on frame sampling, token budgets, and evaluation design. A central challenge is making models rely on temporal evidence and maintain coherent reasoning over long or multiple videos rather than exploiting static cues or struggling with context limits.

Latest papers

CardsList

122 papers11 papers in the last four weeks, down 48% on the four weeks before. 0.1% of all new papers.

  1. Beyond Head Importance: Collective Computation in Video Transformers

    Nov 9, 2026Yi Yang, Siyuan Liu, Bingbing NieAttention Head AnalysisTransformer Interpretability

  2. TraCE: Trajectory-Aware Concepts for Explainable Video Understanding

    Nov 4, 2026Wooil Lee, Junpyo Hong, Jongseo Lee +3Video UnderstandingConcept-Based Explanations

  3. SEEK-VAU: Towards Evidence-Faithful Video Anomaly Understanding via Agentic Search

    Nov 3, 2026Minghe Wang, Weijun Zeng, Dapeng Luo +2Active Perception for Video UnderstandingVideo Anomaly Detection

  4. DramaVQA: Probing Detailed Understanding of Ultra-Long TV Dramas

    Oct 31, 2026Xinyue Huo, Pengfei Chen, Jiannan Ge +5Video UnderstandingLong-Video QA

  5. Think in 360°: Benchmarking and Bootstrapping Panoramic Video Reasoning

    Oct 30, 2026Lidong Lu, Lidong Zou, Xinyi Mao +5Video UnderstandingVideo Reasoning

  6. Confident First, Reflective Second: Reinforcement Learning of Confidence-Preserving Reflection for Video Understanding

    Oct 27, 2026Zihao Lin, Jingyuan Liu, Yucheng Zhang +2RL for Video ReasoningVideo Understanding

  7. Joint Latent Reasoning and Observation for Agentic Video Understanding

    Oct 26, 2026Junzhang Liu, Hao Yuan, Yiming Wu +3Active Perception for Video UnderstandingVideo Understanding

  8. MarineLife-16K: An Attribute-Rich Video-Text Dataset for Motion-Aware Marine Video Understanding

    Oct 24, 2026Quang-Trung Truong, Thanh-Nghia Nguyen, Phu-Phong Vo +7Video UnderstandingMultimodal Dataset

  9. OpenHome: From Perception to Insight in Smart-Home Camera Videos

    Oct 22, 2026Xianglong Wang, Neil Rodrigues, Pei Guo +9Video UnderstandingVLM Reasoning

  10. PsycheMME: A Psychology-Informed Benchmark for Social Understanding and Interaction in Videos

    Oct 20, 2026Heran Hu, Yuhang Dong, Zhucun Xue +3Video-Language Model EvaluationVideo Understanding

  11. TRACER: A Multimodal Harness for Patient Care in Hospitals

    Oct 15, 2026Zane Durante, Paul Tang, Keshigeyan Chandrasegaran +11Video UnderstandingTemporal Video Understanding

  12. CoDy: Codec Dynamics as Complementary Priors for Unsupervised Temporal Action Localization

    Oct 14, 2026Tianyu Yan, Qingyi Yang, Qin Yang +5Temporal Action LocalizationVideo Understanding

  13. VideoHarness-SRSI: Structured Recursive Self-Improvement for Video Understanding Harness

    Oct 14, 2026Zihao Lin, Haoyu Dong, Haobo Xu +10LLM Agent HarnessesRecursive Self-Improvement

  14. Meta-Video: Efficient Standardized Evaluation for Video Understanding

    Oct 13, 2026Soumya Shamarao Jahagirdar, Anna Kukleva, Vishaal Udandarao +6Video-Language Model EvaluationVideo Understanding

  15. EventLens: Event-Structure Reinforcement Learning for Video Understanding

    Oct 12, 2026Guowen Zhang, Boshen Xu, Zihao Yue +6Video UnderstandingTemporal Video Understanding

  16. VideoEvoHarness: Self Evolving Agent Harness from Unlabeled Videos for Video Understanding

    Oct 11, 2026Weixin Xu, Yifan Liu, Yi Wang +5Agent Harness OptimizationSelf-Evolving Agents

  17. CowCare: A Unified Visual Framework for Viewpoint-Robust Cattle Tracking and Spatiotemporal Heat-Stress Analysis

    Oct 10, 2026Xin Liu, Weiran Li, Minghao Zhu +2Precision Livestock FarmingAgricultural Image Analysis

  18. DataVista: Diagnosing Multimodal LLMs on Data Video Understanding

    Oct 8, 2026Yupeng Xie, Zhenyang Wang, Jiayi Zhu +4Video UnderstandingMultimodal Large Language Models

  19. Foresight: planning future perception in streaming VLMs without retraining

    Oct 2, 2026Ashok Prasad Neupane, Dipan Bartaula, Ankit Belbase +5Vision-Language ModelsStreaming Inference

  20. Video-Index: A Curated Meta-Benchmark for Video Understanding

    Oct 1, 2026Enxin Song, Yinuo Xu, Shusheng Yang +2Video UnderstandingVideo-Language Model Evaluation

  21. CoEvoWhen: Policy-Tool Coevolution for Ultra-Long Video Temporal Grounding

    Sep 30, 2026Yiduo Jia, Muzhi Zhu, Jinchuan Shi +4Temporal Video GroundingVideo Understanding

  22. CoVisco: Codec-Native Vision Encoder with Native Token Compression for Unified Image-Video Understanding

    Sep 30, 2026Yulong Liu, Xiaotian Han, Junyuan Shang +6Video UnderstandingEfficient VLM Inference

  23. Video-RSI: Recursive Self-Improvement of Video Understanding Agents via Harness Evolution

    Sep 29, 2026Bingjun Luo, Jialin Guo, Siqi LiVideo UnderstandingAgent Harness Optimization

  24. Does Local Video Understanding Transfer Across Encounters? The EgoGears Benchmark

    Sep 29, 2026Yuedong Tan, Lei Qi, Yu Liu +13Video UnderstandingEgocentric VQA

  25. Rethinking Visual Token Compression for Video Large Language Models: A Simple Yet Strong Baseline

    Sep 28, 2026Xiao Zhang, Wang Zeng, Sheng Jin +3Video UnderstandingVideo-Language Models

  26. CinematicVQA: Benchmarking Film-Grammar Reasoning in Large Vision-Language Models

    Sep 23, 2026Shuo Xing, Pooja Verlani, Balu Adsumilli +1Video UnderstandingVideo-Language Model Evaluation