Temporal Video Understanding

Overview

Temporal video understanding studies how actions and events unfold, how objects and states change, and how evidence across frames supports tasks such as action localization, moment retrieval, video question answering, and future prediction. Approaches include temporal transformers and state-space models, motion and trajectory representations, event graphs and memory, and adaptive frame or token selection; vision-language models are also trained or guided to reason over video evidence. Evaluation spans action-localization datasets, long-video question answering, temporal-order and counting tests, and streaming settings where models must decide when to respond. A central tension is that models can perform well on broad benchmarks while relying on static cues or language priors rather than temporal evidence, and efficient sampling or compression must retain the brief events needed for reliable reasoning.

Latest papers

CardsList

392 papers20 papers in the last four weeks, down 55% on the four weeks before. 0.2% of all new papers.

  1. EgoCaptioner: Towards Comprehensive and Fine-Grained Egocentric Video Captioning

    Nov 19, 2026Yuqi Tang, Yang Shi, Yuran Wang +14Video CaptioningEgocentric Video Understanding

  2. VidToC: Rethinking Video Token Compression for Multimodal Large Language Models

    Nov 19, 2026Zihan Xiao, Boshen Xu, Pei Fu +4Video Token CompressionMLLM Evaluation

  3. Online Mistake Detection in Instruction-Conditioned Procedural Video

    Nov 19, 2026Amit Feldman, Ilan Shimshoni, Rotem DrorEgocentric Video UnderstandingTemporal Video Understanding

  4. MV-JEPA: Post-Training Video Encoders for Cross-View Understanding

    Nov 18, 2026Yangchen Zhao, Ali Payani, Caiwen DingVideo Representation LearningJoint-Embedding Predictive Architecture

  5. RoughCut-Bench: A Benchmark for Automatic Video Rough-Cut with Large Vision-Language Models

    Nov 18, 2026Ruisi Zhao, You Li, Sihan Chen +7Video Editing BenchmarksVideo Editing

  6. NullTok: empirical-null token selection for training-free long video understanding

    Nov 18, 2026Zhang Ziyue, Juwei Jian, Baiyang Song +4Long-Video UnderstandingVisual Token Pruning

  7. VIVID: Beyond Static Multi-Image Reasoning with Localized Visual Differences and Textual Entity Dynamics in Video MLLMs

    Nov 17, 2026Chengqian Ma, Yuanze Zeng, Hongyu Wang +8Video Question AnsweringTemporal Video Understanding

  8. Cross-Modal StateKV Injection for Communication in Multi-LLM Systems across Temporal Segments

    Nov 17, 2026Xiaoqian Zhang, Zhihao ZhangMultimodal Large Language ModelsTemporal Video Understanding

  9. Unlocking Cross-Attention in Label-Scarce Regimes: A Weakly Supervised Approach for Video Semantic Segmentation

    Nov 13, 2026Andrea Marelli, Alessandro Ardenghi, Giacomo Boracchi +1Weakly Supervised Semantic SegmentationCross-Attention

  10. Looking One Step Ahead: An Anticipation-guided Agent for Object State Readout in Videos

    Nov 10, 2026Yuanhe Tian, Yan SongTemporal Video UnderstandingProactive Video Understanding

  11. From Weapons to Events: Evaluating Evidential Support in Multimodal AI for Policing

    Nov 6, 2026Armin Mustafa, Syed Sameed HusainMultimodal Model EvaluationTemporal Video Understanding

  12. Sekai: Incremental Event Memory for Generalized Reasoning Video Segmentation

    Nov 4, 2026Zhengtao Liu, Jinrong Xu, Shiu-Hong Kao +2Temporal Video UnderstandingReferring Video Object Segmentation

  13. Beyond Scene Shortcuts: Action-Centric Video-Text Alignment for Fine-Grained Action Understanding

    Nov 4, 2026Zhiliang Zhu, Jiazheng Huang, Dawei ZhaoFine-Grained Action UnderstandingVideo-Language Models

  14. SEEK-VAU: Towards Evidence-Faithful Video Anomaly Understanding via Agentic Search

    Nov 3, 2026Minghe Wang, Weijun Zeng, Dapeng Luo +2Active Perception for Video UnderstandingVideo Anomaly Detection

  15. Think in 360°: Benchmarking and Bootstrapping Panoramic Video Reasoning

    Oct 30, 2026Lidong Lu, Lidong Zou, Xinyi Mao +5Video UnderstandingVideo Reasoning

  16. DUGO: Dual-Scale Gaussian Memory for Online Temporal Action Localization in Egocentric Procedural Videos

    Oct 30, 2026Mohammed Aman Bhuiyan, Md. Sazzad Hossain Adib, Samiul Basir Bhuiyan +4Temporal Action LocalizationTemporal Video Understanding

  17. Perceiving and Predicting Dynamic Sports: Spatiotemporal Intelligence in Vision Models for Egocentric Tennis Video

    Oct 27, 2026Tudor Braicu, Joseph DelPreto, Jared Boyer +4Egocentric Video UnderstandingTemporal Video Understanding

  18. As the Story Unfolds: Watching a Film and Identifying Characters as a Human Does

    Oct 27, 2026Zhongrui Gui, Junyu Xie, Tengda Han +2Video-Language ModelsMultimodal Grounding

  19. P2P-Former: Bridging the Supervision Gap Between Point and Proposal in Temporal Action Localization

    Oct 25, 2026Yunchuan Ma, Laiyun Qing, Sai Xu +2Temporal Action LocalizationTemporal Video Understanding

  20. DeformStruct: Deformable Structure Modeling for Point-Supervised Temporal Action Localization

    Oct 23, 2026Mengnan Liu, Sanping Zhou, Kun Xia +3Temporal Action LocalizationTemporal Video Understanding

  21. OpenHome: From Perception to Insight in Smart-Home Camera Videos

    Oct 22, 2026Xianglong Wang, Neil Rodrigues, Pei Guo +9Video UnderstandingVLM Reasoning

  22. StateBridge: Phase- and Track-Grounded Reasoning for Synthetic-to-Real Traffic Video Captioning and VQA

    Oct 22, 2026Blessing Agyei Kyem, Richard Dzinyela, Joshua Kofi Asamoah +5Video CaptioningVisual Question Answering

  23. Reading, Writing, Sharing, and Interacting with the World through Video

    Oct 22, 2026Shengqiong Wu, Meng Luo, Bobo Li +5Video Foundation ModelsVideo World Models

  24. Through a PRISM, Clearly: Step-Sequence Reasoning for Mistake Recognition in Procedural Videos

    Oct 22, 2026Olga Loginova, Oleksandr Bezrukov, Frank KellerVideo ReasoningTemporal Video Understanding