Video Action Recognition

Ask about this topic

Overview

Video action recognition studies how to identify activities from sequences of frames, where timing and interactions can distinguish actions that look similar in a single image. Common approaches include 3D convolutional networks and video transformers, RGB and optical-flow streams, pose or skeleton models, and pretrained vision-language models adapted through prompts or limited fine-tuning. Research also addresses few-shot and open-vocabulary recognition, temporal localization, and transfer across domains such as egocentric, medical, sports, and surveillance video. A central challenge is improving temporal understanding and generalization under viewpoint changes, occlusion, scarce labels, and background shortcuts while keeping models efficient and reliable in real-world settings.

Latest papers

CardsList

112 papers13 papers in the last four weeks, up 86% on the four weeks before. 0.1% of all new papers.

  1. Learnable Motion-Focused Tokenization for Effective and Efficient Video Unsupervised Domain Adaptation

    Apr 10, 2026Tzu Ling Liu, Ian Stavness, Mrigank RochanUnsupervised Domain AdaptationVideo Action Recognition

    University of Saskatchewan, Canada

  2. Fine-Grained Action Segmentation for Renorrhaphy in Robot-Assisted Partial Nephrectomy

    Apr 10, 2026Jiaheng Dai, Huanrong Liu, Tailai Zhou +7Temporal Action SegmentationSurgical Video Understanding

    Fudan University, Shanghai, China · University of Macau, Macau, China · The Chinese PLA General Hospital, Beijing, China +1

  3. Insights from Visual Cognition: Understanding Human Action Dynamics with Overall Glance and Refined Gaze Transformer

    Apr 8, 2026Bohao Xing, Deng Li, Rong Gao +2Video Action RecognitionSpatiotemporal Transformers

    Department of Computational Engineering, Lappeenranta-Lahti University of Technology LUT, Finland · Faculty of Information Technology, Brno University of Technology, Czech Republic

  4. SBF: An Effective Representation to Augment Skeleton for Video-based Human Action Recognition

    Apr 4, 2026Zhuoxuan Peng, Yiyi Ding, Yang Lin +1Video Action RecognitionSkeleton-Based Action Recognition

    The Hong Kong University of Science and Technology · The Hong Kong University of Science and Technology (Guangzhou)

  5. ViTs for Action Classification in Videos: An Approach to Risky Tackle Detection in American Football Practice Videos

    Apr 1, 2026Syed Ahsan Masud Zaidi, William Hsu, Scott DietrichSports Video AnalysisVideo Action Recognition

    Kansas State University, Manhattan , KS ,66506, USA · Albright College, Athletics/Athletic training, Reading, PA,1962, USA

  6. PrivHAR-Bench: A Graduated Privacy Benchmark Dataset for Video-Based Action Recognition

    Apr 1, 2026Samar AnsariPrivacy-Utility TradeoffHuman Activity Recognition

    School of Computer and Engineering Sciences University of Chester Chester, CH1 4BJ, United Kingdom

  7. From Pixels to Privacy: Temporally Consistent Video Anonymization via Token Pruning for Privacy Preserving Action Recognition

    Mar 27, 2026Nazia Aslam, Abhisek Ray, Joakim Bruslund Haurum +2Privacy-Preserving MLVisual Token Pruning

    Aalborg University, Denmark · Aarhus University, Denmark · Poineer Centre for AI, Denmark +2

  8. SurgPhase: Time efficient pituitary tumor surgery phase recognition via an interactive web platform

    Mar 26, 2026Yan Meng, Jack Cook, X. Y. Han +12Surgical Video UnderstandingVideo Action Recognition

    Children’s National Hospital, 111 Michigan Ave NW, Washington, 20010, DC, USA · Surgical Data Science Collective, 2000 Pennsylvania Ave. NW, Suite 7000, Washington, 20006, DC, USA · Booth School of Business, University of Chicago, 5807 S Woodlawn Ave, Chicago, 60637, IL, USA +7

  9. B-MoE: A Body-Part-Aware Mixture-of-Experts "All Parts Matter" Approach to Micro-Action Recognition

    Mar 25, 2026Nishit Poddar, Aglind Reka, Diana-Laura Borza +4Video Action RecognitionMixture-of-Experts Models

    INRIA · Birla Institute of Technology & Science, Hyderabad · Université Côte d’Azur

  10. Privacy-Preserving Federated Action Recognition via Differentially Private Selective Tuning and Efficient Communication

    Mar 22, 2026Idris Zakariyya, Pai Chet Ng, Kaushik Bhargav Sivangi +3Differentially Private Federated LearningVideo Action Recognition

    School of Computing Science, University of Glasgow, G12 8QQ, Glasgow, UK · Infocomm Technology Cluster, Singapore Institute of Technology, Singapore · The Edward S. Rogers Sr. Dept. of Electrical and Computer Engineering, University of Toronto, Canada

  11. Generalized Recognition of Basic Surgical Actions Enables Skill Assessment and Vision-Language-Model-based Surgical Planning

    Mar 13, 2026Mengya Xu, Daiyun Shen, Jie Zhang +19Surgical Video UnderstandingVideo Action Recognition

    Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China · Department of Biomedical Engineering, National University of Singapore, Singapore · Department of Surgery, The Chinese University of Hong Kong, Hong Kong, China +13

  12. Human-AI Divergence in Ego-centric Action Recognition under Spatial and Spatiotemporal Manipulations

    Mar 9, 2026Sadegh Rahmaniboldaji, Filip Rybansky, Quoc C. Vuong +3Video Action RecognitionHuman Activity Recognition

    University of Surrey, Guildford, UK · Newcastle University, Newcastle upon Tyne, UK

  13. Novel Semantic Prompting for Zero-Shot Action Recognition

    Mar 9, 2026Salman Iqbal, Waheed RehmanZero-Shot Action RecognitionVideo Action Recognition

  14. A Baseline Study and Benchmark for Few-Shot Open-Set Action Recognition with Feature Residual Discrimination

    Mar 4, 2026Stefano Berti, Giulia Pasquale, Lorenzo NataleOpen-Set RecognitionFew-Shot Learning

    Humanoid Sensing and Perception, Istituto Italiano di Tecnologia, Genoa, Italy

  15. OpenMarcie: Dataset for Multimodal Action Recognition in Industrial Environments

    Mar 2, 2026Hymalai Bello, Lala Ray, Joanna Sorysz +2Human Activity RecognitionVideo Action Recognition

    DFKI Kaiserslautern · RPTU Kaiserslautern-Landau · Korea University

  16. BigMaQ: A Big Macaque Motion and Animation Dataset Bridging Image and 3D Pose Representations

    Feb 23, 2026Lucas Martini, Alexander Lappe, Anna Bognár +2Video Action RecognitionMarkerless Motion Capture

    Hertie Institute, University of Tübingen · IMPRS-IS · KU Leuven

  17. Frame2Freq: Spectral Adapters for Fine-Grained Video Understanding

    Feb 21, 2026Thinesh Thiyakesan Ponbagavathi, Constantin Seibold, Alina RoitbergVideo Action RecognitionVision Foundation Model Adaptation

    Institute for Artificial Intelligence, University of Stuttgart, Germany · Intelligent Assistive Systems Lab, University of Hildesheim · University Hospital Heidelberg, Diagnostic and Interventional Radiology

  18. Spatio-temporal Decoupled Knowledge Compensator for Few-Shot Action Recognition

    Feb 20, 2026Hongyu Qu, Xiangbo Shu, Rui Yan +3Video Action RecognitionFew-Shot Learning

    School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China. · State Key Lab of Brain-Machine Intelligence, Zhejiang University, China.

  19. Why Can't I Open My Drawer? Mitigating Object-Driven Shortcuts in Zero-Shot Compositional Action Recognition

    Jan 22, 2026Geo Ahn, Inwoong Lee, Taeoh Kim +3Zero-Shot LearningVideo Action Recognition

    1Kyung Hee University · ∗This work was done during an internship at NAVER Cloud. · 2NAVER Cloud

  20. MGCA-Net: Multi-Grained Category-Aware Network for Open-Vocabulary Temporal Action Localization

    Nov 17, 2025Zhenying Fang, Richang HongVideo Action RecognitionTemporal Action Detection

    School of Computer Science and Information Engineering, Hefei University of Technology, 188065, China

  21. Hierarchical Relation-augmented Representation Generalization for Few-shot Action Recognition

    Apr 14, 2025Hongyu Qu, Ling Xing, Jiachao Zhang +3Temporal ReasoningHierarchical Representation Learning

    School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China · Artificial Intelligence Industrial Technology Research Institute, Nanjing Institute of Technology, Nanjing 211167, China

  22. XOV-Action: Towards Generalizable Open-Vocabulary Action Recognition

    Mar 3, 2024Kun-Yu Lin, Henghui Ding, Jia-Run Du +6Open-Set RecognitionCross-Domain Generalization

    School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou 510275, China · Institute of Big Data, Fudan University, Shanghai 200433, China · AI Thrust, Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511400, China +1