Temporal Action Segmentation

Ask about this topic

Overview

Temporal action segmentation assigns action labels and boundaries to frames in long, untrimmed videos, making it possible to analyze how procedures and everyday activities unfold. Methods include temporal convolutional networks and transformers, hierarchical models, sequence decoding, and optimal-transport approaches; research also explores weakly supervised, unsupervised, skeleton-based, and vision-language methods. Systems are evaluated on benchmarks such as Breakfast, 50Salads, GTEA, and Assembly101, as well as surgical, egocentric, and robotic workflows. A central challenge is locating precise boundaries without excessive labels while avoiding fragmented predictions and maintaining performance across settings, viewpoints, and online use.

Latest papers

CardsList

49 papers1 paper in the last four weeks, down 83% on the four weeks before. 0.0% of all new papers.

  1. ATLAS: An Annotation Tool for Long-horizon Robotic Action Segmentation

    Apr 29, 2026Sergej Stanovcic, Daniel Sliwowski, Dongheui LeeTemporal Action SegmentationHuman-in-the-Loop Annotation

    Autonomous Systems Lab, Technische Universität Wien (TU Wien), Vienna, Austria · Institute of Robotics and Mechatronics, German Aerospace Center (DLR), Wessling, Germany

  2. HOI-aware Adaptive Network for Weakly-supervised Action Segmentation

    Apr 29, 2026Runzhong Zhang, Suchen Wang, Yueqi Duan +3Temporal Action SegmentationHuman Activity Recognition

    1Nanyang Technological University · 2Tsinghua University · 3Beijing Jiaotong University

  3. Deep kernel video approximation for unsupervised action segmentation

    Apr 23, 2026Silvia L. Pintea, Jouke DijkstraTemporal Action SegmentationKernel Methods

    Department of Intellgient Systems (DIS), Tilburg University, Tilburg, NL · Division of Image Processing, LUMC, Leiden, NL

  4. Understanding Human Actions through the Lens of Executable Models

    Apr 20, 2026Rimvydas Rubavicius, Manisha Dubey, N. Siddharth +1Temporal Action SegmentationHuman Activity Recognition

    School of Informatics The University of Edinburgh

  5. Unsupervised Skeleton-Based Action Segmentation via Hierarchical Spatiotemporal Vector Quantization

    Apr 16, 2026Umer Ahmed, Syed Ahmed Mahmood, Fawad Javed Fateh +3Temporal Action SegmentationVector Quantization

    Retrocausal, Inc. Redmond, WA

  6. Boundary-Centric Clip-Budgeted Active Learning for Temporal Action Segmentation

    Apr 16, 2026Halil Ismail Helvaci, Sen-ching Samson CheungTemporal Action SegmentationActive Learning

    Department of Electrical and Computer Engineering, University of Kentucky, Lexington, KY, USA

  7. Fine-Grained Action Segmentation for Renorrhaphy in Robot-Assisted Partial Nephrectomy

    Apr 10, 2026Jiaheng Dai, Huanrong Liu, Tailai Zhou +7Temporal Action SegmentationSurgical Video Understanding

    Fudan University, Shanghai, China · University of Macau, Macau, China · The Chinese PLA General Hospital, Beijing, China +1

  8. Combining Boundary Supervision and Segment-Level Regularization for Fine-Grained Action Segmentation

    Apr 2, 2026Hinako Mitsuoka, Kazuhiro HottaTemporal Action SegmentationBoundary-Aware Segmentation

    Meijo University, 1-501 Shiogamaguchi, Tempaku-ku, Nagoya 468-8502, Japan

  9. Spectral Scalpel: Amplifying Adjacent Action Discrepancy via Frequency-Selective Filtering for Skeleton-Based Action Segmentation

    Mar 25, 2026Haoyu Ji, Bowen Chen, Zhihao Yang +6Temporal Action SegmentationBoundary-Aware Segmentation

    Harbin Institute of Technology, Shenzhen · Shenzhen H&T Intelligent Control Co., Ltd. · Southern University of Science and Technology +1

  10. LaDy: Lagrangian-Dynamic Informed Network for Skeleton-based Action Segmentation via Spatial-Temporal Modulation

    Mar 25, 2026Haoyu Ji, Xueting Liu, Yu Gao +5Temporal Action SegmentationBoundary-Aware Segmentation

    Harbin Institute of Technology, Shenzhen · Southern University of Science and Technology · Southeast University

  11. FEEL (Force-Enhanced Egocentric Learning): A Dataset for Physical Action Understanding

    Mar 16, 2026Eadom Dessalene, Botao He, Michael Maynord +5Action Representation LearningEgocentric Video Understanding

    University of Maryland, College Park, USA

  12. InterventionLens: A Multi-Agent Framework for Detecting ASD Intervention Strategies in Parent-Child Shared Reading

    Mar 14, 2026Xiao Wang, Lu Dong, Ifeoma Nwogu +2Autism Spectrum DisorderTemporal Action Segmentation

    State University of New York at Buffalo, Buffalo, NY 14260, USA

  13. Point-Supervised Skeleton-Based Human Action Segmentation

    Mar 6, 2026Hongsong Wang, Yiqin Shen, Pengbo Yan +2Temporal Action SegmentationPseudo-Labeling

    School of Computer Science and Engineering, Southeast University, Nanjing 210096, China · School of Cyber Science and Engineering, Southeast University, Nanjing 210096, China · Southeast University - Monash University Joint Graduate School, SuZhou 215123, China +1

  14. From Phase Grounding to Intelligent Surgical Narratives

    Mar 5, 2026Ethan Peterson, Huixin ZhanSurgical Video UnderstandingTemporal Action Segmentation

    New Mexico Institute of Mining and Technology Socorro, New Mexico, USA

  15. MMTA: Multi Membership Temporal Attention for Fine-Grained Stroke Rehabilitation Assessment

    Mar 1, 2026Halil Ismail Helvaci, Justin Huber, Jihye Bae +1Temporal Action SegmentationTemporal Attention

    Department of Electrical and Computer Engineering, University of Kentucky, Lexington, KY, USA · College of Medicine, University of Kentucky, Lexington, KY, USA · Department of Electrical and Computer Engineering, University of California, Davis, CA, USA

  16. Hierarchical Action Learning for Weakly-Supervised Action Segmentation

    Feb 27, 2026Junxian Huang, Ruichu Cai, Hao Zhu +5Temporal Action SegmentationLatent Action Learning

    Guangdong University of Technology · Carnegie Mellon University · Mohamed bin Zayed University of Artificial Intelligence +1

  17. Multimodal Optimal Transport for Training-free Temporal Segmentation in Surgical Robotics

    Feb 27, 2026Omar Mohamed, Edoardo Fazzari, Ayah Al-Naji +5Temporal Action SegmentationSurgical Video Understanding

    Authos are with the Dept. of Robotics, Mohamed bin Zayed University of AI, Abu Dhabi, UAE.

  18. Exploring Vision-Language Models for Open-Vocabulary Zero-Shot Action Segmentation

    Feb 24, 2026Asim Unmesh, Kaki Ramesh, Mayank Patel +2Temporal Action SegmentationZero-Shot Learning

    Purdue University, USA · Birla Institute of Technology and Science (BITS) Hyderabad, India

  19. TEMPURA: Temporal Event Masked Prediction and Understanding for Reasoning in Action

    May 2, 2025Jen-Hao Cheng, Yi-Hao Peng, Huapeng Zhou +11Temporal Action SegmentationTemporal Video Grounding

    University of Washington · Carnegie Mellon University · National Yang Ming Chiao Tung University +1