Temporal Action Segmentation

Overview

Temporal action segmentation assigns action labels and boundaries to frames in long, untrimmed videos, making it possible to analyze how procedures and everyday activities unfold. Methods include temporal convolutional networks and transformers, hierarchical models, sequence decoding, and optimal-transport approaches; research also explores weakly supervised, unsupervised, skeleton-based, and vision-language methods. Systems are evaluated on benchmarks such as Breakfast, 50Salads, GTEA, and Assembly101, as well as surgical, egocentric, and robotic workflows. A central challenge is locating precise boundaries without excessive labels while avoiding fragmented predictions and maintaining performance across settings, viewpoints, and online use.

Latest papers

CardsList

50 papers1 paper in the last four weeks, down 67% on the four weeks before. 0.0% of all new papers.

  1. Adaptive Residual Quantization for Memory-Efficient Temporal Action Segmentation

    Nov 9, 2026Gerard Donahue, Guven Gergerli, Ayush Gupta +3Temporal Action SegmentationVector-Quantized Variational Autoencoder

  2. KiRoSe: Using Kinematic Data for Efficient Segmentation of Robot Demonstrations

    Oct 22, 2026Shuangyu Xie, Kaiyuan Chen, Muhammad Zubair Irshad +3Motion SegmentationTemporal Action Segmentation

  3. Groupwise Selective State-Space Filtering for Accurate and Streaming Action Boundary Detection

    Sep 27, 2026Mustafa Bora ÇelikTemporal Action SegmentationSelective SSMs

  4. Seeing Semantic Shift: Difference-Aware Sentence-Level Temporal Segmentation of Sign Language Videos

    Sep 25, 2026Bowen Guo, Shiwei Gan, Yafeng Yin +4Boundary-Aware SegmentationTemporal Action Segmentation

  5. CE4^4L: Continual Ego, Exo, and Ego-Exo Learning

    Sep 20, 2026Hongwei Yan, Kanglei Zhou, Yuchen Liu +3Temporal Action SegmentationNext-Action Prediction

  6. Boundary Voting Network for Ambiguity-Aware Timestamp-Supervised Action Segmentation

    Sep 8, 2026Runzhong Zhang, Yueqi Duan, Yang Chen +4Temporal Action SegmentationBoundary-Aware Segmentation

  7. A Framework for Egocentric and Exocentric Procedural Understanding via Temporal Segmentation and Semantic Abstraction

    Sep 4, 2026Vivek Chavan, Jörg KrügerMultimodal MemoryEgocentric Video Understanding

  8. FIS-OT: Feature-Induced Optimal Transport for Unsupervised Action Segmentation

    Aug 30, 2026Linxiang Peng, Xinyao Qin, Jinhan Li +2Temporal Action SegmentationOptimal Transport

  9. MASQ: Mask-Aware Spatiotemporal Quantization for Unsupervised Skeleton Action Segmentation

    Aug 30, 2026Xinyao Qin, Linxiang Peng, Youbao Ye +2Temporal Action SegmentationUnsupervised Representation Learning

  10. VidParse: Online Parsing of Egocentric Procedures Like a Pro

    Aug 27, 2026Anubhav Gupta, Archit Kambhamettu, Vatsal Agarwal +2Video UnderstandingEgocentric Video Understanding

  11. ConsensusTAS: Self-Supervised Temporal Action Segmentation for Long-Horizon Construction Videos

    Aug 25, 2026Xiaoshan Zhou, Yafei SunTemporal Action SegmentationHuman Activity Recognition

  12. Artificial Intelligence for Workflow Analysis in Colorectal Surgery: A Multicentric, Cross-Procedural Development and Generalization Study

    Aug 20, 2026Pietro Mascagni, Julia Alekseenko, Pooja P Jain +10Surgical Video UnderstandingTemporal Action Segmentation

  13. D-CLOT: Double Closed Loop Optimal Transport for Unsupervised Action Segmentation

    Aug 6, 2026Elena Bueno-Benito, Mariella DimiccoliTemporal Action SegmentationGraph Regularization

  14. Lag-aware cross-hand alignment for dual-hand action segmentation

    Jul 28, 2026Fatemeh ZiaeetabarTemporal Action SegmentationBoundary-Aware Segmentation

  15. Adaptive Latent Trajectory Anchoring for Action Segmentation Dataset Condensation

    Jul 10, 2026Artheme Gauthier-Villar, Guodong Ding, Angela YaoTemporal Action SegmentationDataset Condensation

  16. Learning Probabilistic Embeddings for Unsupervised Action Segmentation

    Jul 6, 2026Shuai Li, Duc Manh Vu, Juergen GallTemporal Action SegmentationRepresentation Learning

  17. Grammar-Guided Hierarchical Parsing for Long-form Audio Activity Recognition

    Jun 26, 2026Peng Zhang, Qingyu Luo, Philip J. B. Jackson +1Temporal Action SegmentationHuman Activity Recognition

  18. P-JEPA: Procedural Video Representation Learning via Joint Embedding Predictive Architecture

    Jun 22, 2026Felix Tristram, Stefano Gasperini, Benjamin Killeen +4Temporal Action SegmentationVideo Understanding

  19. The Unreasonable Effectiveness of VLMs for Zero-shot Procedural Mistake Detection

    Jun 19, 2026Serdar Ozsoy, Lars Doorenbos, Federico Spurio +2Temporal Action SegmentationVision-Language Models

  20. OR-Action: Multi-Role Video Understanding with Fine-Grained Actions

    Jun 11, 2026Felix Tristram, Ege Özsoy, Christian Benz +3Temporal Action SegmentationHuman Activity Recognition

  21. Intention Driven Identification of In-Possession Match Phases in Association Football through Temporal Graph Learning

    Jun 8, 2026Yuesen Li, Daniel LinkTemporal Action SegmentationTemporal GNNs

  22. Uncertainty-Aware Action Segmentation for Human-to-Robot Assembly Teleoperation

    Jun 6, 2026Fnu Heman, Yixuan Wang, Kolin Xu +7Temporal Action SegmentationRobot Teleoperation

  23. VOLT: Vision and Language Trajectory Segmentation for Faster-than-Demonstration Policies

    Jun 4, 2026Robert Ramirez Sanchez, Daniel J. Evans, Dylan P. Losey +1Temporal Action SegmentationImitation Learning

  24. Ego-METAS: Egocentric online Multimodal Energy-efficient Temporal Action Segmentation benchmark

    May 29, 2026Maria Santos-Villafranca, Jesus Bermudez-cameo, Alejandro Perez-Yus +2Temporal Action SegmentationEnergy-Efficient ML

  25. Polyphony: Diffusion-based Dual-Hand Action Segmentation with Alternating Vision Transformer and Semantic Conditioning

    May 29, 2026Hao Zheng, Hu Wang, Tiantian Zheng +2Temporal Action SegmentationVision Transformer

  26. Visualizing Latent Phase Structures in Locomotion Policies: A Multi-Environment Study with Temporal Feature Extension

    May 27, 2026Daisuke Yasui, Toshitaka Matuki, Hiroshi SatoTemporal Action SegmentationRobot Locomotion

  27. EgoInteract: Synthetic Egocentric Videos Generation for Interaction Understanding and Anticipation

    May 18, 2026Rosario Leonardi, Francesco Ragusa, Daniele Materia +4Temporal Action SegmentationNext-Action Prediction

  28. Improving Temporal Action Segmentation via Constraint-Aware Decoding

    May 11, 2026Yeo Keat Ee, Debaditya Roy, Chen Li +2Temporal Action SegmentationConstrained Decoding