cs.CVOct 30, 2026

PAM-Mamba - Efficient Dense Action Detection using Mamba

Authors: Dhruvraj Singh Rawat, Faegheh Sardari, Armin Mustafa

Organizations: Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, Guildford, UK

Abstract

Dense action detection requires precisely localising multiple overlapping actions in long untrimmed videos. Transformer-based methods achieve strong accuracy but their O(T2)O(T^2) self-attention cost limits scalability. We present PAM-Mamba, a linear-time O(T)O(T) architecture built on Mamba with a multi-branch temporal encoder. Gated Learnable Positional Encoding (GLPE) controls positional signal injection via a scalar gate initialised to zero, starting training without added positional signal. A Hub-and-Spoke Funnel extends the non-hierarchical multi-scale design of PAT with a direct residual from the original full-resolution input to each coarse branch, providing each coarse branch with a projection of the input features. Bi-TemBa (Bidirectional Temporal Mamba) first extracts local motion cues with a shallow Temporal Convolution Module, then applies a cascaded bidirectional Mamba scan; the backward pass refines the forward-scanned representation from the opposite direction, giving each temporal position access to context from both directions in linear time. Uncertainty-Based Mixup uses prediction entropy to guide the selection of mixup partners. Using the PAT evaluation implementation, we obtain 26.82 (Charades I3D), 34.59 (Charades CLIP), 46.56 (MultiTHUMOS I3D), and 46.92 mAP (MultiTHUMOS CLIP), with 26.1-27.9M parameters, 61-63% fewer than PAT.

Figures & tables

Explore similar work

CardsList
  1. MamBOA: State-Space Architecture for Video Recognition

    Jun 13, 2026Mustafa Bora ÇelikVisual SSMsVideo Action Recognition

  2. Faster-WAM: Do World Action Models Need Deep Action Modules?

    Aug 3, 2026Liheng Ma, Rui Heng Yang, Amin Abyaneh +8World Model LearningEfficient Inference for World Action Models