cs.CVOct 30, 2026

PAM-Mamba - Efficient Dense Action Detection using Mamba

Authors: Dhruvraj Singh Rawat, Faegheh Sardari, Armin Mustafa

Organizations: Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, Guildford, UK

Abstract

Dense action detection requires precisely localising multiple overlapping actions in long untrimmed videos. Transformer-based methods achieve strong accuracy but their O(T2)O(T^2) self-attention cost limits scalability. We present PAM-Mamba, a linear-time O(T)O(T) architecture built on Mamba with a multi-branch temporal encoder. Gated Learnable Positional Encoding (GLPE) controls positional signal injection via a scalar gate initialised to zero, starting training without added positional signal. A Hub-and-Spoke Funnel extends the non-hierarchical multi-scale design of PAT with a direct residual from the original full-resolution input to each coarse branch, providing each coarse branch with a projection of the input features. Bi-TemBa (Bidirectional Temporal Mamba) first extracts local motion cues with a shallow Temporal Convolution Module, then applies a cascaded bidirectional Mamba scan; the backward pass refines the forward-scanned representation from the opposite direction, giving each temporal position access to context from both directions in linear time. Uncertainty-Based Mixup uses prediction entropy to guide the selection of mixup partners. Using the PAT evaluation implementation, we obtain 26.82 (Charades I3D), 34.59 (Charades CLIP), 46.56 (MultiTHUMOS I3D), and 46.92 mAP (MultiTHUMOS CLIP), with 26.1-27.9M parameters, 61-63% fewer than PAT.

Figures & tables

Explore similar work

Jun 13, 2026

MamBOA: State-Space Architecture for Video Recognition

Fine-grained action recognition demands temporal reasoning that general-purpose architectures address through different cost-accuracy tradeoffs: 3D dense operators couple computation to the input volume, while difference-based methods approximate motion through rigid, hand-crafted subtraction of uncontextualized features - each reflecting a deliberate design choice with corresponding limitations in expressiveness or flexibility. We present MamBOA, a backbone-agnostic temporal framework built upon a novel interleaved scan structure that recasts the selective state-space recurrence (S6) as a native motion synthesizer. By interleaving consecutive feature representations extracted from a pretrained backbone into a single alternating sequence, the proposed scan structurally drives the recurrence to encode both temporal observations of each position within a shared hidden state, separated by only a single decay step - rendering the inter-frame transition an intrinsic component of the state dynamics rather than an externally computed quantity. A cascade of dedicated alignment and decoding operations then distills this joint encoding into an explicit motion representation, which a dual-path pooling mechanism adaptively aggregates by balancing attention-driven selection with uniform temporal coverage. The framework interfaces seamlessly with CNN, Transformer, and Mamba backbone families, adding only ~2.1 GFLOPs per feature pair. On Diving48, MamBOA achieves 85.02% Top-1 accuracy with an image-pretrained backbone and 86.24% with a video-pretrained backbone processing the entire video in a single forward pass - demonstrating that structurally induced state-space dynamics constitute a principled and general foundation for motion modeling.
Apr 20, 2026

LiquidTAD: Efficient Temporal Action Detection via Parallel Liquid-Inspired Temporal Relaxation

Temporal Action Detection (TAD) requires precise localization of action boundaries within long, untrimmed video sequences. While current high-performing methods achieve strong accuracy, they are often characterized by excessive parameter counts, substantial computational overhead, and a reliance on specialized operators that hinder deployment across diverse hardware platforms. This paper presents LiquidTAD, a framework that distills the exponential relaxation prior of liquid neural dynamics into a parallel temporal operator, rather than reproducing full Liquid Neural Network (LNN) dynamics. By introducing a Parallel Liquid-inspired Relaxation mechanism, sequential ODE solving is avoided through a fully vectorized, non-recursive formulation built entirely upon standard neural operations, enabling hardware-agnostic deployment with linear complexity with respect to the temporal length. A complementary Hierarchical Decay-Rate Sharing Strategy further adapts this relaxation prior across feature pyramid levels, stabilizing optimization and implicitly compensating for temporal compression in deeper layers. Experimental evaluations on THUMOS-14 and ActivityNet-1.3 demonstrate that LiquidTAD achieves accuracy competitive with strong baselines while substantially lowering the model footprint. Specifically, on THUMOS-14, LiquidTAD achieves 69.46% average mAP with only 10.82M parameters and 27.17G FLOPs, reducing the parameter count by over 60% compared with ActionFormer.
Aug 3, 2026

Faster-WAM: Do World Action Models Need Deep Action Modules?

World Action Models (WAMs) build on pretrained video models, whose representations are grounded in physical dynamics and provide a natural basis for action prediction. Despite this natural foundation, many WAMs still rely on deep, parameter-heavy action-prediction modules that incur high inference latency and may overfit to limited robot demonstrations, restricting their real-world applicability. In this paper, we advocate a world-model-centric principle that concentrates capacity and computation in the video world model, while a lightweight action expert translates the backbone's representations into executable robot actions. We realize this principle through three key choices: Dock of Transformers (DoT) with Lite KV-Fusion to give the shallow, lightweight action expert access to representations from all video layers; world-model-only conditioning of the action expert; and retracted 1D-RoPE for positional alignment between video keys and action queries. We test this principle using Faster-WAM, a world-model-centric WAM with only a single-layer action expert. Despite this restriction on action-specific computation, Faster-WAM achieves competitive control performance on LIBERO and RoboTwin~2.0 without additional embodied pretraining. It provides approximately 3.7×3.7\times and 1.3×1.3\times inference speedups over Fast-WAM and π0.5π_{0.5}, respectively. Consistent with its world-model-centric design, Faster-WAM demonstrates stronger generalizability under distribution shifts: the same LIBERO-trained policy achieves 78.3%78.3\% success on LIBERO-Plus, exceeding Fast-WAM and LingBot-VA by 26.826.8 and 8.88.8 percentage points, respectively. Finally, real-robot experiments demonstrate success rates comparable to Fast-WAM, with substantially lower inference latency and shorter task-completion times.