Spatio-Temporal Attention

Latest papers

11 papers1 paper in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Oct 17, 2026

$M^2Net$ : Unifying Scale-Specific Structures and Prototype Decomposition for Spatial-Temporal Forecasting

Spatial-temporal forecasting is pivotal for intelligent urban management yet remains challenging due to the intricate coupling of spatial dynamics and temporal non-stationarity. Existing approaches, while effective, often overlook two fundamental characteristics of real-world data: (1) Granularity-dependence, where dominant spatial correlations shift significantly across different time scales, and (2) Signal entanglement, where global common patterns are inextricably mixed with node-specific disturbances. In this paper, we propose a novel framework named

M^2Net

(Multi-Scale Masked attention and Prototype network) to unify scale-specific structural learning and semantic decomposition. First, to capture the evolution of spatial dependencies, we introduce a Multi-Scale Masked attention mechanism. By downsampling temporal features into multiple resolutions, we dynamically generate scale-adaptive spatial masks, allowing the model to distinguish between micro-scale physical neighbors and macro-scale functional correlations. Second, to disentangle unique variations from shared trends, we propose a Prototype Residual Learning module. It retrieves global common patterns from a momentum-updated prototype bank and subtracts them from node representations, explicitly forcing the network to model the residual unique deviations. By integrating these two orthogonal views, i.e., structural multiscaling and semantic decomposition, our approach effectively isolates valid spatial-temporal signals from noise. Extensive experiments on real-world datasets demonstrate that

M^2Net

outperforms state-of-the-art baselines on extensive empirical verifications.

Chengyang Zhou, Zijian Zhang, Hao Miao +5

College of Computer Science and Technology, Jilin University, Changchun, China · Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun, China +2

Spatiotemporal Forecasting Multiscale Time-Series Modeling Spatial Context Modeling Spatio-Temporal Attention Prototype Learning

Ask PDF

Sep 16, 2026

Pose2Muscle: Structured Spatio-Temporal Decoding for Discrete Muscle Activity Estimation from Human Pose

Muscle activity is fundamental to human movement, and understanding its patterns is critical for injury prevention and rehabilitation. Conventional muscle activity monitoring relies on specialized sensors such as surface electromyography, which limits its practicality for long-term real-world use. Existing studies suggest that muscle-related information can be inferred from human pose. However, the substantial gap between externally observable pose and internal muscle activation, limits the accuracy and generalization of current approaches. In this study, we propose Pose2Muscle, a pose-driven framework for discrete muscle activity estimation without requiring sEMG signals at inference time. Instead of directly regressing continuous sEMG signals, Pose2Muscle reformulates muscle estimation as a structured prediction problem over discrete muscle activity states, yielding a more stable and interpretable target space. The framework combines multi-scale spatio-temporal attention to capture motion patterns at complementary spatial and temporal scales with a directed acyclic graph-based decoder that maintains multiple candidate muscle-state hypotheses and performs structured trajectory inference over time. To support this task, we construct PoseEMG-43, a synchronized pose-sEMG dataset containing 2,992 movement instances from 43 daily-life actions performed by 14 participants. Experiments show that Pose2Muscle consistently outperforms representative retrieval- and pose-based baselines. It achieves an Adjacent-level Accuracy of 86.36% and a Pearson correlation coefficient of 0.8821 under the Random Split, and 63.97% and 0.6795, respectively, under the Subject-Level Split. These results demonstrate the feasibility of inferring structured muscle-state patterns from human pose and suggest the potential of Pose2Muscle for muscle-aware movement analysis when direct physiological sensing is impractical

Yuepeng Chen, Jiehong Shi, Kaili Zheng +4

School of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications, Beijing, China · Key Laboratory of Trustworthy Distributed Computing and Service (BUPT), Ministry of Education, Beijing, China · Department of Rehabilitation Medicine, Peking Union Medical College Hospital, Beijing, China +3

Spatio-Temporal Attention Structured Prediction

Ask PDF

Aug 24, 2026

GuidedFlow: An Attention-Guided Framework for Anomaly Detection in Additive Manufacturing

Additive Manufacturing (AM) plays a vital role in the ongoing industrial revolution. However, quality control remains crucial and challenging due to printing defects or potential cyber-physical intrusions. Image or video-based anomaly detection is a key effort towards addressing these challenges. Various approaches have been explored in this domain, including reconstruction-based, embedding-based, and flow-based methods. Though normalizing flow-based methods address some of the core challenges of unforeseen defects and generalization while maintaining detection performance, existing approaches struggle with tiny/stringing defects common in 3D printing. In a small-data setting, this poses a limitation in generalization. To address these limitations, we propose GuidedFlow, a novel attention-guided normalizing flow model for anomaly detection and localization. GuidedFlow employs a pre-trained ResNet model, fine-tuned on the domain dataset. An attention-guided spatial and temporal flow framework models the dynamics across multiple scales and frames. A Spatio-Temporal Attention Network (SAN) enables the flow model to prioritize relevant contextual cues from input frames. We evaluate GuidedFlow on our AM3D-AD dataset, consisting of benign and anomalous real 3D printed object images and videos. We also conduct a comparative study using the MVTec-AD industrial image anomaly detection dataset. Experimental results demonstrate that GuidedFlow outperforms most of the state-of-the-art models with enhanced detection accuracy and AUROC.

Sosmita Paul, Krishna Roy

New Mexico Institute of Mining and Technology Socorro, NM 87801

Spatio-Temporal Attention Anomaly Localization Video Anomaly Detection Normalizing Flows Additive Manufacturing

Ask PDF

Aug 17, 2026

TokenSTFormer: A Tokenized Spatial-temporal Attention Model for Holistic Motion Analysis in Adolescent Idiopathic Scoliosis Screening

Adolescent Idiopathic Scoliosis (AIS) is a prevalent spinal deformity in adolescents that, if left untreated, can result in severe health outcomes. Traditional screening methods are limited by subjective interpretation, reliance on professional expertise and low scalability. To address these challenges, we present ScoliGait dataset, which comprises 1,516 gait video clips paired with corresponding X-ray records. We also introduce TokenSTFormer, a novel model that tokenizes spatial and temporal semantics to enhance feature representation and convergence. Our model achieves state-of-the-art performance, surpassing vanilla Vision Transformer encoder across key metrics, including accuracy of 0.79. This study highlights the potential of leveraging holistic motion features derived from gait video and attention-based models for scalable, cost-effective AIS screening, paving the way for future clinical applications in scoliosis detection.

Dong Chen, Kenneth M. C. Cheung

The University of Hong Kong

Spatio-Temporal Attention Vision Transformer Temporal Video Understanding

Ask PDF

Jul 24, 2026

Learning Spatiotemporal Decision Priors for Efficient Path Planning under Partial Observability

Path planning under partial observability remains challenging because an agent must make long-horizon navigation decisions from only locally bounded observations. Nevertheless, historical trajectories contain reusable experience-guided directional preferences. Classical planners, however, typically solve each instance from scratch and lack an explicit mechanism to exploit such transferable decision knowledge, often leading to redundant node expansions and locally myopic search behaviors. Motivated by this limitation, this paper proposes ImiPath, a prior-guided learning framework that distills reusable spatiotemporal decision priors from demonstration trajectories and uses them as experience-informed directional guidance to bias planners toward reliable and promising search directions under partial observability. Specifically, ImiPath first constructs a local spatiotemporal observation representation, which encodes the spatial information of the local environment and the temporal information of historical trajectories. The SpatioTemporal-Attention Policy Network (STAPNet) then transforms this representation into dicision priors. These priors are further incorporated into heterogeneous planners as directional guidance, biasing the search toward locally promising regions. Extensive experiments demonstrate that ImiPath achieves competitive path quality and improves search efficiency by reducing redundant node expansions under local observability. Additional physical experiments on a magnetic microrobot platform further validate the adaptability and practical deployment potential of the proposed framework.

Yi Liu, Hongda Zhang, Leyao Zou +7

College of Intelligent Robotics and Advanced Manufacturing, Fudan University, Shanghai, 200433, China · School of Information Science and Technology, Fudan University, Shanghai, 200433, China · Department of Information Technology, Old Dominion University, Norfolk, 23529, VA, USA

Imitation Learning Spatio-Temporal Attention Robot Motion Planning

Ask PDF

May 12, 2026

Federated Client Selection under Partial Visibility: A POMDP Approach with Spatio-Temporal Attention

Federated learning relies on effective client selection to alleviate the performance degradation caused by data heterogeneity. Most existing methods assume full visibility of all clients at each communication round. However, in large-scale or edge-based deployments, the server can only access a subset of clients due to communication, mobility, or availability constraints, resulting in partial visibility where only a subset of clients is observable for aggregation in each communication round. In this paper, we formulate federated client selection under partial visibility as a Partially Observable Markov Decision Process (POMDP) and propose a Spatial-Temporal attention-based reinforcement learning framework. By integrating historical global models and client identity embeddings, the proposed method captures both the temporal contexts of training and the persistent characteristics of clients. Experimental results across multiple datasets demonstrate that our approach achieves superior performance compared to existing baselines in heterogeneous and partially visible settings, validating its effectiveness in addressing the challenges of incomplete observations in practical federated learning systems.

Qijun Hou, Yuchen Shi, Pingyi Fan +1

Department of Electronic Engineering, BNRist, Tsinghua University, Beijing, China · Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology, Hong Kong

Client Selection in Federated Learning Spatio-Temporal Attention Federated Learning Heterogeneous Federated Learning

Ask PDF

Mar 23, 2026

Spatio-Temporal Attention Enhanced Multi-Agent DRL for UAV-Assisted Wireless Networks with Limited Communications

In this paper, we employ multiple UAVs to accelerate data transmissions from ground users (GUs) to a remote base station (BS) via the UAVs' relay communications. The UAVs' intermittent information exchanges typically result in delays in acquiring the complete system state and hinder their effective collaboration. To maximize the overall throughput, we first propose a delay-tolerant multi-agent deep reinforcement learning (MADRL) algorithm that integrates a delay-penalized reward to encourage information sharing among UAVs, while jointly optimizing the UAVs' trajectory planning, network formation, and transmission control strategies. Additionally, considering information loss due to unreliable channel conditions, we further propose a spatio-temporal attention based prediction approach to recover the lost information and enhance each UAV's awareness of the network state. These two designs are envisioned to enhance the network capacity in UAV-assisted wireless networks with limited communications. The simulation results reveal that our new approach achieves over 50% reduction in information delay and 75% throughput gain compared to the conventional MADRL. Interestingly, it is shown that improving the UAVs' information sharing will not sacrifice the network capacity. Instead, it significantly improves the learning performance and throughput simultaneously. It is also effective in reducing the need for UAVs' information exchange and thus fostering practical deployment of MADRL in UAV-assisted wireless networks.

Che Chen, Lanhua Li, Shimin Gong +3

School of Intelligent Systems Engineering, Shenzhen Campus of Sun Yat-sen University, Shenzhen 518000, China, and Southern Marine Science and Engineering Guangdong Laboratory (Zhuhai), Zhuhai 519082, China · Department of Equipment Management and Unmanned Aerial Vehicle Engineering, Air Force Engineering University, Xi’an · School of Computing and Artificial Intelligence, Jiangxi University of Finance and Economics, Nanchang 330032, China +1

Spatio-Temporal Attention Multi-Agent Reinforcement Learning Wireless Resource Allocation UAV Trajectory Optimization Sequential MARL

Ask PDF

Mar 17, 2026

STARK: Spatio-Temporal Attention for Representation of Keypoints for Continuous Sign Language Recognition

Continuous Sign Language Recognition (CSLR) is a crucial task for understanding the languages of deaf communities. Contemporary keypoint-based approaches typically rely on spatio-temporal encoding, where spatial interactions among keypoints are modeled using Graph Convolutional Networks or attention mechanisms, while temporal dynamics are captured using 1D convolutional networks. However, such designs often introduce a large number of parameters in both the encoder and the decoder. This paper introduces a unified spatio-temporal attention network that computes attention scores both spatially (across keypoints) and temporally (within local windows), and aggregates features to produce a local context-aware spatio-temporal representation. The proposed encoder contains approximately

70-80\%

fewer parameters than existing state-of-the-art models while achieving comparable performance to keypoint-based methods on the Phoenix-14T dataset.

Suvajit Patra, Soumitra Samanta

RKMVERI, Belur

Spatio-Temporal Attention Sign Language Recognition Temporal Attention

Ask PDF

Mar 14, 2026

Distributed Acoustic Sensing for Urban Traffic Monitoring: Spatio-Temporal Attention in Recurrent Neural Networks

Effective urban traffic monitoring is essential for improving mobility, enhancing safety, and supporting sustainable cities. Distributed Acoustic Sensing (DAS) enables large-scale traffic observation by transforming existing fiber-optic infrastructure into dense arrays of vibration sensors. However, modeling the high-resolution spatio-temporal structure of DAS data for reliable traffic event recognition remains challenging. This study presents a real-world DAS-based traffic monitoring experiment conducted in Granada, Spain, where vehicles cross a fiber deployed perpendicular to the roadway. Recurrent neural networks (RNNs) are employed to model intra- and inter-event temporal dependencies. Spatial and temporal attention mechanisms are systematically integrated within the RNN architecture to analyze their impact on recognition performance, parameter efficiency, and interpretability. Results show that an appropriate and complementary placement of attention modules improves the balance between accuracy and model complexity. Attention heatmaps provide physically meaningful interpretations of classification decisions by highlighting informative spatial locations and temporal segments. Furthermore, the proposed SA-bi-TA configuration demonstrates spatial transferability, successfully recognizing traffic events at sensing locations different from those used during training, with only moderate performance degradation. These findings support the development of scalable and interpretable DAS-based traffic monitoring systems capable of operating under heterogeneous urban sensing conditions.

Izhan Fakhruzi, Manuel Titos, Carmen Benítez +1

Department of Signal Theory, Telematics and Communications, and the Center for Information and Communication Technologies Research (CITIC), University of Granada, 18014 Granada, Spain

Spatio-Temporal Attention Distributed Acoustic Sensing Recurrent Neural Networks Temporal Attention

Ask PDF

Feb 22, 2026

LWM-Temporal: Sparse Spatio-Temporal Attention for Wireless Channel Representation Learning

LWM-Temporal is a new member of the Large Wireless Models (LWM) family that targets the spatiotemporal nature of wireless channels. Designed as a task-agnostic foundation model, LWM-Temporal learns universal channel embeddings that capture mobility-induced evolution and are reusable across various downstream tasks. To achieve this objective, LWM-Temporal operates in the angle-delay-time domain and introduces Sparse Spatio-Temporal Attention (SSTA), a propagation-aligned attention mechanism that restricts interactions to physically plausible neighborhoods, reducing attention complexity by an order of magnitude while preserving geometry-consistent dependencies. LWM-Temporal is pretrained in a self-supervised manner using a physics-informed masking curriculum that emulates realistic occlusions, pilot sparsity, and measurement impairments. Experimental results on channel prediction across multiple mobility regimes show consistent improvements over strong baselines, particularly under long horizons and limited fine-tuning data, highlighting the importance of geometry-aware architectures and geometry-consistent pretraining for learning transferable spatiotemporal wireless representations.

Sadjad Alikhani, Akshay Malhotra, Shahab Hamidi-Rad +1

Spatio-Temporal Attention Wireless Communications Wireless Foundation Models Representation Learning Sparse Attention Self-Supervised Time Series Representation Learning

Ask PDF

Feb 19, 2026

EA-Swin: An Embedding-Agnostic Swin Transformer for AI-Generated Video Detection

Recent advances in foundation video generators such as Sora2, Veo3, and other commercial systems have produced highly realistic synthetic videos, exposing the limitations of existing detection methods that rely on shallow embedding trajectories, image-based adaptation, or computationally heavy MLLMs. We propose EA-Swin, an Embedding-Agnostic Swin Transformer that models spatiotemporal dependencies directly on pretrained video embeddings via a factorized windowed attention design, making it compatible with generic ViT-style patch-based encoders. Moreover, we construct the EA-Video dataset, a benchmark dataset comprising 130K videos that integrates newly collected samples with curated existing datasets, covering diverse commercial and open-source generators and including unseen-generator splits for rigorous cross-distribution evaluation. Extensive experiments show that EA-Swin achieves 0.97-0.99 accuracy across major generators, outperforming prior SoTA methods (typically 0.8-0.9) by a margin of 5-20%, while maintaining strong generalization to unseen distributions, establishing a scalable and robust solution for modern AI-generated video detection.

Hung Mai, Loi Dinh, Duc Hai Nguyen +6

N2TP Technology Solution JSC, Hanoi, Vietnam · College of Technology, National Economics University, Hanoi, Vietnam · University of Science, Vietnam National University, Ho Chi Minh City, Vietnam +3

Spatio-Temporal Attention Deepfake Detection Vision Transformer

Ask PDF