Spatiotemporal Representation Learning

Follow this topic Ask about this topic

Latest papers

24 papers3 papers in the last four weeks, down 40% on the four weeks before. 0.0% of all new papers.

Spatio-Temporal Multi-Order Interaction for Video Recognition
Nov 3, 2026Kevin D. Nguetche, Lilian Hollard, Eric Granger +1Video Action Recognition Spatiotemporal Representation Learning
Beyond Modeling Order: Dual-Path Spatiotemporal Representation Learning for Cross-Subject EEG Emotion Recognition
Oct 26, 2026Zehui Xiao, Yi Zheng, Guixing Fu +7Spatiotemporal Representation Learning EEG Emotion Recognition
School of Biomedical Engineering, Shenzhen Campus of Sun Yat-sen University, Shenzhen 518107, China · School of Computer Science and Technology, Tianjin University, Tianjin 300354, China · Faculty of Applied Sciences, Macao Polytechnic University, Macao, China +1
Let's Do Turbulence Right: In Space and Time!
Oct 23, 2026Maximilian Plattner, Fabian Paischer, Johannes BrandstetterSpatiotemporal Representation Learning Computational Fluid Dynamics
ELLIS Unit, LIT AI Lab, Institute for Machine Learning, JKU Linz · Mistral AI
T-JEPA: A Temporal Joint-Embedding Predictive Architecture for Learning Better Remote Sensing Representations
Oct 5, 2026Bowen Peng, Li Liu, Yongxiang Liu +3Geospatial Representation Learning Self-Supervised Time Series Representation Learning
National University of Defense Technology, Changsha, China
Planetary Feature Fields are Scalable Earth Representations
Sep 29, 2026Arjun Rao, Sebastian Loeschcke, Anthony Fuller +3Implicit Neural Representations Geospatial Representation Learning
University of British Columbia · University of Copenhagen · Carleton University +2
W2Rep: Learning Visual Representations by Watching the World Change
Sep 28, 2026Wen Huang, Hang Guo, Jiarui Yang +3Video Representation Learning Self-Supervised Visual Representation Learning
Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Nankai University, Tianjin, China · Shenzhen University, Shenzhen, China
Addressing Spatial Indistinguishability in Spatiotemporal Prediction via Optimal Transport-Guided Masking
Sep 28, 2026Guangyu Wang, Jiawei TongSelf-Supervised Time Series Representation Learning Spatiotemporal Forecasting
Data Science and Artificial Intelligence, Dongbei University of Finance and Economics, Dalian, Liaoning, China · Institute of Systems, Molecular & Integrative Biology, University of Liverpool, Liverpool, United Kingdom
MotionJEPA: Preventing Temporal Feature Collapse by Capturing Visual Changes in Latent Space
Sep 20, 2026Markus Karmann, Shile Li, Christian Internò +10Joint-Embedding Predictive Architecture Latent World Models
University of Oxford · vivo Tech Research GmbH · Bielefeld University +5
Chronosphere: Space-Time Tessellation of Local Climate Experts
Sep 18, 2026Daniel Cher, Eric Xing, Kexing Li +3Spatiotemporal Representation Learning Geospatial Representation Learning
Washington University in St. Louis · Taylor Geospatial Institute
What, Where, and How: Probing Spatiotemporal Representations in Video Foundation Models
Sep 1, 2026Sharon S. Musa, Fereshteh Forghani, Harrish Thasarathan +3Physical Reasoning in Video Vision Foundation Models
Lassonde School of Engineering, York University, Toronto, Canada · Vector Institute, Toronto, Canada · FAIR, Meta Superintelligence Labs +2
CoST: Semantic-Aware Urban Understanding via Spatial-Temporal Alignment
Aug 21, 2026Yutian Jiang, Jiabo Liu, Xixuan Hao +1Urban Representation Learning Geospatial Representation Learning
The Hong Kong University of Science and Technology (Guangzhou)
You Don't Need Strong Assumptions: Visual Representation Learning via Temporal Differences
Jun 14, 2026Ninad Daithankar, Alexi Gladstone, Yann LeCun +1Visual Representation Learning Video Representation Learning
1UIUC · 2New York University
Learning Long Range Spatio-Temporal Representations over Continuous Time Dynamic Graphs with State Space Models
Jun 3, 2026Ayushman Raghuvanshi, Thummaluru Siddartha Reddy, Sundeep Prabhakar Chepuri +1Long-Context Modeling Temporal Link Prediction
Department of Electrical Communication Engineering, Indian Institute of Science, Bangalore
SiST-GNN: Simultaneous Spatial-Temporal Message Passing for Dynamic Graph Representation Learning
May 25, 2026Shubhajit Roy, Anirban DasguptaGraph Representation Learning Temporal Link Prediction
Department of Computer Science and Engineering Indian Institute of Technology Gandhinagar
GEESE: Genotype-aware End-to-End Spatio-temporal Embedding for Behavioral Phenotyping
May 23, 2026Yiran Ding, Yuen Gao, Chunqi Qian +1Spatiotemporal Representation Learning Animal Behavior Analysis
Department of Computer Science and Engineering, Michigan State University, East Lansing, Michigan, USA · Department of Radiology, Michigan State University, East Lansing, Michigan, USA
The TIME Machine: On The Power of Motion for Efficient Perception
May 21, 2026Mantas Skackauskas, Xinyue Hao, Laura Sevilla-LaraMotion Representation Learning Self-Supervised Visual Representation Learning
School of Informatics University of Edinburgh
TrajTok: Adaptive Spatial Tokenization for Trajectory Representation Learning
May 19, 2026Zhen Xiong, Shang-Ling Hsu, Cyrus ShahabiGeospatial Representation Learning Trajectory Representation Learning
University of Southern California
Location Is All You Need: Continuous Spatiotemporal Neural Representations of Earth Observation Data
Apr 8, 2026Mojgan Madadikhaljan, Jonathan Prexl, Isabelle Wittmann +2Geospatial Representation Learning Spatiotemporal Representation Learning
University of the Bundeswehr Munich · IBM Research – Europe · Columbia University +1
EventFace: Event-Based Face Recognition via Structure-Driven Spatiotemporal Modeling
Apr 8, 2026Qingguo Meng, Xingbo Dong, Zhe Jin +1Spatiotemporal Representation Learning Face Recognition
State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology, the Anhui Provincial Key Laboratory of Secure Artificial Intelligence, Anhui Provincial International Joint Research Center for Advanced Technology in Medical Imaging, and the School of Artificial Intelligence, Anhui University, Hefei, China · Computer Vision Laboratory, University of Sassari, 07100 Sassari, Italy
Mixture-of-Modality-Experts with Holistic Token Learning for Fine-Grained Multimodal Visual Analytics in Driver Action Recognition
Apr 7, 2026Tianyi Liu, Yiming Li, Wenqian Wang +4Spatiotemporal Representation Learning Multimodal Understanding
Nanyang Technological University, 50 Nanyang Avenue, 639798, Singapore · Singapore University of Technology and Design, 8 Somapah Rd, 487372, Singapore · The Hong Kong Polytechnic University, 6/F, Core D, Lui Che Woo Building, Hung Hom, Hong Kong SAR, China
STRNet: Visual Navigation with Spatio-Temporal Representation through Dynamic Graph Aggregation
Apr 3, 2026Hao Ren, Zetong Bi, Yiming Zeng +3Spatiotemporal Representation Learning Robot Navigation
Sun Yat-sen University · Insta360 Research · Wuhan University
Representation Learning for Spatiotemporal Physical Systems
Mar 13, 2026Helen Qu, Rudy Morel, Michael McCabe +4Representation Learning Spatiotemporal Representation Learning
Flatiron Institute · Princeton University · Université Paris-Saclay, Université Paris Cité, CEA, CNRS, AIM +1
TerraFlow: Multimodal, Multitemporal Representation Learning for Earth Observation
Mar 13, 2026Nazar Puriy, Johannes Jakubik, Benedikt Blumenstiel +1Representation Learning Geospatial Foundation Models
IBM Research Europe, Zurich, Switzerland · ETH Zurich, Zurich, Switzerland
Task Aware Modulation Using Representation Learning for Upsaling of Terrestrial Carbon Fluxes
Mar 10, 2026Aleksei Rozanov, Arvind Renganathan, Vipin KumarSpatiotemporal Representation Learning Climate Science
University of Minnesota, Twin Cities

Nov 3, 2026J/K move · Enter open · S save

Spatio-Temporal Multi-Order Interaction for Video Recognition

Kevin D. Nguetche, Lilian Hollard, Eric Granger, Marco Pedersoli

Video Action Recognition Spatiotemporal Representation Learning Temporal Convolutional Networks

Open paper AskPDF arXiv

Many temporal approaches for video recognition rely on explicit cross-channel aggregation projections, which have quadratic complexity in the number of channels, introducing significant computational overhead. In addition, the spatio-temporal representation used for video recognition is biased either toward local order interactions (for CNNs) or toward global order interactions (for transformers). Starting from the multi-order spatial representation of MogaNet, our method introduces two complementary contributions to tackle these problems. First, for the temporal modelling, we introduce parallel multi-order processing with depthwise-only convolutions and implicit gating-based aggregation, which is linear in the number of channels, inducing a considerable computational reduction. Second, we propose a dual interaction between spatial and temporal dimensions, which allows a better spatio-temporal representation without increasing the computational cost. By combining these two contributions, our proposed VideoMogaNet achieves performance on par with or better than state-of-the-art action recognition approaches across multiple video benchmarks (UCF101, HMDB51, Kinetics-400, Diving48), while considerably reducing the cost of the temporal branch.