Video Representation Learning

Follow this topic Ask about this topic

Momentum

15 papers in the last four weeks, up 50% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 87

All topics

HuC-VideoMAE: Human-Centric Video Masked Autoencoding from synthetic data
Oct 6, 2026Ricardo Pizarro, Roberto Valle, José M. Buenaposada +2Synthetic Data Pretraining Masked Autoencoders
Universidad de Alcal´a, Alcal´a de Henares, Spain · Universidad Polit´ecnica de Madrid, Madrid, Spain · Universidad Rey Juan Carlos, M´ostoles, Spain
Video Encoders Built on Image Representations
Oct 5, 2026Jusheng Zhang, Wenhao Wang, Longqi Cai +3Video-Language Models Video Representation Learning
Stanford University · Vast Intelligence Lab · Google DeepMind
Affine-Aligned Atlas for Canonical Gaussian Construction in Video Representation
Oct 1, 2026Masaya Takabe, Hiroshi Watanabe, Sujun Hong +13D Gaussian Splatting Dynamic Scene Reconstruction
Graduate School of FSE, Waseda University Tokyo, Japan · Sharp Corporation Chiba, Japan
Image Classifiers are Efficient Self-Supervised Video Representation Learners
Sep 30, 2026Owais Iqbal, Sudipta Sarkar, Shyam Marjit +3Self-Supervised Pre-Training Video Representation Learning
Indian Institute of Technology Kharagpur, India · Indian Institute of Science Bangalore, India · École de technologie supérieure Montreal, Canada
I Have a Stream: Making Self-Supervised Learning Work on Continuous Video
Sep 30, 2026Ivan Martinović, Lukas Knobel, Yuki M. AsanoMasked Autoencoders Self-Supervised Learning
Faculty of Electrical Engineering and Computing, University of Zagreb · Fundamental AI Lab, University of Technology Nuremberg
CoVisco: Codec-Native Vision Encoder with Native Token Compression for Unified Image-Video Understanding
Sep 30, 2026Yulong Liu, Xiaotian Han, Junyuan Shang +6Video Understanding Efficient VLM Inference
ERNIE Team, Baidu Inc. · The Hong Kong University of Science and Technology · Institute of Automation, Chinese Academy of Sciences (CASIA)
RESUME: Recurrent State Updates from Motion and Residual Signals for Efficient Video Language Modeling
Sep 30, 2026Can Zhang, Xiaotian Han, Junyuan Shang +5Video-Language Models Video Representation Learning
Beijing University of Chemical Technology · Baidu, Inc.
CurvSpec: Adaptive Multi-Curvature Learning for Partial Relevant Video Retrieval
Sep 29, 2026Zhen Liu, Letian Li, Jinpeng Wang +4Cross-Modal Retrieval Video Representation Learning
SIGS, Tsinghua University Shenzhen, China · Harbin Institute of Technology, Shenzhen Shenzhen, China
W2Rep: Learning Visual Representations by Watching the World Change
Sep 28, 2026Wen Huang, Hang Guo, Jiarui Yang +3Video Representation Learning Self-Supervised Visual Representation Learning
Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Nankai University, Tianjin, China · Shenzhen University, Shenzhen, China
Generative Uncertainty as a Self-supervised Signal for Semantic Similarity Learning
Sep 28, 2026Enrico Pallotta, Sina Raoufi, Lars Doorenbos +2Video Representation Learning Feature Selection
University of Bonn · Lamarr Institute for ML & AI · ENSTA Paris, Institut Polytechnique de Paris
$λ$ -JEPA Spectral Anti-Collapse Regularization for Self-Supervised Learning
Sep 28, 2026Berker Demirel, Clémentine Dominé, Valentino Maiorca +3Spectral Regularization Joint-Embedding Predictive Architecture
Institute of Science and Technology Austria (ISTA) Am Campus 1, 3400 Klosterneuburg, Austria
ReDrive: Shaping Representations with World Modeling for End-to-End Driving
Sep 27, 2026Yueting Zhu, Shaoyu Chen, Yuehao Song +4World Model Learning End-to-End Autonomous Driving
Huazhong University of Science & Technology · Horizon Robotics
TT-VidT: Decoupling the Temporal Axis for Efficient Motion-Centric Video Pretraining
Sep 27, 2026Shih-Ying Yeh, Daniel Z. Kaplan, Xuehai Wang +3Efficient ViTs Video Representation Learning
National Tsing Hua University · Comfy Org Research · Karolinska Institutet +2
Video-STLayout Pre-training
Sep 21, 2026Akash Abdu Jyothi, Greg MoriVideo Action Recognition Video Representation Learning
School of Computing Science Simon Fraser University Burnaby, Canada
Resolution-Flexible Decoding for Hybrid Neural Video Representations
Sep 20, 2026Taiga Hayami, Masaya Takabe, Hiroshi WatanabeVideo Representation Learning
Graduate School of FSE, Waseda University Tokyo, Japan
GeoLAM: Learning Geometry-Grounded Latent Actions from Unlabeled Human Videos
Sep 15, 2026Yifan Xie, Hekun Tian, Jinkun Liu +3Latent Action Learning Geometric Representation Learning
Tsinghua University · Beijing Institute of Technology · Fudan University
Arti-JEPA: Adapting Video World Model to Real-Time MRI of the Vocal Tract for Speech-Production Analysis
Sep 9, 2026Hong Nguyen, Sean Foley, Christina Hagedorn +4Unsupervised Domain Adaptation Joint-Embedding Predictive Architecture
Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, CA, USA · Department of English, College of Staten Island, City University of New York, New York, NY, USA · Department of Linguistics, University of Potsdam, Germany +1
VidaForge: Open Research Infrastructure for Video Pretraining Data Recipes
Sep 6, 2026Yan Ma, Jiadi Su, Zhulin Hu +4Vision Foundation Models Training Data Curation
Fudan University · Generative Artificial Intelligence Research Lab (GAIR) · Shanghai Jiao Tong University +2
The Shape of Time: Video-Token Contrast for Temporal Understanding in VideoLMs
Sep 3, 2026Yumeng Shi, Quanyu Long, Yin Wu +1Video-Language Models Video Representation Learning
Nanyang Technological University
What, Where, and How: Probing Spatiotemporal Representations in Video Foundation Models
Sep 1, 2026Sharon S. Musa, Fereshteh Forghani, Harrish Thasarathan +3Vision Foundation Models Video Representation Learning
Lassonde School of Engineering, York University, Toronto, Canada · Vector Institute, Toronto, Canada · FAIR, Meta Superintelligence Labs +2
Analytic Dynamics: Learning Physics-Grounded Representation for Fast Intrinsic Dynamics Inference from Monocular Videos
Aug 31, 2026Jailing Lin, Jikuan Zhang, Jianhua SunMaterials Property Prediction Video Representation Learning
School of Artificial Intelligence, Shanghai Jiao Tong University
MR-JEPA: A General Purpose Video Foundation Model for Cardiac MRI
Aug 31, 2026Athira J. Jacob, Puneet Sharma, Dorin Comaniciu +1Medical Imaging Foundation Models Self-Supervised Pre-Training
Digital Technology and Innovation, Siemens Healthineers, Princeton, NJ, USA · Chair for Al in Healthcare and Medicine, Technical University of Munich (TUM) · TUM University Hospital, Munich, Germany +1
PRISM: Predictive Recomposition via Semantic Latent Decomposition for View-invariant Video Representation Learning
Aug 31, 2026Youngchae Chee, Hosu Lee, Sungjune Park +2Disentangled Representation Learning Egocentric Video Understanding
KAIST · University of Illinois Urbana-Champaign
V-RAE: Rethinking Video Latent Spaces for Generation
Aug 13, 2026Minghui Guo, Shengqiong Wu, Hao FeiAutoencoders Video Representation Learning
National University of Singapore · University of Oxford
A Controlled Study of Self-Supervised Image and Video Pretraining under Limited Resources
Aug 13, 2026Brunó B. Englert, Gijs DubbelmanVision Foundation Models Self-Supervised Pre-Training
Eindhoven University of Technology
Learning Unified Video and Image Representation for Video Face Forgery Detection
Aug 13, 2026Haotian Liu, Yang Liu, Guoying Zhao +1Video Representation Learning Deepfake Detection
AVA-Encoder: Towards Agent-Native Video Representation Learning
Aug 12, 2026Chuyue Li, Jinpeng Yu, Haozhe Wang +7Video Representation Learning
Qwen Business Unit of Alibaba · ShanghaiTech University · The Hong Kong University of Science and Technology +2
Persistent Object Narratives for Token-Efficient Video Language Models
Aug 5, 2026Junzhe Chen, Siyuan Meng, Xiaojie GuoVisual Tokenization Video-Language Models
Tianjin University
Mitigating Compounding Error via Video Representation Regularization
Jul 29, 2026Taiye Chen, Qi Zhang, Yisen WangVideo Diffusion Models Long-Horizon Video Generation
Peking University
Explicit Layer Modeling for Video Object Insertion and Video Layer Decomposition
Jul 28, 2026Kyujin Han, Seungjoo Shin, Sunghyun ChoVideo Diffusion Models Video Editing
POSTECH, Republic of Korea

Oct 6, 2026, cs.CVJ/K move · Enter open · S save

HuC-VideoMAE: Human-Centric Video Masked Autoencoding from synthetic data

Ricardo Pizarro, Roberto Valle, José M. Buenaposada, Luis M. Bergasa+1

Universidad de Alcal´a, Alcal´a de Henares, Spain · Universidad Polit´ecnica de Madrid, Madrid, Spain · Universidad Rey Juan Carlos, M´ostoles, Spain

Synthetic Data Pretraining Masked Autoencoders Self-Supervised Pre-Training Video Action Recognition Video Representation Learning

Open paper AskPDF arXiv

Modern action recognition models rely on video transformers pretrained on massive collections of web-crawled videos, such as Kinetics-700. However, the use of such data raises ethical concerns, as subjects' consent is typically not obtained. Recent high-quality synthetic video datasets generated from motion-capture data, such as BEDLAM2.0, offer a promising ethical alternative. In this work, we investigate self-supervised pretraining of video transformers on synthetic human-motion datasets. We first show that directly applying the standard VideoMAE masking strategy leads to substantially worse performance than pretraining on Kinetics. To address this limitation, we propose a human-centric masking scheme that leverages body keypoints and person bounding box regions. Our approach encourages the model to focus on the structure and dynamics of human motion during pretraining. Experiments on NTU RGB+D and Toyota-Smarthome demonstrate that our method significantly outperforms standard VideoMAE pretraining on synthetic data, closing 49% of the gap to Kinetics pretraining on NTU RGB+D cross-view-subject without using a single real frame during pretraining. To promote the use of ethical action recognition models, we will publicly release our pretrained models.

Momentum

Related topics

Latest papers 87