Self-Supervised Pre-Training

Follow this topic Ask about this topic

Momentum

18 papers in the last four weeks, up 64% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 155

All topics

MorphCL: Morphological Contrastive Learning for Inertial-based Human Activity Recognition
Oct 7, 2026Marius Bock, Yuwei Zhang, Juergen Gall +3Human Activity Recognition Motion Representation Learning
University of Bonn · University of Cambridge · Lamarr Institute for Machine Learning and Artificial Intelligence +1
HuC-VideoMAE: Human-Centric Video Masked Autoencoding from synthetic data
Oct 6, 2026Ricardo Pizarro, Roberto Valle, José M. Buenaposada +2Synthetic Data Pretraining Masked Autoencoders
Universidad de Alcal´a, Alcal´a de Henares, Spain · Universidad Polit´ecnica de Madrid, Madrid, Spain · Universidad Rey Juan Carlos, M´ostoles, Spain
SpecBraM: What Should an EEG Foundation Model Predict? Masked Band-Power Prediction versus Waveform Reconstruction
Oct 5, 2026Peng Xie, Yequan Bie, Jianda Mao +1Electroencephalography Sleep Stage Classification
The Hong Kong University of Science and Technology
dIon: Fragmentation-Based Invariance for Self-Supervised Learning of Tandem Mass Spectra
Oct 5, 2026Alfred Nilsson, Joel Lapin, Samuel H. Payne +2Self-Supervised Pre-Training Mass Spectrometry
Science for Life Laboratory, KTH Royal Institute of Technology Stockholm, Sweden · Computational Mass Spectrometry, TUM School of Life Sciences Technical University of Munich, Freising, Germany · Biology Department, Brigham Young University Provo, Utah 84602, United States +1
Lightweight Semantic EEG Foundation Model for Frozen Cross-Disorder Transfer
Oct 4, 2026Rita Huan-Ting Peng, Nhat BuiEEG Seizure Detection Self-Supervised Pre-Training
University of Illinois Urbana–Champaign, Urbana, IL, USA · Carle Foundation Hospital, Urbana, IL, USA
Artifact Annotations Partially Substitute for Per-User Calibration: SAFE-EDA and a Normalization-Controlled Evaluation of Wrist-EDA Affect Recognition
Oct 1, 2026Haochen Chai, Xinbi Luo, Zining Liu +1Self-Supervised Pre-Training Affective Computing
College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China
Image Classifiers are Efficient Self-Supervised Video Representation Learners
Sep 30, 2026Owais Iqbal, Sudipta Sarkar, Shyam Marjit +3Self-Supervised Pre-Training Video Representation Learning
Indian Institute of Technology Kharagpur, India · Indian Institute of Science Bangalore, India · École de technologie supérieure Montreal, Canada
VANDAM: Viewing a nucleotide sequence with DNA molecular priors
Sep 30, 2026Jeremy Levy, Ariel Larey, Yury Nahshan +18Self-Supervised Pre-Training Genomics
Applied AI Architecture, NVIDIA, Israel. · Worldwide Field Ops, NVIDIA, Israel. · Developer Programs, NVIDIA, Israel. +3
Flattening the Connectome Spectrum: A Spectral Filter for FC Induces a Pretraining Target for fMRI Encoders
Sep 29, 2026Giovanni Marraffini, Victoria Shevchenko, Carlo Alberto Barbano +1Kernel Ridge Regression Medical Imaging Foundation Models
UNITO · MIND
Emergent Specialization in Populations of Self-Supervised Collaborative Vision Experts Without a Shared Gate or Cross-Agent Gradients
Sep 29, 2026Aram Davtyan, Pablo Acuaviva, Sebastian Stapf +1Self-Supervised Pre-Training Multi-Agent Collaboration
u b University of Bern Computer Vision Group, Institute of Computer Science University of Bern, Switzerland
Handwritten Text Recognition Lives in the High-Pixel Variance Subspace
Sep 28, 2026Carlos Garrido-Munoz, Jorge Calvo-ZaragozaSelf-Supervised Pre-Training Image Reconstruction
University of Alicante, Spain
RD-JEPA: Predictive latent pretraining for few-trajectory transfer across reaction--diffusion equations
Sep 24, 2026Chenhao Si, Ming YanDynamical Systems Self-Supervised Pre-Training
School of Data Science, The Chinese University of Hong Kong, Shenzhen, Shenzhen, China
EMGBlend: Heterogeneity-Aware Self-Supervised Pretraining for Gesture and Force Decoding
Sep 22, 2026Yuwei Jia, Cheng Zhong, Jinyang Yu +1EMG-Based Gesture Recognition Self-Supervised Pre-Training
Beijing University of Posts and Telecommunications, China · Dexwise, China · Shenzhen University, China
A JEPA Recipe for Tabular Foundation Models
Sep 22, 2026Mingyu Jeon, Suwan Cho, Jae Young SuhTabular Foundation Models Self-Supervised Pre-Training
Modulabs, Seoul, Republic of Korea
Toward a foundation model for forest point clouds
Sep 21, 2026Yuanwen Yue, Stefano Puliti, Damien Robert +73D Foundation Models Point Cloud Classification
ETH Zurich · University of Oxford · Norwegian Institute of Bioeconomy Research (NIBIO) +1
Procedural Pretraining for Molecular Property Prediction
Sep 15, 2026Moritz Friedemann, Zachary Shinnick, Philip Torr +1Synthetic Data Pretraining Self-Supervised Pre-Training
University of Oxford · Adelaide University · Slater Labs
Similarity Pairing with Energy Mover's Distance for Self-Supervised Pre-Training at the LHC
Sep 15, 2026Ho Fung Tsoi, Dylan RankinRepresentation Learning Self-Supervised Pre-Training
Department of Physics and Astronomy University of Pennsylvania, USA
MAETrack: Unleashing the Potential of Pretrained Geometric Priors for 3D Single Object Tracking
Sep 15, 2026Sifan Zhou, Qiwei Wang, Linyue Tan +3Masked Autoencoders Self-Supervised Pre-Training
School of Automation, Southeast University, Nanjing, 210096, China · Key Laboratory of Measurement and Control of Complex Systems of Engineering, Ministry of Education, Nanjing, 210096, China · Harbin Institute of Technology Shenzhen, Shenzhen, 518000, China +1
Which Pretext Task Transfers? Self-Supervised Pretraining Objectives for Lung Ultrasound
Sep 15, 2026Moein Heidari, Junbo Rao, Jai Choraria +3Contrastive Learning Masked Autoencoders
University of British Columbia, Vancouver, BC, Canada · Rutgers Cancer Institute of New Jersey, New Brunswick, NJ, USA
BRIDGE-EEG: Bridging Self-Supervised Pretraining and Efficient Deployment for Cross-Dataset EEG Classification
Sep 14, 2026Meghna Roy Chowdhury, Chengwei Zhou, Haotian Yu +2Self-Supervised Pre-Training Edge Inference
Purdue University, Indiana, USA · Case Western University, Ohio, USA
Geometric-to-Semantic Spherical Transfer Learning for Cortical Sulci Labeling
Sep 14, 2026Saeb Tounsi, Joël Chavas, Pietro Gori +3Self-Supervised Pre-Training Geometric Representation Learning
Paris-Saclay University, CEA, NeuroSpin, Baobab, Saclay, France · LTCI, Télécom Paris, Institut Polytechnique de Paris, France
Self-supervised Pre-training Helps Retinal Disease Progression Modelling Most When Data Is Scarce
Sep 14, 2026Ifeoma Veronica Nwabufo, Julius Gervelmeyer, Sarah Müller +1Self-Supervised Pre-Training Longitudinal Medical Image Analysis
Hertie Institute for AI in Brain Health, Faculty of Medicine, University of Tübingen, Germany · Tübingen AI Center, University of Tübingen, Germany
Revisiting Cross-View Completion: Self-Supervised Pre-Training via Reconstruction Error Comparison
Sep 1, 2026Thibaut Loiseau, Guillaume Bourmaud, Vincent LepetitSelf-Supervised Pre-Training 3D Representation Learning
LIGM, Ecole Nationale des Ponts et Chaussées, IP Paris, Univ Gustave Eiffel, CNRS, France · Univ. Bordeaux, CNRS, Bordeaux INP, IMS, UMR 5218, France
User Representation via Cross Multi-source Behavior Pre-training for Mobile Games
Sep 1, 2026Chengqi Yang, Yiran Qiao, Feng Liu +7Self-Supervised Pre-Training User Modeling
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences (CAS), Beijing, China. · University of Chinese Academy of Sciences, CAS, Beijing, China. · OPPO Research Institute, Shenzhen, China +1
Panda Diplomacy: Foundation Model Pre-training across Particle Imaging Detectors for High Energy and Nuclear Physics
Sep 1, 2026Samuel Young, César Jesús-Valls, Kazuhiro TeraoSelf-Supervised Pre-Training Point Cloud Learning
Stanford University · Stanford, CA 94305 · European Organization for Nuclear Research (CERN) +3
EEG-VID: Task-Guided Latent Predictive Pretraining for EEG Decoding and Assistive Target Selection
Sep 1, 2026Guanzhong Sun, Junyi Ma, Yuxuan Wu +2Cross-Subject EEG Decoding Self-Supervised Pre-Training
School of Information and Control Engineering, China University of Mining and Technology, Xuzhou, China · School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai, China · Shanghai Innovation Institute, Shanghai, China +1
MR-JEPA: A General Purpose Video Foundation Model for Cardiac MRI
Aug 31, 2026Athira J. Jacob, Puneet Sharma, Dorin Comaniciu +1Medical Imaging Foundation Models Self-Supervised Pre-Training
Digital Technology and Innovation, Siemens Healthineers, Princeton, NJ, USA · Chair for Al in Healthcare and Medicine, Technical University of Munich (TUM) · TUM University Hospital, Munich, Germany +1
Self-Supervised Pretext Tasks for Infant Cry Analysis: A Controlled Comparison and a Cautionary Result on Donateacry
Aug 31, 2026Luigi SimeoneData Leakage Self-Supervised Pre-Training
Independent researcher
Into the ORBIT for Time Series: Training Regimes for Foundation Models
Aug 13, 2026Hongjie Xia, Yiding Liu, Yifan Hu +2Zero-Shot Time Series Forecasting Self-Supervised Pre-Training
A Controlled Study of Self-Supervised Image and Video Pretraining under Limited Resources
Aug 13, 2026Brunó B. Englert, Gijs DubbelmanVision Foundation Models Self-Supervised Pre-Training
Eindhoven University of Technology

Oct 7, 2026, cs.LGJ/K move · Enter open · S save

MorphCL: Morphological Contrastive Learning for Inertial-based Human Activity Recognition

Marius Bock, Yuwei Zhang, Juergen Gall, Michael Moeller+2

University of Bonn · University of Cambridge · Lamarr Institute for Machine Learning and Artificial Intelligence+1

Human Activity Recognition Motion Representation Learning Contrastive Learning Self-Supervised Pre-Training

Open paper AskPDF arXiv

Despite the ubiquity of sensors in wearable and mobile devices and the abundance of human movement data they generate, translating unlabeled recordings into foundational motion models remains an open challenge. Self-supervised learning (SSL) has alleviated the need for costly annotations, yet existing approaches leave the global structure of large-scale motion data largely untapped, relying on randomly sampled batches and local comparisons that become particularly problematic for in-the-wild inertial data dominated by stationary, low-variance behaviors. Here we introduce Morphological Contrastive Learning (MorphCL), a self-supervised pretraining framework that uses structure-aware grouping to inject explicit modeling of global structure into inertial-based SSL approaches. Building on two well-established pillars of motion analysis, the discovery of motion primitives, or motifs, and domain-specific feature descriptors, we show that MorphCL substantially improves linear probing and finetuning results of learned encoders by up to 15 percentage points in F1-score. In a comparison with existing foundation models, we demonstrate that MorphCL-pretrained encoders match or surpass them models in linear probing performance while trained on

4600\times

less data. Qualitative analysis of the resulting embedding spaces further reveals morphologically meaningful cluster structure, with improved separation of kinematically similar activity classes.

Momentum

Related topics

Latest papers 155