Self-Supervised Pre-Training

Follow this topic Ask about this topic

Momentum

18 papers in the last four weeks, up 64% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 154

All topics

MorphCL: Morphological Contrastive Learning for Inertial-based Human Activity Recognition
Oct 7, 2026Marius Bock, Yuwei Zhang, Juergen Gall +3Human Activity Recognition Motion Representation Learning
University of Bonn · University of Cambridge · Lamarr Institute for Machine Learning and Artificial Intelligence +1
HuC-VideoMAE: Human-Centric Video Masked Autoencoding from synthetic data
Oct 6, 2026Ricardo Pizarro, Roberto Valle, José M. Buenaposada +2Synthetic Data Pretraining Masked Autoencoders
Universidad de Alcal´a, Alcal´a de Henares, Spain · Universidad Polit´ecnica de Madrid, Madrid, Spain · Universidad Rey Juan Carlos, M´ostoles, Spain
SpecBraM: What Should an EEG Foundation Model Predict? Masked Band-Power Prediction versus Waveform Reconstruction
Oct 5, 2026Peng Xie, Yequan Bie, Jianda Mao +1Electroencephalography Sleep Stage Classification
The Hong Kong University of Science and Technology
dIon: Fragmentation-Based Invariance for Self-Supervised Learning of Tandem Mass Spectra
Oct 5, 2026Alfred Nilsson, Joel Lapin, Samuel H. Payne +2Self-Supervised Pre-Training Mass Spectrometry
Science for Life Laboratory, KTH Royal Institute of Technology Stockholm, Sweden · Computational Mass Spectrometry, TUM School of Life Sciences Technical University of Munich, Freising, Germany · Biology Department, Brigham Young University Provo, Utah 84602, United States +1
Lightweight Semantic EEG Foundation Model for Frozen Cross-Disorder Transfer
Oct 4, 2026Rita Huan-Ting Peng, Nhat BuiEEG Seizure Detection Self-Supervised Pre-Training
University of Illinois Urbana–Champaign, Urbana, IL, USA · Carle Foundation Hospital, Urbana, IL, USA
Artifact Annotations Partially Substitute for Per-User Calibration: SAFE-EDA and a Normalization-Controlled Evaluation of Wrist-EDA Affect Recognition
Oct 1, 2026Haochen Chai, Xinbi Luo, Zining Liu +1Self-Supervised Pre-Training Affective Computing
College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China
Image Classifiers are Efficient Self-Supervised Video Representation Learners
Sep 30, 2026Owais Iqbal, Sudipta Sarkar, Shyam Marjit +3Self-Supervised Pre-Training Video Representation Learning
Indian Institute of Technology Kharagpur, India · Indian Institute of Science Bangalore, India · École de technologie supérieure Montreal, Canada
VANDAM: Viewing a nucleotide sequence with DNA molecular priors
Sep 30, 2026Jeremy Levy, Ariel Larey, Yury Nahshan +18Self-Supervised Pre-Training Genomics
Applied AI Architecture, NVIDIA, Israel. · Worldwide Field Ops, NVIDIA, Israel. · Developer Programs, NVIDIA, Israel. +3
Flattening the Connectome Spectrum: A Spectral Filter for FC Induces a Pretraining Target for fMRI Encoders
Sep 29, 2026Giovanni Marraffini, Victoria Shevchenko, Carlo Alberto Barbano +1Kernel Ridge Regression Medical Imaging Foundation Models
UNITO · MIND
Emergent Specialization in Populations of Self-Supervised Collaborative Vision Experts Without a Shared Gate or Cross-Agent Gradients
Sep 29, 2026Aram Davtyan, Pablo Acuaviva, Sebastian Stapf +1Self-Supervised Pre-Training Multi-Agent Collaboration
u b University of Bern Computer Vision Group, Institute of Computer Science University of Bern, Switzerland
Handwritten Text Recognition Lives in the High-Pixel Variance Subspace
Sep 28, 2026Carlos Garrido-Munoz, Jorge Calvo-ZaragozaSelf-Supervised Pre-Training Image Reconstruction
University of Alicante, Spain
RD-JEPA: Predictive latent pretraining for few-trajectory transfer across reaction--diffusion equations
Sep 24, 2026Chenhao Si, Ming YanDynamical Systems Self-Supervised Pre-Training
School of Data Science, The Chinese University of Hong Kong, Shenzhen, Shenzhen, China
EMGBlend: Heterogeneity-Aware Self-Supervised Pretraining for Gesture and Force Decoding
Sep 22, 2026Yuwei Jia, Cheng Zhong, Jinyang Yu +1EMG-Based Gesture Recognition Self-Supervised Pre-Training
Beijing University of Posts and Telecommunications, China · Dexwise, China · Shenzhen University, China
A JEPA Recipe for Tabular Foundation Models
Sep 22, 2026Mingyu Jeon, Suwan Cho, Jae Young SuhTabular Foundation Models Self-Supervised Pre-Training
Modulabs, Seoul, Republic of Korea
Toward a foundation model for forest point clouds
Sep 21, 2026Yuanwen Yue, Stefano Puliti, Damien Robert +73D Foundation Models Point Cloud Classification
ETH Zurich · University of Oxford · Norwegian Institute of Bioeconomy Research (NIBIO) +1
Procedural Pretraining for Molecular Property Prediction
Sep 15, 2026Moritz Friedemann, Zachary Shinnick, Philip Torr +1Synthetic Data Pretraining Self-Supervised Pre-Training
University of Oxford · Adelaide University · Slater Labs
Similarity Pairing with Energy Mover's Distance for Self-Supervised Pre-Training at the LHC
Sep 15, 2026Ho Fung Tsoi, Dylan RankinRepresentation Learning Self-Supervised Pre-Training
Department of Physics and Astronomy University of Pennsylvania, USA
MAETrack: Unleashing the Potential of Pretrained Geometric Priors for 3D Single Object Tracking
Sep 15, 2026Sifan Zhou, Qiwei Wang, Linyue Tan +3Masked Autoencoders Self-Supervised Pre-Training
School of Automation, Southeast University, Nanjing, 210096, China · Key Laboratory of Measurement and Control of Complex Systems of Engineering, Ministry of Education, Nanjing, 210096, China · Harbin Institute of Technology Shenzhen, Shenzhen, 518000, China +1
Which Pretext Task Transfers? Self-Supervised Pretraining Objectives for Lung Ultrasound
Sep 15, 2026Moein Heidari, Junbo Rao, Jai Choraria +3Contrastive Learning Masked Autoencoders
University of British Columbia, Vancouver, BC, Canada · Rutgers Cancer Institute of New Jersey, New Brunswick, NJ, USA
BRIDGE-EEG: Bridging Self-Supervised Pretraining and Efficient Deployment for Cross-Dataset EEG Classification
Sep 14, 2026Meghna Roy Chowdhury, Chengwei Zhou, Haotian Yu +2Self-Supervised Pre-Training Edge Inference
Purdue University, Indiana, USA · Case Western University, Ohio, USA
Self-supervised Pre-training Helps Retinal Disease Progression Modelling Most When Data Is Scarce
Sep 14, 2026Ifeoma Veronica Nwabufo, Julius Gervelmeyer, Sarah Müller +1Self-Supervised Pre-Training Longitudinal Medical Image Analysis
Hertie Institute for AI in Brain Health, Faculty of Medicine, University of Tübingen, Germany · Tübingen AI Center, University of Tübingen, Germany
Geometric-to-Semantic Spherical Transfer Learning for Cortical Sulci Labeling
Sep 14, 2026Saeb Tounsi, Joël Chavas, Pietro Gori +3Self-Supervised Pre-Training Geometric Representation Learning
Paris-Saclay University, CEA, NeuroSpin, Baobab, Saclay, France · LTCI, Télécom Paris, Institut Polytechnique de Paris, France
Revisiting Cross-View Completion: Self-Supervised Pre-Training via Reconstruction Error Comparison
Sep 1, 2026Thibaut Loiseau, Guillaume Bourmaud, Vincent LepetitSelf-Supervised Pre-Training 3D Representation Learning
LIGM, Ecole Nationale des Ponts et Chaussées, IP Paris, Univ Gustave Eiffel, CNRS, France · Univ. Bordeaux, CNRS, Bordeaux INP, IMS, UMR 5218, France
User Representation via Cross Multi-source Behavior Pre-training for Mobile Games
Sep 1, 2026Chengqi Yang, Yiran Qiao, Feng Liu +7Self-Supervised Pre-Training User Modeling
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences (CAS), Beijing, China. · University of Chinese Academy of Sciences, CAS, Beijing, China. · OPPO Research Institute, Shenzhen, China +1
Panda Diplomacy: Foundation Model Pre-training across Particle Imaging Detectors for High Energy and Nuclear Physics
Sep 1, 2026Samuel Young, César Jesús-Valls, Kazuhiro TeraoSelf-Supervised Pre-Training Point Cloud Learning
Stanford University · Stanford, CA 94305 · European Organization for Nuclear Research (CERN) +3
EEG-VID: Task-Guided Latent Predictive Pretraining for EEG Decoding and Assistive Target Selection
Sep 1, 2026Guanzhong Sun, Junyi Ma, Yuxuan Wu +2Cross-Subject EEG Decoding Self-Supervised Pre-Training
School of Information and Control Engineering, China University of Mining and Technology, Xuzhou, China · School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai, China · Shanghai Innovation Institute, Shanghai, China +1
MR-JEPA: A General Purpose Video Foundation Model for Cardiac MRI
Aug 31, 2026Athira J. Jacob, Puneet Sharma, Dorin Comaniciu +1Medical Imaging Foundation Models Self-Supervised Pre-Training
Digital Technology and Innovation, Siemens Healthineers, Princeton, NJ, USA · Chair for Al in Healthcare and Medicine, Technical University of Munich (TUM) · TUM University Hospital, Munich, Germany +1
Self-Supervised Pretext Tasks for Infant Cry Analysis: A Controlled Comparison and a Cautionary Result on Donateacry
Aug 31, 2026Luigi SimeoneData Leakage Self-Supervised Pre-Training
Independent researcher
Into the ORBIT for Time Series: Training Regimes for Foundation Models
Aug 13, 2026Hongjie Xia, Yiding Liu, Yifan Hu +2Zero-Shot Time Series Forecasting Self-Supervised Pre-Training
A Controlled Study of Self-Supervised Image and Video Pretraining under Limited Resources
Aug 13, 2026Brunó B. Englert, Gijs DubbelmanVision Foundation Models Self-Supervised Pre-Training
Eindhoven University of Technology
DINO-A: Adapting Self-Distillation Vision Transformers to General Audio Representation Learning
Aug 11, 2026Tomasz Radzikowski, Mateusz Modrzejewski, Przemysław RokitaAudio Representation Learning Self-Supervised Pre-Training
Warsaw University of Technology, Warsaw, Poland
DegradeQuery: Counterfactual Tuple Pretraining for Context-Aware PROTAC Degradation Prediction
Aug 11, 2026Dong Xu, Zhangfan Yang, Jiantao Wu +3Contrastive Learning Self-Supervised Pre-Training
School of Artificial Intelligence, Shenzhen University · EasternDawn · School of Computer Science, University of Nottingham Ningbo
The Announcement Carries the Cue: Markup, Boundaries, and the Notation of Pre-Training Corpora
Aug 10, 2026E. M. FreeburgLanguage Model Pretraining Self-Supervised Pre-Training
Independent Researcher
Vernata: Self-Supervised Learning of LiDAR Point Representations
Aug 7, 2026Oliver Lemke, Alexander Liniger, Abel Gawel +1Self-Supervised Learning Self-Supervised Pre-Training
1Robotics and AI Institute · 2ETH Zurich
CellWorld: From Gene-Level Reconstruction to Latent Cell Prediction in Spatial Transcriptomics Foundation Models
Aug 7, 2026Haiping Liu, Qian Zhao, Lijing Lin +2Spatial Transcriptomics Self-Supervised Pre-Training
University of Manchester
Lesion Detection in CT with Frozen Self-Distilled Features: SALT, a Spatially Adaptive Label-Guided Temperature
Aug 5, 2026Mahmut S. Gokmen, Evan W. Damron, Mitchell A. Klusty +3Self-Supervised Pre-Training Medical Image Analysis
University of Kentucky Lexington, KY, USA · University of Louisville Louisville, KY, USA
Helping Music Co-Creation Agents 'Listen' Well: Hierarchical Self-Supervised World Models for Understanding and Generation
Aug 5, 2026Scott H. HawleyRepresentation Learning Self-Supervised Pre-Training
Department of Chemistry & Physics Belmont University Nashville, TN, USA
ED-DiT: Physics-Guided Diffusion Pretraining for Transferable Molecular Representations from Electron Density
Aug 4, 2026Liang Shuang, Haocheng Wang, Jiayi Song +2Diffusion Transformer Self-Supervised Pre-Training
Fudan University, Shanghai, China · Shenzhen Loop Area Institute, Shenzhen, China · The Chinese University of Hong Kong, Hong Kong SAR, China
Understanding and Correcting Low-Frequency Bias in EEG Foundation Model
Aug 3, 2026Junjie Yu, Zihan Deng, Jianyu Zhang +8Masked Autoencoders Self-Supervised Pre-Training
Department of Biomedical Engineering, Southern University of Science and Technology, Shenzhen, China · Omni-Intelligence, Shenzhen, China · College of Design and Engineering, National University of Singapore, Singapore +4
Beyond Gene Reconstruction: Learning Cell Representations through Complementary Transcriptomic Views
Aug 2, 2026Jiaqi Xiong, Yuntao hu, Yu Zheng +3Contrastive Learning Representation Learning
University of Oxford, Oxford, United Kingdom · Tongji University, Shanghai, China · Hunan University, Changsha, China +2
Geometric Self-Supervised Pre-training for Neural Combinatorial Optimization
Jul 31, 2026David Aguado, Daniel Fuertes, Carlos R. del-Blanco +1Traveling Salesperson Problem Self-Supervised Learning
Grupo de Tratamiento de Imágenes, Information Processing and Telecomunications Center, ETSI Telecomunicación, Universidad Politécnica de Madrid, 28040, Madrid, Spain
On the Efficacy of Self-Supervised Point Cloud Encoders for Efficient 3D Large Language Models
Jul 31, 2026Yao Zheng, Tian ZhangSelf-Supervised Pre-Training Point Cloud Learning
Beijing University of Posts and Telecommunications
A Lightweight Foundation Model for Collider Physics with Multi-Domain Adaptation
Jul 29, 2026Liangyu Wu, Qibin Liu, Alexander Yue +1Self-Supervised Pre-Training High-Energy Physics
Department of Physics, Stanford University, 450 Jane Stanford Way, Stanford, 94305, CA, USA. · SLAC National Accelerator Laboratory, 2575 Sand Hill Road, Menlo Park, 94025, CA, USA.
SARATR-X-v2: Scale-Aware Structural Pre-Training for SAR Foundation Models
Jul 25, 2026Weijie Li, Yafei Song, Yongxiang Liu +7Self-Supervised Pre-Training Synthetic Aperture Radar
College of Electronic Science and Technology, National University of Defense Technology, Changsha 410073, China
IQ-JEPA: A Joint-Embedding Predictive Architecture with a Hermitian Vision Transformer for Sound Speed and Attenuation Estimation from Ultrasound IQ Data
Jul 24, 2026Masashi Sode, Gianmarco PintonSelf-Supervised Pre-Training Joint-Embedding Predictive Architecture
Joint Department of Biomedical Engineering, University of North Carolina at Chapel Hill and North Carolina State University, Chapel Hill, NC 27599 USA
MSBraM: A Multi-scale Self-supervised Brain Foundation Model for Hierarchical EEG Dynamics Learning
Jul 23, 2026Tao Zhou, Jing Han, Lingyu Shu +1Electroencephalography Self-Supervised Pre-Training
Hunan University Changsha, China
Multimodal Pretraining for Generalizable EEG Representation Learning
Jul 23, 2026Targol Bakhtiarvand, Jugal Kalita, Adham AtyabiCross-Subject EEG Decoding Electroencephalography
Department of Computer Science, University of Colorado Colorado Springs, Colorado Springs, CO, USA
Self-supervision drives representational convergence in medical foundation models more than clinical supervision
Jul 22, 2026Soroosh Tayebi Arasteh, Sebastian Ziegelmayer, Mahshad Lotfinia +4Medical Imaging Foundation Models Self-Supervised Pre-Training
Lab for AI in Medicine, RWTH Aachen University, Aachen, Germany · Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany · Department of Diagnostic and Interventional Radiology, TUM University Clinic, School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Munich, Germany +4
Not All Patches are Equal: Sampling Matters for Visible-Infrared Pre-Training
Jul 22, 2026Qiwei Ma, Bin Deng, Junjie Zhu +5Visual Representation Learning Self-Supervised Pre-Training
School of Artificial Intelligence and Robotics, Hunan University, Changsha, 410082, Hunan, China · Intelligent Game and Decision Lab, Beijing, 100166, Beijing, China · Yuelushan Center for Industrial Innovation, Changsha, 410082, Hunan, China
On the Effectiveness of Pretraining for Graph Combinatorial Optimization
Jul 21, 2026David Aguado, Daniel Fuertes, Carlos R. del-Blanco +1Traveling Salesperson Problem Self-Supervised Pre-Training
Grupo de Tratamiento de Imágenes, Information Processing and Telecomunications Center, ETSI Telecomunicación, Universidad Politécnica de Madrid, 28040, Madrid, Spain
BrainNext: A General-Purpose Self-Supervised Foundation Model for Brain MRI Analysis
Jul 20, 2026Moona Mazher, Abdul Qayyum, Steven A. Niederer +1Medical Imaging Foundation Models Self-Supervised Pre-Training
UCL Hawkes Institute, Department of Computer Science, University College London, London, United Kingdom · National Heart & Lung Institute, Imperial College London, London, United Kingdom · Division of Cardiovascular Medicine, Stanford Cardiovascular Institute, Stanford University, California, USA
A Framework for Early Sepsis Prediction via Self-Supervised (JEPA) and Federated Representation Learning
Jul 18, 2026Umair bin Mansoor, Munaf Rashid, Roomi NaqviSelf-Supervised Pre-Training Joint-Embedding Predictive Architecture
Electrical Engg. Department, DHA Suffa University, Karachi, Pakistan · Salim Habib University, Karachi, Sindh, Pakistan · Intel Foundry Services, Intel, Tempe, Arizona, United States
Multimodality as Supervision: Self-Supervised Specialization to the Test Environment via Multimodality
Jul 16, 2026Kunal Pratap Singh, Ali Garjani, Rishubh Singh +6Cross-Modal Learning Cross-Modal Representation Learning
Swiss Federal Institute of Technology Lausanne (EPFL) · Apple
MIDI-RAE-JEPA: Hierarchical Representation Learning and Generation for Symbolic Music
Jul 16, 2026Scott H. HawleyRepresentation Learning Self-Supervised Pre-Training
Department of Chemistry & Physics, Belmont University, Nashville, TN, USA
Leveraging unlabelled data for generalizable neural population decoding
Jul 15, 2026Ximeng Mao, Nanda H. Krishna, Avery Hee-Woon Ryoo +2Masked Autoencoders Self-Supervised Pre-Training
Mila – Quebec AI Institute · Université de Montréal · Canada CIFAR AI Chair
Self-supervised training for high-resolution close-range multispectral remote sensing imagery
Jul 13, 2026Leon-Friedrich Thomas, Mikael Änäkkälä, Antti LajunenAgricultural Image Analysis Remote Sensing Image Segmentation
University of Helsinki, Department of Agricultural Sciences, P.O. Box 28, Fi-00014 Helsinki, Finland · University of Helsinki, Department of Forest Sciences, P.O. Box 27, Fi-00014 Helsinki, Finland
Scalable Visual Pretraining for Language Intelligence
Jul 10, 2026Yiming Zhang, Zhonghan Zhao, Wenwei Zhang +13Language Model Pretraining Self-Supervised Pre-Training
Shanghai Artificial Intelligence Laboratory · University of Science and Technology of China · Zhejiang University +1
CoCoT-EEG: Contrastive-Pretrained Multiscale Convolutional Transformer for EEG Decoding
Jul 10, 2026Gabriel Mahuas, Victoria Shevchenko, Ugo Tanielian +2Contrastive Learning Hybrid CNN-Transformer Architectures
Sigma Nova Paris, France · Goethe University Frankfurt Frankfurt, Germany
JEPA for AI-Native 6G: Predictive Representations and Open Challenges
Jul 9, 2026Sheikh Salman Hassan, Irshad A. Meer, Almoatssimbillah Saifaldawla +96G Networks Self-Supervised Pre-Training
IDCoM, University of Edinburgh, Edinburgh EH9 3BF, UK · KTH Royal Institute of Technology, SE-100 44 Stockholm, Sweden · SnT, University of Luxembourg, L-1855 Luxembourg +6
Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence
Jul 8, 2026Shuailei Ma, Jiaqi Liao, Xinyang Wang +24Video Diffusion Models Self-Supervised Pre-Training

Oct 7, 2026, cs.LGJ/K move · Enter open · S save

MorphCL: Morphological Contrastive Learning for Inertial-based Human Activity Recognition

Marius Bock, Yuwei Zhang, Juergen Gall, Michael Moeller+2

University of Bonn · University of Cambridge · Lamarr Institute for Machine Learning and Artificial Intelligence+1

Human Activity Recognition Motion Representation Learning Contrastive Learning Self-Supervised Pre-Training

Open paper AskPDF arXiv

Despite the ubiquity of sensors in wearable and mobile devices and the abundance of human movement data they generate, translating unlabeled recordings into foundational motion models remains an open challenge. Self-supervised learning (SSL) has alleviated the need for costly annotations, yet existing approaches leave the global structure of large-scale motion data largely untapped, relying on randomly sampled batches and local comparisons that become particularly problematic for in-the-wild inertial data dominated by stationary, low-variance behaviors. Here we introduce Morphological Contrastive Learning (MorphCL), a self-supervised pretraining framework that uses structure-aware grouping to inject explicit modeling of global structure into inertial-based SSL approaches. Building on two well-established pillars of motion analysis, the discovery of motion primitives, or motifs, and domain-specific feature descriptors, we show that MorphCL substantially improves linear probing and finetuning results of learned encoders by up to 15 percentage points in F1-score. In a comparison with existing foundation models, we demonstrate that MorphCL-pretrained encoders match or surpass them models in linear probing performance while trained on

4600\times

less data. Qualitative analysis of the resulting embedding spaces further reveals morphologically meaningful cluster structure, with improved separation of kinematically similar activity classes.

Momentum

Related topics

Latest papers 154