Self-Supervised Pre-Training

Follow this topic Ask about this topic

Self-supervised pre-training studies how models can learn useful structure from unlabeled data, making it possible to adapt them to new tasks with fewer annotations. Common approaches include masked autoencoders, contrastive learning, teacher-student distillation, and joint-embedding predictive architectures such as JEPA, applied to images, signals, video, text, and scientific data. Researchers test these representations through frozen-feature probes, fine-tuning, and few-label or cross-domain transfer on benchmarks in areas such as medical imaging, speech, and autonomous driving. Results show that pre-training can improve data efficiency and robustness, but its benefits depend on the match between objective, data, and downstream task; greater model size or training time does not reliably help.

Latest papers

251 papers34 papers in the last four weeks, level with the four weeks before. 0.2% of all new papers.

A Self-Supervised Approach for Enhanced Feature Representations in Object Detection Tasks
Feb 18, 2026Santiago C. Vilabella, Pablo Pérez-Núñez, Beatriz RemeseiroObject Detection Self-Supervised Visual Representation Learning
Menéndez Pelayo International University, Spain · Artificial Intelligence Center, University of Oviedo, Gijón, Spain
BAT: Better Audio Transformer Guided by Convex Gated Probing
Feb 18, 2026Houtan Ghaffari, Lukas Rauch, Christoph Scholz +1Audio Representation Learning Self-Supervised Pre-Training
Ghent University · University of Kassel · Now at Earth Species Project +1
AFFMAE: Scalable Vision Pre-Training for High-Resolution Microscopy Segmentation on Desktop Hardware
Feb 18, 2026David Smerkous, Zian Wang, Behzad NajafianMasked Autoencoders Self-Supervised Pre-Training
Department of Laboratory Medicine & Pathology, University of Washington, Seattle WA, USA · Electrical Engineering & Computer Science, Oregon State University, Corvallis, OR, USA · Paul G. Allen School of Computer Science & Engineering, University of Washington, Seattle WA, USA
Non-Contact Physiological Monitoring in Pediatric Intensive Care Units via Adaptive Masking and Self-Supervised Learning
Feb 17, 2026Mohamed Khalil Ben Salah, Philippe Jouvet, Rita NoumeirRemote Photoplethysmography Heart Rate Estimation
Biomedical Information Processing Laboratory, École de Technologie Supérieure, University of Quebec, Montreal, QC, Canada · Pediatric Intensive Care Unit, CHU Sainte-Justine, Montreal, QC, Canada
Vascular anatomy-aware self-supervised pre-training for X-ray angiogram analysis
Feb 12, 2026De-Xing Huang, Chaohui Yu, Xiao-Hu Zhou +8Medical Image Representation Learning Medical Imaging
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · DAMO Academy, Alibaba Group, Hangzhou 310023, China +2
RadJEPA: Radiology Encoder for Chest X-Rays via Joint Embedding Predictive Architecture
Jan 22, 2026Anas Anwarul Haq Khan, Mariam Husain, Pratik Jalan +1Radiology Report Generation Medical Imaging
Department of Computer Science (CSE), Indian Institute of Technology Bombay, India · Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · Department of Biomedical Engineering, Johns Hopkins University, USA +1
Stochastic Siamese MAE Pretraining for Longitudinal Medical Images
Dec 29, 2025Taha Emre, Arunava Chakravarty, Thomas Pinetz +9Masked Autoencoders Self-Supervised Pre-Training
Institute of Artificial Intelligence, Center for Medical Data Science, Medical University of Vienna, Austria · Department of Ophthalmology and Optometry, Medical University of Vienna, Austria · BioMedIA, Department of Computing, Imperial College London, London, United Kingdom +8
Radiance-Field Guided Pretraining: Scaling Localization Models with Unlabeled Wireless Signals
Dec 8, 2025Guosheng Wang, Shen Wang, Lei YangSelf-Supervised Pre-Training Wireless Localization
Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China
Pianist Transformer: Towards Expressive Piano Performance Rendering via Scalable Self-Supervised Pre-Training
Dec 2, 2025Hong-Jie You, Jie-Jing Shao, Xiao-Wen Yang +3Efficient Transformer Inference Self-Supervised Pre-Training
State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing 210023, China · School of Artificial Intelligence, Nanjing University, Nanjing 210023, China · School of Intelligence Science and Technology, Nanjing University, Suzhou 215163, China
Pre-train to Gain: Robust Learning Without Clean Labels
Nov 25, 2025David Szczecina, Nicholas Pellegrino, Paul FieguthSelf-Supervised Pre-Training Learning with Noisy Labels
Systems Design Engineering University of Waterloo Waterloo, Ontario, Canada
BrainPro: Process-conditioned Pretraining with Topographic Retrieval for Large EEG Models
Sep 26, 2025Yi Ding, Muyun Jiang, Weibang Jiang +6EEG Foundation Models Representation Learning
Nanyang Technological University · Shanghai Jiao Tong University · Advanced Telecommunications Research Institute International +1

Feb 18, 2026J/K move · Enter open · S save

A Self-Supervised Approach for Enhanced Feature Representations in Object Detection Tasks

Santiago C. Vilabella, Pablo Pérez-Núñez, Beatriz Remeseiro

Menéndez Pelayo International University, Spain · Artificial Intelligence Center, University of Oviedo, Gijón, Spain

Object Detection Self-Supervised Visual Representation Learning Self-Supervised Pre-Training

Open paper AskPDF arXiv

In the fast-evolving field of artificial intelligence, where models are increasingly growing in complexity and size, the availability of labeled data for training deep learning models has become a significant challenge. Addressing complex problems like object detection demands considerable time and resources for data labeling to achieve meaningful results. For companies developing such applications, this entails extensive investment in highly skilled personnel or costly outsourcing. This research work aims to demonstrate that enhancing feature extractors can substantially alleviate this challenge, enabling models to learn more effective representations with less labeled data. Utilizing a self-supervised learning strategy, we present a model trained on unlabeled data that outperforms state-of-the-art feature extractors pre-trained on ImageNet and particularly designed for object detection tasks. Moreover, the results demonstrate that our approach encourages the model to focus on the most relevant aspects of an object, thus achieving better feature representations and, therefore, reinforcing its reliability and robustness.