Self Supervised Learning

Self-supervised learning (SSL) aims to train machine learning models using unlabeled data by formulating pretext tasks that encourage the model to learn useful representations. Current research focuses on improving SSL's performance and generalization across diverse data types (images, audio, graphs, point clouds) and downstream tasks, employing techniques like contrastive learning, masked autoencoders, and generative models within various architectures such as transformers and convolutional neural networks. These advancements are significant because they reduce the reliance on expensive and time-consuming data labeling, enabling the development of robust models for applications ranging from medical image analysis and speech recognition to geospatial AI and protein function prediction. The efficiency gains from SSL are also a key focus, with research exploring optimal model and data sizes for given computational budgets.

Papers

December 21, 2022

December 20, 2022

Exploring Effective Fusion Algorithms for Speech Based Self-Supervised Learning Models
Changli Tang, Yujin Wang, Xie Chen, Wei-Qiang Zhang
Self Supervised Learning Automatic Speech Recognition Self Supervised Speech Processing Speaker Identification Model Fusion Fusion Algorithm

December 19, 2022

Boosting Automatic COVID-19 Detection Performance with Self-Supervised Learning and Batch Knowledge Ensembling
Guang Li, Ren Togo, Takahiro Ogawa, Miki Haseyama
Self Supervised Learning Covid 19 Chest X Ray Model Ensembling COVID 19 Chest X Ray

December 16, 2022

Toward Improved Generalization: Meta Transfer of Self-supervised Knowledge on Graphs
Wenhui Cui, Haleh Akrami, Anand A. Joshi, Richard M. Leahy
Convolutional Neural Network Self Supervised Learning Graph Drawing Improved Generalization Self Supervised Task Meta Transfer

December 14, 2022

Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Language
Alexei Baevski, Arun Babu, Wei-Ning Hsu, Michael Auli
Self Supervised Learning Human Language Speech Analysis Vision Paper Masked Autoencoders Contextualized Target Representation Convolutional Decoder

December 13, 2022

Semantics-Consistent Feature Search for Self-Supervised Visual Representation Learning
Kaiyou Song, Shan Zhang, Zihao An, Zimeng Luo, Tong Wang, Jin Xie
Self Supervised Learning Contrastive Self Supervised Learning Semantic Search Self Supervised Visual Representation Feature Augmentation

December 12, 2022

December 11, 2022

Accelerating Self-Supervised Learning via Efficient Training Strategies
Mustafa Taha Koçyiğit, Timothy M. Hospedales, Hakan Bilen
Self Supervised Learning Self Supervised Training Strategy

December 7, 2022

December 6, 2022

December 5, 2022

MAP-Music2Vec: A Simple and Effective Baseline for Self-Supervised Music Audio Representation Learning
Yizhi Li, Ruibin Yuan, Ge Zhang, Yinghao Ma, Chenghua Lin, Xingran Chen, Anton Ragni, Hanzhi Yin, Zhijie Hu, Haoyu He, Emmanouil Benetos, Norbert Gyenge, Ruibo Liu, Jie Fu
Self Supervised Learning Self Supervised Effective Baseline Deep Learning Community

December 4, 2022

Joint Self-Supervised Image-Volume Representation Learning with Intra-Inter Contrastive Clustering
Duy M. H. Nguyen, Hoang Nguyen, Mai T. N. Truong, Tri Cao, Binh T. Nguyen, Nhat Ho, Paul Swoboda, Shadi Albarqouni, Pengtao Xie, Daniel Sonntag
Self Supervised Learning Joint Framework 3D Brain Large Scale Medical Contrastive Clustering Volume Representation Unsupervised Joint

December 1, 2022

CL4CTR: A Contrastive Learning Framework for CTR Prediction
Fangye Wang, Yingxu Wang, Dongsheng Li, Hansu Gu, Tun Lu, Peng Zhang, Ning Gu
Self Supervised Learning Contrastive Loss Click Through Rate Prediction Contrastive Learning Framework Feature Representation Learning Model Agnostic Contrastive

November 29, 2022

Survey on Self-Supervised Multimodal Representation Learning and Foundation Models
Sushil Thapa
Deep Learning Timely Survey Self Supervised Learning Foundation Model Multimodal Learning Multimodal Agent Multimodal Self Supervised Learning

Self Supervised Learning

Papers

Joint Embedding of 2D and 3D Networks for Medical Image Anomaly Detection

Continual Contrastive Finetuning Improves Low-Resource Relation Extraction

Exploring Effective Fusion Algorithms for Speech Based Self-Supervised Learning Models

Boosting Automatic COVID-19 Detection Performance with Self-Supervised Learning and Batch Knowledge Ensembling

Toward Improved Generalization: Meta Transfer of Self-supervised Knowledge on Graphs

Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Language

Semantics-Consistent Feature Search for Self-Supervised Visual Representation Learning

CbwLoss: Constrained Bidirectional Weighted Loss for Self-supervised Learning of Depth and Pose

TriNet: stabilizing self-supervised learning from complete or slow collapse on ASR

Accelerating Self-Supervised Learning via Efficient Training Strategies

Self-Supervised PPG Representation Learning Shows High Inter-Subject Variability

Spatio-Temporal Self-Supervised Learning for Traffic Flow Prediction

Pre-trained Encoders in Self-Supervised Learning Improve Secure and Privacy-preserving Supervised Learning

Giga-SSL: Self-Supervised Learning for Gigapixel Images

Label-free Knowledge Distillation with Contrastive Loss for Light-weight Speaker Recognition

Self-supervised Graph Representation Learning for Black Market Account Detection

MAP-Music2Vec: A Simple and Effective Baseline for Self-Supervised Music Audio Representation Learning

Joint Self-Supervised Image-Volume Representation Learning with Intra-Inter Contrastive Clustering

CL4CTR: A Contrastive Learning Framework for CTR Prediction

Survey on Self-Supervised Multimodal Representation Learning and Foundation Models