Self Supervised Learning
Self-supervised learning (SSL) aims to train machine learning models using unlabeled data by formulating pretext tasks that encourage the model to learn useful representations. Current research focuses on improving SSL's performance and generalization across diverse data types (images, audio, graphs, point clouds) and downstream tasks, employing techniques like contrastive learning, masked autoencoders, and generative models within various architectures such as transformers and convolutional neural networks. These advancements are significant because they reduce the reliance on expensive and time-consuming data labeling, enabling the development of robust models for applications ranging from medical image analysis and speech recognition to geospatial AI and protein function prediction. The efficiency gains from SSL are also a key focus, with research exploring optimal model and data sizes for given computational budgets.
Papers
SSTFB: Leveraging self-supervised pretext learning and temporal self-attention with feature branching for real-time video polyp segmentation
Ziang Xu, Jens Rittscher, Sharib Ali
Self-Supervised and Few-Shot Learning for Robust Bioaerosol Monitoring
Adrian Willi, Pascal Baumann, Sophie Erb, Fabian Gröger, Yanick Zeder, Simone Lionetti
POWN: Prototypical Open-World Node Classification
Marcel Hoffmann, Lukas Galke, Ansgar Scherp
ML-SUPERB 2.0: Benchmarking Multilingual Speech Models Across Modeling Constraints, Languages, and Datasets
Jiatong Shi, Shih-Heng Wang, William Chen, Martijn Bartelds, Vanya Bannihatti Kumar, Jinchuan Tian, Xuankai Chang, Dan Jurafsky, Karen Livescu, Hung-yi Lee, Shinji Watanabe
Self-supervised Learning of Neural Implicit Feature Fields for Camera Pose Refinement
Maxime Pietrantoni, Gabriela Csurka, Martin Humenberger, Torsten Sattler
SCDNet: Self-supervised Learning Feature-based Speaker Change Detection
Yue Li, Xinsheng Wang, Li Zhang, Lei Xie
A deep cut into Split Federated Self-supervised Learning
Marcin Przewięźlikowski, Marcin Osial, Bartosz Zieliński, Marek Śmieja
SimSAM: Simple Siamese Representations Based Semantic Affinity Matrix for Unsupervised Image Segmentation
Chanda Grover Kamra, Indra Deep Mastan, Nitin Kumar, Debayan Gupta
Exploring Self-Supervised Multi-view Contrastive Learning for Speech Emotion Recognition with Limited Annotations
Bulat Khaertdinov, Pedro Jeuris, Annanda Sousa, Enrique Hortal
GenDistiller: Distilling Pre-trained Language Models based on an Autoregressive Generative Model
Yingying Gao, Shilei Zhang, Chao Deng, Junlan Feng