Self Supervised Contrastive Learning

Self-supervised contrastive learning aims to learn robust feature representations from unlabeled data by contrasting similar and dissimilar data points. Current research focuses on improving the efficiency and effectiveness of this learning process, exploring techniques like synthetic hard negative generation, novel loss functions (e.g., incorporating local alignment or f-divergences), and adaptive batch processing to enhance representation quality. This approach has shown significant promise across diverse applications, including image classification, video analysis, medical image segmentation, and time series forecasting, by reducing the reliance on large labeled datasets and improving model generalizability.

Papers

June 7, 2023

On the Generalization of Multi-modal Contrastive Learning
Qi Zhang, Yifei Wang, Yisen Wang
Contrastive Learning Strong Generalization Multimodal Information Language Pair Self Supervised Contrastive Learning Modality Pair

May 30, 2023

A Computational Account Of Self-Supervised Visual Learning From Egocentric Object Play
Deepayan Sanyal, Joel Michelson, Yuan Yang, James Ainooson, Maithilee Kunda
Self Supervised Egocentric Video Self Supervised Contrastive Learning Visual Learning Similar Representation Robust Feature Computational Account

May 5, 2023

AmGCL: Feature Imputation of Attribute Missing Graph via Self-supervised Contrastive Learning
Xiaochuan Zhang, Mengran Li, Ye Wang, Haojun Fei
Graph Contrastive Learning Self Supervised Contrastive Learning Tabular Data Imputation Attribute Graph

April 26, 2023

Tissue Classification During Needle Insertion Using Self-Supervised Contrastive Learning and Optical Coherence Tomography
Debayan Bhattacharya, Sarah Latus, Finn Behrendt, Florin Thimm, Dennis Eggert, Christian Betz, Alexander Schlaefer
Deep Neural Network Optical Coherence Tomography Self Supervised Contrastive Learning Needle Insertion Tissue Classification

April 12, 2023

Learning Transferable Pedestrian Representation from Multimodal Information Supervision
Liping Bao, Longhui Wei, Xiaoyu Qiu, Wengang Zhou, Houqiang Li, Qi Tian
Contrastive Learning Person Re Identification Self Supervised Contrastive Learning Reference Dataset Multimodal Supervision Pedestrian Representation

March 24, 2023

Best of Both Worlds: Multimodal Contrastive Learning with Tabular and Imaging Data
Paul Hager, Martin J. Menten, Daniel Rueckert
Contrastive Learning World Event Table Semantics Self Supervised Contrastive Learning Supervised Contrastive Learning Multimodal Pre Manual Morphometry Imaging Data

March 20, 2023

Exploring Representation Learning for Small-Footprint Keyword Spotting
Fan Cui, Liyong Guo, Quandong Wang, Peng Gao, Yujun Wang
Representation Learning Contrastive Loss Keyword Spotting Self Supervised Contrastive Learning Contrastive Siamese

March 2, 2023

ArCL: Enhancing Contrastive Learning with Augmentation-Robust Representations
Xuyang Zhao, Tianqi Du, Yisen Wang, Jun Yao, Weiran Huang
Contrastive Learning Self Supervised Learning Self Supervised Contrastive Learning Representation Augmentation

February 18, 2023

Data-Efficient Contrastive Self-supervised Learning: Most Beneficial Examples for Supervised Learning Contribute the Least
Siddharth Joshi, Baharan Mirzasoleiman
Contrastive Learning Self Supervised Supervised Learning Self Supervised Contrastive Learning High Quality Representation Positive Example

January 31, 2023

Affinity Uncertainty-based Hard Negative Mining in Graph Contrastive Learning
Chaoxi Niu, Guansong Pang, Ling Chen
Adversarial Attack Self Supervised Contrastive Learning Hard Negative Mining

January 27, 2023

Understanding Self-Supervised Pretraining with Part-Aware Representation Learning
Jie Zhu, Jiyang Qi, Mingyu Ding, Xiaokang Chen, Ping Luo, Xinggang Wang, Wenyu Liu, Leye Wang, Jingdong Wang
Self Supervised Pretraining Object Representation Self Supervised Contrastive Learning Part Aware Self Supervised Representation

December 14, 2022

Establishing a stronger baseline for lightweight contrastive models
Wenye Lin, Yifeng Ding, Zhixiong Cao, Hai-tao Zheng
Contrastive Learning Strong Baseline Self Supervised Contrastive Learning EfficientNet Algorithm Contrastive Model

December 5, 2022

Location-Aware Self-Supervised Transformers for Semantic Segmentation
Mathilde Caron, Neil Houlsby, Cordelia Schmid
Semantic Segmentation Pixel Level Self Supervised Contrastive Learning Self Supervised Transformer Dense Pseudo Label

November 25, 2022

Copy-Pasting Coherent Depth Regions Improves Contrastive Learning for Urban-Scene Segmentation
Liang Zeng, Attila Lengyel, Nergis Tömen, Jan van Gemert
Contrastive Learning Self Supervised Contrastive Learning Contextual Representation Unsupervised Semantic Segmentation Self Supervised Depth Urban Scene Segmentation

November 15, 2022

Unsupervised Feature Clustering Improves Contrastive Representation Learning for Medical Image Segmentation
Yejia Zhang, Xinrong Hu, Nishchal Sapkota, Yiyu Shi, Danny Z. Chen
Contrastive Learning Self Supervised Medical Image Segmentation Self Supervised Contrastive Learning Contrastive Representation Learning Feature Clustering Self Supervised Instance

October 28, 2022

cRedAnno+: Annotation Exploitation in Self-Explanatory Lung Nodule Diagnosis
Jiahao Lu, Chong Yin, Kenny Erleben, Michael Bachmann Nielsen, Sune Darkner
Self Supervised Contrastive Learning Annotation Strategy Lung Nodule Cancer Prediction Semi Supervised Active Learning Scarce Annotation

October 23, 2022

Rethinking Rotation in Self-Supervised Contrastive Learning: Adaptive Positive or Negative Data Augmentation
Atsuyuki Miyai, Qing Yu, Daiki Ikami, Go Irie, Kiyoharu Aizawa
Contrastive Learning Data Augmentation Adaptive Importance Self Supervised Contrastive Learning Augmentation Method Negative Data Augmentation

October 19, 2022

QA Domain Adaptation using Hidden Space Augmentation and Self-Supervised Contrastive Adaptation
Zhenrui Yue, Huimin Zeng, Bernhard Kratzwald, Stefan Feuerriegel, Dong Wang
Contrastive Learning Self Supervised Contrastive Learning Self Supervised Domain Adaptation Self Supervised Adaptation Question Answering Domain Adaptation

October 10, 2022

Using Whole Slide Image Representations from Self-Supervised Contrastive Learning for Melanoma Concordance Regression
Sean Grullon, Vaughn Spurrier, Jiayi Zhao, Corey Chivers, Yang Jiang, Kiran Motaparthi, Michael Bonham, Julianna Ianni
Self Supervised Contrastive Learning Cancer Detection Skin Cancer Classification Slide Representation Concordance Index

October 8, 2022

Revisiting Self-Supervised Contrastive Learning for Facial Expression Recognition
Yuxuan Shu, Xiao Gu, Guang-Zhong Yang, Benny Lo
Facial Expression Facial Expression Recognition Expression Recognition Self Supervised Contrastive Learning Facial Representation

Self Supervised Contrastive Learning

Papers

On the Generalization of Multi-modal Contrastive Learning

A Computational Account Of Self-Supervised Visual Learning From Egocentric Object Play

AmGCL: Feature Imputation of Attribute Missing Graph via Self-supervised Contrastive Learning

Tissue Classification During Needle Insertion Using Self-Supervised Contrastive Learning and Optical Coherence Tomography

Learning Transferable Pedestrian Representation from Multimodal Information Supervision

Best of Both Worlds: Multimodal Contrastive Learning with Tabular and Imaging Data

Exploring Representation Learning for Small-Footprint Keyword Spotting

ArCL: Enhancing Contrastive Learning with Augmentation-Robust Representations

Data-Efficient Contrastive Self-supervised Learning: Most Beneficial Examples for Supervised Learning Contribute the Least

Affinity Uncertainty-based Hard Negative Mining in Graph Contrastive Learning

Understanding Self-Supervised Pretraining with Part-Aware Representation Learning

Establishing a stronger baseline for lightweight contrastive models

Location-Aware Self-Supervised Transformers for Semantic Segmentation

Copy-Pasting Coherent Depth Regions Improves Contrastive Learning for Urban-Scene Segmentation

Unsupervised Feature Clustering Improves Contrastive Representation Learning for Medical Image Segmentation

cRedAnno+: Annotation Exploitation in Self-Explanatory Lung Nodule Diagnosis

Rethinking Rotation in Self-Supervised Contrastive Learning: Adaptive Positive or Negative Data Augmentation

QA Domain Adaptation using Hidden Space Augmentation and Self-Supervised Contrastive Adaptation

Using Whole Slide Image Representations from Self-Supervised Contrastive Learning for Melanoma Concordance Regression

Revisiting Self-Supervised Contrastive Learning for Facial Expression Recognition