Supervised Contrastive Learning

Supervised contrastive learning (SCL) is a machine learning technique aiming to learn robust and discriminative feature representations by pulling together similar data points (positives) and pushing apart dissimilar ones (negatives), all guided by labeled data. Current research focuses on applying SCL to diverse tasks, including image classification, natural language processing, and time-series analysis, often incorporating techniques like data augmentation and novel loss functions to address challenges such as imbalanced datasets and label noise. The effectiveness of SCL in improving model performance and generalizability across various domains makes it a significant area of research with broad implications for numerous applications, from medical image analysis to fraud detection.

Papers

December 22, 2023

Joint Self-Supervised and Supervised Contrastive Learning for Multimodal MRI Data: Towards Predicting Abnormal Neurodevelopment
Zhiyuan Li, Hailong Li, Anca L. Ralescu, Jonathan R. Dillman, Mekibib Altaye, Kim M. Cecil, Nehal A. Parikh, Lili He
Multimodal Data Functional Magnetic Resonance Imaging Supervised Contrastive Learning Multimodal Feature Multimodal Deep Learning Neurodevelopmental Disorder

December 12, 2023

Transformer-based No-Reference Image Quality Assessment via Supervised Contrastive Learning
Jinsong Shi, Pan Gao, Jie Qin
Image Quality Assessment Supervised Contrastive Learning Reference Image Quality Assessment Image Quality Score

December 10, 2023

Temporal Supervised Contrastive Learning for Modeling Patient Risk Progression
Shahriar Noroozizadeh, Jeremy C. Weiss, George H. Chen
Contrastive Learning Tabular Data Supervised Contrastive Learning Patient Trajectory ADNI Dataset

November 25, 2023

Elucidating and Overcoming the Challenges of Label Noise in Supervised Contrastive Learning
Zijun Long, George Killick, Lipeng Zhuang, Richard McCreadie, Gerardo Aragon Camarasa, Paul Henderson
Technical Challenge Supervised Learning Label Noise Supervised Contrastive Learning Low Complexity Explanation Labeling Error

November 21, 2023

A Supervised Contrastive Learning Pretrain-Finetune Approach for Time Series
Trang H. Tran, Lam M. Nguyen, Kyongmin Yeo, Nam Nguyen, Roman Vaculin
Contrastive Learning Time Series Pre Training Supervised Contrastive Learning Finetune Paradigm Learned Dynamic Similarity Metric Probabilistic Metric

November 9, 2023

Hard-Negative Sampling for Contrastive Learning: Optimal Representation Geometry and Neural- vs Dimensional-Collapse
Ruijie Jiang, Thuan Nguyen, Shuchin Aeron, Prakash Ishwar
Contrastive Learning Negative Sampling Neural Collapse Supervised Contrastive Learning Adam Optimizer unsupeRvised Contrastive Learning Feature Normalization Dimensional Collapse

October 29, 2023

Retrofitting Light-weight Language Models for Emotions using Supervised Contrastive Learning
Sapan Shah, Sreedhar Reddy, Pushpak Bhattacharyya
Language Model Pre Trained Language Model Sentiment Analysis Experienced Emotion Target Emotion Supervised Contrastive Learning Lightweight Large Language Model

October 25, 2023

SSLCL: An Efficient Model-Agnostic Supervised Contrastive Learning Framework for Emotion Recognition in Conversations
Tao Shi, Xiao Liang, Yaoyuan Liang, Xinyi Tong, Shao-Lun Huang
Emotion Recognition Community Conversation Supervised Contrastive Learning Label Embeddings Model Agnostic Contrastive

September 28, 2023

Mixup Your Own Pairs
Yilei Wu, Zijian Dong, Chongyao Chen, Wangchunshu Zhou, Juan Helen Zhou
Contrastive Learning Representation Learning Supervised Contrastive Learning Spatio Temporal Mixup Mechanism Contrastive Pair Contrastive Regression

September 27, 2023

Transferability of Representations Learned using Supervised Contrastive Learning Trained on a Multi-Domain Dataset
Alvin De Jun Tan, Clement Tan, Chai Kiat Yeo
Contrastive Learning Meaningful Representation Cross Entropy Loss Task Transferability Multi Domain Supervised Contrastive Learning

September 21, 2023

BitCoin: Bidirectional Tagging and Supervised Contrastive Learning based Joint Relational Triple Extraction Framework
Luyao He, Zhongbao Zhang, Sen Su, Yuxin Chen
Supervised Contrastive Learning Relational Triple Bitcoin User Relational Triple Extraction Triple Extraction

September 2, 2023

Pretraining Representations for Bioacoustic Few-shot Detection using Supervised Contrastive Learning
Ilyass Moummad, Romain Serizel, Nicolas Farrugia
Supervised Contrastive Learning Pre Trained Representation Sound Event Detection Bioacoustic Research

August 28, 2023

Joint Multiple Intent Detection and Slot Filling with Supervised Contrastive Learning and Self-Distillation
Nguyen Anh Tu, Hoang Thi Thu Uyen, Tu Minh Phuong, Ngo Xuan Bach
Contrastive Learning Spoken Language Understanding Intent Detection Self Distillation Supervised Contrastive Learning Slot Filling Intent Assurance Joint Intent Detection

August 24, 2023

FaceTouch: Detecting hand-to-face touch with supervised contrastive learning to assist in tracing infectious disease
Mohamed R. Ibrahim, Terry Lyons
Vision Model Supervised Contrastive Learning Infectious Disease Face Touch

August 21, 2023

SupEuclid: Extremely Simple, High Quality OoD Detection with Supervised Contrastive Learning and Euclidean Distance
Jarrod Haas
Distribution Detection Distribution Data Supervised Contrastive Learning Euclidean Distance Distribution Detection Benchmark Larger OCTOPUS

August 19, 2023

Robust Fraud Detection via Supervised Contrastive Learning
Vinay M. S., Shuhan Yuan, Xintao Wu
Contrastive Learning Open Set Supervised Contrastive Learning

July 23, 2023

ASCON: Anatomy-aware Supervised Contrastive Learning Framework for Low-dose CT Denoising
Zhihao Chen, Qi Gao, Yi Zhang, Hongming Shan
Supervised Contrastive Learning Low Dose Low Dose Computed Tomography Normal Dose

July 14, 2023

Gastrointestinal Disease Classification through Explainable and Cost-Sensitive Deep Neural Networks with Supervised Contrastive Learning
Dibya Nath, G. M. Shahariar
Supervised Contrastive Learning Disease Classification Cost Sensitive Gastrointestinal Tract

June 21, 2023

What Constitutes Good Contrastive Learning in Time-Series Forecasting?
Chiyu Zhang, Qi Yan, Lili Meng, Tristan Sylvain
Contrastive Learning Time Series Forecasting Supervised Contrastive Learning Self Supervised Contrastive Learning

May 20, 2023

Mitigating Catastrophic Forgetting in Task-Incremental Continual Learning with Adaptive Classification Criterion
Yun Luo, Xiaotian Lin, Zhen Yang, Fandong Meng, Jie Zhou, Yue Zhang
Continual LEArning Catastrophic Forgetting Contrastive Loss Supervised Contrastive Learning Task Learning Adaptive Classification

Supervised Contrastive Learning

Papers

Joint Self-Supervised and Supervised Contrastive Learning for Multimodal MRI Data: Towards Predicting Abnormal Neurodevelopment

Transformer-based No-Reference Image Quality Assessment via Supervised Contrastive Learning

Temporal Supervised Contrastive Learning for Modeling Patient Risk Progression

Elucidating and Overcoming the Challenges of Label Noise in Supervised Contrastive Learning

A Supervised Contrastive Learning Pretrain-Finetune Approach for Time Series

Hard-Negative Sampling for Contrastive Learning: Optimal Representation Geometry and Neural- vs Dimensional-Collapse

Retrofitting Light-weight Language Models for Emotions using Supervised Contrastive Learning

SSLCL: An Efficient Model-Agnostic Supervised Contrastive Learning Framework for Emotion Recognition in Conversations

Mixup Your Own Pairs

Transferability of Representations Learned using Supervised Contrastive Learning Trained on a Multi-Domain Dataset

BitCoin: Bidirectional Tagging and Supervised Contrastive Learning based Joint Relational Triple Extraction Framework

Pretraining Representations for Bioacoustic Few-shot Detection using Supervised Contrastive Learning

Joint Multiple Intent Detection and Slot Filling with Supervised Contrastive Learning and Self-Distillation

FaceTouch: Detecting hand-to-face touch with supervised contrastive learning to assist in tracing infectious disease

SupEuclid: Extremely Simple, High Quality OoD Detection with Supervised Contrastive Learning and Euclidean Distance

Robust Fraud Detection via Supervised Contrastive Learning

ASCON: Anatomy-aware Supervised Contrastive Learning Framework for Low-dose CT Denoising

Gastrointestinal Disease Classification through Explainable and Cost-Sensitive Deep Neural Networks with Supervised Contrastive Learning

What Constitutes Good Contrastive Learning in Time-Series Forecasting?

Mitigating Catastrophic Forgetting in Task-Incremental Continual Learning with Adaptive Classification Criterion