Supervised Contrastive Loss

Supervised contrastive loss is a machine learning technique that improves model performance by learning representations where data points of the same class are closer together than those of different classes. Current research focuses on enhancing its robustness to noisy labels, long-tailed data distributions, and class imbalances, often integrating it with other loss functions or employing techniques like data augmentation and class-aware attention mechanisms within various architectures, including autoencoders, graph convolutional networks, and transformers. This approach has shown significant improvements in diverse applications such as image classification, speech emotion recognition, and recommendation systems, highlighting its value for learning effective and generalizable representations from complex data.

Papers

September 15, 2024

Towards Multi-view Graph Anomaly Detection with Similarity-Guided Contrastive Clustering
Lecheng Zheng, John R. Birge, Yifang Zhang, Jingrui He
Contrastive Loss Contrastive Method Graph Anomaly Detection Supervised Contrastive Loss Contrastive Clustering

July 23, 2024

Distribution-Aware Robust Learning from Long-Tailed Data with Noisy Labels
Jae Soon Baik, In Young Yoon, Kun Hoon Kim, Jun Won Choi
Noisy Label Long Tailed Data Distributionally Robust Supervised Contrastive Loss Class Centroid Confidence Aware Contrastive

June 10, 2024

Taxes Are All You Need: Integration of Taxonomical Hierarchy Relationships into the Contrastive Loss
Kiran Kokilepersaud, Yavuz Yarici, Mohit Prabhushankar, Ghassan AlRegib
Representation Learning Contrastive Loss Representation Space Supervised Contrastive Loss Taxonomic Classification Local Income Tax

May 31, 2024

Textual Inversion and Self-supervised Refinement for Radiology Report Generation
Yuanjiang Luo, Hongxiang Li, Xuan Wu, Meng Cao, Xiaoshuang Huang, Zhihong Zhu, Peixi Liao, Hu Chen, Yi Zhang
Radiology Report Generation Supervised Contrastive Loss Textual Inversion

April 24, 2024

Mixed Supervised Graph Contrastive Learning for Recommendation
Weizhi Zhang, Liangwei Yang, Zihe Song, Henry Peng Zou, Ke Xu, Yuanjie Zhu, Philip S. Yu
Contrastive Learning Effective Recommendation Graph Contrastive Learning Supervised Contrastive Loss Pairwise Loss Item Based Collaborative Filtering

April 23, 2024

Additive Margin in Contrastive Self-Supervised Frameworks to Learn Discriminative Speaker Representations
Theo Lepage, Reda Dehak
Self Supervised Learning Contrastive Loss Speaker Verification Margin Maximization Contrastive Self Supervised Learning Supervised Contrastive Loss Discriminative Speaker

March 27, 2024

OrCo: Towards Better Generalization via Orthogonality and Contrast for Few-Shot Class-Incremental Learning
Noor Ahmed, Anna Kukleva, Bernt Schiele
Contrastive Learning Class Incremental Learning Representation Space Better Generalization Post Hoc Orthogonalization Supervised Contrastive Loss Mini ImageNet

March 18, 2024

Investigating the Benefits of Projection Head for Representation Learning
Yihao Xue, Eric Gan, Jiayi Ni, Siddharth Joshi, Baharan Mirzasoleiman
Representation Learning Complementary Benefit Feature Space Supervised Contrastive Learning Supervised Contrastive Loss High Quality Representation Feature Weighting Projection Head

March 10, 2024

Decoupled Contrastive Learning for Long-Tailed Recognition
Shiyu Xuan, Shiliang Zhang
Contrastive Learning ImageNet Dataset Visual Representation Learning Long Tailed Recognition Supervised Contrastive Loss

December 4, 2023

Long-Tail Learning with Rebalanced Contrastive Loss
Charika De Alvis, Dishanika Denipitiyage, Suranga Seneviratne
Contrastive Learning Contrastive Loss Long Tailed Learning Supervised Contrastive Loss

October 2, 2023

Engineering the Neural Collapse Geometry of Supervised-Contrastive Loss
Jaidev Gill, Vala Vakilian, Christos Thrampoulidis
Jina Embeddings Contrastive Loss Feature Embeddings Position Engineering Supervised Contrastive Loss Learned Embeddings

August 31, 2023

Supervised Contrastive Learning with Nearest Neighbor Search for Speech Emotion Recognition
Xuechen Wang, Shiwan Zhao, Yong Qin
Contrastive Learning Loss Function Speech Emotion Recognition Cross Entropy Loss Nearest Neighbor Supervised Contrastive Loss

July 31, 2023

Contrastive Learning for API Aspect Analysis
G. M. Shahariar, Tahmid Hasan, Anindya Iqbal, Gias Uddin
Contrastive Learning API Usage Aspect Term Extraction Supervised Contrastive Loss

June 26, 2023

Histopathology Image Classification using Deep Manifold Contrastive Learning
Jing Wei Tan, Won-Ki Jeong
Contrastive Learning Image Classification Manifold Learning Supervised Contrastive Loss Geodesic Distance

June 13, 2023

Symmetric Neural-Collapse Representations with Supervised Contrastive Loss: The Impact of ReLU and Batching
Ganesh Ramachandra Kini, Vala Vakilian, Tina Behnia, Jaidev Gill, Christos Thrampoulidis
Global Impact Contrastive Loss Cross Entropy Loss ReLU Layer Supervised Contrastive Loss Rank Collapse Representational Geometry

June 4, 2023

Towards Robust Feature Learning with t-vFM Similarity for Continual Learning
Bilan Gao, YoungBin Kim
Continual LEArning Contrastive Loss Robust Representation Supervised Contrastive Loss Similarity Score Similarity Metric Robust Feature

May 24, 2023

Clinically Labeled Contrastive Learning for OCT Biomarker Classification
Kiran Kokilepersaud, Stephanie Trejo Corona, Mohit Prabhushankar, Ghassan AlRegib, Charles Wykoff
Contrastive Learning Contrastive Loss Optical Coherence Tomography Biomarker Selection Supervised Contrastive Loss Clinical Language Patient Level Label

April 17, 2023

DisCo-CLIP: A Distributed Contrastive Loss for Memory Efficient CLIP Training
Yihao Chen, Xianbiao Qi, Jianan Wang, Lei Zhang
Contrastive Loss Contrastive Training Supervised Contrastive Loss Adaptive Clipping

January 27, 2023

Bayesian Self-Supervised Contrastive Learning
Bin Liu, Bang Wang, Tianrui Li
Contrastive Learning Self Supervised Negative Sample Contrastive Self Supervised Learning Supervised Contrastive Loss

January 14, 2023

FedSSC: Shared Supervised-Contrastive Federated Learning
Sirui Hu, Ling Feng, Xiaohan Yang, Yongchao Chen
Contrastive Learning Feature Space Supervised Contrastive Loss Federated Contrastive Learning