Contrastive Learning
Overview
Contrastive learning teaches a model which examples should have similar representations and which should be distinct, often using different views of the same item as positive pairs. Common approaches include InfoNCE methods such as SimCLR and MoCo, supervised and prototype-based losses, hard-negative mining, and CLIP-style alignment across modalities. These representations support classification, retrieval, and transfer when labels are scarce, across images, language, time series, graphs, and scientific data. A central challenge is choosing positives and negatives that capture useful invariances without treating semantically related examples as negatives or discarding information needed for downstream tasks; theory also studies representation collapse, geometry, and generalization.
Latest papers
348 papers35 papers in the last four weeks, down 12% on the four weeks before. 0.2% of all new papers.
A Width-Matched Comparison of Hybrid Quantum-Classical Self-Supervised Learning for Fingerprint Recognition
RefCon: Iterative Refinement and Contrastive Memory Extraction for Context-Evolving Agent
Optimal VC Dimension of Contrastive Learning with Margin
anchor--positive--negative'' triplets $(i,j^{+},k^{-})$, indicating that item is closer to than to .'' Despite its success, understanding why contrastive learning leads to representations of high generalization quality---beyond the often pessimistic predictions from PAC-learning---remains a central question. Recently, \citet*{alon2024optimal} proved that, for PAC-learning -dimensional Euclidean representations of -point datasets, triplets are necessary and sufficient, while they posed as an open question whether their VC dimension bounds for the more realistic setting of contrastive learning with a margin can be improved. For a margin parameter , a triplet is satisfied by the embedding , if . In this work, we resolve their question by proving that the VC dimension of contrastive learning under any margin is in fact , improving on the previous bound of . We also establish that the bounds are optimal up to constant factors, by providing a matching lower bound of (the previously known lower bound was ), for .