Self-Supervised Learning

Self-supervised learning creates training signals from the data itself, for example by matching augmented views, predicting masked content, or using temporal, geometric, and physical consistency. Contrastive learning, masked autoencoders, teacher–student distillation, and joint-embedding predictive architectures are common approaches across images, video, audio, language, medical data, and scientific measurements. These methods can reduce dependence on costly labels and support transfer, but useful representations depend on how well the pretext signal fits the data and downstream task. A central tension is balancing invariance and non-collapse with retaining fine-grained information, while establishing reliable transfer under domain shifts and limited compute remains an open challenge.