Predictive Representation Learning

Predictive representation learning studies how models can build useful internal states by predicting masked, future, or cross-view representations rather than reproducing every input detail. Common approaches include joint-embedding predictive architectures such as JEPA, contrastive predictive coding, masked latent prediction, and models that forecast temporal dynamics; they are used in vision, speech, health data, and robotics. Researchers evaluate learned features with transfer tasks, linear probes, forecasting benchmarks, and measures of control or planning performance. A central open tension is that predictability alone may discard rare but important information, so preventing collapse and aligning representations with causal, clinical, or action-relevant structure remain key challenges.