Multimodal Contrastive Learning

Multimodal contrastive learning trains encoders to bring related examples from different modalities closer in an embedding space and separate unrelated ones, supporting retrieval, classification, and transfer. Common approaches include CLIP-style dual encoders with contrastive losses, fine-grained local or hierarchical alignment, and objectives that combine contrastive learning with reconstruction or captioning. Work increasingly addresses noisy or incomplete pairs, false negatives, long-tail data, and the need to preserve modality-specific information alongside shared semantics. A central tension is that stronger alignment can improve cross-modal matching but may also erase useful differences between modalities or reduce robustness and downstream transfer.