Multimodal Contrastive Learning

Overview

Multimodal contrastive learning trains encoders to bring related examples from different modalities closer in an embedding space and separate unrelated ones, supporting retrieval, classification, and transfer. Common approaches include CLIP-style dual encoders with contrastive losses, fine-grained local or hierarchical alignment, and objectives that combine contrastive learning with reconstruction or captioning. Work increasingly addresses noisy or incomplete pairs, false negatives, long-tail data, and the need to preserve modality-specific information alongside shared semantics. A central tension is that stronger alignment can improve cross-modal matching but may also erase useful differences between modalities or reduce robustness and downstream transfer.

Latest papers

CardsList

262 papers16 papers in the last four weeks, down 48% on the four weeks before. 0.1% of all new papers.

  1. AggCLIP: Aggregating CLIP for Zero-Shot Multimodal Anomaly Detection

    Nov 18, 2026Zhi Li, Hanyu Wang, Fei Shen +1Multimodal Anomaly DetectionZero-Shot Anomaly Detection

  2. Learning What Pairs Miss for Higher-Order Multimodal Alignment

    Nov 16, 2026Kunyu Zhang, Shenghui Feng, Xiangjie Shi +2Multimodal Contrastive LearningContrastive Learning

  3. Attribution-Guided Shared-Private Decoupling for Noise-Reduced Audio-Visual Representation Learning

    Nov 14, 2026Linge Wang, Yingying Chen, Bingke Zhu +4Audio-Visual Representation LearningShared Representation Learning

  4. CroCoMAE: Cross-modal self-supervised collaborative representation learning for SAR-optical remote sensing data

    Nov 10, 2026Vito Recchia, Giuseppe Guarino, Cássio Fraga Dantas +3Cross-Modal Representation LearningMultimodal SSL

  5. LUMIR: Decoupled Multimodal Representation Learning from Heterogeneous Data for Next-POI Recommendation

    Nov 10, 2026Zehui Wang, Wolfram Höpken, Dietmar JannachMultimodal RecommendationCross-Modal Representation Learning

  6. Complementarity-Driven Dual-Teacher Hierarchical Alignment for Zero-Shot EEG-to-Image Retrieval

    Nov 9, 2026Lukun Wu, Jie Li, Ziqi Ren +3EEG-Based Visual DecodingCross-Modal Alignment

  7. UniOMA: Unified Optimal-Transport Multi-Modal Structural Alignment for Robot Perception

    Nov 9, 2026Xinrui Zu, Mark Hoogendoorn, Kevin Sebastian Luck +1Cross-Modal AlignmentRobotic Perception

  8. LAION-Audio-XL: A Web-Scale Corpus and Scaling Studies for Contrastive Language-Audio Pretraining

    Nov 8, 2026Gijs Wijngaard, Felix Förster, Ross Wightman +4Audio-Language PretrainingAudio-Language Models

  9. Permute-then-Adapt: Weak-to-Strong Contrastive Image-Text Adaptation

    Nov 7, 2026Jinhao Li, Sarah Erfani, Lei Feng +3VLM AdaptationVision-Language Alignment

  10. FusionCLIP: Multimodal Contrastive Pretraining for Fusion Plasma Diagnostics

    Nov 6, 2026Qiang Chen, Xiao Wang, Qingquan Yang +4Multimodal PretrainingMultimodal Contrastive Learning

  11. CAPE-JEPA - Cross-modal Adversarial Probabilistic Embedding JEPA

    Nov 4, 2026Aniket Junghare, Biplab BanerjeeCross-Modal Representation LearningCross-Modal Retrieval

  12. LoCA: Enabling Multi-Modality in Geospatial Location Encoders

    Oct 23, 2026Panagiotis Koromilas, Eleanna Vali, Diogenis Antonopoulos +7Geospatial Representation LearningMultimodal Contrastive Learning

  13. Closing the Modality Gap via Simplex-Constrained Representations

    Oct 23, 2026Shubham Gupta, Siva Reddy, Perouz Taslakian +2Modality Gap in VLMsCross-Modal Alignment

  14. GeoLEON-CLIP: Graph-Anchored Multimodal Contrastive Learning for Geographic Region Representation

    Oct 23, 2026Radoslaw Malek, Szymon Soltysiak, Aleksandra Kawala-Sterniuk +1Geospatial Representation LearningMultimodal Contrastive Learning

  15. UNIVERS: Learning UNIversal Voxel-wise Embedding via Randomized Synthesis for generalizable one-shot landmark localization

    Oct 21, 2026Derong Yu, Qingsong Zhao, Yanbing Yue +3Anatomical Landmark DetectionMedical Image Representation Learning

  16. CIM-Embed: Learning Instruction-Conditioned Multimodal Embeddings for Clustering

    Oct 15, 2026Yinghua Xu, Pengyang Xie, Yanxiang Zhou +4ClusteringMultimodal Embedding

  17. Semantic Alignment of EEG and Text: A Contrastive Learning Framework for Decoding Imagined Speech

    Oct 14, 2026Shahryar G. Namdari, Francesco Iacomi, Riccardo Barbieri +1EEG DecodingEEG-to-Text Decoding

  18. ACR-Net: Mitigating Semantic Dominance via Contrastive Acoustic-Semantic Decoupling

    Oct 14, 2026Mengke Zhang, Yanda Shao, Tianhe Wu +1Speech Emotion RecognitionAudio-Language Model Evaluation

  19. Revisiting Triplet Contrastive Learning for 3D Shape Understanding

    Oct 14, 2026Chaoyang Zhu, Zhe Liao, Jiayi Zhang +33D Representation LearningCross-Modal Alignment

  20. MicroCLIP: Connecting Text and Microscopy Images via Contrastive Language--Image Pretraining

    Oct 13, 2026Yanran Ruan, Zhixiang Wei, Shiyao Hong +5Fluorescence MicroscopyMicroscopy Image Analysis

  21. Multimodal Contrastive Representation Learning Based Voice Conversion for Electrolaryngeal Speech Enhancement

    Oct 13, 2026Ding Ma, Jinyi Mi, Jiajun He +3Voice ConversionSpeech Enhancement

  22. Beyond Sparse Captions: Aligning Slide-Level Text and Patch-Level Vision in Pathology

    Oct 12, 2026Daniel Shao, Dongmin Bang, Luca L. Weishaupt +6Computational PathologyMedical VLMs

  23. Approximation in Contrastive Representation Learning

    Oct 11, 2026Yuanfan Li, Zihan Zhang, Yiming Ying +1Contrastive LearningMultimodal Contrastive Learning

  24. What Really Works for Universal Multimodal Embeddings? A Controlled Study of Contrastive Training Strategies

    Oct 10, 2026Duo Li, Haojun Pan, Zuhao Yang +4Multimodal EmbeddingMultimodal Contrastive Learning

  25. Prior-Fitted and Frozen Scorers for Multimodal Contrastive Learning

    Oct 10, 2026Tillmann Rheude, Roland Eils, Benjamin WildMultimodal Contrastive LearningMultimodal Learning