Multimodal Contrastive Learning

Follow this topic Ask about this topic

Momentum

16 papers in the last four weeks, up 45% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 153

All topics

Dual-Space Modality Consistency Learning for Universal Cross-Modal Re-Identification
Aug 7, 2026Yujian Zhao, Yukang Zhao, Hankun Liu +4Cross-Modal Alignment Multimodal Contrastive Learning
When Modalities Fail to Tango: Conformal Backdoor Detection in Multimodal Contrastive Learning
Aug 4, 2026Yiming Chen, Kemou Li, Haiwei Wu +1Backdoor Detection Multimodal Contrastive Learning
State Key Laboratory of Internet of Things for Smart City, University of Macau · School of Computer Science and Engineering, University of Electronic Science and Technology of China
Bayesian Data Reweighting Improves Multimodal Retrieval for Knowledge-Based Visual Question Answering
Aug 3, 2026Jingchen Sun, Shaobo Han, Ruiyi Zhang +5Multimodal IR Contrastive Learning
University at Buffalo · NEC Laboratories America · Adobe Research +2
Douyin Multimodal Embedding Model Technical Report
Aug 3, 2026Haonan Chen, Chu Li, Zhicheng Wang +4Multimodal IR Multimodal Pretraining
ByteDance Douyin Search Multimodal Team · Renmin University of China GSAI
Discriminative Axis, Not Data Volume: What a Contrastive Corpus Teaches an Audio Embedding
Aug 3, 2026Abdul Basit TonmoyContrastive Learning Audio Representation Learning
Eximius Labs · Wabash College
Logographic Character Visual Pretraining via Semantic-based Contrastive Learning
Jul 30, 2026Daqian Shi, Wei Cao, Xiaoyu Zheng +3Contrastive Learning Visual Representation Learning
Queen Mary University of London London, UK · Jilin University Changchun, China · King’s College London London, UK +1
SCALPEL: Semantic Cross-modal Alignment via LLM-Powered Encoder Learning for Medical Vision-Language Representation
Jul 29, 2026Yunzhan Fu, Enyu Bao, Xiangyu Shen +4Radiology VLMs Medical VLMs
School of Computer Science, Hangzhou Dianzi University, China
EchoBridge: Long-Tail-Aware ECG-Echocardiography Text Alignment for Echocardiography-Derived Cardiac Findings
Jul 27, 2026Xiaocheng Fang, Jieyi Cai, Guangkun Nie +9Long-Tailed Classification Cross-Modal Alignment
School of Intelligence Science and Technology, Peking University Beijing, China · State Key Laboratory of General Artificial Intelligence, Peking University · University of the Chinese Academy of Sciences Beijing, China +2
TextSLIP: Text Self-Supervised CLIP for Medical Report Generation
Jul 24, 2026Haoyu Jiang, Ziping CongRadiology Report Generation Self-Supervised Learning
Centre for Artificial Intelligence and Robotics (CAIR) · Hong Kong Institute of Science & Innovation · Chinese Academy of Sciences
Distribution-Alignment Bridge for Uncertainty-Aware Text-to-Video Retrieval
Jul 23, 2026Kyeongmo Chae, Jihoon Lee, Sangtae AhnRepresentation Learning Cross-Modal Alignment
School of Electronic and Electrical Engineering, Kyungpook National University, Daegu, South Korea
Not All Patches are Equal: Sampling Matters for Visible-Infrared Pre-Training
Jul 22, 2026Qiwei Ma, Bin Deng, Junjie Zhu +5Visual Representation Learning Self-Supervised Pre-Training
School of Artificial Intelligence and Robotics, Hunan University, Changsha, 410082, Hunan, China · Intelligent Game and Decision Lab, Beijing, 100166, Beijing, China · Yuelushan Center for Industrial Innovation, Changsha, 410082, Hunan, China
Beyond Objective Expressivity: Geometry Preservation in Multimodal Contrastive Learning
Jul 20, 2026Tillmann Rheude, Roland Eils, Benjamin WildGeometric Representation Learning Multimodal Contrastive Learning
Berlin Institute of Health, Charité - Universitätsmedizin Berlin · Department of Mathematics and Computer Science, Freie Universität Berlin · Intelligent Medicine Institute, Fudan University
COLIP-2: Olfaction-Vision-Language Embeddings
Jul 20, 2026Kordel Kade FranceMultimodal Embedding Multimodal Contrastive Learning
Scentience, Inc.
MIME: Multimodal Interactive Motion Encoder
Jul 18, 2026Addison Zucek, Prerit Gupta, Kamila Kuatova +1Text-to-Motion Generation Cross-Modal Alignment
Purdue University
Knowledge-Guided Cross-Modal Fusion for Adult-to-Pediatric ECG Transfer via Label-Conditioned Contrastive Alignment
Jul 17, 2026Xinran Liu, Yuwen Li, Hongxiang Gao +4Electrocardiography Cross-Modal Alignment
School of Instrument Science and Engineering, Southeast University, Nanjing, 210096, China · Nanjing Medical University, Nanjing, 211166, China · Zhengzhou University, Zhengzhou, 450001, China
Region-Grounded Vision-Language Learning for Detection-Guided Mammographic Lesion Classification
Jul 17, 2026Zhengbo Zhou, Jiren Li, Dooman Arefan +2Radiology VLMs Vision-Language Models
Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, USA · Department of Radiology, University of Pittsburgh, Pittsburgh, PA, USA · Department of Biomedical Informatics and Bioengineering, University of Pittsburgh, Pittsburgh, PA, USA
Multimodal Semantic-Aware Contrastive Learning For False Negative Mitigation in 3D Medical Imaging
Jul 16, 2026Sara Ketabi, Matthias W. Wagner, Cynthia Hawkins +3Contrastive Learning 3D Medical Imaging
University of Toronto · The Hospital for Sick Children · University Hospital Augsburg +1
Fine-Grained Vision-Language Pretraining with Organ-Conditioned Pattern Tokens for CT Understanding
Jul 15, 2026Guoliang You, Xiaomeng ChuMedical Image Classification Medical VLMs
University of Pennsylvania · Yale University
AspectCLIP: Optimizing CLIP Representation Space via Aspect-Guided Consistency Regularization
Jul 15, 2026Yiyang Yao, Shanglin Liu, Jianming Lv +4Vision-Language Alignment Multimodal Contrastive Learning
School of Computer Science and Technology, South China University of Technology, Guangzhou, China
Learning Anatomy-Grounded CT Vision-Language Representations with Organ-Hierarchical Report Knowledge
Jul 12, 2026Guoliang You, Hongming Li, Yuanwang Zhang +1Radiology VLMs Medical VLMs
Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA · School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, PA 19104, USA
On the modality gap and the contrastive loss in multi-modal representation learning
Jul 12, 2026Fabian Mager, Hiba Nassar, Lars Kai HansenCross-Modal Alignment Multimodal Contrastive Learning
Department of Applied Mathematics and Computer Science Technical University of Denmark
Local Multimodal Music Alignment from Global Supervision
Jul 10, 2026Irmak Bukey, Zachary Novack, Jongmin Jung +2Cross-Modal Alignment Cross-Modal Retrieval
Carnegie Mellon University · University of California San Diego · Neutune +1
MatBind: A Shared Embedding Space for Multimodal Materials Characterization
Jul 9, 2026Le Yang, Anoop K. Chandran, Jona Östreicher +8Cross-Modal Alignment Multimodal Embedding
Institute for Advanced Simulations · Institute for Advanced Simulations (IAS-9), Forschungszentrum Jülich GmbH, 52425 Jülich, Germany · Jülich Supercomputing Centre, Forschungszentrum Jülich +15
Time Imprint: Learning Time-Aware Representations in Multi-Modal Knowledge Graphs
Jul 8, 2026Pengyu Zhang, Klim Zaporojets, Congfeng Cao +2Multimodal Graph Learning Multimodal Contrastive Learning
University of Amsterdam Amsterdam, The Netherlands · Aarhus University Aarhus, Denmark
MonoIR-RS: Infrared Remote Sensing Vision-Language Learning with CLIP and VLM Adaptation
Jul 7, 2026Jiaju Han, Ma Yaqi, Yahui Chai +8Remote Sensing Image Understanding VLM Adaptation
China University of Petroleum-Beijing at Karamay, Karamay, Xinjiang, China · Guizhou University, Guiyang, China · Shenzhen Research Institute of Big Data, Shenzhen, China +1
CMDR: Contextual Multimodal Document Retrieval
Jul 7, 2026Ryota Tanaka, Taku Hasegawa, Kyosuke NishidaMultimodal IR Multimodal Embedding
Human Informatics Labs., NTT, Inc., Tokyo, Japan
KinEMbed: Decoding Kinematics from Electromyography via Cross-Modal Contrastive Learning
Jul 6, 2026Sofia Gilardini, Chenfei Ma, Kianoush NazarpourCross-Modal Representation Learning Multimodal Contrastive Learning
Department of Statistics, University of Oxford, Oxford, United Kingdom · School of Informatics, University of Edinburgh, Edinburgh, United Kingdom
Omni-Sleep: A Sleep Foundation Model via Hierarchical Contrastive Learning of CNS-ANS Dynamics
Jul 4, 2026Zhoujie Hou, Song Wang, Kexin Lou +3Sleep Stage Classification Latent Dynamics Modeling
Department of Biomedical Engineering, Southern University of Science and Technology, Shenzhen, China · Omni-Intelligence, Shenzhen, China · Shenzhen Loop Area Institute, Shenzhen, China
$C^3$ ASD: Multi-Level Consistency-Driven Representation Learning
Jul 3, 2026Jin Hong, Jisoo Park, Junseok KwonMultimodal Robustness Audio-Visual Representation Learning
Chung-Ang University, Seoul, Republic of Korea
Attribute-Prompted Kernel Hashing for Unsupervised Data-Efficient Cross-Modal Retrieval
Jul 1, 2026Runhao Li, Xiaoxu Ma, Zhenyu Weng +5Cross-Modal Alignment Cross-Modal Retrieval
School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore · Shien-Ming Wu School of Intelligent Engineering, South China University of Technology, Guangzhou, China · College of Computer and Information Engineering, Henan Normal University, China +2

Aug 7, 2026, cs.CVJ/K move · Enter open · S save

Dual-Space Modality Consistency Learning for Universal Cross-Modal Re-Identification

Yujian Zhao, Yukang Zhao, Hankun Liu, Haoxuan Xu+3

Cross-Modal Alignment Multimodal Contrastive Learning Person Re-Identification

Open paper AskPDF arXiv

Cross-modal Re-Identification (ReID) aims to retrieve the same identity across heterogeneous imaging modalities and has been widely studied in visible-infrared person ReID and cross-modal ship ReID. Existing methods have achieved promising performance by learning modality consistency in the spatial embedding space, yet often overlook frequency-domain modality discrepancy, particularly in high-frequency representations that are both highly discriminative and modality-sensitive. In addition, most approaches are tailored to specific modality settings, limiting their applicability across diverse cross-modal scenarios. To address these challenges, we propose a Dual-Space Modality Consistency Learning (DSMCL) framework for universal cross-modal ReID. Specifically, DSMCL jointly models spatial feature distribution consistency and frequency-domain discriminative consistency. A Spatial Modality Consistency Learning (SMCL) branch performs Gaussian-based feature alignment, while a Frequency-aware Discriminative Consistency Learning (FDCL) strategy regularizes high-frequency representations through identity-aware cross-modal contrastive learning. By jointly capturing modality-specific characteristics and modality-shared identity cues, DSMCL learns robust representations and establishes a unified framework capable of accommodating diverse heterogeneous modality settings. Moreover, DSMCL is a plug-and-play framework that can be readily integrated into existing cross-modal ReID architectures. Extensive experiments on SYSU-MM01, RegDB, LLCM, HOSS-ReID, and CMShipReID across seventeen evaluation protocols show that DSMCL consistently improves multiple representative baselines.

Momentum

Related topics

Latest papers 153