Multimodal Contrastive Learning

Follow this topic Ask about this topic

Momentum

16 papers in the last four weeks, up 45% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 153

All topics

Dynamic Alignment and Calibration for Multimodal Learning
Oct 6, 2026Jinghao Xu, Zhenhua Guo, Xiaofeng Zhu +1Multimodal Robustness Cross-Modal Alignment
School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu,China · Tianyijiaotong Technology Ltd.,China · School of Computer Science and Technology, Hainan University, Haikou, China
MS-ECG-FM: Towards a More Universal Electrocardiogram Foundation Model for Health Monitoring using Multi-source Contrastive Learning
Oct 6, 2026Robert A. Lewis, I-Min Chiu, Kyle Verrier +7Electrocardiogram Foundation Models Electrocardiogram Classification
Massachusetts Institute of Technology (work done while at Apple). · Apple, Inc. · Division of Cardiology, Johns Hopkins Medicine. +1
fMRI-TAMCL: Text-Anchored Supervised Multimodal Contrastive Learning for fMRI-Based Brain Disorder Classification
Oct 5, 2026Juliana Mantebea Danso, Enoch Opanin Gyamfi, Mylene C. Q. FariasNeuroimaging Multimodal Contrastive Learning
Department of Computer Science, Texas State University, San Marcos, TX 78666 USA · University of Technology and Applied Sciences, Navrongo, Ghana
Cross-Modal Contrastive Learning for the Retrieval of Immunotherapy-Associated Molecular Signatures from Histopathology
Oct 4, 2026Sigrid Vila-Bagaria, Mar Teixidó, Miquel Piñol +3Computational Pathology Cross-Modal Retrieval
Signal Theory and Communications Department (TSC) Universitat Politècnica de Catalunya - BarcelonaTech (UPC) · Cancer Biomarkers Research Group (GReBiC) - IRB Lleida
Omni-Embed-Mini: Binding Modalities Without Forgetting via Dense Distillation
Oct 1, 2026Mohammed Irfan Kurpath, Jaseel Muhammad Kaithakkodan, Sahal Shaji Mullappilly +2Cross-Modal Knowledge Distillation Multimodal Embedding
Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI)
PLRS-IC: A Dual-Calibration Framework for Chest X-Ray Vision-Language Alignment
Sep 30, 2026Qixing Zhao, Jinpeng LiRadiology VLMs Vision-Language Alignment
South China University of Technology
End-to-End Self-Supervised RGB-T Tracking without Modality Misleading
Sep 29, 2026Shenglan Li, Rui Yao, Kunyang Sun +4Multimodal Robustness Visual Object Tracking
School of Computer Science and Technology / School of Artificial Intelligence, China University of Mining and Technology, China. · Mine Digitization Engineering Research Center of the Ministry of Education, China. · University of Auckland, Auckland, New Zealand. +1
Language as the Interface: Foundation-Model Contrastive Learning Links Transcriptomes and Electrophysiology
Sep 29, 2026Junbo Shen, Jinying Gao, Bo LeiCross-Modal Alignment Multimodal Foundation Models
Department of Computer Science and Engineering, The Chinese University of Hong Kong · State Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Institute of Automation, Chinese Academy of Sciences, Beijing, China · Beijing Academy of Artificial Intelligence, Beijing, China +1
Structured Visual Target Learning For Cross-Subject eeg-to-image retrieval
Sep 29, 2026Salini Yadav, Taveena Lotey, Mickaël Coustaty +2Cross-Subject EEG Decoding Visual Representation Learning
Indian Institute of Technology Roorkee, India · University of La Rochelle, France · Indian Institute of Technology (ISM) Dhanbad, India
SyncRA: Learning Temporal Correspondence in Omni-Modal Models
Sep 28, 2026Zelong Xu, Yan Li, Wenhe Hu +1Audio-Visual Representation Learning Audio-Visual Synchronization
University of Wisconsin–Madison · University of Alberta · Arizona State University
Learning Multimodal Embeddings with Evidence-Aligned Readout
Sep 27, 2026Zirong Chen, Fuda Ye, Enjun Du +8Multimodal Large Language Models Multimodal Embedding
The Hong Kong University of Science and Technology (Guangzhou) · Tencent Yuanbao · Tsinghua University +3
Cross-Modal Contrastive Learning from Histopathology and CT for Automated Renal Cell Carcinoma Grading
Sep 22, 2026Amit Das, Tanmay Shukla, Naofumi Tomita +9Medical Image Classification Oncology
Department of Computer Science, Dartmouth College, Hanover, NH 03755, USA · Department of Biomedical Data Science, Geisel School of Medicine at Dartmouth, Hanover, NH 03755, USA · Department of Radiology, Massachusetts General Hospital, Boston, MA 02114, USA +6
MMAP: Multimodal Missing-Aware Pretraining for Longitudinal Alzheimer's Prediction
Sep 22, 2026Fiona Kekwick, Matthew Baugh, Bernhard Kainz +2Missing-Modality Learning Cross-Modal Alignment
Department of Computing, Imperial College London, London, UK · FAU Erlangen-Nürnberg, Erlangen, Germany · Department of Brain Sciences, Imperial College London, London, UK +1
Ovis-Embedding: Pushing the Frontiers of Universal Omni-Modal Embeddings
Sep 21, 2026Ovis-Embedding TeamUnified Multimodal Models Multimodal Embedding
Alibaba Token Hub, Alibaba Group
RegRet: Enhancing Region-Level Retrieval in Large Multimodal Models
Sep 15, 2026Xun Liang, Honghui Yang, Weihang Pan +6Multimodal IR Multimodal Large Language Models
State Key Lab of CAD&CG, Zhejiang University · Xiaohongshu Inc. · School of Software Technology, Zhejiang University
Anatomical Grounding and Leakage-Aware Multimodal Contrastive Learning for Alzheimer's Disease Classification from Structural MRI
Sep 14, 2026Paul-Gabriel Nicolae, Irina Georgiana MocanuData Leakage Alzheimer's Disease Classification
Computer Science and Engineering Department, Faculty of Automatic Control and Computers, National University of Science and Technology POLITEHNICA Bucharest, Romania
PACE: Progressive Angular-to-Norm Contrastive Embedding
Sep 14, 2026Yanping Li, Wei Zhou, Yawen Liu +7Cross-Modal Representation Learning Multimodal Embedding
The Hong Kong University of Science and Technology · Alibaba Group · Huazhong University of Science and Technology +1
Multimodal Floorplan Encoding: Learning Dense Modality-Invariant Representations
Sep 14, 2026Xavier Anadón, Rémi Pautrat, Rui WangRepresentation Learning Cross-Modal Representation Learning
University of Zaragoza, Spain · Microsoft, Switzerland
TeMo: Temperature Modulation for Multimodal Contrastive Learning
Sep 7, 2026Dhimitrios Duka, Bernt Schiele, Hilde Kuehne +1Cross-Modal Retrieval Multimodal Classification
MPI for Informatics, SIC · Tuebingen AI Center/University of Tuebingen · MIT-IBM Watson AI Lab
The Shape of Time: Video-Token Contrast for Temporal Understanding in VideoLMs
Sep 3, 2026Yumeng Shi, Quanyu Long, Yin Wu +1Video-Language Models Video Representation Learning
Nanyang Technological University
Exploring the Potential of Contrastive Language-Image Pre-training for Multi-Source Remote Sensing Data
Sep 3, 2026Xiangyang Miao, Kelu Yao, Yekai Huang +7Remote Sensing Image Understanding Multimodal Pretraining
School of Computer Science and Technology, Zhejiang University · Space-based Computing System Research Center, Zhejiang Lab
AlphaRAD: Grounded Zero-Shot Classification in Chest Radiology via $α$ -Corrected Binary Cross Entropy and Factorized Latent Supervision
Sep 1, 2026Jianzhong You, Yuan Gao, Chris McIntoshChest X-Ray Classification Medical Image Classification
Peter Munk Cardiac Centre, University Health Network (UHN), Toronto, Canada · Ted Rogers Centre for Heart Research, Toronto, Canada · Vector Institute, Toronto, Canada +5
Beyond Language Priors: Diagnosing and Fixing Visual-Origin Hallucinations in Multimodal LLM
Aug 31, 2026Peiyang Xu, Xiaopei Zhu, Jun Zhu +1Object Hallucination in VLMs Multimodal Contrastive Learning
Tsinghua University Beijing, China
Multimodal Shared Latent Representation of Narration, Microscope and iOCT Images for Phase Recognition in Vitreoretinal Surgery
Aug 31, 2026Onur Izmitlioglu, Shervin Dehghani, Tarek Ghannoum +2Surgical Video Understanding Multimodal Contrastive Learning
Computer Aided Medical Procedures, Technical University of Munich, Germany · SynthesEyes GmbH, Munich, Germany · Department of Ophthalmology, LMU University Hospital, Munich, Germany
Multimodal Adaptive Expert Selection with Text Routing and Ordinal Prototype Optimization for Sentiment Analysis
Aug 31, 2026Xiaode Chen, Jiakang Yu, Hongtao Deng +3Representation Learning Mixture of Experts
Jianghan University Wuhan, China
GAAT: Geometry-Aware Alignment Transformer for Multimodal UAV Perception
Aug 28, 2026Jingpu Yang, Debin Tang, Yilin Sun +4Cross-Modal Alignment Multimodal Pretraining
Beihang University, Beijing, China. · Zhongguancun Academy, Beijing, China. · Northeastern University, Shenyang, China.
Low-Interaction-Rank Learning: Unifying Multiplicative Dual-Encoder Heads
Aug 12, 2026Zijian Zhao, Sen LiRepresentation Identifiability Multimodal Contrastive Learning
The Hong Kong University of Science and Technology · The Hong Kong University of Science and Technology (Guangzhou)
Rethinking Text-Based Image Retrieval in Specific Domain
Aug 11, 2026Jingyang Tan, Sheng Yang, Yuanpeng Chen +4Image-Text Retrieval Vision-Language Retrieval
Department of Electronic and Communication Engineering, Harbin Institute of Technology · School of Data Science, Fudan University · Independent Researcher +1
Perception Before Supervision: Self-Contained Visual Distillation from Counterfactual Blind Spots
Aug 10, 2026Shravan Venkatraman, Omkar Thawakar, Ritesh Thawkar +2VLM Distillation Large Vision-Language Models
1Mohamed bin Zayed University of Artificial Intelligence · 2Aalto University
GeoUniPR: A Geometry-Consistent Unified Framework for Cross-Modal Place Recognition
Aug 10, 2026Wonbong Kim, Jiatong Xiao, Rui Li +5Cross-Modal Representation Learning LiDAR Place Recognition
School of Computer Science and Technology, Tongji University, Shanghai, China · Shanghai Research Institute for Intelligent Autonomous System, Tongji University, Shanghai, China · MOE Key Lab of Embedded System and Service Computing, Tongji University, Shanghai, China +1

Oct 6, 2026, cs.CVJ/K move · Enter open · S save

Dynamic Alignment and Calibration for Multimodal Learning

Jinghao Xu, Zhenhua Guo, Xiaofeng Zhu, Xiaoshuang Shi

School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu,China · Tianyijiaotong Technology Ltd.,China · School of Computer Science and Technology, Hainan University, Haikou, China

Multimodal Robustness Cross-Modal Alignment Uncertainty-Aware Multimodal Fusion Multimodal Contrastive Learning

Open paper AskPDF arXiv

Dynamic multimodal learning aims to learn robust representations by adaptively modeling information discrepancies across modalities. However, existing methods still suffer from two limitations: (i) static cross-modal alignment strategies usually impose uniform constraints on all samples while overlooking sample-wise variations, potentially leading to unreasonable over-alignment; and (ii) confidence- or uncertainty-aware fusion methods often fail to adequately account for feature magnitude and confidence differences across modalities. For modality pairs with significant feature magnitude differences or small confidence gaps, it might be unreliable to strictly align fusion weights according to confidence. To address these issues, we propose an Alignment- and Calibration-driven Multimodal Learning framework (ACML). Specifically, ACML incorporates a dynamic cross-modal triplet alignment module, which enforces strong semantic consistency for high-confidence positive pairs while encouraging diverse representation learning between high- and low-confidence positive pairs according to their confidence gaps. Additionally, ACML introduces a difference-aware attention calibration strategy that adaptively adjusts attention regularization based on feature magnitude and confidence differences across modalities, thereby mitigating biases caused by unreasonable fusion constraints. Extensive experiments on multiple multimodal benchmark datasets demonstrate that ACML consistently achieves superior performance and robustness over recent state-of-the-art methods.

Momentum

Related topics

Latest papers 153