Multimodal Contrastive Learning

Follow this topic Ask about this topic

Momentum

16 papers in the last four weeks, up 45% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 153

All topics

Unsupervised Data-Efficient Cross-Modal Retrieval with Global-Neighborhood Alignment Hashing
Jun 30, 2026Runhao Li, Xiaoxu Ma, Zhenyu Weng +5Cross-Modal Alignment Cross-Modal Retrieval
School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore · Shien-Ming Wu School of Intelligent Engineering, South China University of Technology, China · College of Computer and Information Engineering, Henan Normal University, China +2
Data-Efficient Multimodal Alignment for Histopathology-based Molecular Prediction
Jun 29, 2026Dominik Winter, Dominik Vonficht, Loïc Le Bescond +6Computational Pathology Cross-Modal Alignment
AstraZeneca Computational Pathology and Biomarkers, Munich, Germany · AstraZeneca, Early Oncology and Translation Medicine, Cambridge, UK
SonoCLIP: Mask-Guided Region-Aware Vision-Language Pretraining for Fetal Ultrasound Analysis
Jun 28, 2026Hang Su, Chao Sun, Zhaofan Li +3Medical Imaging Foundation Models Ultrasound Imaging
School of Computer Science, Wuhan University, Wuhan, China · Institute of Artificial Intelligence, Wuhan University, Wuhan, China · Department of Ultrasound, Renmin Hospital of Wuhan University, Wuhan, China +2
Rank-Aware Hyperbolic Alignment for Vision-Language Dataset Distillation
Jun 28, 2026Jongoh Jeong, Sun-Kyung Lee, Kuk-Jin YoonDataset Distillation Vision-Language Alignment
Korea Advanced Institute of Science and Technology (KAIST), Republic of Korea · Electronics and Telecommunications Research Institute (ETRI), Republic of Korea
TacGen: Touch Is a Necessary Dimension of Physical-World Representation -- Addressing Tactile Data Scarcity with Scalable Vision-to-Touch Alignment and Generation
Jun 28, 2026Wanghao Ye, Aarosh Das, Sihan Chen +19Visuo-Tactile Representation Learning Multimodal Contrastive Learning
University of Maryland, College Park · Carnegie Mellon University · University of Alabama +2
Learning from Acquisition: Metadata-driven Multimodal Pre-training for Cardiac MRI
Jun 27, 2026Xueyi Fu, Liwei Hu, Zi Wang +1Medical Image Segmentation Multimodal Contrastive Learning
Department of Surgery & Cancer, Imperial College London, London, UK · Bioengineering Department and Imperial-X, Imperial College London, London, UK · National Heart and Lung Institute, Imperial College London, London, UK +2
Text as Illumination: Spatial Contrastive Retinex Learning for Language-guided Medical Image Segmentation
Jun 26, 2026Jian Shi, Cheng Zhen, Pingping Zhang +6Text-Guided Image Segmentation Medical Image Segmentation
School of Software Technology & DUT-RU International School of Information Science and Engineering, Dalian University of Technology, Dalian, China · School of Future Technology, Dalian University of Technology, Dalian, China · Central Hospital of Dalian University of Technology, Dalian, China +2
DFM: Difference Feature Modeling with Text-Guided Gated Contrastive Loss for Remote Sensing Image Change Captioning
Jun 25, 2026Yelin Wang, Zijia Song, Chuanguang Yang +4Remote Sensing Image Understanding Remote Sensing Image Change Captioning
ShanghaiTech University · National University of Defense Technology · Institute of Computing Technology, Chinese Academy of Sciences +1
Jolia: Concept-Level Vision-Language Alignment for 3D CT Contrastive Learning
Jun 23, 2026Julien Khlaut, Charles Corbière, Baptiste Callard +93D Foundation Models Medical Imaging Foundation Models
Raidium, Paris, 75014, France · Department of Vascular and Oncological Interventional Radiology,2026 Hôpital Européen Georges Pompidou, AP-HP, Paris, France · Faculté de Santé, Université Paris-Cité, Paris, France +3
Mind the Heads: Topological Representation Alignment for Multimodal LLMs
Jun 22, 2026Davide Caffagni, Alberto Compagnoni, Federico Melis +5Cross-Modal Alignment Multimodal Large Language Models
University of Modena and Reggio Emilia · University of Pisa · AMD Silo AI
MAPS: Multi-Anchor Projection Similarity for Joint Vision-Language Geo-Localization
Jun 21, 2026Yutong Hu, Siyuan Tan, Shaocheng Yan +3Cross-View Geo-Localization Geometric Representation Learning
School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, China
MultiMem: Measuring and Mitigating Memorization in Multi-Modal Contrastive Learning
Jun 20, 2026Wenhao Wang, Franziska Boenisch, Michael Backes +1Multimodal Contrastive Learning
CISPA Helmholtz Center for Information Security
Beyond Flat Labels: Level-Restricted Contrastive Learning for Hierarchical Fine-Grained Vision Classification
Jun 20, 2026Zhiyuan Tao, Srikumar Sastry, Matthew J Thompson +9Hierarchical Classification Contrastive Learning
The Ohio State University · Washington University in St. Louis · Neuromatch +1
Multi-Modal Contrastive Learning for Implicit Earth Embeddings via Location Tying
Jun 18, 2026Jonathan Hecht, Lukas Arzoumanidis, Ziyue Li +1Self-Supervised Pre-Training Geospatial Representation Learning
Computational Methods Lab, HafenCity University Hamburg, Germany. · Institute of Geodesy and Geoinformation, University of Bonn, Germany. · Dept. of Operations & Technology, Technical University of Munich; Heilbronn Data Science Center; Munich Data Science Institute.
Exploring Multi-Modal Large Language Models and Two-Stage Fine-Tuning for Fashion Image Retrieval
Jun 18, 2026Nguyen Cao Hoang, Hoang Bui Le, Nam Vo Hoang +1Multimodal Large Language Models Image-Text Retrieval
University of Science, VNU-HCM, Ho Chi Minh, Vietnam · Vietnam National University, Ho Chi Minh, Vietnam
REVEAL++: Differentiable Phenotypic Grouping for Vision-Language Retinal Modeling of Alzheimer's Disease Risk
Jun 17, 2026Ethan Elio Meidinger, Seowung Leem, Zeyun Zhao +1Contrastive Learning Medical Image Analysis
University of Virginia, Charlottesville, VA, USA · J. Crayton Pruitt Family Department of Biomedical Engineering, Herbert Wertheim College of Engineering, University of Florida, Gainesville, FL, USA
TactSpace: Learning a Physics-enriched Shared Latent Space for Tactile Sim-to-Real Transfer
Jun 17, 2026Arunim Joarder, Arjun Bhardwaj, René Zurbrügg +6Cross-Modal Representation Learning Tactile Sensing
Robotic Systems Lab, ETH Zürich · ETH AI Center · NVIDIA +1
Contextualizing Biological Language Models across Modalities via Logit-Space Contrastive Alignment
Jun 17, 2026Yanjun Shao, Yundi Chen, Yashvi Patel +2Contrastive Learning Protein Language Models
Biomedical Informatics and Data Science, Yale School of Medicine, United States · Institute of Computational Life Sciences, Zürich University of Applied Sciences (ZHAW), Switzerland
SUP-MCRL: Subject-aware Unified Pseudo-feature Coded Multimodal Contrastive Representation Learning for EEG Visual Decoding
Jun 15, 2026Shengyu Gong, Weiming Zeng, Yueyang Li +4Cross-Subject EEG Decoding Contrastive Learning
Lab of Digital Image and Intelligent Computation, Shanghai Maritime University, Shanghai 201306, China · Department of Language Science and Technology, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong Special Administrative Region, China · Affiliated Lianyungang Hospital of Xuzhou Medical University, Lianyungang 222002, China
CAP: Towards PPG Universal Representation Learning with Patient-level Supervision
Jun 13, 2026Chenyang He, Xinyi Shao, Shun Huang +4Representation Learning Cross-Modal Representation Learning
Nanjing University of Aeronautics and Astronautics Nanjing, Jiangsu, China · Peking University Beijing, Beijing, China · Independent Researcher +1
Trimodal Glioma Representation Alignment via Volumetric Contrastive Learning
Jun 12, 2026Denise Marini, Eleonora Grassucci, Danilo ComminielloCancer Survival Prediction Cross-Modal Alignment
Dept. of Information Engineering, Electronics, and Telecommunications, Sapienza University of Rome, Italy
LaME: Learning to Think in Latent Space for Multimodal Embedding via Information Bottleneck
Jun 11, 2026Peixi Wu, Biao Yang, Feipeng Ma +7Efficient Multimodal Inference Multimodal Embedding
University of Science and Technology of China · 2Kuaishou Technology · 3Zhejiang University +1
Objects Before Words: Object-First Inductive Biases for Grounding Language in Child-View Video
Jun 11, 2026Sathira Silva, Abrham Kahsay Gebreselasie, Muhammad Umer Sheikh +3Object-Centric Representation Learning Egocentric Video Understanding
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · Weizmann Institute of Science, Rehovot, Israel
RespiraMFM: A Multimodal Foundation Model with Contrastive Audio-Language Alignment for Respiratory Disease Identification
Jun 8, 2026Shakhrul Iman Siam, Tiantian Feng, Jiankun Zhang +2Healthcare Audio-Language Models
The Ohio State University · University of Southern California · University of Chicago
A robust PPG foundation model using multimodal physiological supervision
Jun 5, 2026Eloy Geenjaar, Vince Calhoun, Scott Daly +4Multimodal Robustness Multimodal Pretraining
Department of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, USA · Tri-Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State University, Georgia Institute of Technology, Emory University, Atlanta, USA · Dolby Laboratories, San Francisco, USA
FIGMA: Towards FIne-Grained Music retrievAl
Jun 4, 2026Nishit Anand, Ashish Seth, Sreyan Ghosh +2Audio Understanding Audio-Text Retrieval
University of Maryland, College Park, USA
BRepCLIP: Contrastive Multimodal Pretraining on BRep Primitives for CAD Understanding
Jun 3, 2026Muhammad Usama, Didier Stricker, Mohammad Sadil Khan +1Multimodal Pretraining Multimodal Contrastive Learning
1DFKI, Germany · 2RPTU Kaiserslautern-Landau, Germany
Robust Scene Transfer for PointGoal Navigation via Privileged Sensor Guided Contrastive Learning
Jun 3, 2026Amirhossein Zhalehmehrabi, Tiziano Tezze, Alberto Castelini +1Robot Navigation Domain Generalization
Continual Visual and Verbal Learning Through a Child's Egocentric Input
Jun 3, 2026Xiaoyang Jiang, Yanlai Yang, Kenneth A. Norman +2Egocentric Video Understanding Replay-Based Continual Learning
Agentic Learning AI Lab, New York University · Department of Psychology, Princeton University
CP-Agent: Context-Aware Multimodal Reasoning for Cellular Morphological Profiling under Chemical Perturbations
Jun 2, 2026Yuxin Zhang, Yiyao Li, Ping Shu Ho +3AI Agents for Scientific Discovery Multimodal Contrastive Learning
Department of Electrical and Computer Engineering, The University of Hong Kong · School of Computing and Data Science, The University of Hong Kong · Nvidia AI Technology Center +2

Jun 30, 2026, cs.IRJ/K move · Enter open · S save

Unsupervised Data-Efficient Cross-Modal Retrieval with Global-Neighborhood Alignment Hashing

Runhao Li, Xiaoxu Ma, Zhenyu Weng, Yue Zhang+4

School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore · Shien-Ming Wu School of Intelligent Engineering, South China University of Technology, China · College of Computer and Information Engineering, Henan Normal University, China+2

Cross-Modal Alignment Cross-Modal Retrieval Multimodal Contrastive Learning

Open paper AskPDF arXiv

Compared to supervised cross-modal hashing (CMH), unsupervised CMH reduces the reliance on manual labeling by learning binary codes from unlabeled image-text pairs. However, existing unsupervised CMH methods often rely on large-scale image-text pairs, which are costly to collect. To address this limitation, we propose Global-Neighborhood Alignment Hashing (GNAH), a novel approach that preserves the semantic structure of vision-language foundation models within a compact binary Hamming space using only a limited number of image-text pairs. Specifically, GNAH captures global structural information from the continuous latent space and transfers it into the binary Hamming space through a Prototype-Anchored Global Alignment module. In addition, GNAH extends conventional pairwise contrastive learning by modeling stochastic neighborhood relationships via a Contrastive Stochastic Neighborhood Alignment module, thereby alleviating overfitting to sparse pairwise correlations. Extensive experiments demonstrate that GNAH consistently outperforms existing unsupervised cross-modal retrieval methods under data-constrained settings, offering a practical solution for real-world CMH applications.

Momentum

Related topics

Latest papers 153