Cross-Modal Retrieval

Follow this topic Ask about this topic

Momentum

21 papers in the last four weeks, up 250% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 134

All topics

A Strong Baseline for Evaluating Vision Encoders in Multimodal Large Language Models
Oct 4, 2026Yilin Yang, Jun-Tao Tang, Kengyi Wang +3VLM Evaluation Multimodal Large Language Models
School of Artificial Intelligence, Shanghai Jiao Tong University · Nanjing University · Fudan University +1
Cross-Modal Contrastive Learning for the Retrieval of Immunotherapy-Associated Molecular Signatures from Histopathology
Oct 4, 2026Sigrid Vila-Bagaria, Mar Teixidó, Miquel Piñol +3Computational Pathology Cross-Modal Retrieval
Signal Theory and Communications Department (TSC) Universitat Politècnica de Catalunya - BarcelonaTech (UPC) · Cancer Biomarkers Research Group (GReBiC) - IRB Lleida
Omni-Embed-Mini: Binding Modalities Without Forgetting via Dense Distillation
Oct 1, 2026Mohammed Irfan Kurpath, Jaseel Muhammad Kaithakkodan, Sahal Shaji Mullappilly +2Cross-Modal Knowledge Distillation Multimodal Embedding
Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI)
AVSD-Scenes: A Dataset for Audio-Visual Description of Urban Scenes
Oct 1, 2026Dhanunjaya Varma Devalraju, Arshdeep Singh, Mark D. PlumbleyAudio-Visual Understanding Cross-Modal Retrieval
IIT Madras, India · King’s College London, United Kingdom
Spherical Interpolation for Backward-Compatible Multimodal Representations
Sep 30, 2026Simone Ricci, Niccolò Biondi, Federico PerniciCross-Modal Alignment Cross-Modal Retrieval
DINFO (Department of Information Engineering), University of Florence, Italy · MICC (Media Integration and Communication Center) · University of Trento, Italy
CurvSpec: Adaptive Multi-Curvature Learning for Partial Relevant Video Retrieval
Sep 29, 2026Zhen Liu, Letian Li, Jinpeng Wang +4Cross-Modal Retrieval Video Representation Learning
SIGS, Tsinghua University Shenzhen, China · Harbin Institute of Technology, Shenzhen Shenzhen, China
One Geometry, Different Outcomes: Readout-Dependent Effects of the Modality Gap in Vision-Language Models
Sep 28, 2026Aditya Sharma, Divya SaxenaCross-Modal Retrieval Modality Gap in VLMs
Indian Institute of Technology Jodhpur
Revitalizing Medical Time Series with Vision-Informed Retrieval: A Vision-Language Perspective
Sep 28, 2026Guoqi Yu, Juncheng Wang, Shujun WangMultivariate Time Series Classification Time Series Classification
Department of Biomedical Engineering and Sports Technology The Hong Kong Polytechnic University
HUMAN-TCI: Hierarchical Multi-Stream Motion-Aware Network with Torso-Centered Interaction for Text-to-Motion Retrieval
Sep 28, 2026Muhammad Islam, Euijoon Ahn, Usman Naseem +1Cross-Modal Retrieval Motion Representation Learning
College of Science and Engineering, James Cook University, Cairns, 4878, QLD, Australia · Centre for AI and Data Science Innovation, James Cook University, Cairns, 4878, QLD, Australia · School of Computing, Macquarie University, Sydney, 2113, NSW, Australia
PlaylistEval: Can Video-Language Judges Be Trusted at Day Scale and Beyond?
Sep 28, 2026Shayekh Bin Islam, Hwanjun SongLLM-as-a-Judge Video-Language Model Evaluation
KAIST
Learning Multimodal Embeddings with Evidence-Aligned Readout
Sep 27, 2026Zirong Chen, Fuda Ye, Enjun Du +8Multimodal Large Language Models Multimodal Embedding
The Hong Kong University of Science and Technology (Guangzhou) · Tencent Yuanbao · Tsinghua University +3
Discovery-Driven Integration of Disjoint Tables via Text
Sep 22, 2026Md Ataur Rahman, Dimitris Sacharidis, Oscar Romero +1Cross-Modal Retrieval Tabular Representation Learning
UPC, BarcelonaTech Barcelona, Spain · Université Libre de Bruxelles Brussels, Belgium
Ovis-Embedding: Pushing the Frontiers of Universal Omni-Modal Embeddings
Sep 21, 2026Ovis-Embedding TeamUnified Multimodal Models Multimodal Embedding
Alibaba Token Hub, Alibaba Group
The Visual Target Matters: Learning across the Visual Hierarchy for Brain-to-Image Retrieval
Sep 21, 2026Ye Wang, Haokun Ren, Hong Yu +4Visual Representation Learning EEG-Based Visual Decoding
School of Artificial Intelligence, Chongqing University of Posts and Telecommunications, Chongqing, China · College of Information Science and Technology, Beijing University of Chemical Technology, Beijing, China · Department of Land Surveying and Geo-Informatics, The Hong Kong Polytechnic University, Hong Kong, China +1
Adaptive Cortically Constrained EEG-Vision Alignment for Zero-Shot Brain-to-Image Retrieval
Sep 21, 2026Ye Wang, Haokun Ren, Wei Wu +4Electroencephalography EEG-Based Visual Decoding
School of Artificial Intelligence, Chongqing University of Posts and Telecommunications, Chongqing, China · School of Medicine, Shanghai Jiaotong University, Shanghai, China · National Center for Applied Mathematics in Chongqing, Chongqing Normal University, Chongqing, China +1
RPA: Residual Patch-Token Adapter for Image Retrieval from EEG and MEG
Sep 19, 2026Yuhui Jin, Yonghao Song, Bingchuan LiuCross-Subject EEG Decoding Cross-Modal Representation Learning
Vortek Lab Inc. · Department of Computer Science and Technology, Tsinghua University · Biology and Biological Engineering, California Institute of Technology
SCOUT: Sim-to-Real Text-Based Person Retrieval by Embedding-Space Prediction over Frozen Video Features
Sep 16, 2026Abdarahmane Traoré, Andy Couturier, Éric HervetSynthetic-to-Real Domain Adaptation Multimodal Embedding
Embia, Computer Science Department, Faculty of Science, Université de Moncton, Moncton, NB, Canada
Hub-Spectral Activation of Latent Multimodal Knowledge
Sep 15, 2026Ying Guo, Haidong Chen, Linrui Xu +7Cross-Modal Representation Learning Cross-Modal Alignment
Beijing Key Laboratory of Key Technologies for AI+ Domain Applications, North China University of Technology, Beijing 100144, China · School of Geosciences and Info-Physics, Central South University, Changsha 410083, China · NExT++ Research Centre, National University of Singapore, Singapore +2
FLAT: Resampling Image and Text into 1D Flexible-Length Aligned Transmodal Tokens for Retrieval and Generation
Sep 15, 2026Guangyu Sun, Shlok Kumar Mishra, Wentao Bao +8Cross-Modal Representation Learning Cross-Modal Alignment
Meta AI
Query-Conditioned Spherical Centroid Aggregation for Multimodal Retrieval
Sep 14, 2026Ambuj Mehrish, Anindya Nag, Sebastiano VasconMultimodal Robustness Cross-Modal Retrieval
Department of Environmental Science, Informatics and Statistics Ca’ Foscari University of Venice, Italy
TeMo: Temperature Modulation for Multimodal Contrastive Learning
Sep 7, 2026Dhimitrios Duka, Bernt Schiele, Hilde Kuehne +1Cross-Modal Retrieval Multimodal Classification
MPI for Informatics, SIC · Tuebingen AI Center/University of Tuebingen · MIT-IBM Watson AI Lab
CORE: Improving Compositional Reasoning in MLLM Embedding via Reranker Distillation
Sep 3, 2026Tingyu Song, Mingxin Li, Yanzhao Zhang +5Multimodal Embedding Cross-Modal Retrieval
CASIA · Alibaba Group · University of Chinese Academy of Sciences +1
V2TATC: Joint Voice-Trajectory Embedding and Dataset for Air Traffic Controller Situational Awareness
Aug 29, 2026Louis Brusset, Mathurin Petit, Jordan Kam +1Air Traffic Management Cross-Modal Retrieval
Mines Paris–PSL University, Paris, 75006, France · École Polytechnique, Palaiseau, 91128, France · California Institute of Technology, Pasadena, CA, 91125, USA +1
Who Remains, What Changes: Identity Anchored Composed Gait Retrieval
Aug 27, 2026Jingchen Fei, Zengbin Wang, Yukun Liu +3Cross-Modal Retrieval Gait Recognition
Beijing University of Posts and Telecommunications · Huazhong University of Science and Technology
When Images Look Right and Retrieve Wrong: Coverage-Guided Cross-Scale Re-Indexing for Knowledge-Faithful Generative Perception
Aug 21, 2026Guangyuan Dong, Chuang Liu, Haoyu Wang +10Multimodal IR Remote Sensing
1National University of Singapore · 2Wuhan University · 4Nankai University +6
Generative Universal Multimodal Retrieval with Dual-role Identifiers
Aug 13, 2026Kaipeng Li, Haitao Yu, Xuanchen ZhouMultimodal IR Generative Retrieval
Independent Researcher, Tsukuba, Ibaraki, Japan · Institute of Library, Information and Media Science, University of Tsukuba, Tsukuba, Ibaraki, Japan · College of Knowledge and Library Sciences, University of Tsukuba, Tsukuba, Japan
Heterogeneous Vision-Language Ensemble with Disagreement-Aware Reranking for Text-Based Person Anomaly Retrieval
Aug 13, 2026Huu-An Vu, Cam Tu Tran Thi, Thanh Toan Le Ngo +5Vision-Language Models Multimodal Reranking
Hanoi University of Science and Technology, Hanoi, Vietnam · GenAI4E Lab · University of Information Technology, VNU-HCM, Ho Chi Minh City, Vietnam +4
CertBind from Multimodal Connectivity to Certifiable Retrieval Decisions
Aug 6, 2026Shuheng Cao, Zhenhao Zhang, Ruiqi Chen +7Selective Prediction Conformal Prediction
University of California, San Diego · ShanghaiTech University · Tsinghua University +5
Predict, Then Retrieve: Cross-Instance Future-State Retrieval from Video Prefixes
Aug 5, 2026Quynh Vo, Thong Nguyen, Vinh-Hien Do +2Cross-Modal Retrieval Video Prediction
Centre for AI Research, VinUniversity · National University of Singapore
Multimodal Alignment Through Joint Kernel Entropic Gromov--Wasserstein Optimal Transport
Aug 4, 2026Yixuan Florence Wu, Yilun Zhu, Naichen ShiCross-Modal Alignment Multimodal Embedding
Department of Engineering Sciences and Applied Mathematics Northwestern University Evanston, IL 60208, USA · Deparment of Electrical Engineering and Computer Science University of Michigan Ann Arbor, MI 48109, USA · Department of Industrial Engineering and Management Sciences Department of Mechanical Engineering Northwestern University Evanston, IL 60208, USA

Oct 4, 2026, cs.CVJ/K move · Enter open · S save

A Strong Baseline for Evaluating Vision Encoders in Multimodal Large Language Models

Yilin Yang, Jun-Tao Tang, Kengyi Wang, Siyuan Su+2

School of Artificial Intelligence, Shanghai Jiao Tong University · Nanjing University · Fudan University+1

VLM Evaluation Multimodal Large Language Models Cross-Modal Retrieval

Open paper AskPDF arXiv

Evaluating vision encoders requires metrics that reliably predict their downstream performance in multimodal large language models (MLLMs). Although recent studies have shown that cross-modal metrics can better capture such performance, unimodal metrics remain the dominant choice in practice. In this work, we revisit cross-modal evaluation of vision encoders through large-scale experiments. We identify important limitations in both the experimental design and methodological formulation of prior approaches. After addressing these limitations and introducing simple improvements, we propose RAVEL, a training-free method based on cross-modal nearest-neighbor retrieval. Despite its simplicity, RAVEL achieves state-of-the-art performance across our experiments, outperforming prior methods by a substantial margin. Our results demonstrate that simple cross-modal metrics, when evaluated under a careful and comprehensive setup, can provide a strong basis for evaluating vision encoders for MLLMs.

Momentum

Related topics

Latest papers 134