Cross-Modal Retrieval

Ask about this topic

Overview

Cross-modal retrieval studies how to find relevant items when a query and its targets use different modalities, such as searching videos with text or matching brain signals to images. A common approach trains modality-specific encoders to map data into a shared space using contrastive learning, while other methods align local regions, frames, or tokens for finer matches. Systems also use query-aware fusion, candidate reranking, reasoning, or compact hashing to handle complex searches and large collections. A central challenge is balancing scalable global similarity with compositional detail and robustness to noisy, missing, or mismatched modalities; benchmarks increasingly test these harder settings.

Latest papers

CardsList

187 papers25 papers in the last four weeks, up 67% on the four weeks before. 0.1% of all new papers.

  1. SIMMER: Cross-Modal Food Image--Recipe Retrieval via MLLM-Based Embedding

    Apr 17, 2026Keisuke Gomi, Keiji YanaiCross-Modal LearningImage-Text Retrieval

    The University of Electro-Communications

  2. SLQ: Bridging Modalities via Shared Latent Queries for Retrieval with Frozen MLLMs

    Apr 15, 2026Haoran Lou, Ziyan Liu, Chunxiao Fan +6Multimodal IRCross-Modal Retrieval

    School of Electronic Engineering, Beijing University of Posts and Telecommunications, China · Xiaohongshu Inc., China

  3. Geometry-Aware CLIP Retrieval via Local Cross-Modal Alignment and Steering

    Apr 13, 2026Nirmalendu Prakash, Narmeen Fatimah Oozeer, Xin Su +8Cross-Modal RetrievalCross-Modal Alignment

    Singapore University of Technology and Design · Martian · Thoughtworks +2

  4. EmergentBridge: Improving Zero-Shot Cross-Modal Transfer in Unified Multimodal Embedding Models

    Apr 13, 2026Jincheng Xie, Xingchen Xiao, Runheng Liu +3Cross-Modal LearningMultimodal Embedding

    School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China · JD iCity, JD Technology, JD Intelligent Cities Research, Beijing, China

  5. Descriptor-Injected Cross-Modal Learning: A Systematic Exploration of Audio-MIDI Alignment via Spectral and Melodic Features

    Apr 11, 2026Mariano Fernández MéndezCross-Modal RetrievalCross-Modal Attention

    Asociación Civil AlterMundi, Córdoba, Argentina

  6. BRIDGE: Multimodal-to-Text Retrieval via Reinforcement-Learned Query Alignment

    Apr 8, 2026Mohamed Darwish Mounis, Mohamed Mahmoud, Shaimaa Sedek +4Multimodal IRCross-Modal Retrieval

    High institute for computer & information systems · Chungbuk National University · Assiut University +1

  7. Hidden in the Multiplicative Interaction: Uncovering Fragility in Multimodal Contrastive Learning

    Apr 7, 2026Tillmann Rheude, Stefan Hegselmann, Roland Eils +1Multimodal Contrastive LearningCross-Modal Retrieval

    Berlin Institute of Health, Charité - Universitätsmedizin Berlin · Department of Mathematics and Computer Science, Freie Universität Berlin · Intelligent Medicine Institute, Fudan University

  8. Imagine Before Concentration: Diffusion-Guided Registers Enhance Partially Relevant Video Retrieval

    Apr 4, 2026Jun Li, Xuhang Lou, Jinpeng Wang +4Cross-Modal RetrievalInformation Retrieval

    Harbin Institute of Technology, Shenzhen · Tsinghua Shenzhen International Graduate School, Tsinghua University · Peng Cheng Laboratory

  9. MG2^2-RAG: Multi-Granularity Graph for Multimodal Retrieval-Augmented Generation

    Apr 4, 2026Sijun Dai, Qiang Huang, Xiaoxing You +1Cross-Modal LearningMultimodal RAG

    School of Intelligence Science and Engineering, Harbin Institute of Technology (Shenzhen) · School of Computer Science, Hangzhou Dianzi University

  10. FusionBERT: Multi-View Image--3D Retrieval via Cross-Attention Visual Fusion and Normal-Aware 3D Encoder

    Apr 2, 2026Wei Li, Yufan Ren, Hanqing Jiang +6Multi-View LearningCross-Modal Retrieval

    IROOTECH TECHNOLOGY, Hangzhou, Zhejiang, China · Wolf 1069 b Lab, Sany Group, Hangzhou, Zhejiang, China · Zhejiang University, Hangzhou, Zhejiang, China +4

  11. Camouflage-aware Image-Text Retrieval via Expert Collaboration

    Apr 1, 2026Yao Jiang, Zhongkuan Mao, Xuan Wu +2Image-Text RetrievalCross-Modal Retrieval

    College of Computer Science, Sichuan University · National Key Lab of Fundamental Science on Synthetic Vision, Sichuan University

  12. Storing Less, Finding More: How Novelty Filtering Improves Cross-Modal Retrieval on Edge Cameras

    Mar 31, 2026Sherif AbdelwahabCross-Modal RetrievalVideo Frame Selection

  13. MoCHA: Denoising Caption Supervision for Motion-Text Retrieval

    Mar 24, 2026Nikolai Warner, Cameron Ethan Taylor, Irfan Essa +1Cross-Modal RetrievalMultimodal Contrastive Learning

    Georgia Institute of Technology, Atlanta, GA, USA · Amazon, Sunnyvale, CA, USA

  14. HyFI: Hyperbolic Feature Interpolation for Brain-Vision Alignment

    Mar 24, 2026Sangmin Jo, Wootaek Jeong, Da-Woon Heo +2Cross-Modal AlignmentEEG-Based Visual Decoding

    Department of Artificial Intelligence, Korea University, Seoul, South Korea

  15. Evo-Retriever: LLM-Guided Curriculum Evolution with Viewpoint-Pathway Collaboration for Multimodal Document Retrieval

    Mar 17, 2026Weiqing Li, Jinyue Guo, Yaqi Wang +4Multimodal IRCross-Modal Retrieval

    Alibaba Cloud Computing

  16. SDF-Net: Structure-Aware Disentangled Feature Learning for Opticall-SAR Ship Re-identification

    Mar 13, 2026Furui Chen, Han Wang, Yuhan Sun +5Cross-Modal Representation LearningCross-Modal Retrieval

    Technology and Engineering Center for Space Utilization, Chinese Academy of Sciences, Beijing 100094, China · Key Laboratory of Space Utilization, Chinese Academy of Sciences, Beijing 100094, China · University of Chinese Academy of Sciences, Beijing 100049, China +1

  17. Fine-grained Motion Retrieval via Joint-Angle Motion Images and Token-Patch Late Interaction

    Mar 10, 2026Yao Zhang, Zhuchenyang Liu, Yanlan He +2Representation LearningFeature Interaction Modeling

    Aalto University, Espoo, Finland · Fudan University, Shanghai, China · Georgia Institute of Technology, Atlanta GA, USA

  18. Echo2ECG: Enhancing ECG Representations with Cardiac Morphology from Multi-View Echos

    Mar 9, 2026Michelle Espranita Liman, Özgün Turgut, Alexander Müller +3Electrocardiogram Representation LearningMultimodal Learning

    Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital, Germany · Department of Cardiology, TUM University Hospital, Germany · Department of Computing, Imperial College London, UK +1

  19. Optimizing Multi-Modal Models for Image-Based Shape Retrieval: The Role of Pre-Alignment and Hard Contrastive Learning

    Mar 7, 2026Paul Julius Kühn, Cedric Spengler, Michael Weinmann +23D Shape RetrievalCross-Modal Retrieval

    Fraunhofer IGD, 64283 Darmstadt, Germany · Delft University of Technology, 2628 CD Delft, Netherlands

  20. MultiHaystack: Benchmarking Multimodal Retrieval and Reasoning over 40K Images, Videos, and Documents

    Mar 5, 2026Dannong Xu, Zhongyu Yang, Jun Chen +6Multimodal IRMultimodal Model Evaluation

    INSAIT, Sofia University “St. Kliment Ohridski” · Lanzhou University · King Abdullah University of Science and Technology +3

  21. TRACE: Task-Adaptive Reasoning and Representation Learning for Universal Multimodal Retrieval

    Mar 3, 2026Xiangzhao Hao, Shijie Wang, Tianyu Yang +3Multimodal IRMultimodal Embedding

    Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences

  22. Efficient and High-Fidelity Omni Modality Retrieval

    Mar 2, 2026Chuong Huynh, Manh Luong, Abhinav ShrivastavaMultimodal IRCross-Modal Retrieval

    University of Maryland, College Park, USA · Monash University, Australia

  23. Beyond Global Similarity: Towards Fine-Grained, Multi-Condition Multimodal Retrieval

    Mar 1, 2026Xuan Lu, Kangle Li, Haohang Huang +3Multimodal IRCross-Modal Retrieval

    Shanghai Jiao Tong University · Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative +1

  24. SleepLM: Natural-Language Intelligence for Human Sleep

    Feb 27, 2026Zongzhe Xu, Zitao Shuai, Eideen Mozaffari +3Multimodal PretrainingCross-Modal Retrieval

    University of California, Los Angeles

  25. ViCLIP-OT: The First Foundation Vision-Language Model for Vietnamese Image-Text Retrieval with Optimal Transport

    Feb 26, 2026Quoc-Khang Tran, Minh-Thien Nguyen, Nguyen-Khang PhamVision-Language RetrievalCross-Modal Retrieval

    Can Tho University, Can Tho, Vietnam

  26. StructXLIP: Enhancing Vision-language Models with Multimodal Structural Cues

    Feb 23, 2026Zanxi Ruan, Songqun Gao, Qiuyu Kong +2Vision-Language AlignmentCross-Modal Retrieval

    University of Verona · Sapienza University of Rome · University of Trento +2

  27. Closing the gap in multimodal medical representation alignment

    Feb 23, 2026Eleonora Grassucci, Giordano Cicchetti, Danilo ComminielloModality Gap in VLMsVision-Language Alignment

    Dept. of Information Engineering, Electronics, and Telecommunications, Sapienza University of Rome, Italy

  28. CREM: Compression-Driven Representation Enhancement for Multimodal Retrieval and Comprehension

    Feb 22, 2026Lihao Liu, Yan Wang, Biao Yang +10Multimodal Large Language ModelsCross-Modal Retrieval

    Tsinghua University · Kuaishou Technology

  29. Magic-MM-Embedding: Towards Visual-Token-Efficient Universal Multimodal Embedding with MLLMs

    Feb 5, 2026Qi Li, Yanzhe Zhao, Yongxin Zhou +4Efficient Multimodal InferenceMultimodal Large Language Models

    Honor Device Co., Ltd., China

  30. Generating a Paracosm for Training-Free Zero-Shot Composed Image Retrieval

    Jan 31, 2026Tong Wang, Yunhan Zhao, Shu KongImage MatchingCross-Modal Retrieval

    University of Macau, Macau 999078, China · Uiversity of California at Irvine, Irvine, CA 92697, USA · Institute of Collaborative Innovation, University of Macau, Macau 999078, China