Overview

Multimodal embedding studies how to represent different kinds of data so that related items are close in a shared space, enabling tasks such as cross-modal search, recommendation, and classification. Common approaches include contrastive dual encoders, instruction-tuned multimodal language models, and alignment methods based on graphs or optimal transport; multi-vector and late-interaction representations can retain finer detail than a single vector. Benchmarks such as MMEB, ViDoRe, and audio- and video-retrieval suites test these models across tasks and modalities, while newer methods explore compressed representations and latent reasoning to improve quality without costly inference. A central challenge is balancing broad, zero-shot capability with fine-grained, task-specific information, while keeping representations efficient and robust when modalities are missing or unevenly informative.

Latest papers

CardsList

167 papers11 papers in the last four weeks, down 50% on the four weeks before. 0.1% of all new papers.

  1. When Does Reusing Layers Help Multimodal Embeddings?

    Nov 15, 2026Zhizhang Hu, Yue ZhangMultimodal EmbeddingRecurrent Transformers

  2. Provable Cross-Modal Transfer of Hierarchical Concept Structure in CLIP Embeddings

    Nov 13, 2026Nghia Nguyen, Tianjiao Ding, Lachlan Ewen MacDonald +1Vision-Language ModelsMultimodal Embedding

  3. A Technique to Model Product Information and User Reviews (MPIUR)

    Nov 9, 2026Emna Belguith, Fajrian Yunus, Talel AbdessalemRecommender SystemsMultimodal Recommendation

  4. Factorized MLLM Embeddings for Composed Image Retrieval

    Nov 3, 2026WonJun Moon, Seongchan Kim, Dahun Kim +1Composed Image RetrievalMultimodal Embedding

  5. MedHEB: Benchmarking Medical Embeddings Across Heterogeneous Clinical Evidence

    Nov 2, 2026Yingshu Li, Shaoyang Zhou, Zhanyu Wang +6Multimodal IRMultimodal Embedding

  6. From Fragmentation to Fusion: A Survey and Roadmap for Omnimodal Embeddings

    Oct 30, 2026Ziyin Zhang, Yuyu Guo, Ying Zhang +15Multimodal EmbeddingCross-Modal Alignment

  7. Parameter-Efficient Fine-Tuning of Transformer Encoders for Audio-Image-Symbolic Music Retrieval

    Oct 30, 2026Eliseo Fuentes-Martinez, Antonio Hidalgo-Centeno, Jorge Calvo-Zaragoza +1Multimodal IRParameter-Efficient Fine-Tuning

  8. Beyond Post-Hoc Rewards: Privileged Reasoning Exploration for Multimodal Embeddings

    Oct 23, 2026Tianyu Yang, Yijun Tian, Lars Benedikt Kaesberg +4Multimodal EmbeddingCross-Modal Retrieval

  9. OlmoEarth v1.3: Embeddings

    Oct 23, 2026Gabriel Tseng, Favyen Bastani, Patrick Alan Johnson +13Geospatial Representation LearningMultimodal Embedding

  10. pplx-embed-v2-late Technical Report

    Oct 21, 2026Markus Krimmel, Louis Milliken, Ekaterina Glazkova +8Multimodal EmbeddingMultimodal IR

  11. Inducing Fourier Geometry: A Smoothness Prior Imposes Approximate Translation Equivariance in Multimodal Stellar Embeddings

    Oct 16, 2026Ilay Kamai, Alex M. Bronstein, Hagai B. PeretsGeometric Representation LearningEquivariant Representation Learning

  12. Domain-specific language alignment of multimodal embeddings towards explainable predictions

    Oct 16, 2026Gabrielė Tijūnaitytė, Thijs L. van der Plas, Mohamed-Bachir Belaid +3Multimodal EmbeddingConcept-Based Explanations

  13. VTrajEmbed: Aligning Text with Video Object Trajectories via Spatiotemporal Embeddings

    Oct 16, 2026Ruizhi Chen, Zining Fan, Yudi Guo +5Multimodal EmbeddingVision-Language Retrieval

  14. TWIN-MoE: Two-stage Intra-Expert Imputation Network for Incomplete Multimodal Learning

    Oct 15, 2026Liang Nathan Zheng, Jiyuan Liu, Wenhao Liang +3Missing-Modality LearningMultimodal Learning

  15. Stuck on Shuffle: On Attribute-Object Binding in Vision-Language Models

    Oct 15, 2026Netanel Y. Tamir, Michael Green, Issar Tzachor +4Vision-Language RetrievalMultimodal Embedding

  16. CIM-Embed: Learning Instruction-Conditioned Multimodal Embeddings for Clustering

    Oct 15, 2026Yinghua Xu, Pengyang Xie, Yanxiang Zhou +4ClusteringMultimodal Embedding

  17. What Really Works for Universal Multimodal Embeddings? A Controlled Study of Contrastive Training Strategies

    Oct 10, 2026Duo Li, Haojun Pan, Zuhao Yang +4Multimodal EmbeddingMultimodal Contrastive Learning

  18. EarthAlign: Learning Language-Aligned Representations of Earth System Data

    Oct 10, 2026Fadillah Adamsyah Maani, Numan Saeed, Jiaqi Ma +4Cross-Modal Representation LearningMultimodal Embedding

  19. Autoregressive Retriever: Improving Query Understanding from Item Feedback for Universal Multimodal Retrieval

    Oct 8, 2026Jianfei Zhao, Yifan Wang, Feng Zhang +7Information RetrievalCross-Modal Retrieval

  20. Enhancing Long-Video VLM Embeddings with Query-Aware Streaming Latent Reasoning

    Oct 4, 2026Haozhe Chi, Song Jin, Yang Jin +1Multimodal EmbeddingVision-Language Retrieval

  21. World Embedding Benchmark

    Oct 2, 2026Yiqi Liu, Ruifeng Yuan, Yang Wang +7Physical Reasoning in VideoVideo Representation Learning

  22. Omni-Embed-Mini: Binding Modalities Without Forgetting via Dense Distillation

    Oct 1, 2026Mohammed Irfan Kurpath, Jaseel Muhammad Kaithakkodan, Sahal Shaji Mullappilly +2Cross-Modal Knowledge DistillationMultimodal Embedding

  23. Mutual Equilibrium: Multimodal Representation Learning through Reciprocal Feedback

    Sep 30, 2026Ho-min Park, Byungkon KangMultimodal GroundingMultimodal Embedding

  24. GATE-ST: Gene-Aware Text-image Encoder for Spatial Transcriptomics

    Sep 30, 2026Lucas Ni, Jian Luo, Wentao Huang +1Spatial TranscriptomicsMultimodal Embedding

  25. ResComEmb: Effective and Efficient Multimodal Embedding via Residual Homogeneity Compression

    Sep 29, 2026Zijing Cai, Yuzhe Wang, Jingxian Zhu +2Multimodal IRMultimodal Embedding

  26. Binding Multiple Modalities via Multimodal Wasserstein Barycenter

    Sep 27, 2026Xiaole Tang, Jiayi Xu, Xiang Gu +2Cross-Modal AlignmentMultimodal Embedding

  27. Learning Multimodal Embeddings with Evidence-Aligned Readout

    Sep 27, 2026Zirong Chen, Fuda Ye, Enjun Du +8Multimodal Large Language ModelsMultimodal Embedding

  28. VaME: Exploring Variational Latent Reasoning for Multimodal Embeddings

    Sep 27, 2026Peixi Wu, Mingzhou Jiang, Feipeng Ma +11Multimodal IREfficient Multimodal Inference