cs.AIAug 19, 2026

UMER: Unifying Embedding and Ranking via Pair-Aware Discriminative Reasoning for Universal Multimodal Retrieval

Authors: Libiao Chen, Xiyang Liu, Yanheng Wei, Tao Wang, Zhenyu Tang

Organizations: Beihang University, Beijing, China

Abstract

Universal multimodal retrieval aims to support diverse instruction-aware retrieval tasks, demanding both efficient corpus-scale matching and fine-grained semantic reasoning. Recent MLLM-based embedding methods typically derive representations from hidden states, while Chain-of-Thought (CoT) reasoning is emerging as a promising strategy for embedding enhancement by encoding intermediate semantic evidence into the representation space. However, existing CoT methods typically use item-wise reasoning over queries and candidates in isolation, providing no explicit evidence to distinguish a positive from a semantically confusable hard negative. Moreover, contrastive embeddings capture global similarity but struggle with meta-tasks requiring answer verification, category judgment or fine-grained reasoning. In this paper, we propose UMER, a Unified Multimodal Embedding and Ranking framework for universal multimodal retrieval. UMER replaces item-wise reflection with Pair-Aware Discriminative Reasoning, which compares query--candidate pairs to identify instruction-relevant matching and discrepancy evidence. UMER jointly learns contrastive embeddings for efficient global matching and discriminative ranking for explicit pairwise relevance judgment within a single MLLM. A complementary mutual distillation strategy further transfers reliable pairwise preferences between the embedding and ranking functions. On the MMEB-V2 benchmark, UMER achieves state-of-the-art performance under comparable experimental settings while supporting budget-adjustable inference.

Explore similar work

CardsList
  1. Reason What Matters: Retrieval-Grounded Reasoning for Universal Multimodal Embeddings

    Sep 14, 2026Mingzhou Jiang, Peixi Wu, Hang Cheng +7Multimodal IREfficient Multimodal Inference

  2. PLUME: Latent Reasoning Based Universal Multimodal Embedding

    Apr 2, 2026Chenwei He, Xiangzhao Hao, Tianyu Yang +6Multimodal EmbeddingMultimodal IR

  3. Is Reasoning Always Useful? Rethinking Reasoning Utility in Universal Multimodal Embeddings

    Aug 26, 2026Wenxiao Fan, Jingling Fu, Luohang Liu +6Multimodal EmbeddingMultimodal IR