cs.CVNov 4, 2026

Semantic Prior-Guided Incomplete Cross-Modal Retrieval with Noisy Labels

Authors: Jiali Chen, Ruitao Pu, Yingke Chen, Dezhong Peng

Organizations: College of Computer Science, Sichuan University, Chengdu 610065, China · Department of Computer and Information Sciences, Northumbria University, Newcastle upon Tyne NE1 8ST, UK · Tianfu Jincheng Laboratory, Chengdu 610093, China

Abstract

Cross-modal retrieval (CMR) aims to bridge the semantic gaps between modalities by learning shared representations. Unfortunately, due to the dynamism and inconsistencies in real-world data acquisition, multimodal datasets frequently encounter severe modality incompleteness. Existing incomplete CMR methods adhere to a label-driven paradigm, relying heavily on categorical annotations as the semantic benchmark. However, label noise is difficult to avoid and often coexists with modality incompleteness. To address this challenge, we propose a novel semantic prior-guided framework for incomplete cross-modal retrieval with noisy labels, termed SPIRN. Specifically, SPIRN uses fixed semantic priors as a consistent reference for robust representation learning, noisy-label handling, and missing-modality completion. Based on this reference, Semantic Priors Guidance (SPG) establishes a stable category-level semantic space, Quality-aware Robust Learning (QRL) dynamically evaluates sample quality to distinguish clean samples from noisy ones and handles noise via reverse learning, Prior-guided Global Label Calibration (PGLC) rectifies corrupted annotations for refined supervision, and Confidence-driven Selective Imputation (CSI) recovers missing modalities from a cross-modal memory bank while filtering unreliable imputations with an increasing threshold. To the best of our knowledge, this is the first framework to jointly tackle the dual challenges of modality incompleteness and label noise in the CMR community. Extensive experiments on four widely used benchmark datasets demonstrate that SPIRN outperforms 11 state-of-the-art methods in various scenarios involving both missing modalities and noisy labels.

Figures & tables

Explore similar work

CardsList
  1. Intra-Modal Neighbors Never Lie: Rectifying Inter-Modal Noisy Correspondence via Graph-Based Intra-Modal Reasoning

    Jun 2, 2026Yang Liu, Wentao Feng, Shu-Dong Huang +2Cross-Modal AlignmentCross-Modal Retrieval

  2. General Incomplete Multimodal Learning via Dynamic Quality Perception

    Jul 8, 2026Xiangyu Meng, Shicai WeiMissing-Modality LearningMultimodal Robustness

  3. MIDAS: Mutual Information Disentanglement with Uncertainty-Aware Fusion for Incomplete Multimodal Sentiment Analysis

    Aug 6, 2026Yuhua Wen, Yingying Zhou, Qifei Li +4Missing-Modality LearningMultimodal Disentangled Representation Learning