Cross-Modal Retrieval

Cross-modal retrieval studies how to find relevant items when a query and its targets use different modalities, such as searching videos with text or matching brain signals to images. A common approach trains modality-specific encoders to map data into a shared space using contrastive learning, while other methods align local regions, frames, or tokens for finer matches. Systems also use query-aware fusion, candidate reranking, reasoning, or compact hashing to handle complex searches and large collections. A central challenge is balancing scalable global similarity with compositional detail and robustness to noisy, missing, or mismatched modalities; benchmarks increasingly test these harder settings.