Cross-Modal Representation Learning

Overview

The field studies how models can represent different kinds of data so that related observations can be matched, combined, or used to transfer knowledge. Common approaches include CLIP-style contrastive alignment, masked cross-modal reconstruction and predictive learning, teacher–student distillation, and methods that separate shared from modality-specific features; anchors and optimal transport can help when paired examples are scarce. These representations support image-text retrieval, cross-sensor recognition, and transfer across domains or missing-modality settings. A central tension is that shared spaces can improve transfer but discard useful modality-specific structure, and apparent semantic alignment does not always imply fine-grained correspondence; the best choice between alignment and prediction depends on data quality and cross-modal relationships.

Latest papers

CardsList

224 papers17 papers in the last four weeks, down 15% on the four weeks before. 0.1% of all new papers.

  1. CLCR: Cross-Level Semantic Collaborative Representation for Multimodal Learning

    Feb 23, 2026Chunlei Meng, Guanhong Huang, Rong Fu +3Cross-Modal Representation LearningMultimodal Learning

  2. MoBind: Motion Binding for Fine-Grained IMU-Video Pose Alignment

    Feb 22, 2026Duc Duy Nguyen, Tat-Jun Chin, Minh HoaiCross-Modal Representation LearningCross-Modal Alignment

  3. SpectralGCD: Spectral Concept Selection and Cross-modal Representation Learning for Generalized Category Discovery

    Feb 19, 2026Lorenzo Caselli, Marco Mistretta, Simone Magistri +1Cross-Modal LearningCross-Modal Representation Learning

  4. Semantic Purification for Conditional Representation Learning

    Feb 5, 2026Jiaquan Wang, Yan Lyu, Chen Li +1Disentangled Representation LearningRepresentation Learning

  5. Superman: Unifying Skeleton and Vision for Human Motion Perception and Generation

    Feb 2, 2026Xinshun Wang, Peiming Li, Ziyi Wang +5Cross-Modal Representation LearningUnified Multimodal Models

  6. Cross-Modal Visuo-Tactile Representation Learning with Action Chunking Transformers for Contact-Rich Manipulation

    Jan 31, 2026Yaohua Liu, Rong Fu, Amir H. Gandomi +2Representation LearningCross-Modal Representation Learning

  7. PPGFlowECG: Latent Rectified Flow with Cross-Modal Encoding for PPG-Guided ECG Generation and Cardiovascular Disease Detection

    Sep 24, 2025Xiaocheng Fang, Jiarui Jin, Haoyu Wang +8ElectrocardiographyConditional Generation

  8. Inverse-LLaVA: Rethinking Multimodal Alignment via Text-to-Vision Mapping

    Aug 17, 2025Xuhui Zhan, Tyler DerrCross-Modal LearningCross-Modal Representation Learning

  9. Platonic Representations for Poverty Mapping: Unified Vision-Language Codes or Agent-Induced Novelty?

    Aug 1, 2025Satiyabooshan Murugaboopathy, Connor T. Jerzak, Adel DaoudCross-Modal Representation LearningMultimodal Learning

  10. Heterogeneous-Modal Unsupervised Domain Adaptation via Latent Space Bridging

    Jun 19, 2025Jiawen Yang, Shuhao Chen, Shengtao Zhang +2Unsupervised Domain AdaptationCross-Modal Alignment

  11. Multimodal Representation Alignment for Cross-modal Information Retrieval

    Jun 10, 2025Fan Xu, Luis A. LeivaCross-Modal LearningMultimodal IR

  12. A quantitative analysis of semantic information in deep representations of text and images

    May 21, 2025Santiago Acevedo, Andrea Mascaretti, Riccardo Rende +3Representational Similarity AnalysisCross-Modal Representation Learning

  13. On the Value of Cross-Modal Misalignment in Multimodal Representation Learning

    Apr 14, 2025Yichao Cai, Yuhang Liu, Erdun Gao +4Cross-Modal Representation LearningCross-Modal Alignment