Overview

Cross-modal alignment studies how to connect information from different data types, such as images and text, audio and video, or brain signals and visual features, so models can retrieve, interpret, or generate across them. Common approaches use contrastive learning and shared embedding spaces, alongside optimal transport, prototypes, geometric mappings, and local or temporal correspondence; some methods rely on weakly paired or unpaired data. Work is evaluated on tasks such as image-text retrieval, grounding, zero-shot transfer, and multimodal generation, including in medical and scientific settings. A central tension is how to align shared meaning without erasing useful modality-specific information, since stronger alignment scores do not always translate into better downstream performance or consistent behavior across tasks.

Latest papers

CardsList

474 papers30 papers in the last four weeks, down 47% on the four weeks before. 0.2% of all new papers.

  1. MS-Point: Multi-Modal Hierarchical Spectral Learning for Point Cloud Understanding

    Nov 18, 2026Hai-tao Yu, Jun XiaPoint Cloud LearningPoint Cloud Understanding

  2. From Points to Trajectories: Rethinking Multimodal Alignment for Physical Events

    Nov 18, 2026Yiyang Zhu, Guangyuan Liu, Jingdi Lei +2Cross-Modal AlignmentCross-Modal Representation Learning

  3. Stabilizing Neural-to-Image Retrieval with Frequency-Domain Priors and Adaptive Supervision

    Nov 18, 2026YaoYang Liu, ZhuoShi Tang, Haopeng Lai +3EEG-Based Visual DecodingCross-Modal Alignment

  4. SIGNTOK: A Text-Aligned Continuous Pose Tokenizer for Sign Language Generation

    Nov 18, 2026Nabeela Khan, Gaurav Rai, Bowen Wu +4Sign Language GenerationCross-Modal Alignment

  5. TextSphere: Efficient Text-to-Image Generation via Shared Hyperspherical Alignment and Uniformity

    Nov 17, 2026Yixia Zhang, Girish A. Koushik, Xilu Wang +1T2I GenerationGenerative Modeling

  6. Learning What Pairs Miss for Higher-Order Multimodal Alignment

    Nov 16, 2026Kunyu Zhang, Shenghui Feng, Xiangjie Shi +2Multimodal Contrastive LearningContrastive Learning

  7. What Matters for Cross-Modal Alignment

    Nov 15, 2026Yipeng Sun, Jiajun Wang, Siyuan Mei +4Cross-Modal AlignmentVision-Language Alignment

  8. Attribution-Guided Shared-Private Decoupling for Noise-Reduced Audio-Visual Representation Learning

    Nov 14, 2026Linge Wang, Yingying Chen, Bingke Zhu +4Audio-Visual Representation LearningShared Representation Learning

  9. STaG: Human-Annotation-Free Unsupervised Domain Adaptation for Text-Based Person Retrieval

    Nov 13, 2026Ji Yucheng, Bai Yang, Li Yang +2Text-Based Person SearchUnsupervised Domain Adaptation

  10. Sub-actions in Action: Text-Guided Hand-Role Alignment for Sub-Action Recognition

    Nov 11, 2026Sadegh Rahmaniboldaji, Filip Rybansky, Quoc Vuong +2Egocentric Video UnderstandingHand-Object Interaction Detection

  11. DuoJEPA: Cross-Modal Translation Between Unaligned Self-Supervised Embedding Spaces

    Nov 11, 2026Mahmut S. Gokmen, Mitchell A. Klusty, Evan W. Damron +3Multimodal SSLCross-Modal Alignment

  12. Relaxing Cross-Modal Correspondence for Multimodal Remote Sensing Pretraining

    Nov 10, 2026Xianping Ma, Yu Zhang, Mingzhe Jiang +5Multimodal PretrainingMultimodal Learning

  13. Where Does Alignment Emerge and What Does It Mean? A Closer Look at the Platonic Representation Hypothesis

    Nov 9, 2026Julia Herbst, Simon Roschmann, Quentin Bouniot +1Platonic Representation HypothesisCross-Modal Alignment

  14. Complementarity-Driven Dual-Teacher Hierarchical Alignment for Zero-Shot EEG-to-Image Retrieval

    Nov 9, 2026Lukun Wu, Jie Li, Ziqi Ren +3EEG-Based Visual DecodingCross-Modal Alignment

  15. UniOMA: Unified Optimal-Transport Multi-Modal Structural Alignment for Robot Perception

    Nov 9, 2026Xinrui Zu, Mark Hoogendoorn, Kevin Sebastian Luck +1Cross-Modal AlignmentRobotic Perception

  16. Flow-Guided Target-Space Alignment via Path Consistency

    Nov 6, 2026Ruizhi Yuan, Zeqiu Yu, Wei Gao +3Cross-Modal AlignmentCross-Modal Representation Learning

  17. BrainBind: Bridging Synchronized EEG and fMRI through Long-Form Naturalistic Video Representations

    Nov 3, 2026Shixiao Wang, Rining Wu, Jian K. LiuCross-Modal AlignmentfMRI-Based Visual Decoding

  18. The Platonic Brain: Representational Convergence in EEG Foundation Models

    Nov 3, 2026Nicolas Waehner, Enzo Tagliazucchi, Jacobo Sitt +3Representation AlignmentPlatonic Representation Hypothesis

  19. ProAlign: Progressive Positional and Prototype-Guided Alignment for Aerial-Ground Person Re-Identification

    Nov 3, 2026Qiao Li, Jing Chen, Cong Wu +3Person Re-IdentificationCross-Modal Alignment

  20. Unimodal Augmentation-based SSL Objectives Induce Cross-Modal Alignment

    Nov 2, 2026Zohar Mashiah, Nir Ben-Ari, Ronen Talmon +1Cross-Modal AlignmentCross-Modal Representation Learning

  21. IsoAlign: Isoform-Aware DNA-Protein Alignment via Joint-Embedding Predictive Architectures

    Oct 30, 2026Niousha Sadjadi, Arman Seyed-Ahmadi, Tony Tu +8Genomic Foundation ModelsCross-Modal Alignment

  22. From Fragmentation to Fusion: A Survey and Roadmap for Omnimodal Embeddings

    Oct 30, 2026Ziyin Zhang, Yuyu Guo, Ying Zhang +15Multimodal EmbeddingCross-Modal Alignment

  23. CrossWeave: Cross-Modal Scene and Instance Retrieval from Sparse 2D-3D Alignment

    Oct 29, 2026Aadith Warrier, Gnana Prakash Punnavajhala, Siddharth Tourani +3Cross-Modal RetrievalCross-Modal Alignment

  24. Proving and Improving the Platonic Representation Hypothesis with JEPAs

    Oct 29, 2026Randall Balestriero, Yann LeCunPlatonic Representation HypothesisCross-Modal Alignment

  25. Uncertainty-Guided Cross-Modal Alignment for Diffusion-Based IoT Intrusion Detection

    Oct 29, 2026Mahdi Arab Loodaricheh, Anita Raja, Mohammad Hossein ManshaeiIoT Intrusion DetectionCross-Modal Alignment

  26. BASILISC: Bayesian Alignment of Siamese-Integrated Latents for Inferring Single-cell Correspondence

    Oct 28, 2026Sepideh Saran, Mahsa Ghanbari, Uwe OhlerCross-Modal AlignmentSingle-Cell Representation Learning