Overview

Cross-modal alignment studies how to connect information from different data types, such as images and text, audio and video, or brain signals and visual features, so models can retrieve, interpret, or generate across them. Common approaches use contrastive learning and shared embedding spaces, alongside optimal transport, prototypes, geometric mappings, and local or temporal correspondence; some methods rely on weakly paired or unpaired data. Work is evaluated on tasks such as image-text retrieval, grounding, zero-shot transfer, and multimodal generation, including in medical and scientific settings. A central tension is how to align shared meaning without erasing useful modality-specific information, since stronger alignment scores do not always translate into better downstream performance or consistent behavior across tasks.

Latest papers

CardsList

474 papers30 papers in the last four weeks, down 47% on the four weeks before. 0.2% of all new papers.

  1. Separability Is Not Alignability: Representations for Spectrum-to-Molecule Retrieval

    Oct 27, 2026Karol Musielinski, Fanurs C. E. Teh, Takahiko Koyama +1Molecular Representation LearningMass Spectrometry

  2. Correspondence Is Not Transfer: A Pairing-Controlled Study of Clinical Audio-Metadata Alignment

    Oct 26, 2026Liu He, Xi Wang, Jiahong Yuan +1Cross-Modal AlignmentAudio Understanding

  3. Closing the Modality Gap via Simplex-Constrained Representations

    Oct 23, 2026Shubham Gupta, Siva Reddy, Perouz Taslakian +2Modality Gap in VLMsCross-Modal Alignment

  4. GeoLEON-CLIP: Graph-Anchored Multimodal Contrastive Learning for Geographic Region Representation

    Oct 23, 2026Radoslaw Malek, Szymon Soltysiak, Aleksandra Kawala-Sterniuk +1Geospatial Representation LearningMultimodal Contrastive Learning

  5. Hierarchical Fine-Grained Cross-Modal Alignment via Dynamic Granularity Discovery and Optimal Transport

    Oct 22, 2026Pengfeng Song, Shaozhong Zhang, Pengfei ShaoImage-Text RetrievalCross-Modal Alignment

  6. SenseAlign: Understanding Vision and Touch through Language at Test Time

    Oct 20, 2026Chuyang Ye, Qinting Jiang, Jingyan JiangMultimodal UnderstandingCross-Modal Alignment

  7. ODSS-Net: Orthogonally Decoupled Shared-Specific Representations for Multi-Modal Object Re-Identification

    Oct 16, 2026Zhifeng Ma, Yuhong Feng, Chongyi Huang +2Multimodal Disentangled Representation LearningMultimodal Fusion

  8. MSP: Modality Self-Play for Training Multimodal LLMs Without Paired Cross-Modal Data

    Oct 14, 2026Robert Moseley, Md Kaykobad Reza, Ameya Patil +2Multimodal LearningMultimodal Foundation Models

  9. FACES: Fine-grained Gated Semantic Alignment for Multimodal Emotional Support Conversation

    Oct 14, 2026Hang Ren, Peidong Wang, Yichen Gao +4Emotional Support ConversationMulti-Objective Language Model Alignment

  10. Revisiting Triplet Contrastive Learning for 3D Shape Understanding

    Oct 14, 2026Chaoyang Zhu, Zhe Liao, Jiayi Zhang +33D Representation LearningCross-Modal Alignment

  11. Focus, Align, and Diffuse: Time-Series-Aware Keyframe Diffusion for Cardiac Dynamic Synthesis from Sparse Observations

    Oct 13, 2026Junkai Liu, Haofan Wu, Nay Aung +3Medical Image SynthesisVideo Diffusion Models

  12. To Align or Not To Align: Check Your COMPASS Before You Train

    Oct 13, 2026Fabian Gröger, Petar Damjanović, Maria BrbićLanguage Model MergingCross-Modal Alignment

  13. Multimodal Fusion of RGB and Complementary Modalities for Semantic Segmentation

    Oct 13, 2026Hu Cao, Yifeng Cheng, Mu He +3Multimodal Semantic SegmentationMultimodal Fusion

  14. Omni-Modal LLMs Know More Than They Say

    Oct 12, 2026Na Min An, Taekyung Kim, Jaemyung Yu +2Multimodal Large Language ModelsCross-Modal Alignment

  15. Silence and Noise: Resolving Suppression and Overvaluation in Multimodal Entity Alignment

    Oct 12, 2026Xizhe Zhang, Jingfeng Zhang, Meng-Fen ChiangCross-Modal AlignmentMultimodal KGs

  16. Rethinking Human Intent-to-CAD: Parametric CAD Model Generation via Cooperative Multi-Task Alignment and Spatial-Aware Reinforcement Learning

    Oct 11, 2026Qingwang Zhang, Jiahao Li, Xiangdong ZhouParametric CAD ModelingText-to-CAD Generation

  17. TartanMatch: Towards Universal Dense Correspondence Across Modalities

    Oct 10, 2026Hyeokjoon Kwon, Jiting Cai, Ruogu Li +7Cross-Modal Feature MatchingUnified Multimodal Models

  18. ROSETTA: Rotation Subspace Estimation for Training-free Transmodal Alignment

    Oct 9, 2026Alex Junho LeeCross-Modal AlignmentLatent Space Alignment

  19. Representation Alignment from a Few Paired Data via Distilled Optimal Transport

    Oct 8, 2026Julien Kordes, Reuben DorentCross-Modal AlignmentCross-Modal Representation Learning

  20. PXtal: Learning to Align Powder X-Ray Diffraction and Crystal Structures under Information Asymmetry across Modalities

    Oct 7, 2026Zhuoran Yang, Christopher M. Collins, Bei Peng +3Cross-Modal AlignmentMultimodal Learning

  21. ORCA: Hunting Compositional Failures in Text-to-Image Diffusion

    Oct 7, 2026Arshia Hemmat, Amirhossein Vahidi, Amitis Shidani +4Compositional T2I GenerationT2I Diffusion Models

  22. Unpaired Canonical Correlation Analysis

    Oct 7, 2026Nir Ben-Ari, Ronen Talmon, Uri ShahamCanonical Correlation AnalysisMulti-View Learning

  23. Shared Geometry As A Rosetta Stone: Cross-Modal Alignment Without Paired Data

    Oct 7, 2026Dominik Schnaus, Thomas Dagès, Daniel Cremers +2Cross-Modal AlignmentCross-Modal Representation Transfer

  24. Beyond Perturbation Magnitude: Direction-Dependent Responses in Multimodal Geometric Representations

    Oct 6, 2026Yongsheng Luo, Wengan He, Yu Li +2Neural Representation GeometryMultimodal Robustness

  25. Dynamic Alignment and Calibration for Multimodal Learning

    Oct 6, 2026Jinghao Xu, Zhenhua Guo, Xiaofeng Zhu +1Multimodal RobustnessCross-Modal Alignment

  26. From Transformation to Target State: Rethinking Query Representation for Zero-Shot Composed Image Retrieval

    Oct 5, 2026Yihe Zhao, Songhe FengCross-Modal AlignmentComposed Image Retrieval

  27. Kinematics-Centric Continuous Sign Language Retrieval with Gloss-Guided Boundary-Aware Alignment

    Oct 4, 2026Chang Liu, Ke Han, Davide Talon +2Cross-Modal AlignmentMotion Representation Learning