Cross-Modal Learning

Follow this topic Ask about this topic

Momentum

2 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 127

All topics

Multimodal Distribution Matching for Vision-Language Dataset Distillation
May 22, 2026Jongoh Jeong, Hoyong Kwon, Minseok Kim +1Cross-Modal Learning Cross-Modal Alignment
Visual Intelligence Lab., KAIST
Balancing Multimodal Learning through Label Space Reshaping
May 22, 2026Xiaoyu Ma, Weijie Zhang, Yuanhao Gao +3Cross-Modal Learning Cross-Modal Representation Learning
PREPRINT 2026.05
Inconsistency-aware Multimodal Schrödinger Bridge for Deepfake Localization
May 22, 2026Jiayu Xiong, Jing Wang, Qi Zhang +2Cross-Modal Learning Multimodal Robustness
Department of Computer Science and Techonology, Huaqiao University · Xiamen Key Laboratory of Computer Vision and Pattern Recognition, Huaqiao University · Tongji University +1
Supervised Classification Heads as Semantic Prototypes: Unlocking Vision-Language Alignment via Weight Recycling
May 21, 2026David Méndez, Roberto Confalonieri, Natalia Díaz RodríguezCross-Modal Learning VLM Adaptation
Department of Computer Science and Artificial Intelligence, DaSCI Institute, University of Granada, Granada, Spain · Department of Mathematics “Tullio Levi-Civita”, University of Padova, Padova, Italy.
LatentOmni: Rethinking Omni-Modal Understanding via Unified Audio-Visual Latent Reasoning
May 21, 2026Yifan Dai, Zhenhua Wu, Bohan Zeng +18Cross-Modal Learning Audio-Visual Understanding
School of AI, Shanghai Jiao Tong University · 2Kling Team, Kuaishou Technology · 3Peking University +5
STiTch: Semantic Transition and Transportation in Collaboration for Training-Free Zero-Shot Composed Image Retrieval
May 20, 2026Miaoge Li, Dongsheng Wang, Zening Sun +3Cross-Modal Learning Cross-Modal Alignment
Department of COMP/LSGI, The Hong Kong Polytechnic University, Hong Kong SAR · College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China · The Hong Kong University of Science and Technology, Hong Kong SAR
Multimodal LLMs under Pairwise Modalities
May 20, 2026Yan Li, Yunlong Deng, Yuewen Sun +3Cross-Modal Learning Representation Identifiability
Mohamed bin Zayed University of Artificial Intelligence · Carnegie Mellon University
LLM Pretraining Shapes a Generalizable Manifold: Insights into Cross-Modal Transfer to Time Series
May 19, 2026Alexis Roger, Prateek Humane, Zhenghan Tai +4Cross-Modal Learning Language Model Pretraining
Université de Montréal Mila - Quebec AI Institute
MetaEarth-MM: Unified Multimodal Remote Sensing Image Generation with Scene-centered Joint Modeling
May 19, 2026Zhiping Yu, Chenyang Liu, Jinqi Cao +4Cross-Modal Learning Unified Multimodal Models
Department of Aerospace Intelligent Science and Tech-nology, School of Astronautics, Beihang University, Beijing 100191, China · State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing 100191, China · Shenyuan Honors College, Beihang University, Beijing 100191, China
AffectVerse: Emotional World Models for Multimodal Affective Computing
May 19, 2026Bo Zhao, Fanghua Ye, Yixin Ji +3Cross-Modal Learning Multimodal Large Language Models
1Great Bay University · 2Tencent · 3Tsinghua University +1
Fine-tuning language encoding models on slow fMRI improves prediction for fast ECoG
May 19, 2026Aditya R. Vaidya, Richard J. Antonello, Alexander G. HuthCross-Modal Learning Transfer Learning
Department of Computer Science The University of Texas at Austin, USA · Zuckerman Mind Brain Behavior Institute Columbia University, USA · Departments of Neuroscience and Statistics University of California, Berkeley, USA
Chronicle: A Multimodal Foundation Model for Joint Language and Time Series Understanding
May 18, 2026Paul Quinlan, Jeremy Levasseur, Qingguo Li +1Cross-Modal Learning Multivariate Time Series Forecasting
1InertialAI · Department of Electrical and Computer Engineering, Queen’s University · Department of Mechanical and Materials Engineering, Queen’s University
Weakly Supervised Cross-Modal Learning for 4D Radar Scene Flow Estimation
May 18, 2026Jingyun Fu, Zhiyu Xiang, Na ZhaoCross-Modal Learning Autonomous Driving Perception
Zhejiang University, Hangzhou, Zhejiang, China · Zhejiang Provincial Key Laboratory of Multi-Modal Communication Networks and Intelligent Information Processing, Zhejiang, China · Singapore University of Technology and Design, Singapore.
CodeBind: Decoupled Representation Learning for Multimodal Alignment with Unified Compositional Codebook
May 18, 2026Zeyu Chen, Jie Li, Kai HanCross-Modal Learning Multimodal Disentangled Representation Learning
Visual AI Lab, The University of Hong Kong
See What I Mean: Aligning Vision and Language Representations for Video Fine-grained Object Understanding
May 18, 2026Boyuan Sun, Bowen Yin, Yuanming Li +2Cross-Modal Learning Video Object Segmentation
VCIP, CS, Nankai University · Tongyi Lab, Alibaba Group · NKIARI, Shenzhen Futian
LatentUMM: Dual Latent Alignment for Unified Multimodal Models
May 18, 2026Yinyi Luo, Wenwen Wang, Hayes Bai +2Cross-Modal Learning Unified Multimodal Models
Carnegie Mellon University · William & Mary
t-gems: text-guided exit modules for decreasing clip image encoder
May 17, 2026Alberto Presta, Grzegorz Stefanski, Michal Byra +1Cross-Modal Learning Vision-Language Models
Samsung AI Center Warsaw · Institute of Fundamental Technological Research, PAS, Warsaw
Mind the Gap: Learning Modality-Agnostic Representations with a Cross-Modality UNet
May 16, 2026Xin Niu, Enyi Li, Jinchao Liu +3Cross-Modal Learning Cross-Modal Representation Learning
Tianjin Key Laboratory of Intelligent Robotics, College of Artificial Intelligence, Nankai University, China, and also with Engineering Research Center of Trusted Behavior Intelligence, Ministry of Education, Nankai University, China. · VisionMetric Ltd, Canterbury, Kent, UK. · Department of Computer Science, Haifa University, Israel.
Learning Relative Representations for Fine-Grained Multimodal Alignment with Limited Data
May 16, 2026Shiwon Kim, Yu Rang ParkCross-Modal Learning Representation Learning
Yonsei University
Cross-modal Affinity-aligned Multimodal Learning Analytics for Predicting Student Collaboration Satisfaction in Game-Based Learning
May 16, 2026Wen-Hsin Tsai, Chia-Ming Lee, Yuk-Ying TungCross-Modal Learning Multimodal Robustness
Institute of Education, National Cheng Kung University · Institute of Intelligent System, National Yang Ming Chiao Tung University · Department of Computer Science, University at Albany, State University of New York
SOLAR: Self-supervised Joint Learning for Symmetric Multimodal Retrieval
May 15, 2026Wenjie Yang, Hang Yu, Yuyu Guo +1Cross-Modal Learning Multimodal IR
Ant Group
GOMA: Toward Structure-Driven Multimodal Alignment from a Graph Signal Smoothing Perspective
May 15, 2026Xu Wang, Xunkai Li, Yinlin Zhu +2Cross-Modal Learning Multimodal IR
School of Airspace Science and Engineering, Shandong University, WeiHai, China · Department of Computer Science, Beijing Institute of Technology, Beijing, China · School of Computer Science and Engineering, Sun Yat-sen University, GuangZhou, China
Differentially Private Motif-Preserving Multi-modal Hashing
May 14, 2026Zehua Cheng, Wei Dai, Jiahao SunCross-Modal Learning Cross-Modal Retrieval
Department of Computer Science University of Oxford Oxford, United Kingdom · FLock.io London, United Kingdom
GeoFlowVLM: Geometry-Aware Joint Uncertainty for Frozen Vision-Language Embedding
May 13, 2026Mayank Nautiyal, Li Ju, Andreas Hellander +2Cross-Modal Learning Flow Matching
Department of Information Technology, Uppsala University, Uppsala, Sweden · SciLifeLab, Uppsala University, Uppsala, Sweden
SMA: Submodular Modality Aligner For Data Efficient Multimodal Learning
May 13, 2026Truong Pham, Anay Majee, Rishabh IyerCross-Modal Learning Cross-Modal Alignment
The University of Texas at Dallas
SynIB: Informational Bottleneck for Maximizing Synergy in Multimodal Learning
May 12, 2026Konstantinos Kontras, Teodora Gagaleska, Thomas Strypsteen +4Cross-Modal Learning Multimodal Learning
1KU Leuven · 2MIT
Cross-Modal-Domain Generalization Through Semantically Aligned Discrete Representations
May 12, 2026Souptik Sen, Raneen Younis, Zahra AhmadiCross-Modal Learning Representation Learning
Peter L. Reichertz Institute for Medical Informatics, Hannover Medical School, Germany · Lower Saxony Center for AI and Causal Methods in Medicine (CAIMed), Hannover, Germany
Boosting Omni-Modal Language Models: Staged Post-Training with Visually Debiased Evaluation
May 12, 2026Che Liu, Lichao Ma, Xiangyu Tony Zhang +4Cross-Modal Learning Multimodal Large Language Models
StepFun-Audio Team
Probing Cross-modal Information Hubs in Audio-Visual LLMs
May 11, 2026Jihoo Jung, Chaeyoung Jung, Ji-Hoon Kim +1Cross-Modal Learning Multimodal Large Language Models
Department of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea · The Graduate School of Advanced Imaging Science, Multimedia & Film, Chung-Ang University, Seoul, Republic of Korea
Dynamic Cross-Modal Prompt Generation for Multimodal Continual Instruction Tuning
May 11, 2026Tao Hu, Da-Wei ZhouCross-Modal Learning Continual Learning for LLMs
School of Artificial Intelligence, Nanjing University · State Key Laboratory for Novel Software Technology, Nanjing University

May 22, 2026, cs.CVJ/K move · Enter open · S save

Multimodal Distribution Matching for Vision-Language Dataset Distillation

Jongoh Jeong, Hoyong Kwon, Minseok Kim, Kuk-Jin Yoon

Visual Intelligence Lab., KAIST

Cross-Modal Learning Cross-Modal Alignment VLM Distillation Distribution Matching Dataset Distillation

Open paper AskPDF arXiv

Dataset distillation compresses large training sets into compact synthetic datasets while preserving downstream performance. As modern systems increasingly operate on paired vision-language inputs, multimodal distillation must preserve representation quality and cross-modal alignment under tight compute and memory budgets, yet prior methods often require heavy computes and overlook their correlations. To address this, we present Multimodal Distribution Matching (MDM), a geometry-aware framework for efficient and generalizable multimodal distillation. Specifically, MDM integrates complementary components at the data, model, and loss levels. At the data level, it initializes synthetic image-text pairs by sampling from clusters in the joint embedding space. At the model level, it forms a mixed teacher by interpolating independently fine-tuned models in weight space according to their angular deviation from the pretrained anchor. At the loss level, it matches joint distributions on the unit hypersphere using a geometry-aware matching objective that exploits the joint features in the cross-modal agreement and discrepancy directions along with symmetric contrastive learning. Across image-text retrieval benchmarks with cross-architecture evaluation, MDM yields compact synthetic sets that preserve multimodal semantics, substantially reduce distillation cost, and remain robust across architectures.

Momentum

Related topics

Latest papers 127