Cross-Modal Learning

Follow this topic Ask about this topic

Momentum

2 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 127

All topics

Context Unrolling in Omni Models
Apr 23, 2026Ceyuan Yang, Zhijie Lin, Yang Zhao +16Cross-Modal Learning Unified Multimodal Models
MiMIC: Mitigating Visual Modality Collapse in Universal Multimodal Retrieval While Avoiding Semantic Misalignment
Apr 23, 2026Juan Li, Chuanghao Ding, Xujie Zhang +1Cross-Modal Learning Cross-Modal Alignment
State Key Laboratory for Novel Software Technology, Nanjing University
UniCVR: From Alignment to Reranking for Unified Zero-Shot Composed Visual Retrieval
Apr 22, 2026Haokun Wen, Xuemeng Song, Haoyu Zhang +3Cross-Modal Learning Learning to Rank
Harbin Institute of Technology (Shenzhen) Shenzhen, China · Southern University of Science and Technology Shenzhen, China · Pengcheng Laboratory Shenzhen, China +1
DUALVISION: RGB-Infrared Multimodal Large Language Models for Robust Visual Reasoning
Apr 20, 2026Abrar Majeedi, Zhiyuan Ruan, Ziyi Zhao +3Cross-Modal Learning Visual Question Answering
University of Wisconsin-Madison · Amazon · Northeastern University
Back into Plato's Cave: Examining Cross-modal Representational Convergence at Scale
Apr 20, 2026A. Sophia Koepke, Daniil Zverev, Shiry Ginosar +1Cross-Modal Learning Cross-Modal Representation Learning
UC Berkeley · Technical University Munich, MCML · University of Tübingen, Tübingen AI Center +1
ReTrack: Evidence-Driven Dual-Stream Directional Anchor Calibration Network for Composed Video Retrieval
Apr 20, 2026Zixu Li, Yupeng Hu, Zhiwei Chen +4Cross-Modal Learning Cross-Modal Representation Learning
School of Software, Shandong University · School of Computer Science and Technology, Shandong Jianzhu University
BioVLM: Routing Prompts, Not Parameters, for Cross-Modality Generalization in Biomedical VLMs
Apr 19, 2026Mainak Singha, Tanisha Gupta, Ankit Jha +3Cross-Modal Learning Prompt Learning
University of Trento, Italy · Carnegie Mellon University, USA · LNMIIT Jaipur, India +3
Cross-Modal Attention Analysis and Optimization in Vision-Language Models: A Study on Visual Reliability
Apr 19, 2026Lijie ZhouCross-Modal Learning VLM Robustness
School of Computer Science University of Nottingham Ningbo China Ningbo, China
Better with Less: Tackling Heterogeneous Multi-Modal Image Joint Pretraining via Conditioned and Degraded Masked Autoencoder
Apr 18, 2026Bowen Peng, Yongxiang Liu, Jie Zhou +4Cross-Modal Learning Masked Autoencoders
College of Electronic Science and Technology, National University of Defense Technology (NUDT), Changsha 410073, China · Beijing Institute of Remote Sensing Information, Beijing 100192, China
PivotMerge: Bridging Heterogeneous Multimodal Pre-training via Post-Alignment Model Merging
Apr 18, 2026Zibo Shao, Baochen Xiong, Xiaoshan Yang +4Cross-Modal Learning Cross-Modal Alignment
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · Pengcheng Laboratory, Shenzhen 518066, China · School of Artificial Intelligence, University of Chinese Academy Sciences, Beijing 100049, China +1
CrossFlowDG: Bridging the Modality Gap with Cross-modal Flow Matching for Domain Generalization
Apr 18, 2026Antonios Kritikos, Nikolaos Spanos, Athanasios VoulodimosCross-Modal Learning Cross-Modal Alignment
National Technical University of Athens
Channel Attention-Guided Cross-Modal Knowledge Distillation for Referring Image Segmentation
Apr 18, 2026Chen YangCross-Modal Learning Cross-Modal Knowledge Distillation
School of Information Science and Technology, TaiShan University, Taian, China
AVRT: Audio-Visual Reasoning Transfer through Single-Modality Teachers
Apr 17, 2026Edson Araujo, Saurabhchand Bhati, M. Jehanzeb Mirza +5Cross-Modal Learning Audio-Visual Understanding
University of Tübingen, Germany · MIT, Cambridge MA, USA · IBM Research, USA +2
Joint-Centric Dual Contrastive Alignment with Structure-Preserving and Information-Balanced Regularization
Apr 17, 2026Habibeh Naderi, Behrouz Haji Soleimani, Stan MatwinCross-Modal Learning Modality Imbalance
Dalhousie University, Halifax NS, Canada
SIMMER: Cross-Modal Food Image--Recipe Retrieval via MLLM-Based Embedding
Apr 17, 2026Keisuke Gomi, Keiji YanaiCross-Modal Learning Image-Text Retrieval
The University of Electro-Communications
MG $^2$ -RAG: Multi-Granularity Graph for Multimodal Retrieval-Augmented Generation
Apr 4, 2026Sijun Dai, Qiang Huang, Xiaoxing You +1Cross-Modal Learning Multimodal RAG
School of Intelligence Science and Engineering, Harbin Institute of Technology (Shenzhen) · School of Computer Science, Hangzhou Dianzi University
CoLA: Cross-Modal Low-rank Adaptation for Multimodal Downstream Tasks
Apr 1, 2026Wish Suharitdamrong, Tony Alex, Muhammad Awais +1Cross-Modal Learning VLM Adaptation
Surrey Institute for People-Centred AI, University of Surrey, Guildford, GU2 7XH, UK · Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, UK
UniMotion: A Unified Framework for Motion-Text-Vision Understanding and Generation
Mar 23, 2026Ziyi Wang, Xinshun Wang, Shuang Chen +2Cross-Modal Learning Unified Multimodal Models
State Key Laboratory of General Artificial Intelligence, Peking University, Shenzhen Graduate School, China · Donghua University, China · School of Automation Science and Engineering, South China University of Technology
Rethinking Multimodal Fusion for Time Series: Text Modalities Need Constrained Fusion
Mar 23, 2026Seunghan Lee, Jun Seo, Jaehoon Lee +7Cross-Modal Learning Time Series Forecasting
LG AI Research Seoul, South Korea
ITO: Multi-View Alignment and Training-Time Fusion for Image-Text Pretraining
Mar 3, 2026Hanpeng Liu, Zidan Wang, Shuoxi Zhang +6Cross-Modal Learning Cross-Modal Alignment
School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China · Li Auto Inc.
SOTAlign: Semi-Supervised Alignment of Unimodal Vision and Language Models via Optimal Transport
Feb 26, 2026Simon Roschmann, Paul Krzakala, Sonia Mazelet +2Cross-Modal Learning Multimodal Embedding
Helmholtz Munich · Technical University of Munich · Munich Center for Machine Learning +3
MMLoP: Multi-Modal Low-Rank Prompting for Efficient Vision-Language Adaptation
Feb 24, 2026Sajjad Ghiasvand, Haniyeh Ehsani Oskouie, Mahnoosh Alizadeh +1Cross-Modal Learning Prompt Learning
University of California, Santa Barbara · University of California, Los Angeles
Audio-Visual Continual Test-Time Adaptation without Forgetting
Feb 20, 2026Sarthak Kumar Maharana, Akshay Mehra, Bhavya Ramakrishna +2Cross-Modal Learning Source-Free Domain Adaptation
The University of Texas at Dallas · Dolby Laboratories
SpectralGCD: Spectral Concept Selection and Cross-modal Representation Learning for Generalized Category Discovery
Feb 19, 2026Lorenzo Caselli, Marco Mistretta, Simone Magistri +1Cross-Modal Learning Cross-Modal Representation Learning
University of Florence, Media Integration and Communication Center (MICC), Italy
The Geometric Mechanics of Contrastive Representation Learning: Alignment Potentials, Entropic Dispersion, and Cross-modal Divergence
Jan 27, 2026Yichao Cai, Zhen Zhang, Yuhang Liu +1Cross-Modal Learning Contrastive Learning
Australian Institute for Machine Learning, Adelaide University · Responsible AI Research Centre
CLIMP: Contrastive Language-Image Mamba Pretraining
Jan 11, 2026Nimrod Shabtay, Itamar Zimerman, Eli Schwartz +1Cross-Modal Learning VLM Robustness
Tel-Aviv University · IBM Research
VLM2GeoVec: Toward Universal Multimodal Embeddings for Remote Sensing
Dec 12, 2025Emanuel Sánchez Aimar, Gulnaz Zhambulova, Fahad Shahbaz Khan +2Cross-Modal Learning Multimodal Embedding
Linköping University, Sweden
Theoretical Refinement of CLIP by Utilizing Linear Structure of Optimal Similarity
Oct 17, 2025Naoki Yoshida, Satoshi Hayakawa, Yuhta Takida +3Cross-Modal Learning Kernel Mean Embeddings
The University of Tokyo · Sony Group Corporation · Sony AI
FairSSL: Fair Multimodal Self-Supervised Learning
Aug 22, 2025Jiaee Cheong, Abtin Mogharabin, Paul Liang +2Cross-Modal Learning Representation Learning
Harvard University · University of Cambridge · Middle East Technical University +1
Inverse-LLaVA: Rethinking Multimodal Alignment via Text-to-Vision Mapping
Aug 17, 2025Xuhui Zhan, Tyler DerrCross-Modal Learning Cross-Modal Representation Learning
Data Science Institute, Vanderbilt University, 2201 West End Ave, Nashville, 37235, Tennessee, United States. · Computer Science Department, Vanderbilt University, 2201 West End Ave, Nashville, 37235, Tennessee, United States.

Apr 23, 2026, cs.CVJ/K move · Enter open · S save

Context Unrolling in Omni Models

Ceyuan Yang, Zhijie Lin, Yang Zhao, Fei Xiao+15

Cross-Modal Learning Unified Multimodal Models Multimodal Pretraining Multimodal Reasoning Multimodal Generation Multimodal Foundation Models+1

Open paper AskPDF arXiv

We present Omni, a unified multimodal model natively trained on diverse modalities, including text, images, videos, 3D geometry, and hidden representations. We find that such training enables Context Unrolling, where the model explicitly reasons across multiple modal representations before producing predictions. This process enables the model to aggregate complementary information across heterogeneous modalities, facilitating a more faithful approximation of the shared multimodal knowledge manifold and improving downstream reasoning fidelity. As a result, Omni achieves strong performance on both multimodal generation and understanding benchmarks, while demonstrating advanced multimodal reasoning capabilities, including in-context generation of text, image, video, and 3D geometry.

Momentum

Related topics

Latest papers 127