Cross-Modal Learning

Follow this topic Ask about this topic

Momentum

2 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 127

All topics

NEARL: Interacted Query Adaptation with Orthogonal Regularization for Medical Vision-Language Understanding
Aug 6, 2025Zelin Peng, Yichen Zhao, Yu Huang +5Cross-Modal Learning VLM Adaptation
MoE Key Lab of Artificial Intelligence, AI Institute, School of Computer Science, Shanghai Jiao Tong University · Department of Radiology, The First Affiliated Hospital, School of Medicine, Zhejiang University · Mohamed bin Zayed University of Artificial Intelligence +1
ADMC: Attention-based Diffusion Model for Missing Modalities Feature Completion
Jul 8, 2025Yuhan Li, Wei Zhang, Juan Chen +3Cross-Modal Learning Missing-Modality Learning
National University of Defense Technology · The Fifth Electronic Research Institute of MIIT · University of Chinese Academy of Sciences +2
Multimodal Representation Alignment for Cross-modal Information Retrieval
Jun 10, 2025Fan Xu, Luis A. LeivaCross-Modal Learning Multimodal IR
aDepartment of Computer Science, University of Luxembourg, 6, avenue de la Fonte, Esch-sur-Alzette, L-4364, Luxembourg
Bridging the Inter-Domain Gap through Low-Level Features for Cross-Modal Medical Image Segmentation
May 17, 2025Pengfei Lyu, Pak-Hei Yeung, Jing Xia +5Cross-Modal Learning Cross-Modality Medical Image Segmentation
Northeastern University, Shenyang, China · Nanyang Technological University, Singapore
ChronoSteer: Bridging Large Language Model and Time Series Foundation Model via Synthetic Cross-Modal Alignment Dataset
May 15, 2025Chengsen Wang, Qi Qi, Zhongwen Rao +2Cross-Modal Learning Multivariate Time Series Forecasting
Beijing University of Posts and Telecommunications · Huawei Noah’s Ark Lab
CLIP-RD: Relational Distillation for Efficient CLIP Knowledge Distillation
Date pendingJeannie Chung, Hanna Jang, Ingyeong Yang +2Cross-Modal Learning Cross-Modal Knowledge Distillation
Ewha Womans University, Seoul 03760, South Korea
CAT-GS: Balanced Multimodal Learning via Calibrated Gating and Fusion Surgery
Date pendingMahir Shahriar Tamim, Sharjil Khan, Md. Samiul Alim +3Cross-Modal Learning Modality Imbalance
Department of Electrical and Computer Engineering, North South University, Dhaka, Bangladesh

Aug 6, 2025, cs.CVJ/K move · Enter open · S save

NEARL: Interacted Query Adaptation with Orthogonal Regularization for Medical Vision-Language Understanding

Zelin Peng, Yichen Zhao, Yu Huang, Piao Yang+4

MoE Key Lab of Artificial Intelligence, AI Institute, School of Computer Science, Shanghai Jiao Tong University · Department of Radiology, The First Affiliated Hospital, School of Medicine, Zhejiang University · Mohamed bin Zayed University of Artificial Intelligence+1

Cross-Modal Learning VLM Adaptation Cross-Modal Alignment Medical VLMs Cross-Modal Attention Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

Computer-aided medical image analysis is crucial for disease diagnosis and treatment planning. While vision-language models (VLMs) such as CLIP exhibit strong generalization ability, their direct application to medical imaging remains hindered by a substantial domain gap. Existing methods for bridging this gap, including prompt learning and unidirectional modality interaction, typically introduce domain knowledge into only one modality. However, such approaches fail to fully exploit CLIP's inherent dual-modality structure and overlook the synergistic effect of bidirectional cross-modal interaction, resulting in persistent modality misalignment. In this paper, we propose NEARL (iNteracted quEry Adaptation with oRthogonaL Regularization), a novel parameter-efficient VLM framework for bidirectional cross-modal interaction. NEARL consists of two key components: (1) the Unified Synergy Embedding Transformer (USEformer), which dynamically generates compact cross-modal queries to facilitate interaction; and (2) the Orthogonal Cross-Attention Adapter (OCA), which decouples new knowledge into truly novel and incremental components through orthogonal regularization. This design reduces interference from incremental components, enabling more focused learning of novel information and improving modality interaction in VLMs. Notably, NEARL introduces only 1.46M learnable parameters. Extensive experiments on three medical imaging modalities demonstrate state-of-the-art performance (e.g., a 2.3% relative improvement on the pneumonia dataset), along with fast inference and low memory overhead, highlighting its effectiveness for real-world medical vision-language understanding.

Momentum

Related topics

Latest papers 127