Prompt-Based Multimodal Learning

Latest papers

8 papers0 papers in the last four weeks, down 100% on the four weeks before. 0.0% of all new papers.

Multimodal Prompt Learning with Dual-Granularity Contrastive Alignment for Preoperative Prediction of Lung Adenocarcinoma Invasiveness Using 3D CT
Nov 2, 2026Qing Huang, Xiaotong Tan, Lintong Yao +6Prompt-Based Multimodal Learning Medical Image Classification
National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, School of Biomedical Engineering, Thoracic Surgery Department of the First Affiliated Hospital, Shenzhen University Medical School, Shenzhen University, Shenzhen, China · Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, China · Department of Thoracic Surgery, Shenzhen Second People's Hospital, The First Affiliated Hospital of Shenzhen University 518035, Guangdong Province, China +1
Semantic Modality Compensation for Unsupervised Visible-Infrared Person Re-identification under Unpaired Settings
Sep 28, 2026Duanning Chen, Ke He, Bin Yang +1Missing-Modality Learning Multimodal Disentangled Representation Learning
Wuhan University
UniAR: A Unified Framework for Autism Recognition Enhanced by Multi-View Prompt Learning
Sep 25, 2026Lei Xin, Zeheng Wang, Jiayin Zhu +6Autism Spectrum Disorder Prompt-Based Multimodal Learning
Wuhan University Wuhan, Hubei, China · Northeast Normal University Changchun, Jilin, China · The Hong Kong University of Science and Technology (Guangzhou) Guangzhou, Guangdong, China +4
Incentive Noise and Structural Prior Infusion for Multi-modal Object Re-Identification
Sep 21, 2026Weixiang Zhou, Yuhao Wang, Xingguo Xu +4Multimodal Robustness Cross-Modal Alignment
Dalian University of Technology, China · New York University, USA · Nanjing University of Science and Technology, China +1
Whole-Body MRI Classification via Prompt-Based Clinical Conditioning
Aug 31, 2026Laura Daza, Marta Hasny, Cristina González +1Missing-Modality Learning Prompt-Based Multimodal Learning
Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich, Germany · School of Computation, Information and Technology, Technical University of Munich, Germany · School of Biomedical Engineering & Imaging Sciences, King’s College London, UK +1
Multimodal Prompt Learning with Irregular EHRs for Robust Monitoring of Critical Care Patients
Aug 22, 2026Yixin Yang, Yueyang Sun, Weichen Liu +2Prompt-Based Multimodal Learning Missing-Modality Learning
Faculty of Engineering, Shenzhen MSU-BIT University · International School, Beijing University of Posts and Telecommunications · School of Computer Science and Engineering, Southeast University +1
Modality-Agnostic Prompt Learning for Multi-Modal Camouflaged Object Detection
Apr 14, 2026Hao Wang, Jiqing Zhang, Xin Yang +4Prompt-Based Multimodal Learning Camouflaged Object Detection
Information Science and Technology College, Dalian Maritime University, Liaoning, 116026, China · Dalian University of Technology, Liaoning, 116024, China · Beijing University of Technology, Beijing, 100124, China
Evaluation Before Generation: A Paradigm for Robust Multimodal Sentiment Analysis with Missing Modalities
Apr 7, 2026Rongfei Chen, Tingting Zhang, Xiaoyu Shen +1Prompt-Based Multimodal Learning Multimodal Sentiment Analysis
School of Computer Science and Technology, Eastern Institute of Technology, Ningbo, China · School of Mechatronic Engineering and Automation, Shanghai University, China

Nov 2, 2026J/K move · Enter open · S save

Multimodal Prompt Learning with Dual-Granularity Contrastive Alignment for Preoperative Prediction of Lung Adenocarcinoma Invasiveness Using 3D CT

Qing Huang, Xiaotong Tan, Lintong Yao, Wufeng Xue+5

National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, School of Biomedical Engineering, Thoracic Surgery Department of the First Affiliated Hospital, Shenzhen University Medical School, Shenzhen University, Shenzhen, China · Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, China · Department of Thoracic Surgery, Shenzhen Second People's Hospital, The First Affiliated Hospital of Shenzhen University 518035, Guangdong Province, China+1

Prompt-Based Multimodal Learning Medical Image Classification Clinical Prediction Radiology VLMs Multimodal Learning 3D Medical Imaging

Open paper AskPDF arXiv

Accurate preoperative prediction of lung adenocarcinoma invasiveness is important for surgical planning and individualized treatment. However, overlapping CT appearances make it challenging to distinguish invasive adenocarcinoma (IAD) from Pre-IAD lesions. We propose a multimodal prompt-learning framework with dual-granularity image-text alignment for 3D CT-based IAD classification. A 3D ResNet-CBAM encoder extracts global and spatial visual features. A frozen RadBERT backbone is adapted with separate learnable prompts for report-level and keyword-level text encoding. These prompts are initialized using semantic representations derived from real-world clinical reports. Global contrastive alignment connects whole-volume CT representations with reports generated from clinician-annotated radiological attributes to provide overall semantic supervision. A multiple instance learning-based local alignment module associates spatial CT features with vascular-focused radiological keywords, to capture local imaging characteristics potentially associated with tumor invasion. The framework jointly optimizes classification and alignment objectives, while inference requires only CT images. Experiments included 1,593 pathologically confirmed nodules. On the test set, the proposed method achieved an accuracy of 87.92%, F1-score of 87.78%, and AUC of 0.9499. Its AUC significantly exceeded that of the image-only Res-CBAM baseline with the same visual backbone (0.9297; DeLong test, p = 0.0078). Ablation analyses supported the complementary contributions of global and local alignment and the benefit of report-based prompt initialization. Qualitative visualizations showed activation within nodules and adjacent vascular structures. These findings suggest that dual-granularity radiological supervision can enhance CT-based invasiveness prediction while preserving a CT-only inference workflow.